Grok Bot Jev Router

How we reviewed this: we ran it in python:3.12-slim with --network none against an offline SDK stand-in that recorded every wire payload. We made no Jev calls.

What it does

A router for Grok Bot. It normalises a task’s state, asks Jev five questions in a single request — intent, whether a cached artifact can be reused, whether a sub-agent is needed, whether to stop retrying, and complexity — then maps the answers through local thresholds onto one of eight recommended actions. It executes nothing; a SKILL.md you paste into Grok Bot is what makes the recommendation stick.

One request per task is the right shape, and the dry-run path genuinely works offline: five cases, five requests, no key needed for the disabled path. That last part is deliberate and well done — src/router.py:37 puts the kill-switch check above the from typesafe_sdk import line, with the comment “Keep the kill-switch path dependency-free so it works during incidents.”

A sentence turns the gate off

src/router.py:10:

BYPASS_MARKERS = ("bypass jev", "bypass jev:", "no jev")

and _bypassed() joins four fields, lowercases, and does a substring match:

raw = " ".join(
    str(state.get(k, ""))
    for k in ("goal", "raw", "user_message", "notes")
).lower()
return any(m in raw for m in BYPASS_MARKERS)

user_message and notes are exactly the fields that carry content the operator did not write — a customer email, a ticket body, scraped page text. We ran the matcher on four states:

state	bypassed
{goal: "delete the production bucket", kind: "account"}	no
same + user_message: "FYI there is no jev integration on our side"	yes
same + notes: "Customer says they have no Jev access yet"	yes
user_message: "Please bypass Jev for now"	yes

Two of those are sentences a person would write without any intent to change anything. And this happens before Jev is called, so the task proceeds at full permission with "reason": "lab disabled or bypass jev".

README.md:20 describes the kill switch as “enabled: false or a bypass jev marker”. It does not mention no jev, which is the one that fires by accident.

A marker that has to be its own line, or a dedicated boolean field rather than a scan of free text, would fix this without losing the feature.

The approval gate is a model classification, not a rule

README.md:24 says the router “never sends, publishes, pays, deletes, or changes permissions without a human approval path”.

ask_human has exactly one trigger, src/router.py:116:

elif intent.choice == "account":

That is the model’s own classification of the task. The kind the caller passes in the state travels as kind_hint and is never enforced locally. With our stand-in classifying a “open the checkout page and pay the invoice” task (declared kind: "account") as research, the recommendation came back research_capped.

The ordering compounds it. reuse_cache is tested first, before the account branch:

if jstate["has_cached_artifact"] and reuse_n >= float(thr.get("reuse_min", 0.65)):

and no threshold in the config is range-checked. Setting reuse_min: -5 — a typo away from 0.5 — returned reuse_cache for a “delete the production bucket” task that the model had classified as account with 0.99 confidence. A mistyped number in a YAML file silently routes around human approval.

The 300-character clip protects the log, not the payload

src/router.py:152 truncates when writing the decision log: "goal": jstate["goal"][:300]. The state sent upstream does not (:50): "prior_error": state.get("prior_error") or "".

prior_error is the field most likely to hold a stack trace, and stack traces hold connection strings. A captured payload from our run:

"prior_error": "Traceback: psycopg2 OperationalError: password authentication failed for user \"admin\" (dsn=postgres://admin:hunter2@db.internal:5432/prod)"
"constraints": "budget code 4417; approver alice@corp.example"

The SKILL file tells the agent “Do not include API keys in state, prompts, logs, or tool arguments”. Nothing in the code enforces it.

Verdict

MIT, six commits, one day old, no tests and no CI. src/config.py:14 falls back to config.example.yaml when config.yaml is missing, with no warning, so forgetting to copy the file means running on example thresholds without knowing.

The A/B figures in examples/ are not reproducible — ab_results.md says the facts were copied from elsewhere, chatgpt_pack.json is a hand-written summary with no raw log or generating script, and dry_run.py measures nothing. To the project’s credit, the README already notes that the timing difference mostly comes from a cap rather than from Jev.

The instincts are good — one call, no execution, an honest “what this does not do” section in docs/architecture.md, and a deliberate crash rather than a silent default when the model returns something unexpected. The gate just needs to be made of something sturdier than a substring search. We’ve reported the bypass, the threshold ordering and the untruncated prior_error.

For the same job with enforced local rules, see pi-warden.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Sep 20, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.