pi-jev

How we reviewed this: we ran its 44 tests in Docker offline and drove its real router modules against a stand-in, including with the endpoint unreachable, then repeated the failure drive at 549c2bf. We made no Jev calls.

What it does

Three tools for the Pi coding agent: pick which registered-but-inactive tools this turn needs and switch them on, suggest a skill, and pass a question through. Plus a standalone gate that scores git diff HEAD against a criterion you write and exits 0 or 1.

Activating a narrow set of tools per turn is a good idea — it is context you don’t spend and capability you don’t expose.

The failure path expanded what’s available

When the request fails — network, 429, expired key — this runs:

} catch (err) {
  fallbackUsed = true;
}
…
if (fallbackUsed) {
  for (const c of candidates.slice(0, 3)) {
    activated.push(c.name);
    probabilities[c.name] = 1.0;
  }
}

Three things at once. It widens the tool surface on failure rather than narrowing it. It reports 1.0 — a higher confidence than any real answer could carry — for a decision no model took part in. And that number is rendered back to the model as structured data, where it reads as certainty.

We drove it with a query sharing no keywords with anything, so the local scorer ranked everything at zero and slice(0, 3) fell back to registration order:

QUERY: "summarise this README"
  activated    : ["delete_production_db","send_email","git_force_push"]
  probabilities: {"delete_production_db":1,"send_email":1,"git_force_push":1}

The README called this “fails open gracefully to local keyword shortlists”. It is a keyword shortlist when keywords match. When none do, it was whatever happened to be first. The 1.0 was not mentioned. The user-facing note said a local shortlist was used; the structured probability said the opposite.

Fixed in 549c2bf, hours after we reported it. The fallback activates nothing and claims nothing: every candidate gets probability 0, and the skill router falls back only to candidates whose terms actually match. We re-ran the same drive at that commit, with the request failing and again with no key configured:

QUERY: "summarise this README"
  activated    : []
  probabilities: {"delete_production_db":0,"send_email":0,"git_force_push":0,"read_file":0}
  Pi's active tool list: unchanged

An outage now costs you the routing, not the guardrail.

Two smaller things, also fixed

The error message told you to run /jev login, a command that was never registered. It now names the environment variable and the key file instead.

And the headline is “semantic tool routing”, while the model-selection command doesn’t use Jev at all — it is a table of regular expressions with hand-written confidence constants, as is the topology fallback. The README has a “Cost Clarity” line now that lists which paths spend a Jev request and says these two classify locally.

Verdict

The routing idea is right and the key handling is clean: environment first, then a file, never written, never printed. 44 tests, all passing.

Everything we reported was fixed within hours, and the fallback is now the conservative one: an outage narrows what the agent can reach rather than widening it.

One thing hasn’t changed. The publish workflow failed again on 20 September at 21:09 — as it did on the four releases before — and pi-jev@0.5.0 appeared on npm three minutes later. Whatever is publishing these versions is not the workflow that runs the typecheck and tests, which is worth knowing when you install one.

For routing done by a dedicated service, see jev-router and tiershift.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Sep 21, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.