Helm

How we reviewed this: we ran its suite on a clean clone, then pointed it at a stand-in on TYPESAFE_ENDPOINT and gave it a machine with exactly two coding agents installed — and a decision layer that insisted on a third. We made no Jev calls.

What it is

Three Python scripts, standard library only, installable as a Claude Code plugin, a Gemini CLI extension or an Agent Skill. The split is stated on the tin:

The probe observes. Jev judges. The chosen agent generates.

probe.py reads the machine — which agent CLIs are on PATH, their versions, whether each one says it is logged in — and calls no model. route.py turns that inventory into a typed question and asks Jev which agent should take the task. supervise.py dispatches the winner and asks Jev whether the run actually finished, because exit 0 does not distinguish “wrote the code” from “wrote about the code”.

The README claims roughly $0.0001 per decision. That is the author’s figure; we did not measure it, and we make no cost claims of our own.

The claim worth testing

The answer space handed to Jev is built from what the probe found. Recommending an agent you do not have is structurally impossible, not merely unlikely.

This is the same property we spent the day chasing in JevLoop, Canny and dsh-jev, and Helm is the fourth distinct way of arriving at it. So we gave it a machine with codex and aider and nothing else, and a stand-in that answered cursor-agent at confidence 0.99 regardless of what it was asked.

What actually went out on the wire as the agent question’s criteria:

["aider", "codex", "none"]

cursor-agent was never on the menu. And when our stand-in named it anyway:

ESCALATE -- no installed agent fits this task
  signals: kind scaffold | confidence 0.99 | needs_human 0.02
  judged by: jev

  not auto-executing because:
    - selected agent 'cursor-agent' is not in the routable set

Two independent gates, and the second one is labelled as such in the source:

# Defence in depth. With a Jev Choice this is unreachable by
# construction; it would only fire if a future edit let an unprobed
# name into the answer space.

It is unreachable by construction, and it still caught a lie from upstream. That comment is the right way to write a redundant check: state that it should never fire, keep it anyway, say why.

The auth asymmetry

The probe asks each CLI whether it is logged in, and treats the two possible disappointments differently. “We could not find a credential” never disqualifies, because tokens live in keychains no probe can enumerate — those agents show as [auth unverified] and stay routable, which is what both of ours did. Only a tool’s own status command saying it is logged out removes it from the set.

That is a small decision that takes discipline. The easy version drops anything it cannot confirm, and quietly stops routing to half your machine.

When the decision layer misbehaves

jev.py opens by refusing to have a fallback:

Jev is the decision layer, so a Jev that cannot answer means Helm has no answer. Callers surface the failure and stop rather than substituting a weaker judgement the user did not ask for.

Against our stand-in, that held:

what the provider did	what happened
HTTP 500	exit 4, no routing decision
a non-JSON body	exit 4, no routing decision
connection refused	exit 4, no routing decision
no API key at all	exit 2, with the command to fix it
a 200 carrying no answers	exit 0, reported as “no agent fits”

The last row is the one gap. compose() tolerates a missing answer by defaulting every signal to its midpoint, so an empty {"answers":{}} produces confidence 0.0, kind None, and an escalation labelled judged by: jev — a provider failure wearing the clothes of a routing conclusion. It still refuses to dispatch, which is the important half, and the README documents exit 4 for exactly this situation. Reported, alongside the larger finding below.

The tests

230 tests, 18 skipped. The skipped ones are test_jev_live.py, gated behind HELM_LIVE=1, and its docstring explains the policy:

Skipped unless a key is configured, because most runs of this suite should not need the network and must not cost anyone money.

We agree with the policy, which is why it is worth saying that three tests escape it. test_supervise.TestRealDispatch calls the real supervise(), which dispatches a worker and then asks Jev to judge it. Running that class alone, with no HELM_LIVE set, put three requests on our stand-in:

Ran 3 tests in 3.051s
OK

POSTs received: 3
  ['awaiting_input', 'failure_severity', 'needs_human', 'status', 'task_satisfied']  ×3

On a plain discover run, with a real key, those are billed. Separately, a clean clone cannot go green: without a key, seven tests fail, because the API-key check deliberately precedes the card lookup and the precondition tests assert on messages they never reach. Any key string, still offline, fixes six of them; the seventh needs a reachable Jev. There is no CI, which is consistent with nobody having run this on a fresh machine.

None of that is the product — the product behaved correctly in every case we drove. But the suite is currently a poor guide to whether the product works, and that is worth fixing before it grows.

Engineering

MIT, Python 3.10+, no dependencies installed into your environment. Thirteen capability cards, hand-written, each flag carrying its provenance — "source": "--help, verified on this machine 2026-09-20" — and, where a cheat sheet disagreed with reality, the correction is recorded in the card rather than silently applied:

the cheat sheet’s --approval-mode does not exist in the installed codex exec; -s/--sandbox is what it has

The model is pinned to jev-1.13.0 rather than an alias, with a comment explaining that thresholds calibrated against one version are invalidated by an alias shift, and noting that the concrete version cannot be discovered from GET /v1/models and has to be read off a response.

The key lands in ~/.cache/helm/credentials.json, owner-only, outside the skill so it survives updates, and is only ever echoed back masked.

Verdict

The structural claim is real and we could not get around it. As a demonstration that a routing decision belongs to a decision model — over an option set that code, not the model, decides — this sits comfortably beside the best in the category.

Install it if you genuinely have four or five agent CLIs and keep reaching for whichever one is already open. If you have one, there is nothing to route. Either way the capability cards are worth reading as an artefact in their own right: somebody sat down with thirteen CLIs and --help, and wrote down what was actually true on the day.

Ten stars and one author, so weigh it accordingly — but the reasoning in the source is a good deal more careful than the star count suggests.

For the same property proved three other ways, see JevLoop, Canny and dsh-jev; for the version that gets it wrong, Jevmind.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Sep 22, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.