jev (shaharia-lab)

How we reviewed this: we ran its CI — cargo fmt --check, clippy -D warnings, cargo test — then drove the built binary against a loopback stand-in and walked its exit codes as a matrix. We made no Jev calls.

What it is

jev noul "Is this spam?" --state "$MESSAGE" and you get a probability, not a paragraph. Four asking commands — noul, choice, score, and eval for a request file with many questions — plus batch for JSONL and CSV, validate, mcp, profiles, and a self-describing spec.

The README is careful about what it is not: “Unofficial project… not affiliated with, endorsed by, or sponsored by TypeSafe AI.”

The exit codes are the design

docs/user-guide/exit-codes.md calls them a stable contract and promises they change only in a major release after 1.0. The table has two rows we have not seen anywhere else in this directory:

code	meaning
10	gate_false — evaluated fine, the gate condition is false. Never an error.
11	abstain — the answer is inside --abstain-band: the model cannot tell. Ask a person.

We walked it. Same question, same flags, a stand-in returning three different probabilities:

the model returned	no gate	--fail-under 0.6	--fail-under 0.6 --abstain-band 0.4,0.6
0.91	0	0	0
0.20	0	10	10
0.52	0	10	11

That bottom-right cell is the point. A shell script can branch on “yes”, “no”, and “the model was not sure enough to ask” as three separate outcomes, without parsing anything. Every other CLI we have reviewed collapses the third into one of the first two.

The rest of the table is equally deliberate: 2 means “nothing was sent, nothing was billed”, 4 is “the API rejected it, do not retry unchanged”, 5 is rate limiting, 6 is network — and 7 is a batch that partly failed, with --resume.

It lints the question before spending

Running eval on a perfectly valid request file produced this on stderr, before anything else:

warning: question `team`: choice `team` has no way out; the model must pick one
of its options even when none fits [choice-no-escape-option]

Our request had a choice between billing and tech with no “none of these”. That is a real Jev design pitfall — a Choice forces a pick — and the tool noticed without being asked. Other rules we found in the source include score-too-many-levels and state-missing.

validate --file is the same machinery run offline. With TYPESAFE_API_KEY unset it made zero requests against our stand-in and returned structured findings:

{ "file": "/tmp/bad.json", "valid": false,
  "findings": [ { "severity": "error", "rule": "type-unknown",
                  "question": "q", "path": "/questions/q/type" } ] }

A JSON Pointer to the offending field, and a named rule. That is a linter, not an error message.

Errors are objects

Every failure is JSON on stderr with the same shape — code, exit_code, message, hint, request_id, http_status, retryable:

{"error":{"code":"no_api_key","exit_code":3,
  "message":"no API key is configured for profile `default`",
  "hint":"create a key at https://console.typesafe.ai/keys, then either
          `export TYPESAFE_API_KEY=...` or store it with `jev auth …`"}}

A refused connection gave "code":"connection", exit 6, and "Connection refused (os error 111) (after 3 attempts)" — it says how many times it tried.

The base URL rule

Pointing it at a plain http:// host that is not loopback is refused outright:

the base URL must use https://; plain http:// is accepted only for a
loopback address, or when insecure HTTP is explicitly allowed

Exit 2, nothing sent. http://127.0.0.1:8891 works, which is how we reviewed it without a key. That is the right trade — it keeps a key off the wire in plaintext while leaving a local stand-in possible — but it is worth knowing before you point it at an internal proxy.

Engineering

Dual MIT / Apache-2.0, Rust, published to crates.io. 191 tests pass, cargo fmt --all --check is clean and clippy --all-targets --all-features -- -D warnings is clean. Seven CI workflows including a fuzz job, a security audit, and a live smoke test kept separate from the ordinary run.

jev spec dumps the whole command tree as JSON for agents, and jev schema prints JSON Schemas for the request file, the output and a batch record. The MCP documentation states it “contacts no host other than the configured TypeSafe base URL”.

Eleven stars at the time of writing, which is not a useful signal about this one.

Verdict

The best-engineered CLI in this directory, and the only one that gives “the model could not tell” its own exit code instead of rounding it to yes or no.

If you are putting a Jev decision into a shell script or a CI job, start here and read docs/user-guide/exit-codes.md first. The request linter alone will save you the classic mistake of a Choice with no way out.

For semantic search rather than typed questions, see jgrep; for the same care about “not sure” inside an agent loop, JevLoop and Canny.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Sep 22, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.