Jevify

How we reviewed this: we ran the tests and the dry-run path in python:3.12 in Docker with no network, then patched two lines of the client to point at a local stand-in and ran a live scan over a document we wrote, with a canary string planted in it so we could see exactly what left the machine. We reverted and diffed the patch. No Jev calls.

What it is

A single skill — SKILL.md plus a reference library — whose job is to make an agent reach for Jev at the right moment: before loading a large collection into context, or when the same semantic judgment is about to be repeated across many items. Alongside it come three stdlib-only scripts: a client, a case runner, and scan.py, which ranks a long local document by relevance and returns a shortlist instead of the text.

The skill text is good on the conceptual point. Jev is framed as “a semantic condition that code can use in an if or branch”, with a table that draws the distinction most projects get wrong: Noul returns P(yes) and “near 0.5 means uncertainty, not medium intensity”, while Score is a probability-weighted position on an ordered rubric. It also tells the agent that question IDs are invisible to the model, so the target belongs in the question text — the kind of detail you only learn by getting a batch back and not knowing which answer belongs to what.

The instruction worth reading twice

One paragraph in the skill is an instruction to your agent about how to think:

Default to Jev for bulk semantic judgments. Assume it is faster and cheaper than spending your own reasoning-model tokens on the same repeated judgments.

That is a reasonable working assumption and it is stated as an assumption, which we would rather see than a fabricated benchmark. The next paragraph is the reason it works: benchmarking is “a separate task, reserved for an explicit request”, and the agent is told to “report actual usage or time when available; describe expected savings as expected rather than inventing measurements”. So the skill asks for a default, not for a claim.

Still, this is a skill that installs a bias into an agent’s tool selection, and you should know that going in. The prices it quotes are TypeSafe’s published figures with the date the author checked them.

What the scanner does, and what it sends

We built an 8,700-character handbook with a distinctive canary line buried in the middle and asked scan.py what caused the March outage.

The dry run is the part to try first:

{"dry_run": true, "units": 3, "requests": 1, "input_tokens_byte_estimate": 11332,
 "estimated_input_usd_before_retries": 0.000475944,
 "estimate_note": "Conservative JSON byte heuristic, not a tokenizer or a billed cost.
                   Retries add usage. Refresh model prices and limits before live use."}

No key, no network, and an estimate that labels itself an estimate.

The live run, against our stand-in, sent one request carrying all three chunks as the state with one Score question per chunk (u0, u1, u2), each with the same four-level relevance rubric. The body held model, state and questions and nothing else; the headers were Authorization and Content-Type. The canary was in the request, as it must be — this is a tool for reading your documents with someone else’s model, and what it sends is the document.

What comes back is a shortlist with character offsets, a confidence per unit, an audit sample of the lowest-ranked and least certain units, and this line, which we would like to see in more tools:

“Ranked reading shortlist, not exhaustive coverage or a no-match verdict. Reopen original evidence and neighbors before answering.”

Excerpts default to 800 characters and the top 8 units, so “without printing the full input” holds for a large corpus and approaches the whole thing for a small one. Set --excerpt-chars deliberately if the context you are protecting is the point.

The failure behaviour is the best part. Our first stand-in answered a different question shape than the one scan.py asked, and the report said so rather than returning an empty ranking:

"units": 3, "judged": 0, "unjudged": 3

A tool that counts what it could not judge is a tool you can put in a pipeline.

Keys, tests and maintenance

load_key reads the environment first, then a .env in the directory you run from — no parent traversal, and the docstring says “Looks only where it is told to. The value is never printed or logged,” which matched what we saw. With no key at all, the error names the console URL and suggests --dry-run. Provider error bodies are deliberately dropped: “Provider rejected the request; response body omitted to protect submitted data.”

Eight tests pass offline in fourteen milliseconds. There is no .github directory, so nothing runs them; for a five-file repository that is a smaller omission than usual, but the scripts are the part people will run unattended.

MIT, five commits, one author, and an evals/evals.json that defines how the skill should be measured against no-skill and official-skill baselines — a rubric rather than results, but a rubric with the right controls in it.

Verdict

Install this one for the habit rather than the code: it is a compact, well-written argument for pushing repeated judgments out of your reasoning model, and scan.py is a genuinely useful 250 lines. Read the “default to Jev” paragraph before you install it, decide whether you want that bias in your agent, and remember that scanning a document means sending it.

For a larger collection with a more defensive client, see Jev Skills; for the same idea aimed at a specific harness, Jevvy.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Sep 21, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.