Awesome Jev Skills

How we reviewed this: we ran the test suite and every dry run in python:3.12 in Docker with --network none, so nothing could reach an API by accident. For the wire trace we patched the one endpoint constant to a local stand-in of ours, then reverted and diffed to prove nothing else changed. We also stood up a server that answers 302 to check what the client does with a redirect. No Jev calls.

What it is

Six skills for a coding agent — a general jev skill plus focused ones for triage, documents, evaluation, UI and simulation — with a catalogue of 108 scenarios, an evals directory, and a 338-line Python client that does the actual calling.

The framing in SKILL.md is the right one, and unusually disciplined about it: “Jev returns typed answers; the host agent remains responsible for planning, executing, and checking the result. It does not browse, generate prose, or remember earlier requests.” The recipe library is described as “inspiration, not a fixed menu of supported functions”. A skill that tells the agent reading it where its own limits are is a better neighbour than one that doesn’t.

The client is the reason to look

skills/jev/scripts/jev.py is 338 lines with no dependencies, and it is the most careful Jev client we have read. Four properties, each of which we checked rather than took on faith.

It refuses any endpoint but the documented two. http_json holds an allow-list and looks the URL up in it:

http://127.0.0.1:9110/v1/systemone              refused
https://api.typesafe.ai.evil.example/v1/…       refused
https://api.typesafe.ai/v1/systemone/../../x    refused

That last one matters: a lookalike host and a path-traversal suffix are both rejected by exact match rather than by a prefix test. To trace the wire at all we had to patch the constant — a one-line diff we then reverted.

It does not follow redirects, so the key cannot be leaked by one. We pointed the patched constant at a server that answers 302 Location: http://127.0.0.1:9111/… and watched both ends:

$ python -m jev decide checkpoint.json --provider typesafe
{"error": "TypeSafe HTTP 302; no automatic retry was made", "error_kind": "http", …}

requests that reached the redirect target: 0

Most HTTP clients follow a redirect with the Authorization header attached. This one hands you the status code instead.

What it sends is only what it should. With our canary key in the environment, the request body carried model, state, questions and nothing else — 1,147 bytes for the shipped checkpoint example — and the headers were Authorization, Content-Type and X-OpenRouter-Title. (That last one is sent to TypeSafe too, which is a cosmetic wart rather than a leak.) Errors are deliberately blunt — “no automatic retry was made”, a phase and a category — because the code declines to put provider error bodies or keys in a log.

Bad payloads stop before the network. The JSON loader rejects duplicate object keys and non-finite numbers, which most json.loads callers silently accept:

payload	result
NaN inside the state	rejected: “Non-finite JSON number: NaN”
two questions with the same name	rejected: “Duplicate JSON object field”
"type": "essay"	rejected: “type must be choice, noul, or score”
a choice with one option	rejected: “requires 2–255 choice criteria”

A key containing a space is rejected before the request is built, rather than producing a puzzling 401.

Setup, and the mode that isn’t Jev

python -m jev setup reports which keys are present and nothing else:

{"available": {"openrouter": true, "typesafe": true}, "jev_called": false,
 "note": "Presence is not authentication or credit validation. No network call or
          configuration change was made."}

We ran it with canary values in both variables; neither appeared in the output.

The one thing to read carefully is option B in that report: after consent, the host agent can simulate the decision with whatever model it already is, and the text says plainly “no Jev probabilities”. That is an honest escape hatch for someone without a key, and it is also the kind of thing that gets forgotten three turns later. If you use it, keep the distinction in the transcript — the skill’s own wording will help you, but nothing enforces it.

The default route is OpenRouter rather than TypeSafe direct, which adds a hop; the skill explains both and refuses to switch silently.

Tests, evals and the numbers in the badges

124 tests pass offline in three and a half seconds, and CI runs them on Python 3.10, 3.12 and 3.14 plus the same --dry-run smoke command we ran. The suite includes a red-team file that checks the evaluation preparer strips expected labels out of the request, rejects duplicate case ids and refuses ../../escape as a case id.

The badge counts are asserted by the tests — 45 catalogue entries, 108 scenarios, and the number in the badge itself — so the README’s arithmetic can’t drift without CI noticing. That is a small idea worth stealing.

The evals/ directory contains the author’s own BBH calibration pilot, run against real APIs, and it is written the way we would want: 160 items, frozen inputs, receipts, an explicit note that a high-confidence band still contained eight wrong labels, and the conclusion that a threshold “is a routing policy, not permission or a certainty guarantee”. Those are the author’s measurements, not ours, and we did not reproduce them.

Verdict

If you want Jev inside a coding agent and you’d rather not write the client, install this one. The skill text is long — your agent will read all of it as instructions, so skim it yourself first — but the engineering underneath is the most defensive in this directory, and the parts we could test held up: the endpoint allow-list, the redirect refusal, the payload validation and the key hygiene all behaved exactly as documented.

MIT, 34 commits, CI green. For a smaller skill aimed at finding Jev use cases rather than running them, see Jevify.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Sep 21, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.