pi-jev

How we reviewed this: we installed the real extension into pi 0.87.0 in node:24 and let a real agent drive it, with endpoint pointed at a stand-in on loopback that logged every request. The agent’s own model ran on OpenRouter; the decision layer talked only to us. We made no Jev calls.

What it is

A decision layer for the Pi coding agent. Three hooks: a gate that judges bash, write and edit before they run, an output judge that reads what a bash call printed, and a jev_ask tool the model can call itself.

Four questions, one request

The gate asks whether the action is destructive, whether it sends local data off-machine, whether it goes beyond what the user asked for, and how much damage it would do. All four arrive together. From our wire log, for rm -rf /proj/data.txt:

--- request 1  questions(4): destructive, exfiltration, beyond_scope, impact
    state: {"cwd":"/proj","tool":"bash","arguments":{"timeout":30,"command":"rm -rf /proj/data.txt"},
            "platform":"linux","user_request":"Run this exact shell command and nothing else: …"}
--- request 2  questions(2): leaks_secret, failure_class
    state: {…,"is_error":false,"output":"(no output)"}

Two requests for one tool call: one before, one after. Three identical echo hello calls in a row produced the same two — the verdict cache is real and we watched it hold.

The three modes do what the README says

Same destructive command, same flagged verdict, three configurations:

configuration	what happened	file
shadow (default)	notified, ran anyway	deleted
enforce, headless, blockWithoutUI: false	warned that it is not blocking, ran anyway	deleted
enforce, blockWithoutUI: true	blocked, and the model was told why	survived

The blocked run is the one to look at, because the model receives the reason rather than an opaque refusal:

The command was blocked by a safety hook:
pi-jev: destructive 0.99, impact 3.00/3 at confidence 0.95

Note that two of the three rows let the command through, and both are defaults. That is stated plainly in the README — enforcement with no UI “would deadlock automation on a classifier’s opinion” — but it means an out-of-the-box install reports rather than prevents.

What leaves the machine

Long string arguments are elided at 400 characters, so a write call does not carry the file. We had the agent write a 3,226-character file and watched the request:

state bytes 750
content field, last 120 chars: "…filler line 024\nfil…[2826 chars elided]"

The body stayed local. What does go out is the working directory, the platform, the tool arguments in outline — and user_request, the user’s own last message, up to 1,200 characters. That is the right input for a scope question and it is also the most sensitive field in the state, and the README does not mention it.

Every failure path lets the call through

We put the gate in its strictest configuration — enforce, blockWithoutUI: true, a provider answering destructive 0.99 — and then broke the provider:

what the provider did	requests	did rm -rf run?
answered normally	2	no, blocked
HTTP 500	6	yes
HTTP 401	2	yes
a non-JSON body	2	yes
no answers key	2	yes
{"answers":{}}	2	yes, reported as clear

Six requests for two judgements on the 500 row is the retry policy working: three attempts each, and 401 and a non-JSON body correctly not retried.

Failing open is the design, and the extension says so on screen — notifyError appends (failing open). The last row is the one that does not. An empty answers map passes validation, and evaluateGate reads each missing dimension as 0, which is not “unknown” but “certainly not destructive”. The verdict comes back clear, the status line reads jev: clear (enforce), and nothing reports a failure. Driving the two functions directly:

raw answers from provider: {}
verdict: {"flagged":false,"reasons":[],"destructive":0,"exfiltration":0,"beyondScope":0}

Reported, with the smaller shape-validation gap beside it. The end behaviour is the same as every other failure; what differs is that this one claims a judgement was made.

Engineering

MIT, 1,781 lines across five files, shipped as TypeScript source with no build. biome lint is the only script. There are no tests and no CI, which for a file that decides whether rm -rf runs is the gap worth naming — Canny makes the same kind of decision in nine lines and exports them as a pure function you can walk as a truth table.

What is here instead is a written calibration record. gate.ts explains that the first draft of the destructive question asked whether data “cannot be recovered from version control”, which scored a real rm -rf src && git push --force below any useful threshold because the model could reason “it is in git”. The rewritten question is plainer. Thresholds sit at 0.90 and 0.85 rather than a round 0.7 because the author measured where ordinary edits land. That reasoning is in the source, next to the numbers it produced.

Verdict

Install it in shadow mode to watch what your agent is about to do, and read gate.ts for the question phrasing whether or not you run Pi — it is the most careful account of why these words that we have found in this category.

Do not install it expecting a guard. It fails open on every provider failure, it does not block by default, and it has no tests. For a gate that refuses to be configured into allowing, see dsh-jev; for the same decision proved as a pure function, Canny.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Sep 22, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.