Jev Self-Driving Sim

How we reviewed this: we ran it in node:22 with --network none, replacing the SDK with a local shim and /api/decide with a stand-in we wrote. We made no Jev calls.

What it does

A top-down 2D driving simulation in the browser. Sensor readings are compressed into a JSON state and sent with four questions — lane action, speed action, hazard, pedestrian yield — roughly every 180 ms, and the answers drive the car through a confidence gate. There is an optional side-by-side mode that runs a second car on an OpenRouter LLM over the same seeded course.

As a demonstration it is well built. The courses are deterministic from a seed, both cars share one spawn table, and an obstacle you click onto the track appears on both. The gate at public/js/brain.js:153 is explicit — ignore a low-confidence lane change, let a strong pedestrian yield override speed, force a slowdown on a severe hazard — and every rule that fires is written into the notes panel next to the model’s raw answer. Keeping the answer and the executed intent separately visible is the right instinct for a comparison demo.

The key handling is right too: it lives in server.js, the browser never sees it, and /api/decide returns a clean 503 without one. server.js:34 sets retry: { maxRetries: 0 } with the comment “a stale driving decision is useless, never retry”, which is exactly correct in this domain.

A failure looks like a result

public/js/main.js:93:

if (!answers) answers = localDecide(state);

Every failed Jev call becomes a local rule-based decision. Nothing about the run’s headline output says so. We ran the headless harness with a stand-in that returned HTTP 502 to every request:

{"seed":42,…,"distance_m":325,"crashed":null,"decisions":281,
 "brain":"jev avg 3ms, 281 errors, tokens 0/0, cost $0.00000"}
stand-in received 281 requests; exit code 0

Two hundred and eighty-one failures. The distance, the crash count and the average speed are all the local brain’s driving, and the label still says jev. Exit code 0.

In the browser it is worse, because the mode indicator is decided once. public/js/main.js:60:

mode: this.kind === "llm" ? (app.health.llm ? "llm" : "local") : (app.health.jev ? "jev" : "local"),

app.health.jev is whether the server had a key configured at page load. It never updates. So the canvas header keeps reading JEV · jev-latest and the comparison table keeps its Jev column heading while the local brain drives. The only honest surfaces are the answers panel, which uses last.source and will show ANSWERS · LOCAL, and the (N err) count in one cell.

The fallbacks also count toward st.calls (main.js:99), so “avg latency” and “cost per decision” — the two numbers the whole side-by-side exists to compare — get diluted by calls that never happened.

The README does mention a clearly labelled fallback when no key is set. It says nothing about what happens when a key is present and individual calls fail, which is the case that matters.

An empty variable zeroes the cost table

server.js:19 uses ?? where it needs ||:

jev: { input: Number(process.env.JEV_PRICE_INPUT_PER_M ?? 0.042), output: Number(process.env.JEV_PRICE_OUTPUT_PER_M ?? 0) },

?? only falls back on null or undefined, and Number("") is 0. So JEV_PRICE_INPUT_PER_M= with nothing after it — an easy thing to leave in a .env — makes every cost readout zero while tokens are really being spent. We confirmed it: 1,000 input tokens, "cost": 0, and all four cost cells reading $0.00000.

Nothing else validates either: JEV_TIMEOUT_MS=abc starts the server and prints timeout NaNms.

A latent path traversal

server.js:236:

const filePath = path.normalize(path.join(PUBLIC_DIR, pathname));
if (!filePath.startsWith(PUBLIC_DIR)) { res.writeHead(403); return res.end("Forbidden"); }

pathname comes from decodeURIComponent(url.pathname), and the decode happens after the URL parser — which does not treat %2f as a separator. So %2e%2e%2f survives parsing and becomes a real traversal. And because the prefix check has no trailing separator, /app/public also matches /app/publicX. We built a tree and probed it:

200  /                                      "<h1>ok</h1>"
404  /../topsecret.txt                      "Not found"
403  /..%2ftopsecret.txt                    "Forbidden"
200  /%2e%2e%2fpublic-backup%2fleak.txt     "SECRET-SIBLING"
200  /%2e%2e%2fpublicX%2fb.txt              "X-LEAK"
403  /%2e%2e%2f.env2                        "Forbidden"
403  /%2e%2e%2f%2e%2e%2fetc%2fpasswd        "Forbidden"

The reach is narrow: only sibling directories whose names begin with public. .env and /etc/passwd are not reachable. And in the shipped layout there is no such sibling — public/ sits next to scripts/, server.js, README.md and the package files — so nothing is exposed today. Add a public-backup/ or public.old/ and it is. Comparing against PUBLIC_DIR + path.sep fixes it in one character.

What it costs

About 5.1 KB per request, of which the state is 1.0 KB and the questions are 4.0 KB — so roughly 78% of every request is the same static prompt text, resent about 5.5 times a second. At 180 ms that is around 20,000 requests per car per hour, and in comparison mode two cars run at once.

Verdict

Six commits, two days old, no CI, no tests, and no licence: no LICENSE file and no license field in package.json, so none of it can legally be reused.

The simulation design is careful and the gate is a good idea. The problem is that the one thing it exists to measure — how Jev drives compared to an LLM — is the thing its own error handling quietly corrupts. We’ve reported the silent fallback, the cost zeroing and the traversal.

For a driving-shaped demo that separates the model’s claim from the verified outcome, see JevPilot.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Sep 20, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.