Jev Chess Lab

How we reviewed this: we ran its 26 tests in Docker and drove all six scenarios end to end against a local stand-in, then recomputed its published figures from the traces in the repository. We made no Jev calls.

What it does

jev-chess-lab, by denikuchero, hands a model the FEN, an ASCII board, the move history and the full list of legal moves, and asks it to pick one. Python plays the move, a local opponent replies, and the whole exchange is written to disk with an SVG replay. There is no chess reasoning of its own: the model is answering a multiple-choice question, and everything interesting is in how the question is posed.

Two things to know before you clone it. The endpoint is hardcoded at jev.py:19 to openrouter.ai/api/alpha/decisions — “Jev” here is the model name typesafe/jev-1.13, reached through OpenRouter, so the key is an OpenRouter key. And the research write-up and the CLI’s own output are in Russian.

The negative result

The headline mode is deliberate: propose a move, ask three review questions about it, predict the opponent’s reply, then choose. Five calls a turn instead of one.

Running the repository’s own audit script over its own recorded games: the plain mode averaged 0.21 avoidable material losses per turn at $0.000049 a turn. Deliberation averaged 0.61 losses per turn at $0.000391 — 8× the cost, 2.9× the blunders.

The README says only that no reliable strength gain was established. The data in the same repository says it played worse, which is a stronger and more useful statement than the one the author made.

The failure is legible, too. The write-up records the model correctly predicting the opponent’s punishing reply on several turns — and scoring the risk at 0.15, 0.40 and 0.47, all under the 0.65 threshold that would have made it reconsider. Across all 23 turns of the only paid run, the retry branch never fired once.

What we checked, and what held

• Every figure recomputes. Token and cost totals for six games match to the digit; “18 of 38 turns offered only one move” is exactly 18; “changed its first choice on 16 of 23 turns” is exactly 16.
• A hallucinated move can’t be played. We returned z9z9 from the stand-in; the run aborted with “API returned a move outside the legal move list” and exit code 1. There’s no retry, so one bad answer ends a paid game — the README says so.
• The budget reserves before it spends, so a turn never pays for four calls and then runs out before the fifth.
• Redaction works. We ran everything with a fake key and grepped every published artifact: no key, no home directory, no vendor request ids.
• The safety filter is blind to forks, which its own docstring admits in the first line: “Bounded legal same-square exchange analysis, not a full chess engine.” We confirmed a position where a knight fork is rated safe. Disclosed, not oversold.
• The tests test the right things, including three that assert by construction that the default mode can’t quietly use engine help.

What has already rotted: four of the six published traces can’t be reproduced by the current code — fields added, a prompt version string that no longer exists. Two days was enough.

Verdict

Not a tool, and barely a project: 1,300 lines of Python against 28 MB of published output, four commits in a day, then silence. Read it as what it is — a lab notebook someone published with the disappointing result left in, and the arithmetic left checkable.

GPL-3.0, correctly, because python-chess is. For the same instinct applied to a benchmark, see jev-benchmarks.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Sep 20, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.