jev-plays-pokemon-red

How we reviewed this: we ran its 93 tests in Docker, decoded a battle from the repository’s own RAM fixture, and drove the full chain — options, request, validation, button sequence — against a local stand-in. We made no Jev calls.

What it does

jev-plays-pokemon-red, by valentynkit, runs Pokémon Red under PyBoy with Python doing everything except the choice: RAM decoding, pathfinding, the type chart, HP scoring, button sequences. Jev is asked only at branch points, and only to pick from options code has already enumerated and will re-validate.

The state shows the discipline. A battle sends six fields per Pokémon — species, level, HP fraction, condition, status, types — with a comment saying “No inventory, no badges, no event flags”. The arithmetic is done before the request: HP becomes “hurt” or “critical”, type matchups become “not very effective against WATER”. Dialogue text is truncated to 180 characters. Our captured battle request was 2,091 bytes, 522 input tokens, about two thousandths of a cent.

The prompt-injection risk from NPC text is acknowledged in a comment — and then structurally contained: policy.py:235 checks the chosen option against the enumerated list before any button is pressed, so the worst an NPC can do is make it pick a different legal move.

The key follows redirects

_post() builds the Authorization header and calls urllib.request.urlopen with no redirect policy. Python’s default handler strips Content-Length and Content-Type on a redirect — but not Authorization, and not across origins. A 302 from the configured endpoint hands your key to whatever host it names.

The default endpoint makes this unlikely. But the README and .env.example both suggest pointing JEV_BASE_URL at “a gateway shim, a recording proxy, or the local fake”, and every extra hop is another place a redirect can appear. One line fixes it.

The contrast is in this same batch: typesafe-ai-playground sets redirect::Policy::none() explicitly. Reported.

Honest about what it isn’t

The README’s candour is the other reason to read this repository. It says calibration is deliberately not published, and explains why: the free tier allows about five calls a window, the longest annotated run was n=5, and a Brier score on that sample “would be decoration”. The latency figures are marked n=6 and marked as measured through a gateway rather than the API directly. The comparison table writes “not run” and “not published” in the competitors’ columns rather than inventing numbers. The fourth paragraph is “The catch, up front: it cannot plan.”

We found nothing the README claims that the code doesn’t do.

Three of its tests — padded, labels_versus_raw_numbers, injected_control — are exactly the experiments we’d most want to see, and they skip without a key to record fixtures. That’s a shame rather than a fault.

Verdict

Read it for the division of labour. Everything that can be computed is computed, the model is handed a small closed question, and the answer is checked against the same list it was offered. That pattern generalises far beyond a Game Boy.

For the same instinct in a different game, see jev-chess-lab and TypeSafe Mario.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Sep 20, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.