PlayJev

How we reviewed this: we ran the game harness in the Playwright 1.63 Python image on four CPU cores, benchmarked two games, checked episode determinism ourselves, and cross-checked the published results table against the JSON the demo ships. Then we loaded the published 0.8B weights on the same four cores and had it choose moves on frames our own harness run had captured. We did not reproduce the training, which needs an H200. No Jev calls: this is a local model.

What it is

Qwen3.5-0.8B-Base, fully fine-tuned to play ten classic browser games from raw pixels. One frame in, one forward pass, one move out. The prompt contains nothing about the game — no name, no rules — only the image and a lettered option list, and the answer is read from the probabilities of the letter tokens at the last position, which is the same readout trick the open System One models use. Here is the whole prompt, printed by the model wrapper:

Apply the question to the state. Choose exactly one of the listed options.
Respond with only its uppercase letter, with no explanation or reasoning.

<state>
<|vision_start|><|image_pad|><|vision_end|>
</state>

Question: Which move should the player make next?

Options:
A. flap: flap the wings once to rise
B. wait: do nothing and let the bird fall

Answer with one letter: A, B.
Answer:

The training recipe is behaviour cloning on 863k teacher-labelled frames followed by two DAgger rounds, where the model plays 40k frames per game and per-game search programs relabel what it visited.

The harness is the part you can use today

Each game is a vendored HTML5 project with a manifest, a hook, and a shared shim that replaces setTimeout, requestAnimationFrame, Date and performance.now with a virtual clock. The page does not advance unless the driver calls it, so a slow policy — a 0.8B model on a CPU, say — cannot desynchronise the game.

On four CPU cores with four headless pages we measured 67 env-steps/s on Snake and 131–177 on Floppy Bird, random policy, which is plenty to collect frames on a laptop. Episodes reproduced exactly: the same seed and the same action list gave byte-identical JPEG frames, scores and clock values across separate runs.

One documented caveat we ran into. HARNESS.md says randomness that happens at page load precedes the seed, and that is visible: two different seeds gave us an identical short Snake episode, because the first food placement is fixed and our scripted policy died before anything else was drawn. Seeds diversify what happens after start, not the opening position.

The published numbers match the artifacts

We can’t run an H200, so we did the next thing: checked whether the table in the README is the table the harness produced. demo/results.json carries per-game scores for random, teacher and model policies, 16 episodes each, all with same_seed: true. Every row and every vs-teacher ratio agrees with the README, and the mean comes out at 0.535 against the 0.53 quoted:

	Space Invaders	Racer	Sokoban	Snake	Pacman	Mario	Tetris	Flappy	Breakout	2048
vs teacher	1.00	1.00	1.00	0.79	0.52	0.32	0.30	0.16	0.14	0.13

These are the authors’ figures, not ours, and a consistency check is not a reproduction. But it is a check most projects in this directory would fail, and the range is worth reading as carefully as the mean: three games at their teacher, and 2048 at an eighth of it because, by the authors’ own diagnosis, the tile digits aren’t readable at 448 pixels.

What raises the whole thing above a demo is that the write-up runs its own controls. The System Two handover experiment has a random-handover control and an inverted-handover control. The claim that the second frame must be a separate image rather than a temporal patch comes with the games it hurt as well as the one it helped. And there is a paragraph explaining that Snake’s score fell between DAgger rounds while its on-policy agreement rose — with a sign-flip permutation test and the reason (the episodes got shorter, so the average moved to the easy early game). Publishing the measurement that argues against your own headline is rare.

Running the model without a GPU: don’t

requirements.txt pins flash-linear-attention, triton and a cu128 torch build. On CPU those aren’t installed, and transformers says what it is doing:

`causal_conv1d_fn` is falling back to its reference PyTorch implementation …
`chunk_gated_delta_rule` is falling back to its reference PyTorch implementation …

Correct, and very slow. On four cores a single move took 20 seconds in float32 and one to two minutes in the default bfloat16 — bf16 is emulated on CPU, so if you do run it there, pass dtype="float32" and get three to five times the speed. The authors have said that note is going into the README. An oversized frame (a full-page screenshot rather than the harness’s 448-pixel capture) pushed it past five minutes, because the cost is in the visual tokens. The authors report 43 ms per move on an H200. The 2.2 GB of weights load in under two minutes; it is the forward pass that is hopeless. If you want to watch it play, use the hosted demo or the Hugging Face Space — both were up when we checked.

The decisions themselves looked sane on the frames our own harness captured. Two Floppy Bird frames from a random-policy episode, options flap and wait:

frame 0005: choice=wait  probs=[0.097, 0.903]  confidence=0.805
frame 0060: choice=wait  probs=[0.082, 0.918]  confidence=0.835

Worth knowing if you instrument this yourself. Each Decision also carries allowed_mass — the share of the full-vocabulary softmax that lands on the answer letters — and it read about 0.0001 on every run we made, while the top token over the whole vocabulary was still B. It is not a dtype or prompt artefact: float32 and bfloat16 agree to three decimal places on the answer (0.918 against 0.918), and the checkpoint’s own chat template gives the same move at 0.949.

We asked the authors whether it looks different on the hardware they trained on, and it does not: they measured about 1e-5 on the H200 in both dtypes, and explained why it does not matter — the training loss shapes the letters relative to each other, and the decision is the softmax over the letter slots alone, so the share of the whole vocabulary never enters it. They are considering pruning the vocabulary to a few hundred entries, closer to Jev’s option space. So read allowed_mass as a diagnostic for “did the readout find its slots”, not as confidence: the docstring calling it “is the model answering at all?” is the part they plan to reword.

Licences and size

Apache-2.0 on the repository and on the weights, with Qwen3.5-0.8B-Base named as the base model. Every vendored game records its upstream and licence in its manifest — MIT for six of them, Apache-2.0 for Floppy Bird, the Unlicense for Mario, WTFPL for Pacman — and keeps the upstream LICENSE file. That is more care about vendored code than most repositories take.

The cost is size: ten games and their assets make 3,000-plus files. Clone with --depth 1.

Verdict

Two things ship here, and they have different audiences. The harness — virtual clock, seeded episodes, one hook per game, a vectorised Playwright driver — is a genuinely reusable environment for anyone training or evaluating a policy in a browser, and it runs on any laptop. The model is a GPU artifact: worth reading about, not worth loading on a CPU.

Read the results section before the code. It is the most carefully argued experiment we have seen in this ecosystem, including the parts that didn’t work.

For decision models that answer text rather than pixels, see OpenThai-SystemOne and the rest of the open models category.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Sep 21, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.