Dohnuts

How we reviewed this: we ran its own CI — pdm install, pdm run typecheck, and the make test target that deliberately loads no model weights — and read the model card against its own claims. We made no Jev calls.

What it is

Small multimodal models that answer directly: give one a message, a document or an image and ask it to choose, judge or score, and it returns probabilities from a single forward pass, with several questions sharing the input. The README calls this “our take on System 1”. The 0.8B model is meant to run on a consumer GPU.

The model card is the reason to read it

Most model cards are marketing with a table. This one publishes the benchmark where it loses.

MODEL_CARD.md reports a JevBench comparison in which Jev leads and Dohnuts does not, states the result plainly rather than reframing it, and then opens its limitations section with the same fact: “Quality depends on the task. Jev leads on the public JevBench tasks.” (Those are the project’s measurements of another system, so we do not reproduce the figures here; the point is that they are published at all, and in the direction that costs the author something.)

The rest of that section is equally unusual:

Global temperatures do not improve every dataset’s calibration. The API’s confidence field summarizes a distribution; it is not a measured probability of correctness. Check calibration on the intended workload.

Candidate wording, order, and input length can affect decisions.

Laya Vision has possible VQAv2 training-pool exposure and A-OKVQA selection exposure. Backbone pretraining exposure is unverified. These comparisons do not establish performance on unseen data for every reference.

Bub acceptance exercises the decision tool. It does not measure autonomous planning quality.

An author flagging possible benchmark contamination in their own comparison set, and saying that their backbone’s exposure is unverified, is publishing the thing that could be used against them.

And on the headline accuracy figure, the evaluation section adds:

Development data selects weights; calibration data fits temperatures; test data does neither. This is one training seed, with no estimate of variation across seeds.

That last sentence is the one most machine-learning write-ups omit. A single-seed result is a point estimate with unknown variance, and saying so is the difference between a number and a claim.

Engineering

Apache-2.0 for the code; the weights are CC BY-NC-SA 4.0 — non-commercial research only, which is the first thing to check if you were planning to build on this.

pdm run typecheck passes clean. make test is documented as running “pytest without loading model weights” and does: 11 tests pass, covering data isolation, execution, metrics and RLCD. That is a thin suite by the standards of the tools we reviewed today — but this is a training and research repository, not a library, and the test_data_isolation.py file suggests the author is checking the thing that matters most in that setting.

Twenty-two stars, one day old when we tested it.

What we did not do

We did not download the weights or run the model. On the evidence above we would expect the reported numbers to be honest, but we have not checked them, and the review is of the repository and its claims rather than of the model’s quality.

Verdict

We keep saying that the axis separating good projects from bad ones in this directory is whether they publish what could disprove them, regardless of stars or institution. Dohnuts is a clean example: a benchmark it loses, a contamination risk it flags, a confidence field it declines to call a probability of correctness, and a single seed it refuses to dress up.

Read MODEL_CARD.md as a template. Then check the non-commercial licence before you plan anything around the weights.

For an open decision model with a trained head and its evaluation JSON shipped, see Dev-0.4B; for one read straight out of logits, SemIf and JEV-CPU.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Sep 22, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.