Open Alternative to Jev

How we reviewed this: we ran it in python:3.13-slim in Docker on a 4-core CPU with no GPU, loading Qwen/Qwen2.5-0.5B-Instruct at revision 7ae5576. We made no Jev calls.

What it does

The neatest idea in this group. It assembles your state and every question into one ChatML sequence, leaving a reading position per question, then runs one forward pass and reads each question’s answer from the next-token distribution at its position, softmaxed over the option letters A/B/C. The model never generates free text, so an answer outside your option list is not merely unlikely — it is structurally impossible.

It ships no weights. Decider.from_pretrained("<any HF model id>") and it is your choice; the package itself contains no network code at all.

What it is not

It is a library replacement, not an API one, and nothing in the repository suggests otherwise — but the distinction is easy to miss. Grepping the package for fastapi, uvicorn, systemone, http.server, flask or /v1/ returns nothing. There is no endpoint for the official SDK to point at. If you were hoping to change a base_url, this is not that; jev_local is.

What we measured

Three questions in one packed sequence, twenty timed calls after three warm-ups, serial, machine idle, Qwen2.5-0.5B in fp32:

	packed (default)	separate
median	615 ms	3,003 ms
p25 / p75	588 / 656 ms	—
p95	2,994 ms	—
load	4.8 s	—
resident	2.29 GiB	—

Packing is about five times faster than one sequence per question, which is the whole argument for the design. Note the gap between the median and the mean: one or two of twenty runs landed near three seconds, so quote the median with its spread rather than an average.

And a caveat that belongs with every number here: this is a 4-core i3-9100T with no GPU, running a 0.5B model in fp32. It says nothing about the library’s ceiling.

What it does say something about is the model you bring. On a politely worded support ticket, the 0.5B model returned:

{"name": "frustration", "choice": "3", "confidence": 0.5802,
 "probabilities": {"0": 0.2910, "1": 0.0580, "2": 0.0708, "3": 0.5802}}

3 out of 3 — furious — for a polite message. The library worked perfectly; the model is small. That is the honest summary of what this package is: a correct mechanism whose output quality is entirely the weights you point it at.

The author is upfront about the trade-off his own default makes, in a docstring at so1/decider.py:12:

mode=“packed”: all questions of an item in one sequence, one forward pass, state written once. Fastest when questions share a state; later questions can be influenced by earlier ones

That is his measurement of the effect, not ours, so we are not quoting the figure — but documenting a known weakness of your default mode, in the code, is the right instinct.

Rough edges

Option labels must be single tokens. so1/prompting.py:84:

raise ValueError(f"label {letter!r} is not a single token for this tokenizer: {toks}")

Correct to enforce, and the place a tokenizer swap will break. so1/schema.py:24 caps a choice at 26 options.

ChatML is hardcoded — user_turn_start="<|im_start|>user" and friends at prompting.py:35. A non-Qwen template needs editing, and getting it wrong raises “could not find the user turn start in the chat prompt”.

Confidence is uncalibrated. A temperature-calibration module exists at so1/calibration.py, and using it needs a few hundred labelled examples of your own.

Verdict

Apache-2.0 — with LICENSE:190 still carrying the unfilled template line Copyright [yyyy] [name of copyright owner], and the author field a generic “Open Alternative Jev contributors”. 21 commits, two days old, one contributor.

python -m pytest gives 12 passed, 1 skipped; the skip is test_vllm_backend.py because vLLM will not install on a CPU-only box, which is unavoidable rather than papered over.

There was no CI — no .github directory at all — and no lockfile. For a library whose correctness depends on tokenizer and chat-template behaviour, an unpinned transformers is the dependency most likely to change the answers underneath you. The author agreed and asked for a patch; ours was merged the same day. Two workflows now run the suite on every push: a pinned matrix on 3.10 and 3.13, and a weekly job against whatever pip resolves that Monday, so a breaking release surfaces as one failing scheduled run rather than as somebody’s bug report. Both jobs report 12 passed, 1 skipped, which is what we get locally on a warm cache with no network.

The install itself still floats — torch>=2.1, transformers>=4.51, accelerate>=0.30 — so ci/constraints.txt tells you the versions the tests passed on, not the versions you will get.

The benchmarks are scripted and the dataset revisions pinned, but the headline results were produced on an H200 slice with a 27B model and Slurm scripts carrying site-specific paths, and one dataset is not committed for licence reasons. We did not reproduce them and do not repeat them here.

The idea deserves a wider audience.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Sep 22, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.