Von

How we reviewed this: we installed the published package in Docker on four CPU cores with no GPU, ran its tests, ran a third-party benchmark, and drove its server with the official typesafe-sdk. We made no calls to TypeSafe — this model is meant to replace that API, so everything ran locally.

What it is

Von is a local decision model with two backends. The default wraps a fine-tune of tasksource/ModernBERT-large-nli as a zero-shot cross-encoder: each option becomes a hypothesis, one forward pass each. The second, option-marker, puts a [MASK] before every option in a single sequence and scores them from the mask positions in one pass — that is the one the headline numbers come from.

It serves POST /v1/systemone, and the official Python SDK works against it with nothing but a base-URL change. Choice, noul and score all parsed against the official schema. GET /v1/models used to be the exception — von returned an OpenAI-shaped {"object":"list","data":[...]} where the SDK expects {"models":[...]}, so client.models.list() raised. Since bed7e73 the response carries both shapes, so either client works.

The default install answered at random, for a day

This was the finding, and it is fixed. We are leaving it here because a pinned or cached install from that window still carries it. We read the safetensors header from Hugging Face directly at three revisions:

revision	tensors	parameters	classifier head
999e01cf · 09-19 15:19	174	395,834,371	present
b50019a7 · 09-19 22:35	174	395,834,371	present
then-main · 09-20 04:24	170	394,781,696	absent

The commit that removed it is titled “release Von-1.2 Option-Marker converged weights”. The option-marker weights live in a separate file, option_marker.pt; what landed on model.safetensors was the encoder body with the classification head stripped off.

So AutoModelForSequenceClassification newly initialised a random three-class head on every load. Transformers said so in a warning and then continued. Three runs of the README’s own first example, same input, separate processes:

infrastructure 0.479 / 0.365 / 0.403

Different answers each time. On a third-party benchmark shipped in the repository — 78 cases, byte-identical to the upstream suite it came from — we measured:

what we ran	micro
pip install von-sdk, README example, today	0.256
the same weights pinned to b50019a7	0.923
option-marker, hand-assembled	0.936

So the model was good and the published default was broken. The 93.5% macro figure in the README reproduced for us exactly — but only after downloading a second 1.58 GB file by hand, placing it in an undocumented directory, and setting an environment variable the README never mentions. von serve --backend option-marker was rejected outright: the CLI’s choices were modernbert, laya, needle, berta-v3.

Fixed in bed7e73, hours after we reported it. We re-read the safetensors header on Hugging Face at 94f2207, the current main:

revision	tensors	parameters	classifier head
94f2207 · 09-20 13:35	174	395,834,371	present

classifier.weight [3, 1024], classifier.bias [3], head.dense.weight and head.norm.weight are all back, so AutoModelForSequenceClassification has nothing left to initialise. Three more things landed with it: option-marker is in the CLI’s choices and is now the default backend, OptionMarkerBackend downloads option_marker.pt from the Hub when there is no local checkpoint, so the flagship path no longer needs a manual file, and GET /v1/models answers in both shapes.

One thing to know about that download path — or rather, one thing that used to be true. If hf_hub_download failed while the encoder was already available, except Exception built the model anyway with a freshly initialised OptionMarkerScorer, and nothing was logged; we ran three option representations through six fresh scorers and the winner moved (1, 0, 0, 2, 0, 0). Both load paths also passed strict=False, so a checkpoint missing its scorer.* keys loaded quietly.

Fixed in d3874e3, four hours after we reported it. We re-ran both halves at 2656a69. With no local checkpoint and no network:

RuntimeError: Failed to load Option-Marker decision weights: could not find local
'...option_marker.pt' and failed to fetch 'option_marker.pt' from Hugging Face Hub
('wfzyx/von-1.0'). Refusing to run with an untrained random scoring head.

And a checkpoint whose scorer.* keys we stripped by hand is now rejected rather than accepted:

RuntimeError: Error(s) in loading state_dict for OptionMarkerModel:
  Missing key(s) in state_dict: "scorer.input_norm.weight", ... "scorer.out_proj.bias".

A successful load also prints where the weights came from, which is the part that was silent before.

The repository’s own tests caught the original regression, and a CI workflow now runs them on every push. Its first runs failed before reaching a test — uv sync --dev installs the dev dependency group, while pytest is declared as an optional-dependency extra — which the author fixed in 13fbfc4; the suite has been green since.

The confidence you get, and the confidence it fitted

When we reviewed this, the Hub shipped marker_calibration.json beside the weights with a temperature of 2.2 fitted on the project’s own validation set — and the default install never read it. OptionMarkerBackend looked for the file in self.checkpoint_dir, but the Hub branch runs precisely when that directory holds no weights, and it downloaded only option_marker.pt. So after pip install von-sdk the temperature stayed at 1.0.

Same weights, same questions, on four cores. A was the default install; B was the identical files placed in a local checkpoint directory, so the calibration was found:

ticket	winner	top P at T=1.0	top P at T=2.2	confidence
“charged twice for order 4417”	billing	1.0000	0.9906	1.000 → 0.985
tracking page erroring, already paid	shipping	0.9985	0.9384	0.997 → 0.892
“hello, this is not working, please help”	technical	0.9994	0.9564	0.999 → 0.926

The winner never moved; only the distribution did. That is the whole point of a temperature, and it is the number you act on: confidence is the margin between the top two options, so a caller routing on confidence >= 0.95 got a different set of decisions depending on whether a file happened to be on disk.

Reported with a two-line suggestion — fetch the calibration in the same branch that fetches the weights, and name the temperature in the line that already names the weight source. Both were taken, in 047ff26, with the local file preferred when both exist.

We re-checked at 14be898, and the calibration file is now in the snapshot the default install downloads, next to the weights. The load line says what it is using:

[von] Loaded von-1.1.0 weights from Hugging Face Hub 'wfzyx/von:option_marker.pt'
      (…/snapshots/d8bb5e0/option_marker.pt) (input-conditioned calibration map active)

and marker_calibration.json is in that snapshot directory, which it was not before.

We are not republishing the table above as a before-and-after, because two things moved at once: the fix landed, and so did new weights. The calibration is no longer one number. marker_calibration.json still carries temperature: 2.2, but beside it is a calibration_map that computes the temperature per request from the answer’s entropy, the state’s token count and the number of options, clamped to [0.3, 12.0], with the fitting set named and a per-tier ECE recorded in the same file. Comparing today’s numbers against yesterday’s would be comparing two models, not two code paths.

What we can say about the default install today is that it is no longer pinned near certainty. The three tickets above come back at confidence 0.865, 0.253 and 0.960 — and that last one is the caution that survives the fix. A ticket reading, in full, “hello, this is not working, please help” is still answered at 0.976 on the winning option. Calibration moved the number; it did not give the model information the ticket does not contain.

What we measured

Four cores of an i3-9100T, fp32, no GPU, 30 timed calls after five warm-ups, timing only the library call:

	default backend	option-marker
median latency	369 ms	231 ms
p95	420 ms	269 ms
resident memory	1.77 GB	1.97 GB
load, page cache warm	1.6 s	2.3 s

The README’s “sub-25ms” and the repository description’s “Sub-15ms” are GPU or Metal figures, and they disagree with each other. On CPU, expect a few hundred milliseconds.

Other things worth knowing

von serve binds 0.0.0.0:8000 and only checks a key if VON_API_KEY is set, so the default is an unauthenticated inference endpoint on every interface; CORS is allow_origins=["*"] with credentials. Dependencies are unbounded >=, and the tokenizer config requires transformers 5.x despite a declared floor of 4.48. Weight loading uses weights_only=True, which is the right default and worth crediting.

The training story is muddled: the README says 250,000 examples, the script defaults to 500,000, and the training log published beside the weights says 62,991. The calibration temperatures are no longer dead code — train_option_marker.py writes marker_calibration.json and the backend reads it — but only for a local checkpoint, as above.

The repository is Apache-2.0, properly. The weights on Hugging Face carry no licence tag at all, and neither the README nor the model card credits tasksource, whose model this is a fine-tune of.

Verdict

Three days old, one author, moving fast enough to have broken the front door at 04:24 UTC and repaired it by 13:35 the same day. The architecture is real and the numbers reproduce — we got them ourselves — and as of bed7e73 you get them from the install the README describes.

That speed cuts both ways, and the thing to watch is the quiet path rather than the loud one. The silent-fallback path is closed — a failed fetch is now a RuntimeError — but the quiet path that remains is calibration: if you threshold on confidence, download the weights into a checkpoint directory of your own so the published temperature is applied, and pin the Hugging Face revision you benchmarked against.

For other local alternatives, see SemIf, NanoJev and the rest of the open models category.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Sep 23, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.