OpenDecision

How we reviewed this: we ran it in Docker on a 4-core i3-9100T with no GPU, measured with the machine otherwise idle, and pointed the official SDK at it. We made no Jev calls.

What it does

A FastAPI service that answers /v1/systemone by translating choice, noul and score questions into repeated calls to an existing zero-shot NLI classifier — MoritzLaurer/ModernBERT-large-zeroshot-v2.0, downloaded from Hugging Face on first use. It trains nothing itself; all of the judgement comes from third-party weights.

It works. One request with three questions came back correct on all three:

"department": {"choice":"technical","probabilities":{"billing":0.008,"technical":0.891,"sales":0.101}}
"urgent":     {"noul":0.901}
"frustration":{"score":1.109,"legend":{"0":"Calm","1":"Frustrated","2":"Extremely angry"}}

It is also unusually careful about what it claims. confidence is defined as one minus normalised entropy and the code says so — “This is OpenDecision’s confidence definition, not TypeSafe’s” — and the README states that its benchmark numbers are not an official comparison. The evaluation harness, the 500-case set and the per-case results are all committed, with a SOURCES.md marking which cases are paraphrased. That is the most auditable evaluation setup in this group.

It has no authentication

src/opendecision/api/app.py declares exactly two routes:

@app.get("/health")
@app.post(

and nothing in src/ mentions an API key, an Authorization header or a bearer token — the only hits anywhere in the tree are in an example script. A bare POST to /v1/systemone with no headers returns 200.

The README tells you to bind loopback, and the code does not enforce it. Anyone who types --host 0.0.0.0 — which is exactly what you do to test it from another container — has published an unauthenticated inference endpoint. Compare jev_local, which defaults to 127.0.0.1 and requires a token.

Two things break the drop-in story

The official SDK does work against it, after two adjustments.

Its default timeout is 10 seconds, and this cannot answer that fast on a CPU. Out of the box we got TypeSafeAPITimeoutError after three retries and 31 seconds. You have to pass timeout=600.0.

/v1/models returns 404, because it is not implemented. client.models.list() raises.

Neither is OpenDecision’s fault exactly — the SDK’s defaults assume a hosted service — but they are the first two things that break, and the README does not mention either.

What we measured

One request carrying three questions, twenty timed calls after three warm-ups, serial, machine idle:

median	8,358 ms
p25 / p75	8,115 / 8,481 ms
p95	11,242 ms
model load (warm cache)	4.9 s
resident	1.06 GiB

Two caveats that matter more than the number. First, this machine is a 4-core i3-9100T with AVX2 and no AVX-512 or AMX, and the checkpoint is bfloat16 — so bf16 arithmetic here is software-emulated. On a machine with AMX this would look very different. Second, we measured with nothing else running: an earlier run with a Docker build in the background gave about 40 seconds for the same payload on the same container.

The structural reason it is slow is visible in the code: a choice question runs two “semantic compilers” and a third arbitration pass when they disagree, so one question is two or three full classifier invocations.

Which leads to the accounting. usage.input_tokens counts the serialised payload once, and the code is upfront about it:

"""
Approximate logical API input size.

This is NOT currently intended to represent actual transformer
compute, because zero-shot classification may evaluate the same
state against multiple candidate hypotheses.
"""

Fine as documented behaviour; just don’t use that field for cost modelling.

Verdict

Apache-2.0 — though LICENSE:189 still has the unfilled template line Copyright [yyyy] [name of copyright owner], so nobody is actually named as the copyright holder. Nine commits, one author, three days old, CI on Python 3.13 with 10 tests passing.

The dependency floor is steep: Python ≥3.13, torch ≥2.14.0, transformers ≥5.17.0 — all the newest releases available, with no backward margin. The uv.lock at least makes it reproducible.

We did not re-run its 500-case evaluation, so its self-reported accuracy is unverified here. What we can say is that the harness and data are present and the claims are scoped honestly, which puts it ahead of most.

We’ve reported the missing authentication and the /v1/models gap. Don’t expose it beyond loopback until there’s a key.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Sep 20, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.