OpenSourceJev

How we reviewed this: we ran it in python:3.11-slim in Docker on four CPU cores, against llama.cpp b11065 and Qwen2.5-0.5B-Instruct-Q8_0 — not the author’s Windows/CUDA setup, deliberately, because the README lists Linux x86_64 as supported. We ran its test suite, drove the native engine directly, and finished with a run in a container with no network. We made no Jev calls; nothing here talks to an API.

What it does

OpenSourceJev is a FastAPI service and browser playground around one good idea: rather than asking a model to emit JSON, tokenise each allowed answer, run the prompt once, and read llama_get_logits_ith for the candidate tokens. A multi-token option such as technical_support is scored by accumulating conditional log-probabilities. The answer space is closed, so a malformed answer isn’t unlikely, it’s unrepresentable.

The binding is hand-written ctypes — 380 lines mirroring llama.cpp’s llama_model_params, llama_context_params and llama_batch structures field by field — which is both the interesting part and the fragile part.

It offers four primitives (noul, choice, score, text), a small conditional workflow language (when: "urgent == true"), and a trace that reports each candidate’s probability, logprob and token count. text generates, and the README says so; the guarantee applies to the other three.

Getting it running on Linux

The README’s system requirements say “Windows 10 / 11 (64-bit) or Linux x86_64”. The loader disagrees:

$ JEV_LLAMA_RUNTIME=/lc python -c "from app.llama_ctypes import native_status; print(native_status())"
{'available': False, 'library': '/lc/llama.dll', 'error': 'llama.dll not found'}

$ JEV_LLAMA_DLL=/lc/libllama.so python -c "…"
{'available': False, 'error': 'Could not load the prebuilt llama.cpp runtime:
                               /lc/ggml.dll: cannot open shared object file'}

_library_path() defaults to llama.dll and _load_library() then loads ggml.dll from the same directory unconditionally, so pointing JEV_LLAMA_DLL at the .so doesn’t help. Two symlinks do:

ln -s libllama.so llama.dll && ln -s libggml.so ggml.dll   # plus apt install libgomp1

After that it works properly. A three-option choice on the duplicate-charge example from the README returned billing at 0.930, with technical_support and account_access at 0.006 and 0.064, each candidate carrying its logprob and token count. Median time for that step across seven runs on four CPU cores with a 0.5B Q8 model: 512 ms (the author reports sub-100 ms on an RTX 3050 with Qwen3-1.7B, which is a different machine and a bigger model). The final run, in a container with --network none, completed in 286 ms — the “100% offline” claim in the UI is accurate.

The fix upstream is three lines: pick the library name from os.name. Everything else about the Linux path already works.

One model’s calibration, applied to every model

jev_calibration.json records a temperature of 9.4705 fitted on BoolQ, and is explicit about its scope: "model": "Qwen/Qwen3-1.7B", noul only. The README reports it cutting Expected Calibration Error from 0.20 to 0.09 — the author’s figure, on the author’s model.

It is applied to whatever GGUF you load, with no check on which one. Every noul answer goes through sigmoid((logit_true − logit_false) / 9.4705), and dividing by 9.47 compresses the output into a narrow band around 0.5 by construction. On our model:

question	shipped T = 9.4705	T = 1
“Does this request convey urgency?” (clearly yes)	0.518	0.668
“Is this message about domestic cats?” (clearly no)	0.412	0.033

The boolean verdict survives — the transform is monotone, so True/False came out right both times — but the number attached to it is nearly uninformative, and 0.412 on a question whose answer is plainly no is the kind of figure people put in a threshold. If you swap in a different model, refit or set JEV_LLAMA_NOUL_TEMPERATURE=1 and calibrate yourself.

One feature we expected to bite didn’t. The README lists “multi-token candidate scoring with length normalization (α-penalty)”, but _length_penalty_alpha() returns 0.0 unless you set JEV_LENGTH_PENALTY_ALPHA, so summed logprobs are used raw and longer labels ought to be at a disadvantage. We relabelled the winning option from billing (1 token) to billing department escalation team (4 tokens): it still won, 0.921 against 0.930. Worth knowing the default is off; not, on this evidence, worth worrying about.

The default mode is a keyword heuristic

RunRequest.mode defaults to "mock". That mode doesn’t load a model — it matches words like “urgent”, “outage” and “invoice” against the context and returns values with confidences attached:

{"outputs": {"urgent": false, "category": "technical"},
 "meta": {"mode": "mock", "model": "qwen3:1.7b"},
 "trace": [{"id": "urgent", "value": false, "confidence": 0.76}, …]}

That input was “The checkout is failing for every customer and card authorizations are failing”, and the mock decided it was not urgent, with 76% confidence, because “failing” isn’t in its word list. meta.mode does say mock, which is the saving grace — but meta.model names qwen3:1.7b, a model that was never loaded. The bundled playground always sends native, so this only reaches you if you post to /api/run yourself and leave mode out, as a reader of the quickstart might.

Maintenance and risk

MIT, eight commits, one author, 16 tests passing — and no .github directory, so nothing runs them. The FastAPI app sets allow_origins=["*"]; there are no keys involved, but it is still an open local endpoint that will load any GGUF path you hand it.

The part we would watch is the ctypes struct mirror. llama_version is bound but never compared against anything, and the struct layouts are copied from one llama.cpp build; a library with a different layout won’t fail loudly, it will read the wrong fields. Ours matched b11065 (the library reported 0.4.1-dev) and the numbers came out sane, but that is a property of the pair, not a guarantee. Pin the llama.cpp build you tested with.

Verdict

The engine is real and the trace it produces is the most informative in this group — logprob and token count per candidate is exactly what you want when a decision surprises you. Treat the calibration as a placeholder until you refit it for your own model, set mode explicitly on every request, and if you are on Linux, expect to spend five minutes on symlinks before anything loads.

For a local engine that installs the way its README says, see Laya for Node or the rest of the open models category.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Sep 21, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.