openjev-sglang

How we reviewed this: the full server needs an NVIDIA B200 and a Modal account, which we don’t use. We read the code at commit 604664a, ran its test suite, and ran the API server in Docker against a small mock of the SGLang backend. The mock returns random probabilities, so our tests cover the API layer and its compatibility with Jev’s request format, not answer quality. We didn’t use the author’s hosted demo endpoint.

What it does

openjev-sglang is a server that implements Jev’s HTTP API (POST /v1/systemone with Noul, Choice, and Score questions) on an open model, Qwen3.6-35B-A3B, served by SGLang. No requests go to TypeSafe.

Instead of generating text, it reads probabilities directly. The shared state is sent once to warm SGLang’s prefix cache, then each question runs as a separate request that produces exactly one token. The server reads the probability of each answer label, such as A, B, and C, and normalizes them into Jev-shaped answers. So N questions cost N+1 one-token calls. Answer labels are checked against the real tokenizer at startup, so even a 64-option Choice is a single-token readout.

Like SemIf, it reproduces Jev’s interface, not Jev itself. The README says plainly that its probabilities “are not calibrated estimates of correctness”. Its confidence is an entropy measure of how peaked the distribution is.

Setup

The documented path deploys to Modal:

uv sync
uv run modal run modal_app.py     # temporary server plus a smoke test
uv run modal deploy modal_app.py  # stable public endpoint

Each container uses one B200 GPU. The first start downloads the model weights and compiles kernels, and a scaled-to-zero server returns 503 while it starts. You can also point the API at an existing SGLang backend with openjev serve --connect, which is how we ran it.

Using it

• The tests pass. All 70 offline tests, the integration test (which downloads the real tokenizer), and the linter passed.
• TypeSafe’s own SDK works with it. We pointed the official Python SDK (0.7.0) at the server with base_url. It handled all three question types, chat-message and JSON state, Noul criteria, Choice options without descriptions, a 64-option Choice, the model list, and the SDK’s new response_model parsing. Errors mapped cleanly too: a bad key raised the SDK’s authentication error, and a 65-option Choice was rejected with a clear 422.
• A few differences from Jev’s API. OpenJev requires instructions on every question, while TypeSafe’s schema makes them optional, so a request that works with Jev can fail with 422. It accepts only jev-latest or the Qwen model name, so code that pins a Jev version such as jev-1.13.0, as TypeSafe’s docs advise once you’ve tuned thresholds, needs a change. And where Jev’s response reports the versioned model that answered, OpenJev echoes the name you sent. We’ve reported these to the author.
• Limits are explicit. Up to 64 questions and 64 options, a 2 MiB body cap, per-question and total token budgets, and concurrency limits, with 413, 422, 504, and 529 responses documented. When one question fails, the others are cancelled.
• The README is precise. It explains the prompt layout, the cache behavior and its limits, and what the usage counts mean. It’s worth reading if you’re building anything similar.

The repository’s evals/ folder compares OpenJev with Jev on public datasets. Consistent with our policy, we don’t reproduce the Jev figures here.

Permissions and data

• Your data stays on your deployment. Requests go to your server and the model runs there. Beyond your own deployment, the only download is the model from Hugging Face.
• The Modal deployment is public by default. modal_app.py sets unauthenticated=True with no container cap, and the README says the default endpoint “intentionally has no auth.” Anyone who finds the URL can send requests, and each new container is another B200 on your bill. Set OPENJEV_API_KEY (through a Modal Secret, since the launcher doesn’t forward it) and consider a max_containers limit. With a key set, we confirmed that the API, model list, and documentation pages all return 401 without it, while the two health checks stay open.
• No license. The repository has no license file, so under default copyright you have no explicit right to reuse the code. Ask the author before building on it. The Qwen weights have their own terms.

Maintenance

Eighteen commits on September 17 and 18, most of them evaluation work, and one open issue. There’s a solid offline test suite but no CI. It reads as a well-built experiment rather than a supported product.

Verdict

openjev-sglang is a close reproduction of Jev’s API on your own hardware: close enough that TypeSafe’s own SDK works against it after one base URL change, and carefully engineered underneath. It suits teams that want Jev’s request format on infrastructure they control, and have the budget for a B200. Before deploying it, add an API key and a container cap, get the author’s permission to use the code, and remember that its scores need your own thresholds. Our confidence thresholds guide explains how.

For a lighter option that runs on one consumer GPU, see SemIf. For a small model trained end to end, see NanoJev. Our Jev alternatives guide compares these approaches.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Sep 19, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.