Janus

How we reviewed this: we ran its 38 tests in Docker and drove a full measure() over 40 rows with a local stand-in in place of the SDK, then reproduced the budget and timezone bugs directly. On 6 October we retested the fixes in 0.3.2: all 47 tests pass, and we reran our budget scenarios with our own stand-in on a machine set to Tokyo time. We made no Jev calls.

What it does

Janus, by Firas Ayadi, answers a question most routing projects skip: should you route at all? You give it labelled data and two providers; it runs both, sweeps every confidence threshold, and writes an operating point to janus.json that the Router then executes. It will tell you the answer is DO NOT ROUTE, which is the feature.

The repository reads like a lab notebook — French comments recording what was tried and abandoned, a RESEARCH.md, two method documents — and the gate experiment (should this tool call run without asking?) has genuinely careful hygiene: arguments truncated at 2,000 characters, six secret patterns redacted, and values from your local .env matched exactly and removed.

The budget flag, fixed

At 9cb66c4, cost_of() returned None for any model missing from the price table, which had one Jev entry, jev-1.13.0, and the budget check skipped unknown costs. With the stand-in answering as jev-latest and a budget of $0.00001, all 40 calls went out. Meanwhile pricing_tier used the local hour against UTC peak windows, so on a machine not set to UTC the tier in the log disagreed with the one used for billing. Reported; the author fixed both the same day in 0.3.2, with regression tests.

We reran our scenarios:

model asked / answered	budget	calls	outcome
jev-latest / jev-latest	$0.00001	0	refused before the first call: “no price on file”
jev-1.13.0 / jev-1.13.0	$0.00001	10	stopped: projected cost exceeds the budget
jev-1.13.0 / an unpriced id we made up	$1	1	stopped: the answering model has no price
jev-1.13.0 / jev-1.13.0	none	40	finished

The budget now fails closed. The side effect, which the docs now state, is that jev-latest with --budget is refused: an alias’s cost cannot be bounded in advance, so pin the version. pricing_tier now reads the UTC clock, the same one billing uses.

What it sends

The library sends state exactly as you pass it — no truncation, no redaction. That’s defensible for a library, but the same repository’s gate experiment shows what care looks like, and measure.py:79 writes the raw input text to disk artifacts with none of it.

Verdict

The most intellectually honest framing in this category: measure, then decide, and accept “don’t” as an answer. Pin a resolved version rather than an alias: the budget flag now requires it, and the threshold only means something for the model it was measured on.

For the calibration half of the same problem, see jevcal.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Oct 6, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.