pijev

Checked, not fully reviewed. We ran its tests — 9 pass — and drove it against a stand-in built to be biased. No Jev calls.

Jev’s answer can change when the same options are listed in a different order. pijev asks each Choice in several orders inside one system_one request and averages the label-aligned probabilities; you change only the import. By default the orders form a balanced design — every option in every position, and after every other option, equally often — which takes K orders for K options and 2K when K is odd.

The step that could go wrong is mapping each order’s answer back to the right label. So our stand-in put 0.6 on whichever option came first, whatever it was. Averaged over pijev’s orders, that bias cancelled exactly: 0.3333 each for three options, 0.25 each for four. Each option led two of the six three-option orders. The questions sent matched the README’s table — 6 for three options, 4 for four, 7 with a Noul added, which passed through untouched.

The README’s guarantee is stated precisely and is correct: by Jensen’s inequality, the average’s log loss and Brier score are no worse than those of a randomly chosen included order — not a promise of better calibration in general. Two cautions: confidence becomes the selected label’s mean probability, so the tie above reads 0.33 where TypeSafe’s published formula gives 0; and "all" allows up to 720 questions in a request, while perch found TypeSafe’s gateway rejecting more than 128. Apache-2.0, 34 stars.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Oct 4, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.