Jev vs. ML

Checked, not fully reviewed. We did not rerun the benchmark, which makes paid API calls and needs two GPUs on Kaggle. We checked that what it publishes is what its run produced. The results are the author’s, and as with every project’s own Jev figures we do not repeat them here.

The V3 run compares Jev 1.13.0 with eleven classical pipelines, from logistic regression to CatBoost and a voting ensemble, plus a majority baseline. It uses eight datasets, three training seeds and one shared test set per dataset, and reports balanced accuracy raw and threshold-adjusted.

The audit trail holds. The notebook and the release bundle match the SHA-256 values in manifest.json. Its three local validators pass: the artifact, the backends and the GPU process isolation. Independently of those, we matched all 112 mean-and-spread cells of the published raw table against the notebook’s saved outputs, and every one is there verbatim.

Its caveats are the right ones, and they are in the README, not a footnote. The adjusted Jev scores use labelled policy data for the binary tasks, so they are not zero-shot end to end. The “best classical” column in the results summary is, as it says, picked “retrospectively from the displayed test means”, which flatters the classical side; the full tables show every pipeline. The seed spreads are not confidence intervals. The run’s cost figure is a configured estimate, “not an invoice”. The manifest lists what is not published: per-example predictions, bootstrap intervals, run diagnostics and dataset snapshots. That last list matters most. Without per-example predictions, a difference between two rows cannot be tested for significance from the repository. Read close results as unresolved.

What a reader should take from it is the shape of the comparison, not a leaderboard. A classical pipeline gets the training split, and Jev gets none. So where Jev holds its own zero-shot, it saves you a labelling and training project. Where it does not, a trained classifier is the cheaper answer. The V4 tests in progress add policy, temporal and iterative-decision tasks in response to feedback.

17 stars, no licence file.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Oct 10, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.