Typed Evals

Python toolkit that puts evaluation of LLM, RAG and agent output as typed questions, with optional calibration against human labels.

View on GitHub →

We haven't reviewed this project hands-on yet. The description above is our own summary of its README. Numbers it reports are the author's, not ours.

More in Evaluation & Benchmarks

JevBench

★ 327▲ 16

fstandhartinger/jevbench

Benchmark for typed decision models across several suites, with confidence cascades and committees reported separately.

PythonReviewed

jev-align

★ 313▲ 2

sutro-sh/jev-align

CLI from Sutro that finds the examples a Jev function is least sure about, asks you to label them, and uses GEPA to improve the question.

PythonReviewed

PZ_Optimization

★ 157▲ 18

xD3I/PZ_Optimization

Performance work on a game, notable here for the harness: the arithmetic and the noise floors are computed in code, and Jev is asked only for the verdict on what the numbers mean. No licence file.

Java

Get new Jev projects every week

New Jev releases, pricing changes, and the best new projects, once a week. No spam; unsubscribe anytime.

Powered by Buttondown. See our privacy policy.