jev-libero

How we reviewed this: we installed the core package and ran its tests, then built the full simulator stack — LIBERO at the pinned commit, robosuite and MuJoCo, CPU PyTorch — in Docker on four cores with no GPU. We replayed a recorded trajectory offline, then ran two live episodes against the real API. We don’t publish our own measurements of Jev’s accuracy or latency; the project’s own figures are labelled as its author’s.

What it does

jev-libero runs LIBERO manipulation tasks — close a microwave, open a drawer, put the alphabet soup in the basket — with Jev choosing each move. The interesting part is what happens before the question is asked.

Each step it takes the simulator state, enumerates candidate inputs, and previews each one in a reversible branch of the physics, up to eight environment steps. MuJoCo supplies the dynamics and FCL measures collision-shape distances, so every option arrives at the model with its measured consequence attached: this translation closes the door by so many degrees, that one puts the fingerpad through the frame. If nothing useful is reachable, two-step previews look for a route.

Jev then picks from that annotated list. It never proposes a motion of its own, and the state it sees is numbers and labels — no images.

It reproduced its own recorded run

The README lists one recorded example per task with seeds and counts. We ran the microwave configuration at the same seed and initial state, live, against TypeSafe:

	the project’s record	our run
outcome	success	success
decisions	14	14
environment steps	111	111
estimated cost	$0.001249	$0.001270

Same decision count, same step count, and a cost within rounding of theirs — their record used OpenRouter, ours TypeSafe, which is where the small difference comes from. Reproducing a published robotics result exactly on unrelated hardware is not something we get to write often.

The offline path is just as good. jev-libero replay examples/records/top_drawer_seed1 re-applies the saved controls through the simulator and checks the outcome:

{ "steps": 155, "max_state_error": 1.27e-11, "success": true, "api_calls": 0 }

A recorded trajectory reproduces the physics to eleven decimal places and costs nothing to verify. Anyone doubting a claim here can check it without a key.

Cost and consent

The CLI is unusually blunt about which commands spend money. run is documented as “Start a NEW, paid Jev-controlled episode”; tasks, validate-task, inspect and replay all say no API key or model calls are needed. A --budget-usd guard defaults to ten cents and aborts the episode when the running estimate would cross it, and the estimate is printed after every decision.

The output directory is created with exist_ok=False, with the comment “Never overwrite or accidentally rebill a run”. We hit that on purpose while testing and it did exactly what it says.

One thing to know: the default provider is OpenRouter, not TypeSafe. --provider typesafe switches it. It’s in the help text, but a reader who pastes the quick-start command is sending the scene state through a third party.

Setup friction we hit

The core package installs cleanly and its 34 tests pass, with 19 skipped until a simulator is configured. The robot extra was harder than the README suggests, in two ways that cost us an hour:

• pip install -e '.[robot]' fails on a slim image with Failed building wheel for evdev — a robosuite dependency that needs a C compiler and Linux headers. The README lists the pip commands but not the system packages.
• On a headless CPU box, importing robosuite dies in mujoco.egl with 'NoneType' object has no attribute 'eglQueryString', because the import chain reaches for EGL whether or not you render. MUJOCO_GL=osmesa with libosmesa6-dev installed fixes it; the README mentions neither.

Neither is jev-libero’s bug — both come from the LIBERO/robosuite stack — but they stand between a reader and the first episode, and two lines in the setup section would remove them.

Maintenance and licensing

Two days old, nine commits, one author, MIT, with a THIRD_PARTY.md that names LIBERO, robosuite and MuJoCo with their licences and the pinned LIBERO revision. CI runs the tests and the docs site build; both green. Dependencies are bounded rather than pinned in the core, and pinned exactly in the robot extra. Credentials are read from an environment variable or a key file, never written; the error message when the key is missing says “never commit credentials”.

Verdict

Of the robotics projects we have looked at, this is the one to read if you care about the shape of the integration: enumerate in code, measure in physics, let the model pick, record everything, and make the recording replayable without paying again. The replay path is the part other projects should copy.

What it is not is a robot controller. Three bundled tasks, one seed each in the published record, a simulator, and two days of history. Take the pattern, not the success rate.

For the same pattern with a lighter setup see RoboJEV, and for the browser version of “measure, then pick” see fastbrowse.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Sep 21, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.