RoboJEV

How we reviewed this: we built it in Docker on Python 3.12 with four CPU cores and no GPU, fetched the Panda assets, ran its test suite, ran a full pick-and-place episode against the real API, and ran the rule baseline in a container with no network. We do not publish our own measurements of Jev’s accuracy or latency; the numbers in the project’s README are its author’s, and labelled as such below.

What it does

RoboJEV puts a decision model in the control loop of a Franka Panda arm in MuJoCo. Each step it measures the scene, asks Jev what the arm should be doing, asks again which way to move, and hands the result to a Cartesian controller that MuJoCo executes with real contacts.

The state is the part to notice: positions, quaternions, gripper width, finger contacts, velocities and a set of precomputed relations — no images. A cube is “amber cube” with coordinates, not a picture of a cube. That makes the question cheap to ask and the answer easy to audit, and it is the same trick as Jev-cu on a desktop and Mobile Jev on a phone, applied to a robot.

Two stages per decision. The first picks an intent from eight — approach, grasp, lift, carry, lower, release, withdraw, finish — each with criteria written against measured fields. The second picks a direction per axis and a gripper command, with the instruction not to revisit the intent. Code then clamps the result to a workspace box and a step no larger than two centimetres.

It ran

One episode, seed 0, the default configuration, against the real API:

success: true   decisions: 122   executed: 122   rejected: 0
empty_grasps: 0   direction_reversals: 0   api_requests: 244

The arm approached, closed on the cube, lifted, carried, lowered, released and withdrew, and the task’s own check — contacts, gripper width, the cube resting inside the target — called it done. Two requests per decision, because of the two stages.

The same episode with --policy rule runs with no key and no network at all, on the same scene and the same success check. That is worth more than it sounds: you can watch the whole loop, read the state it would have sent, and decide whether you want to pay for the model half. Most projects in this directory cannot be looked at without a key.

What we’d check before trusting it

The cost of a decision is the whole scene. Every request carries the full state — one completed pick-and-place was 244 requests. That is the design, not a bug, but it is the number to think about before pointing this at anything long-running.

The run directory is a full history. runs/<timestamp>/ keeps the initial state, every step, and a result file. Nothing sensitive in a simulator, but this is a template people will copy into a real cell, where the state is a real workspace. The key itself is not written: environment.json records api_key_present: true and nothing else, and we grepped the whole run directory for our key and found no copy.

Failure stops the arm. A transport error or an HTTP error raises after the configured retries and the episode ends with “no action executed”. There is no fallback to the rule policy, which the docs state deliberately: the baseline exists to compare against, not to rescue a failed request. For a robot that is the right choice, and it is the opposite of what we found in pi-jev, where a failed request used to widen what the agent could reach.

Answers are validated before they move anything. validate_answer requires the choice to be one of the offered options, the probabilities to cover exactly those options, sum to one within 0.02, and put the returned choice at the maximum. Anything else raises rather than executes. Config validation refuses a step above two centimetres, an empty workspace, more than two retries, and — a nice touch — any api_url that is not HTTPS.

What it claims

The README reports 10/10 on pick and place, 10/10 on surface push and 8/10 on stacking, over fixed seeds 0–9 per task, with the rule baseline at 10/10 everywhere. Those are the author’s numbers, not ours. What is unusual is the framing around them: Wilson intervals on every rate, an explicit note that ten seeds is a small sample and not a claim of general-purpose manipulation, and a line stating that the demo videos follow simulation time and omit API waits, so they are not real-time inference. Projects usually leave that last one for a reviewer to discover.

The same care shows in the code. tasks.py opens with “never choose a JEV intent here” and decides success from contacts and geometry; the finish intent’s own description says it “is an intention, not an authoritative success label”. The model cannot mark its own homework.

Maintenance and licensing

One author, six commits, created 20 September — one day old when we reviewed it. Apache-2.0, with a THIRD_PARTY.md that names MuJoCo, the Menagerie Panda description and its pinned revision, and lists the hosted API as a separate service with its own terms. fetch_panda.py downloads the robot assets, verifies checksums against a manifest, and never runs implicitly at startup. Dependencies are pinned to exact versions. CI runs the tests; all 93 passed for us in a clean container.

Verdict

This is the first robotics project we have reviewed here, and it is a good one to start with, because it is honest about being a laboratory rather than a controller. The interesting claim — that a model which only picks from a list can run a manipulation loop if the state is measured and the checks are physical — is demonstrated, offline as well as online, with the baseline sitting next to it for comparison.

Treat the numbers as ten seeds in a simulator, keep the two-hundred-request-per-task cost in mind, and read execution.py before you adapt any of it to hardware, since that file is the only thing standing between a probability and a motor.

For the same idea on a desktop see Jev-cu, on a phone Mobile Jev, and in a browser jev-ultrafast.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Sep 21, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.