Passage (Working Memory Jev)

How we reviewed this: we ran its documented setup and its full test suite, and read its evaluation documentation against its claims. We made no Jev calls — its tests are designed not to need any.

What it is

Passage is a localhost tool for educators. Paste instructional text and it proposes where a reader is being asked to hold too many ideas or relationships at once — active groups, connections between them, and how the demand changes as the passage unfolds. Jev makes the judgements.

The disclaimer comes first, and holds everywhere

The README’s second paragraph, before any feature:

This is an experimental teaching aid. Its slot estimates and confidence labels are not validated measurements of a student’s memory or calibrated probabilities of overload. Research motivates the design; it does not validate the app’s numerical predictions.

That is the correct claim for a piece of software about a contested psychological construct, and the discipline does not lapse once the README is over. From the testing section: “Tests establish implementation behavior, not psychological accuracy.” From the evaluation document: “These are engineering checks of source-linked distinctions, not learner measurements or a calibrated memory benchmark.”

It cites primary literature — McElree, Speer and Zacks, Van Dyke and McElree, Chen and Cowan — and after each one says what that work does not establish: “Direct-access retrieval experiments support that distinction, without validating this app’s count conversion.” “Their findings do not establish exact slot counts for these passages.” “the authors also caution that slower responses alone do not prove reduced memory availability.”

We have not seen another project in this directory cite research and then immediately bound what the citation buys.

The evaluation document is the review

docs/active-memory-evaluation.md is the most epistemically careful thing we have read this month, and it is worth listing what it does:

• Eight cases frozen before any response was inspected, with the fixture set’s SHA-256 printed in the document.
• “The cases are visible to implementers, so they are not a hidden holdout.” — it discloses its own weakness rather than waiting to be caught.
• A publication note stating that five raw JSON work records are not in the repository, and that “No claim of independently reproducing these five historical records follows from cloning the repository.”
• “Fixtures must not be adjusted after results are observed.”
• “Report failures and partial coverage explicitly; do not compute an overall accuracy percentage.”

And then the result itself, in bold, at the top of the run:

The current grouping model fails two important semantic tests. It handles completed familiar actions and a topic reset, but collapses fresh mappings at their comparison and discards an unfinished condition. Favorable passage-level numerical directions obscure both failures.

An author leading their own evaluation with the two things their model gets wrong — and naming the mechanism by which the good aggregate numbers hide them — is the behaviour this directory exists to reward. It is the same instinct as jev-ultralightspeed’s note that aggregate agreement is exactly the measurement that would not notice a poisoned item.

The honest counterweight: because those raw records are unpublished, this is a narrative you are trusting rather than an artifact you can recompute. The document says so itself, which is the point.

Engineering

348 tests pass on the documented setup, with no key. The README is explicit that they “use mocks and saved fixtures; they do not need a real API key or intentionally make paid provider calls” — which is why a reviewer can check the implementation without funding it.

Python 3.12+, FastAPI, spaCy, a small web front end. The backend is split sensibly — extraction, memory, relational memory, repair, confirmation, reading trace — and the API and architecture are documented.

One fragility we tripped over: backend/server.py computes WORKSPACE = APP_ROOT.parents[1], which raises IndexError if the repository sits fewer than two directories below the filesystem root. That only bites in an unusual layout — ours — and a normal clone is fine.

The licence is the thing to check first

This is not open source, and the project says so plainly: “This is publicly readable source under custom terms, not an open-source license.”

The Working Memory Jev License is free for individuals, one-person organizations, and non-commercial use. “An organization with more than one employee, or any commercial use, needs written permission first” — by email, saying who you are and how you will use it.

For a tool aimed at educators that is a defensible choice, and it is stated in the README’s third paragraph rather than buried. But if you are at a school district, an edtech company, or anywhere with a second employee, that is a conversation before an install, and licence scanners will not tell you.

Verdict

The most careful piece of writing about its own limits in this directory, attached to a working local tool with 348 passing tests.

Whether Passage’s slot counts tell an educator anything real is not a question this review can answer, and — unusually — not one the project claims to have answered either. What we can say is that everything around the claim is built the way you would want: frozen fixtures, published failures, no accuracy percentage, and a refusal to let the research it cites carry more weight than it can.

Read docs/active-memory-evaluation.md even if you never teach. Then check the licence before you install it anywhere with colleagues.

For the same refusal to over-claim from a benchmark, see Rizzo Flow and DocJev; for a project that measures whether its questions work at all, jev-calibrate.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Sep 22, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.