Abide

How we reviewed this: we built the monorepo in Docker on Node 24, ran its test suite and type check, then wrote a small repository with its own AGENTS.md, a hand-written rubric, one compliant API handler and one that breaks two rules, and ran audit, check and report against the real Jev API. We also intercepted the outgoing HTTP requests to see exactly what leaves the machine. We didn’t run it inside a live coding session.

What it does

Abide, from Coldtea AI, enforces the rules in your instruction files — AGENTS.md, CLAUDE.md and friends — on the agent that’s editing your code. It installs hooks into Claude Code, Codex and OpenCode, and on every edit (or once per turn) it asks Jev one question per rule about the diff. Break a rule and the agent is told which one, with the line of your instruction file it came from, and repairs it in the same turn.

The rules it targets are the ones nothing else can check: “never let a raw error reach a user”, “don’t create an abstraction with one caller”, “use Yup, not hand-written validation”. Rules a linter could enforce are marked as such and handed back to your linter — the model is never asked about them.

The rubric is the good idea here. .abide/rubric.json is a committed, readable file, compiled from your instruction files, where each rule carries the text you wrote, the file and line it came from, an optional scope of globs, whether it runs per edit or per turn, and the typed question to ask. Verdicts are banded: at or above 0.8 the agent is told to repair, 0.5 to 0.8 you see a note and the agent doesn’t, below that nothing happens. calibrate scores each rule against your own git history and switches off the ones that never fire; tune has the agent rewrite them.

The README leads with a replay benchmark over real sessions in two repositories, with its own latency and cost figures and a second reviewer checking the catches. We don’t republish those numbers — they’re the author’s, measured on their machines — but the method is written up in the repository, which is more than most projects offer.

Using it

• The suite passes. 101 of 102 tests, plus the type check. The one failure asserts that an unreadable file can’t be written; in our container the tests ran as root, for whom permissions don’t apply. As a normal user, all 8 tests in that file pass. Not a bug.
• audit found exactly the right file. On our two handlers, it flagged src/api/bad.ts for both the Yup rule and the raw-error rule, and cleared the compliant one. Two files, one call each, both rules in the same request.
• check runs the same path the hooks do, on uncommitted changes: per-file edit rules first, then the turn rule over the whole diff.
• report is the part you’d keep. Every rule, how it’s checked, when it runs, its scope and calibration status, and a count of what has fired. A rule that never fires is visible rather than invisible.

The data-retention claim didn’t hold on the direct path

Under “Cost, privacy, safety” the README says:

Changed lines go to TypeSafe under your key, with zero data retention requested on every call, and nowhere else.

The first and last parts check out. The middle one doesn’t, on the path the quickstart recommends.

In the code, the evaluation call passes providerOptions: { gateway: { zeroDataRetention: true } }. That option is namespaced to the Vercel AI Gateway provider. With a direct TypeSafe key — abide login with a TypeSafe key, the first option the README offers — the request goes through @ai-sdk/typesafe-ai instead, which has no retention option at all, so the gateway namespace is simply ignored.

We confirmed it on the wire. Intercepting abide’s own requests during an audit with a direct key:

url: https://api.typesafe.ai/v1/systemone
headers: {"authorization":"<redacted>","content-type":"application/json","user-agent":"ai/7.0.105 …"}
body keys: [ 'model', 'state', 'questions' ]
mentions retention? false

No header, no field, nothing. Zero data retention is requested only when you route through the Vercel AI Gateway with AI_GATEWAY_API_KEY. If retention matters to you, use the gateway path, or ask TypeSafe about it for your account — and either way don’t rely on the README’s sentence.

Worth being precise about what this was: the data still goes only to TypeSafe under your own key, the same as any other tool here. It was the retention request that was missing.

Corrected the same day we reported it. The author’s fix is the right one: rather than inventing a flag, the README now states that TypeSafe’s API takes no per-request retention option and that zero data retention is an account arrangement on their enterprise tier, while the gateway path explicitly asks for zeroDataRetention: true so it routes only to providers under such an agreement. A test asserts that option is set.

What actually gets sent

The rest of the data story is exactly as documented, and we could see it in the captured requests:

• Changed lines only. The state is the file path and the unified diff — not the conversation, not the file’s other contents. That’s deliberate: Jev sees the rule and the change, so edit 200 is judged like edit 1.
• Your rules go too, as the questions. If a rule quotes something confidential, that text is in every request.
• Untracked files count. Our scratch file in the working tree was checked along with the real edits. Anything not gitignored is fair game, which is worth knowing before running check in a directory with stray notes in it.
• Nothing goes to Coldtea. No server of theirs is involved; the only endpoint is TypeSafe’s, or the gateway if you choose it.
• The key. TYPESAFE_AI_API_KEY — the AI SDK’s name, not TypeSafe’s own TYPESAFE_API_KEY, which tripped us up for a moment. Lookup order is the environment, then .env.local and .env at the repo root, then ~/.abide/.env, owner-only. Never a flag, never logged.
• Failures fail open. No key or no network means the edit goes through unchecked, and the miss is recorded in .abide/events.jsonl where report counts it. Every hook path exits 0 with a deadline, so it can’t wedge your session — a choice we’d make too, as long as you know a silent miss is possible.

One thing to plan for

Compiling the rubric runs Claude Code: abide compile spawns claude -p with --permission-mode acceptEdits to turn your instruction files into rules. In a session the agent does the compiling itself via the bundled skill, so Codex and OpenCode users are covered in the normal flow — but the standalone command expects Claude Code on the path. Worth knowing if that isn’t your setup. The rubric is plain JSON, so writing or editing rules by hand works fine; that’s how we made ours.

Maintenance

Created September 18, 17 commits, MIT licensed, published as @coldtea/abide 0.0.5, with a schema package, 100-plus tests, a documented replay benchmark and no open issues. It’s the most finished-looking project of its age we’ve reviewed — an npm package, three agent integrations, calibration and tuning commands — while still being a 0.0.x from one week.

Verdict

Abide is aimed at a real gap: the rules teams actually write down are the ones no tool enforces, and a per-edit typed check is a sensible way to enforce them. In our tests it caught the handler that broke our rules, ignored the one that didn’t, and explained every verdict by pointing at the rule and the line it came from. The rubric-as-a-committed-file design means a wrong verdict is something you can argue with and edit, which matters more than any accuracy claim.

Before you install it: know that a failed check quietly lets the edit through, which is a reason to read .abide/events.jsonl now and then rather than a reason to avoid it. The retention wording that was wrong when we reviewed it was corrected the same day.

For a different take on agent quality gates, see pi-warden and Foreman; for reviewing the diff itself, Jev Review and Jev Review MCP.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Sep 20, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.