jeff (Alurith)

How we reviewed this: we built it in Docker with Go 1.25, ran its tests, and pointed it at a directory we made — source, a private key, a config file with a password, a .env and a JSON file of secrets — with a local stand-in in place of the API, to see exactly what it uploads. We first reviewed it at ca03fc4 and re-ran everything at 656ca43, which is v0.1.0.

What it does

jeff, by Alurith, checks source files against twenty written rules, one Noul each, batched into one request per file:

This source file mixes multiple conceptually unrelated responsibilities.

with criteria spelling out both sides. The verdict is three-way and that is the best idea in the project: below 0.20 passes, at or above 0.80 is a violation, and the band between is inconclusive, which exits 2. A CI job can tell “we’re not sure” from “this is wrong”, which most tools can’t.

Around that sits solid Go: retry with backoff and Retry-After, a response size cap, symlink refusal, a per-answer cache written atomically at 0600, and the key stripped from the environment of every git subprocess it runs.

What it uploads

At ca03fc4 the selector excluded markdown, dotfiles and vendor directories and honoured .gitignore, but everything else was fair game. Our directory, and what left the machine:

$ jeff check .
SENT FILE (first line): package main
SENT FILE (first line): db:
SENT FILE (first line): -----BEGIN PRIVATE KEY-----

Fixed in v0.1.0, the day after we reported it. The selector is an allow-list now — IsSourceFile in internal/files/files.go accepts sixty-three source extensions and ten well-known filenames, and every rule is skipped for a file outside it unless the rule opts in with allow-non-source, which no shipped rule does. A test asserts that no embedded rule matches a non-source file.

We re-ran the same directory at 656ca43 against our stand-in, with a distinct canary string planted in each file, and watched the wire:

file	at ca03fc4	at 656ca43
src/app.go	sent	sent
server.pem	sent whole	not sent
config.yaml (password)	sent whole	not sent
secrets.json	sent whole	not sent
.env	not sent	not sent

One request left the machine, carrying src/app.go. Naming a file explicitly bypasses discovery ignores, but not the allow-list: jeff check server.pem and jeff check .env config.yaml secrets.json each made no request at all, because no rule applies to those files.

What is still true is the ordinary part: a source file a rule applies to is sent whole, as the request state. That is what the tool is for, and the README now has a privacy paragraph that names what is excluded — certificates and keys, .env, YAML, JSON, TOML, TFVars, Markdown and hidden paths — and says an external rule can opt back in with allow-non-source: true. Treat the source you point it at as source you are handing to a third party.

The same run still leaves a .jeff-cache/ directory in the scanned tree. (The cache itself is fine: it stores only a model name and a probability, keyed by a hash.)

Its tests, and its licence

At ca03fc4 five tests across three packages failed — real assertion failures, expecting a two-rule catalogue the repository no longer shipped — and CI had been red on every run. Both are resolved. At 656ca43 every package passes:

ok  jeff/cmd/jeff  ok  jeff/internal/check  ok  jeff/internal/files
ok  jeff/internal/rules  ok  jeff/internal/cache  ok  jeff/internal/typesafe

CI has been green since 82b6b0c. The repository now carries an Apache-2.0 licence, the README’s link to a documentation directory that did not exist is gone, and v0.1.0 is tagged with checksummed release archives.

Verdict

The design deserves attention — written rules, a probability per rule, and an inconclusive band that a build can act on. If you want a semantic check in CI, this is the right shape, and after two days of fixes it is a reasonable thing to try on a repository you would be comfortable sending to an API.

It is still three days old and one person’s project, so pin the version, read the twenty rules before you trust a verdict, and add .jeff-cache/ to your .gitignore.

For a JavaScript-only screen, see Jev Review; for rules from your own instruction files, Abide.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Sep 20, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.