perch

How we reviewed this: we built it in Docker, ran its 96 tests, lint and typecheck, scanned its own repository against a local stand-in, and reproduced three findings ourselves. On 2026-09-28 we re-ran those findings against eaa137b and against the released v0.3.5, in the same image and against the same stand-in. We made no Jev calls and did not touch perchscan.com.

Update, 2026-09-28: every finding is fixed

We re-ran the review’s three findings eight days later against eaa137b, the tip of main, in the same Docker image and against the same local stand-in. All three are fixed, and the suite has grown from 96 tests to 279, all passing.

Two of the fixes are not in a published release yet. We built the released v0.3.5 (2026-09-24) from its tag and ran the same two commands against it:

	v0.3.5, the current release	eaa137b on main
perch scan generated/thing.js, with ignore: generated/**	✓ nothing to report, exit 0, nothing read	reads the file, reports on it, exit 3
the same scan with the API unreachable	✓ nothing to report, exit 0	nothing could be read: 2 failed; last error: fetch failed, exit 1

So if you install perch from npm today, both of the gate failures below are still there. They are fixed in #177, merged 2026-09-27, and #50 closed with it.

The file-rule findings shipped earlier, in v0.3.4 (2026-09-21). Method scanning and file rules now share one src/exclusions.js, and a scan of a repository with a committed node_modules, an app.min.js and an app.min.mjs sent none of the three to our stand-in — we counted a planted marker on the wire, and only src/app.js was named in any request. The minified pattern is now /\.min\.(?:[cm]?[jt]s|[jt]sx)$/, and a Choice over more than 255 options raises rather than truncating.

The cost documentation changed too. “One HTTP request per method read” is gone, and docs/rules.md has a section called The cost of a search that says searches are the most expensive thing perch does, that a rule with no answer anywhere reads every unit it covers, that this is the case a clean codebase hits every run — and that a rule stops at 400 units, “past that the rest are skipped without a word”. That last sentence is the one we would have written.

Worth noting for its own sake: the maintainer turned our findings into perch rules in perch’s own repository. .perch/rules/scan-scope.yaml now carries named-target-overrides-ignore, a-scan-that-read-nothing-fails and empty-scope-is-said, so perch scans itself for the regressions.

The sections below are the original review at 54a38d6, kept as written.

What it does

perch, by Lakeday, parses a git commit into a call graph with tree-sitter, then walks methods in descending risk order. For each one it builds a state containing the method’s source, its imports, the file’s module-level code, up to eight callees and eight callers with their source, and the call-graph edges — and asks about 29 questions in a single request. Twenty-three are built in (scan.yaml), the rest are yours, written as sentences in perch.yaml.

The built-in questions are good:

Does method contain a concrete behavioral defect that a caller can reach?

and, for each callee:

Does method call callee in a way that violates the contract evident from the callee’s source: wrong argument order, type, or shape, an unchecked result, or an ignored error?

That second one is the reason to care about this tool. Most semantic linters show the model one function. perch shows it the neighbours, which is where contract bugs actually live.

Your own rules get wrapped: write ensure: "The file contains no plaintext credentials." and what goes out is instructions: "Is \rule` true of the code below?“with your sentence ascriteria.trueand“Not so: “ + your sentenceascriteria.false`.

What we liked, measured

• One outbound host. We grepped every network primitive in src/, bin/ and the build: two lines, both in src/systemone.js, both to api.typesafe.ai. No telemetry, no results upload, no licence check, no analytics, nothing to perchscan.com. For a project with a “Perch Cloud, coming soon” signup on its homepage, that is worth saying out loud.
• The key stays put. Read from TYPESAFE_API_KEY, sent only as a bearer header. We scanned a repository with a marked key and grepped everything it wrote: no hits. perch doctor prints the key’s length and nothing else.
• Supply chain. Published to npm through OIDC trusted publishing — we checked the attestation rather than the workflow comment, and the provenance names the repository and workflow. Three runtime dependencies, two pinned exactly.
• It’s genuinely tested. 96 tests passing, lint and typecheck clean. (The suite needs git in the image; on a slim Node image a third of it fails with spawn git ENOENT, which is the image’s fault, not perch’s.)
• The cache works. Re-scanning an unchanged repository made zero requests.

Name an ignored path and it passes

This is the finding that matters, because perch is meant to gate a build.

const inScope = path => covered(path) && !ignored.some(glob => matches(glob, path));

A path you name on the command line still has to survive ignore. If it doesn’t, there are no candidates — and the comment just below says an empty scope “is an ordinary run, not a failure”. We reproduced it:

$ perch scan fixtures/broken.py        # with ignore: fixtures/**
✓ nothing to report
repo at commit c1f7aa8: 0 methods, read 0
EXIT=0

It did not look at the file. It said the equivalent of “pass”. Issue #50 is now closed and the fix is on main, but not in a release — see the update above.

Running the same command with ignore removed and the API unreachable, we got a second version of the same shape:

✓ nothing to report
repo at commit 8275d7b: 1 methods, read 0, 1 could not be read (perch doctor)
EXIT=0

There is a guard — scan.js:267 throws once parallel * 2 methods fail in a row, so sixteen by default. Below that, a total API outage is a green build. (Fixed on main, not in a release; see the update above.) A CI job scanning only the files a small pull request touched is exactly the case that sits under the threshold. We added it to #50.

File rules go around the scanner’s own rules

Method scanning filters the git tree through sourceFile, which skips vendor, node_modules, dist, build, coverage, anything over 1 MiB, and *.min.js. File-level rules don’t:

return tree.filter(item => item.type === 'blob' && matches(source, item.path))

That’s the raw tree. So in a repository with a committed node_modules, one rule matching **/*.js sends those files — whole, with no size limit, because the 48 KiB state budget is only applied on the method path. The same run correctly skips node_modules for method scanning and ships it for file rules. Reported, along with the two below; all three were fixed in v0.3.4, and we re-verified them at eaa137b.

Two smaller ones in the same family. The minified-file exclusion is /\.min\.js$/ while the scanner reads eight JS/TS extensions, so app.min.mjs is parsed and sent and app.min.js isn’t — the same bug we found in JevLint two days ago, in a different project. And where_window can build a Choice with more than the 255 options its own constant declares, once a method has more than 65,025 candidate lines.

What it costs

Batching is real: we measured an average of 29.2 questions per request, up to 37. But the arithmetic in the docs wasn’t, at the time we tested. docs/ci.md said “One HTTP request per method read”; scanning perch’s own repository — 420 methods, 26 rules — took 1,689 requests, about four per method, because file-level rules, defect-location follow-ups and search rules each cost their own.

The expensive case is counter-intuitive: an ensure_absent rule is cheapest when it finds something and most expensive when it doesn’t, because proving absence means asking every unit up to a cap of 400. Three such rules in perch’s own config burned 336 requests each and reported nothing — 60% of the run. Nothing in the documentation warns that adding one costs a few hundred calls per CI run.

Also worth knowing, at the time we tested: the base URL was hardcoded and the CLI never overrode it, so there was no way to point perch at a proxy, a gateway or a local stand-in — we patched one line to review it at all. And the README’s “.perch, the one directory perch writes to” wasn’t quite true: it appended to .git/info/exclude. Both changed in v0.3.4: PERCH_BASE_URL and PERCH_MODEL_ID are documented environment variables now, and perch writes .perch/.gitignore instead. The re-check above needed no patch.

Verdict

The idea is right and the engineering around it is better than the age of the project suggests. Four days old and 161 stars when we tested it, twelve days and 303 now, one person, and it already has OIDC publishing with provenance, a clean test suite, and a cache that makes a re-scan free. Every finding we reported was fixed, most of them within two days, and the ones about what a scan reads are now rules perch checks against itself.

Before you gate a build on it: on the published v0.3.5, a path you both name and ignore reports a pass, and so does a short scan against a dead API. Both are fixed on main and neither is in a release yet, so pin the commit or wait for the next version. The file-rule leak is fixed and shipped.

For the single-file version of the same idea, see JevLint and jev-lint. For rules taken from your agent instructions instead, see Abide.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Sep 28, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.