Foreman

What it does

Foreman puts a fast decision model above a slower coding agent. You give it a job (“Add rate limiting to the API and make sure it is properly tested”) and a repository. A Codex worker does the engineering while Foreman runs a second, independent loop that watches the work and asks Jev nine yes/no questions at a time: Is the implementation complete? Are the tests sufficient? Is the worker stuck, off track, or in need of a human?

Jev only assesses. A deterministic Python policy turns the probabilities into one of a small set of actions: continue, start a worker, start an independent verifier, steer the active Codex turn, stop, retry, finish, or escalate to a human. The authors call it “an architectural experiment, not a claim that this design is already better than a conventional coding-agent harness,” and the documentation keeps to that framing throughout.

Setup

python -m pip install '.[dev]'
cp .env.example .env          # add TYPESAFE_API_KEY
foreman run --repo ./my-project --job "Add rate limiting to the API and make sure it is properly tested."

It needs Python 3.11+, the Codex CLI with a working codex login, and a TypeSafe key. By default it talks to Codex through the experimental codex app-server protocol, which lets Foreman send guidance into a turn that’s still running. A plainer codex exec backend is available but can’t be steered. Jev is called through TypeSafe’s official Python SDK.

foreman demo runs the whole runtime, policy, persistence, and terminal UI with simulated models and workers, and needs no keys or network.

Using it

We had no OpenAI account in our test environment, so we couldn’t run a real Codex worker. Instead we tested everything around it.

• The offline test suite mostly passed. 70 of 71 tests passed. The one failure was a persistence test that expects files to be listed in a particular order, which depends on the filesystem; it’s a test issue, not a runtime bug, and v0.3.0 fixed it after our report.
• The demo works. On a fresh Git repository it ran through its simulated sequence to a finished state, and foreman runs listed the result.
• The real Jev path works end to end. We put a stub codex executable on the path that makes a small, real edit to the repository, and ran foreman run with a real Jev key on the exec backend. Foreman launched the stub worker, observed the repository, assessed it with Jev, started a verification pass, and reached a final policy decision. With a stub worker the outcome itself isn’t meaningful, so we don’t report it.
• Failures fail safe. If a Jev assessment fails, the code records the error and escalates to a human rather than letting the worker continue unsupervised.
• Steering messages are templates. When Foreman steers a worker, the text it sends Codex is a fixed template filled in with a few of the assessment’s probabilities and a direction. Nothing is generated.
• Limits are everywhere. Worker count, retries, iterations, steering attempts, per-worker and overall timeouts are all bounded and configurable, and per the README, Ctrl-C stops workers and saves a final state.

We didn’t test live steering through the App Server protocol, since it needs a real Codex session.

Permissions and data

• Your diff goes to TypeSafe. Each assessment sends the original job, git status, the last 20,000 characters of git diff, the last 12,000 characters of the worker’s output, recent events, and the previous assessment. Changes to tracked files are included; new untracked files show up only by name. There’s no redaction, so if Codex prints a secret, it can end up in an assessment.
• Codex runs unattended. Workers start with approvals set to never in Codex’s workspace-write sandbox, so Codex edits files and runs commands in your repository without asking. The README is explicit that Foreman “is not a security sandbox and does not make untrusted repositories safe.” Run it on a branch or a disposable copy.
• The worker inherits your environment, minus the TypeSafe key. Foreman loads .env from the target repository and from the current directory, then launches Codex with that environment. In v0.1.0, which we tested, Codex could therefore read your TYPESAFE_API_KEY. After our report, v0.3.0 (released September 19) removes every TYPESAFE_* variable before starting Codex. Anything else in those .env files still reaches the worker, so keep unrelated secrets out of them.
• The key stays out of Foreman’s own records. In our run the key didn’t appear in the terminal log or in the saved state and event files.
• Worker output is kept locally. Each run is saved under .foreman/runs/ in the repository, including every line of Codex output (up to 4,000 characters per line). Foreman adds /.foreman/ to .git/info/exclude, so it stays out of git status without changing tracked files, but it’s still on disk.

Maintenance

Foreman was published on September 17 under the thruwire organization and had four commits when we tested it, the latest adding live steering. It’s MIT licensed, with thorough docs on its theory, runtime, and steering design, and a well-organized offline test suite. The maintainers fixed both problems we reported in v0.3.0 within a day, with tests for each. The Codex App Server protocol it depends on is itself experimental, so expect updates when Codex changes.

Verdict

Foreman is one of the more thoughtfully designed Jev experiments we’ve reviewed: a clear separation between judgment and action, conservative defaults, bounded everything, and unusually honest documentation about what it doesn’t prove. The trade-off is inherent to the idea: a supervisor needs to see the work, so your diffs and agent output go to TypeSafe, and Codex runs without asking for approvals. Try it on a disposable copy of a repository, and treat its thresholds as starting points to calibrate, as the README itself suggests. Our confidence thresholds guide covers how.

For a lighter way to use Jev with Codex, Jev Codex Router picks the model per turn, and jev-router does the same for both Claude Code and Codex. Jev Review applies Jev to reviewing finished work instead of supervising it in progress. See our best Jev tools roundup for the tools we’ve tested.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Sep 19, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.