typesafe-computer-use

How we reviewed this: typesafe-computer-use drives macOS through Quartz, the accessibility API, and Apple’s Vision OCR, so it can’t run on our Linux test machine. We read the code at commit cc7b506 and ran its test suite on Linux with the macOS-only modules stubbed out: all 144 tests passed. The project’s own CI runs the same suite on macOS. We didn’t drive a real Mac, and we didn’t call Jev.

What it does

You give clicker a goal in plain English, and it works toward it on your Mac. Each step builds a picture of the screen without a large model:

• Apple Vision OCR reads the frontmost window, reusing unchanged regions from the previous step to save time.
• The accessibility tree adds labelled controls that OCR can’t see, like icon buttons, and even controls that are scrolled out of view.
• Deterministic context adds the frontmost app, the browser’s current URL, the focused field, the local time, and hints like “dated 2026-10-13 (in 27 days)” next to any text that contains a date.

Everything becomes one numbered list, and a single Jev request answers several Choice questions at once: which kind of action to take (click, open a site, type, scroll, wait, done), which item to click, and which website to open. The action itself is carried out by ordinary code.

Jev never writes text. When a field needs free text, a “writer” model (Claude Haiku 4.5 by default) composes it; after typing, a Jev yes/no check confirms the field holds a sensible value and clears it otherwise. When the run ends, a stronger model (Claude Sonnet 5 by default) reads the final screen and reports the answer.

The author reports a cost of about a fiftieth of a cent per step and includes a side-by-side comparison with a frontier model in the README. Those are the author’s measurements. As with all our reviews, we don’t reproduce performance figures for Jev.

Setup

It needs macOS 14 or newer, Python 3.12, and uv:

git clone https://github.com/awlevin/typesafe-computer-use
cd typesafe-computer-use
uv sync
cp .env.example .env   # TYPESAFE_API_KEY required; ANTHROPIC_API_KEY optional
uv run clicker "open the Playground"          # dry run: one step, prints what it would do
uv run clicker "open the Playground" --act    # actually drives the machine

Your terminal needs Screen Recording and Accessibility permission. Without an Anthropic key, the parts that need free text (typing, suggested URLs, and the final answer) are unavailable. The README’s advice to clear the terminal first is worth heeding: the terminal is on screen, so its text becomes part of what gets read.

Design highlights

• A dry run is the default. Without --act, it takes one step and prints what it would do.
• Several ways to stop. Press Ctrl-C, or move the mouse into the top-left corner of the screen from any app. The loop also stops itself when it’s done, when Jev’s confidence falls below 0.4, after two consecutive actions that change nothing, or after 100 steps by default.
• Clicks land on the real control. Items that came from the accessibility tree are pressed through it, so a click can’t hit whatever happens to be covering the control; a mouse click at the center of the box is the fallback.
• Choices are kept distinct. The action, the item, and the site are separate questions, so screen noise doesn’t leak into the choice of action. The README explains how overlapping options lower confidence, a lesson worth reading for anyone designing Jev questions.
• Everything is replayable. A saved capture can be re-run offline with --image, without touching the screen.
• The platform code is isolated. Only macos.py touches macOS APIs, and the README sketches how a Linux port would swap it out. An open pull request from another contributor adds a browser backend that reads the page structure instead of using OCR.

Permissions and data

• Screen Recording and Accessibility are powerful. Together they let the tool read your screen and operate any app. Using the machine during a live run fights it for the mouse and focus.
• Each step sends screen text to TypeSafe. A request includes your goal, the text of every item on screen (limited to the frontmost window and menu bar), accessibility labels, the frontmost app, the browser URL, the local time, the last eight actions, and the focused field’s label, placeholder, and first 200 characters of its value. After the writer’s text is typed, that text and the field’s new value are sent for the check.
• Anthropic receives more in three places. Typing sends the goal, recent actions, and text near the field. A suggested URL sends the goal and history. The final answer sends the goal, every action, the last screen’s text, and the screenshot itself, downscaled. So the claim that it “never sends a screenshot to a big model” holds for each step, but not for the one final answer.
• Passwords are avoided by instruction, not by code. The writer is told never to fill credential fields. We didn’t find a code-level check for password fields, so rely on your browser’s password manager as the README advises.
• Runs are recorded locally. Every run writes runs/<timestamp>/ with the raw capture of each step, an annotated copy, the exact payload sent to TypeSafe, and every probability returned. That’s excellent for debugging, but those files hold whatever was on your screen. The repository’s .gitignore excludes runs/; delete old runs you don’t need.
• Keys live in .env, which is also ignored by git. An optional CLICKER_EMAIL enables a type_email action that fills your address into the focused field.

Maintenance

Developed between September 15 and 18 and published on September 16, with 26 commits, pinned dependencies, linting, and macOS CI. The author opened three issues about the tool’s own limits (similar link names splitting the vote, deeper accessibility use, and portable OCR). MIT licensed.

Verdict

typesafe-computer-use is a carefully engineered, honestly documented computer-use loop. It’s cheap per step, stops readily, and makes every decision inspectable after the fact. Before running it on a real account, understand what leaves your Mac: on-screen text goes to TypeSafe on every step, and the final answer sends a screenshot to Anthropic. Keep runs/ tidy and let your password manager handle logins.

For another macOS approach where you supply every string it types, see agent-desktop. For a browser-only agent with a similar “choose, don’t generate” loop, see jev-ultrafast. Our best Jev tools roundup compares the tools we’ve tested.

See how it compares with other tools in Best Jev tools, tested hands-on.

Review updated Sep 19, 2026. Numbers quoted from the project are its author's own; we don't publish our own measurements of Jev.