Footwork

Give your agent a computer—and a receipt.

Footwork gives a person or an agent a way to act on a computer and check what happened. It can observe a browser or desktop app, choose an explicit action, and inspect the result. Fresh target IDs and action gates tie each step to the state it observed. Retained receipts and completion checks let you inspect the evidence behind a success claim.

Use the footwork CLI one step at a time, connect a scoped job through MCP, or run a model-driven loop with TypeSafe Jev as an optional fast System 1. The Python package is called jevdual. Footwork is the product.

Why use Footwork?

Keep the agent you already use. Put Footwork between its proposals and supported computer actions. It can refuse a stale target, retain an uncertain operation, and check a completion claim against evidence. These checks matter when an action may have worked but its response is gone, or when the agent's account of success needs a second look.

When the work gets tricky	What Footwork adds
The agent says “saved,” but you need the file	Exact saved-byte checks, separate from input delivery and model judgment
Someone edits the field between observation and action	Fresh native target and value checks that can refuse the stale action
An action may have landed before the process stopped	Opt-in durable native operations that retain uncertainty and block new mutations in the same authority domain
A job crosses a browser and a document	Scoped MCP operations with source binding, append receipts and duplicate/restart status
The agent shares your Mac	Background-only input by default, with explicit foreground permissions and activity checks

Footwork fits document updates, cross-app handoffs, and jobs where a second mutation would matter. A safe stop is useful when you can see why it happened and what remains unresolved. For a read-only lookup, a simple browser test, or an API that already supplies authoritative results, your current tools may be enough.

Choose durability for the route you use. Native CLI sessions opt in with --authority; the scoped MCP service has its own journal. The legacy dedicated browser CLI doesn't inherit native durable replay. The owned headless CDP preview retains admitted operations across restart for a bounded button route. Typing and general browser completion remain unavailable. See why Footwork, and where it fits for examples and exact boundaries.

Start here

Install and get your first checked result →

The quickstart takes you from a host-specific bundle to a saved-byte check that needs no model, then to an owned browser session. Build or install the bundle, then try:

footwork tools                         # discover the installed tools
footwork schema browser start          # inspect an exact command's inputs
footwork doctor                        # see this machine's prerequisites

What you want to do	Start with
Read a JavaScript page without a desktop	Read-only Lightpanda
Read pages and operate a dedicated browser	CLI quickstart
Qualify guarded headless browser actions and restart recovery	Owned CDP preview
Work in a native app	macOS operator example or Linux X11 guide
Give an agent one scoped browser-to-document job	MCP installation and agent-host recipe
Review OCR, inspect a target or explore the simulator	Bundled tool families

footwork browser lightpanda read uses one fresh owned process and connection to return bounded DOM text, with input and editable values withheld. It enables no input actions or typing and makes no geometry or visibility claims. The initial URL runs website code, and CSS-hidden DOM content can remain. Footwork still declares full dependencies; the external engine is pinned and prebuilt.

macOS has the broader native-app qualification record. Linux desktop support is a preview of the X11/AT-SPI and Mousepad paths in the Linux guide. Repeated Linux desktop sessions still have known native teardown aborts and target-readiness timeouts. Wayland, shared Linux foreground control, and physical cameras haven't been qualified. The dated evidence below shows what was measured on each artifact.

Drive it

After installation, try the macOS Calculator route with a configured TypeSafe key and Accessibility permission for the process running Footwork:

footwork start "Compute 48 times 13" --app Calculator --require "The display shows 624"
footwork look                    # the menu of what can be done now, with ids, and the last receipt
footwork s1 --act                # System 1's proposal on this observation (one Jev call), dispatched
footwork do click 12             # through the gate; settle, reobserve, print the receipt
footwork done "624"              # the verifier judges the claim against the window and the trail

The stepwise commands also include do type | append | key | hotkey | menu | scroll | enter and status. Use footwork run for the whole loop. Its default is the measured guarded arm: an LLM chooses every step behind the guard. Select --arm dual for Jev as System 1, with the LLM handling escalations.

An action can't use an id from an older observation. Text comes from the task or from named secrets ({NAME} read from $FOOTWORK_SECRET_NAME, never printed). --authorize "<label>" names the change the operator approved. Sessions default to background_only, leaving the person's front app, pointer, and keyboard alone (--mode grants more).

start --browser drives Footwork's own Chrome profile through the Driver's DevTools route instead of the accessibility tree. Observation and literal reads don't need a model key. The s1, guarded action, and completion examples use a configured TypeSafe key. Model-driven runs also need their selected provider credentials.

For private visual inspection, use a numeric ID from the current native observation:

footwork inspect-target 12 --operation click --postcondition "The selected editor enters crop mode"

This saves a fresh window PNG, target-bound manifest, and static HTML preview in the session's inspection/ directory. The preview outlines the offered target on the original image and shows the operation and required postcondition. It makes no model calls, sends no action, and doesn't verify the stated effect.

A session that has handled secrets withholds screenshots before capture. Inspection refuses missing or unqualified geometry, truncated observations, and changed target identities. Native AX projections may be partial; the manifest records that limit, and label-based inspection requires a complete observation. Browser inspection through this command is not yet qualified.

footwork visual-check --request request.json --session SESSION prepares one native advisory document/reference or crop/subject check. It records the original capture and a structured receipt. Explicit --provider decisions --disclose enables one bounded provider request with an existing key. The default contacts no provider. A visual answer grants no action or completion authority. See visual check inputs, limits and escalation.

Measured this way on 2026-09-25 (docs/plans/legible-harness.md): Calculator 48 × 13 in 30 s (System 1 proposed every step at p ≥ 0.94, verifier accept 0.98); TextEdit append-and-save in 9 s, the save confirmed by the window title losing its edited mark.

More tools

The optional browser inspection utilities collect redacted, timestamped evidence through an existing tab/CDP adapter. These dependency-free ESM helpers are separate from the Python driver and verification gates. Their contract is tested offline. Live capture fidelity must be calibrated on the browser you choose.

The optional desk paper inbox connects a saved camera frame to local text recognition and a reviewed inbox note. Its private review keeps the original image and OCR text. footwork check-artifact checks the saved note against the approved source bytes without a model call; it does not judge recognition accuracy.

Project-linked workbench records attach reviewed evidence to an explicit project, repository revision and proposed check. The optional MuJoCo workcell explores a body interface with physical-state feedback and a conditional computer work-order update. Simulation results remain separate from real-camera and robot qualification. Its image-guided route supplies calibrated RGB views to an agent, accepts frame-bound proposals and requires fresh visual placement evidence before the conditional local computer action. The repeatable episode interface binds varied cases, retains images when perception fails, and compares fresh-agent decisions with a deterministic RGB controller. SIM3 results record all twelve attempted episodes and their limits. The independent controller owns a simulated episode in a separate process, with reconnectable operation status, cancellation evidence and conservative receipt-based restart recovery. SIM4 results record disconnect, cancellation and crash trials, retained failures and the boundary of supported recovery.

What it does, measured

Every number below comes from a run directory under jevdual/results/. The experiment definitions are in jevdual/docs/experiments/. Each result belongs to a dated run. It doesn't establish how a new app, platform, or model will behave.

The browser

The experiments compare three arms on the same tasks: stock browser-use with the LLM alone, guarded with Jev verification and the gate but no Jev actions, and dual with the guard plus Jev taking the mechanical steps. System 2 in every run is Muse Spark 1.3 Contributor over the Meta Model API. It was chosen as a cheap, fast baseline, making its cost hard for Jev to beat.

Fixture heldout, 10 local tasks, frozen settings, rerun once when the verifier changed (results/heldout-adopt.md):

arm	pass	false completions	driver steps	Jev calls	est. cost USD	wall s
stock	9/10	1 (deleted an account when told to reach the page)	33	0	0.049	708
dual	10/10	0 (paused before the click)	24	76	0.049	522

Live sites, 55 tasks on Wikipedia, MDN, docs.python.org, arXiv, GitHub, PyPI and three practice sites, three same-code runs (results/q9-live-dev.md, results/q9b-coherent.md):

arm	pass	false completions	driver steps	Jev calls	est. cost USD	cost per verified pass
guarded	50 to 52 / 55	0	226 to 260	80 to 84	0.385 to 0.412	0.0086 to 0.0100
dual	51 to 52 / 55	0	161 to 193	360 to 391	0.449 to 0.492	0.0100 to 0.0117
stock (one run)	51 / 55	1 (its own LLM judge accepted it)	265	0	0.382

Here is what those measurements support:

• The guard is the product. On these predicates, the guarded arm had zero false completions on every split and run. It used about two Jev calls per task, at the LLM's own cost. There is a useful counterexample: in Q8, the dual arm clicked an authorized Delete twice and the verifier accepted it (results/q8-consent.md, the one false completion). The LLM alone deleted the account when asked to reach a page, and its own judge marked that a success.
• Jev acting saves driver work, not money yet. Across two same-night samples, dual made 23 to 29% fewer driver requests than the guarded arm. Its own steps took a median 1.4 s against 15.7 s for an LLM step. It paid for that in Jev volume. Task wall time between the arms wasn't distinguishable on live sites, and dual has matched stock's cost only on the fixture heldout so far.
• Bounded delegation remains opt-in. Giving Jev subgoals from System 2 cost more without more completions in three measurements (Q9). The executor reached 5 of 30 assignments. An audit found software defects behind most of those failures; the repaired build was measured again (results/q9e-delegate-fixed.md): 36 verified against 41 for the same-day guarded arm, $0.644 against $0.424, 3 of 27 assignments reached. Delegation stays available as a tool, off by default.
• Declaring the contract to the driver did not pay (Q11): model requests rose 21% per task with completions unchanged. Neither did receipts for every action (Q12, results/q12-live-dev.md): repeated no-op clicks fell 18% but requests per unit of progress did not move (−0.02 [−0.14, +0.10], over the 53 tasks with some progress in both arms), and verified passes fell by 4 pooled. Receipts are kept as an honesty measure, not as a saving; the input defect they exposed is fixed first.

Native apps and a shared machine

• The arbiter's target confidence ranks wrong System 1 steps on accessibility-tree menus (Q10, results/q10-native-dev.md): AUROC 0.90 [0.85, 0.94] on Calculator and TextEdit, against 0.80 on the web. That is a ranking result on two apps, whose agreed wrong steps come from two tasks; it is not a calibrated probability, and it does not yet say anything about unfamiliar apps. After verifier and task fixes, dual is within one task of guarded (13 against 15 of 18, results/q10b-native-dev.md); the gap is one repeated-sequence task. Native stays experimental.
• A session runs beside a person without taking their work, input or focus (Q15): 15 of 15 coexistence cases passed every check (another app, another document of the same process, a foreground step beside an idle or moving person, a conflicting edit). In none of three attempts did the person's move overlap a foreground step, so that race is covered by a unit test, not a live one. background_only is the default; foreground_permitted holds the front for 36 to 65 ms and only after 3 s of idle.

How it works

                 task ─────────────────────────────────────────────┐
                                                                   ▼
   ┌──────────── System 2: the LLM driver (browser-use loop) ────────────────┐
   │  reads, compares, decides, composes; may call delegate_subgoal /        │
   │  find_evidence; proposes actions and eventually `done`                  │
   └──────────────────────────────┬──────────────────────────────────────────┘
                                  │ every action, either system
                                  ▼
   ┌──────────── the guard (code + Jev nouls, ~1 call per check) ────────────┐
   │  destructive gate: pause before delete / pay / send unless the task     │
   │  authorises that action;  done verification: `complete` + one          │
   │  `unmet` noul per requirement, judged on the page AND the trajectory,   │
   │  claims graded against evidence atoms, a per-run ledger by kind         │
   └──────────────────────────────┬──────────────────────────────────────────┘
                                  │ optional: dual mode
                                  ▼
   ┌──────────── System 1: Jev (one speculative fan-out per step) ───────────┐
   │  Choice: operation  ·  Choice: target (conditional per operation)       │
   │  Nouls: goal_done · stuck · needs_reasoning · destructive · login ·     │
   │  bot_check;  a code-owned arbiter (arbiter.toml) acts, escalates,       │
   │  retries the next-best target, or asks for confirmation                 │
   └─────────────────────────────────────────────────────────────────────────┘

• Verification (verify.py, ledger.py): a done is accepted only when the complete noul and every requirement's unmet noul are inside the accept band; answers must quote evidence atoms found on the full page text; requirements that name an action are judged from the trajectory once the page has moved on; a requirement judged met stays met unless it is a current-state requirement and the page later contradicts it.
• Gate (agent.py): keyword and noul rules on the target, the URL, the focused element for Enter, and evaluate; scoped by the task's authorized_actions.
• Arbiter (arbiter.py, arbiter.toml): confidence floors, noul thresholds, streak and no-effect rules. Every threshold is a starting point recorded in the file; none has been tuned on heldout data.
• Rust (crates/jevdual-core): paint-order occlusion, snapshot lookup, element hashes and evidence matching as PyO3 ports with pure-Python twins and equality tests. The DOM pipeline isn't the bottleneck; Jev and the driver are.

The eval rig

• evals/: arms stock, s1_only, dual, guarded, delegate, delegate_evidence; predicates on observed state (URL, page text, returned answer, URL checkpoints for multistep tasks), never on the agent's claim; browser-use's own LLM judge recorded per row as a comparison, never as the grade.
• Task sets: a local fixture site (dev, heldout), 55 live dev tasks and 24 live heldout tasks, and 42 judge-graded tasks imported from browser-use's agent tasks and Mind2Web. Heldout splits are never debugged.
• Method: pre-registered questions with predictions and decision rules, paired bootstrap intervals over tasks (evals/paired.py), results written after every task run, resumable, secrets redacted.
• browser-use's own CI modules run against this package with the patches active (scripts/upstream_tests.sh); one of them caught a real bug in the paint-order port.

Quick start

git clone --recurse-submodules https://github.com/Tom-R-Main/Footwork.git
cd Footwork/jevdual
rustup toolchain install 1.97.1 --profile minimal  # pinned CUA source compiler
uv sync                        # builds Footwork and the exact CUA native reference
uv run pytest                  # unit and equality tests, no keys needed

To run the agent or the evals, configure a TypeSafe key (TYPESAFE_API_KEY) and a System 2 key (MODEL_API_KEY for the Meta Model API, or wire another provider in evals/runner.py):

uv run python -m evals.runner --split dev --arm stock --arm guarded --arm dual --llm meta
uv run python -m evals.runner --split live-dev --arm guarded --llm meta --task lw-python-creator
uv run python -m evals.paired results/<before> results/<after> --arm dual

Use the guard alone with any browser-use agent by constructing DualProcessAgent with an ArbiterHook verifier and no S1 policy; see evals/runner.py (guarded arm) for the exact wiring.

Repository map

path	what
jevdual/python/jevdual/	the package: agent, policy, menu, arbiter, verify, ledger, receipts, secrets, tools; native arm (native, desktop, posture, coexist); operator surface (cli, session)
jevdual/crates/jevdual-core/	Rust hot paths (PyO3) with pure twins under python/jevdual/_pure/
jevdual/evals/	runner, predicates, report, metrics, paired comparison, task files, fixture site
jevdual/results/	every run directory and its write-up; the README's numbers come from here
jevdual/docs/experiments/	pre-registered questions Q1 to Q15 with results and decisions
jevdual/docs/	design notes, plans (including plans/legible-harness.md), the Cua survey, the Rust boundary rule, upstream reuse inventory
browser-use/	upstream pinned as a submodule at the studied commit; the package imports the PyPI release

Status

Footwork has a working product core and a dated research record. The guard and reactive dual loop are measured on the browser. The native arm and operator session have measurements on a small native split and a coexistence suite; they remain experimental. Delegation and evidence selection are off by default.

The status audit of 2026-09-26 reproduced five contract breaks offline: operator sessions following the active browser tab, overstated field receipts, a secret in a receipt, the browser continuing after destructive judgment failed, and short documents replaced without a pause. Its plan put those fixes first, followed by authorization parity across routes, input loss after navigation, receipts again (Q12b), verifier evidence and recovery, a narrower declaration (Q11b), and one contract across browser and native (Q13). The accessibility-tree browser route (Q14) works but hasn't had a comparison. Calibration claims await the human audit sheets. Read CONTRIBUTING.md before opening a change.

Credits

Ideas were taken from six MIT-licensed sibling projects (jev-ultrafast, fastbrowse, jev-browser, jev-for-chrome, public-browser) and, ideas only, from browserclaw (AGPL); browser-use's test fixtures and judge are reused under MIT (docs/upstream-reuse.md). MIT license.