Skip to content

Keep tests off the operator's credential store and the network - #179

Merged
oakimov merged 2 commits into
mainfrom
hermetic-test-credentials
Nov 10, 2026
Merged

oakimov merged 2 commits into
mainfrom
hermetic-test-credentials

Conversation

@oakimov

@oakimov oakimov commented Nov 10, 2026 •

Copy link
Copy Markdown
Owner

Problem

plugin.config loads models: on a cache miss it reads the stored Cursor credential (authFromStore → readStoredAuth, src/plugin.ts) and calls discoverModels. Three tests in CursorPlugin shell result hooks (test/plugin.test.ts) call it inside withTempHostCache, which is always a cache miss, and leave the data root alone. On a machine with a Cursor login they read the real auth.json and ran live model discovery.

A full run with a recorder that blocks and logs every non-loopback fetch and socket found two more callers of live Cursor:

  • opencode2 setup > the aisdk sdk hook resolves the token per Run… sent its placeholder token to AvailableModels.
  • pollForTokens > honours abort signal polled /auth/poll on the default API host.

The guard then found a third on CI: with no Cursor CLI installed the client version comes from https://cursor.com/install (src/protocol/client-version.ts), and six Run tests fetched it.

One slow discovery reached Bun's 5 s timeout, and 14 later tests plus the TMPDIR leftover check failed with it. Bun fails a timed-out test and keeps running its body. withTempHostCache therefore restored the process-wide cache root late: the scope entered meanwhile restored the removed directory as the root for the rest of the file.

Fix

  • test/preload.ts points XDG_DATA_HOME into the run's private TMPDIR and clears OPENCODE_AUTH_CONTENT, as it already does for XDG_CACHE_HOME. openCodeGlobalDataDir (src/context/paths.ts) reads that variable first on every platform, so the store is empty unless a test writes a fake credential. Anything left in the data root fails the run.
  • It also fixes CURSOR_CLIENT_VERSION, so the client version depends on neither a local Cursor CLI install nor the installer page.
  • test/network-guard.ts refuses every fetch and every node:net / node:tls / node:http2 connection that leaves the machine. Bun follows redirects inside fetch, so the guard makes each hop itself and checks the destination first. The preload's afterEach fails the test that tried, so code that swallows the refusal is caught too. It also fails a test that leaves XDG_DATA_HOME or OPENCODE_AUTH_CONTENT changed and puts them back. The guard covers the test process; processes a test spawns are outside it.
  • test/host-cache.ts puts a temp host cache's roots back when its test ends (test/test-end.ts), so a test that times out in its body or while the cache is cleaned up no longer hands its root to later tests.
  • test/conversation-maintenance.test.ts: one test created a quarantine file while the maintenance started by its own first read could still recover and remove it (the ENOENT in this PR's first CI run). It now waits for that maintenance.
  • The two tests above now use a fetch stub and a local server.

No source change, no changelog entry.

Verification

Measured on macOS with a stored Cursor credential present, bun test ./test/plugin.test.ts -t 'keeps soft-background Bash command original', three runs each:

stored credential empty XDG_DATA_HOME
before 1330, 845, 965 ms 68, 61, 63 ms
after 74, 61, 87 ms 63, 71, 62 ms
  • The timeout cascade reproduces on main without live Cursor: a fixture credential under a fake HOME and a local server whose first model list answer is 7 s late give 1 timeout, 14 later failures and the leftover check in test/plugin.test.ts. With this branch the same setup passes 61/61, also when the credential comes through the caller's XDG_DATA_HOME or OPENCODE_AUTH_CONTENT.
  • test/test-isolation.test.ts covers the guards in-process, redirects included, and runs four fixtures in a child bun test: a swallowed remote fetch fails its test; a test that leaves the store redirected fails and the next one is isolated again; a test that times out inside withTempHostCache, in its body or during cleanup, fails alone. Against the previous test/host-cache.ts each timeout fixture fails all four of its tests.
  • With a home that has no Cursor CLI install, the six files that failed on CI give one guard failure before the client version is fixed and 135/135 after.
  • bun run typecheck, bun run build, and bun test ./test three times in a row, plus once with an empty home: 3418 pass, 5 skip, 0 fail across 127 files each time.

@greptile-apps

greptile-apps Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no new actionable issues were found.

Summary

The PR keeps tests away from stored operator credentials and remote hosts.

  • Each test run uses a private credential store and refuses remote network requests.
  • Timed-out tests restore temporary cache roots before later tests start.
  • The token-poll test uses localhost, and setup answers model requests with a stub.
  • The quarantine test waits for recovery before arranging its files.

Both earlier, unnumbered findings are addressed in the current code. Cache scopes stay registered through cleanup, and redirects are checked one hop at a time. No new actionable issues were found. Verification here was by code inspection; no tests were run.

Reviews (2) · Last reviewed commit: "Close the test isolation gaps CI and rev..." · Reviewed by Greptile

Comment thread test/host-cache.ts Resolved
Comment thread test/network-guard.ts Resolved
@codecov

codecov Bot commented Nov 10, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Three CursorPlugin shell hook tests called plugin.config with an empty
temp cache and the native data root, so each read the developer's
auth.json and ran live model discovery. A 2.x setup test sent its
placeholder token to Cursor and the pollForTokens abort test polled it.

test/preload.ts now points XDG_DATA_HOME into the run's TMPDIR, clears
OPENCODE_AUTH_CONTENT and refuses every fetch and socket that leaves the
machine; the test that tried one, or left the store redirected, fails.

A timed-out test's body keeps running under Bun, so withTempHostCache
restored the process-wide cache root late and the scope entered
meanwhile restored a removed dir as the root for the rest of the file.
The roots now go back when the test ends.
Without a Cursor CLI install, as in CI, the client version came from
Cursor's installer page, so six Run tests fetched it. The preload now
fixes CURSOR_CLIENT_VERSION.

Bun follows redirects inside fetch, past the guard. The guard makes each
hop itself and refuses one that leaves the machine.

A temp host cache stays registered until its cleanup finishes, so a test
that times out while leaving it no longer hands its root on.

A conversation-maintenance test created a quarantine file while the
maintenance its own first read started could still recover and remove
it. It waits for that maintenance first.
@oakimov
oakimov force-pushed the hermetic-test-credentials branch from 4744b74 to 265f8d0 Compare November 10, 2026 19:57
@oakimov
oakimov merged commit 423abf6 into main Nov 10, 2026
7 of 8 checks passed
@oakimov
oakimov deleted the hermetic-test-credentials branch November 10, 2026 20:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant