diff --git a/README.md b/README.md index c4edbc5..8cad748 100644 --- a/README.md +++ b/README.md @@ -34,7 +34,7 @@ Authenticate to your registry before selecting the remote context: the gateway d An organization admin must provision the builder first with `com docker-builder create ci --org acme`. The token needs membership in that organization. Setup fails if the builder is missing, unavailable, or refused; it never falls back to a local build or creates a cache accidentally. -Inputs are `token` and `org` (required), `builder` (default `main`), `use` (default `true`), `api-url` (default `https://composal.ai`), and `install-cli` (default `true`). The installed CLI is added to the job PATH. Set `install-cli: 'false'` to use an already installed released `com` supporting background Docker builder setup. Docker and Buildx must be installed on the runner. Outputs are `builder`, `context`, and `driver` (`docker`). Each job isolates CLI supervisor state in its temporary directory. Setup checks the remote daemon and Buildx readiness, then selects the context and builder for subsequent job steps using `DOCKER_CONTEXT` and `BUILDX_BUILDER`. It leaves Docker's saved defaults unchanged and restores the previous job environment during cleanup. +Inputs are `token` and `org` (required), `builder` (default `main`), `wake-timeout` (default `900` seconds, range `30`–`3600`), `use` (default `true`), `api-url` (default `https://composal.ai`), and `install-cli` (default `true`). The installed CLI is added to the job PATH. Set `install-cli: 'false'` to use an already installed released `com` supporting background Docker builder setup. Docker and Buildx must be installed on the runner. Outputs are `builder`, `context`, and `driver` (`docker`). Each job isolates CLI supervisor state in its temporary directory. Setup logs the CLI version, checks gateway health and waits for the shared builder to become ready before opening native Docker connections. Cold starts can take several minutes; `wake-timeout` controls this wait. Progress includes the builder state and generation. Canceling a job disconnects only that job; it does not cancel the shared wake. It retries transient status-read server errors, then checks the remote daemon with bounded, retryable probes and verifies Buildx readiness before selecting the context and builder for subsequent job steps using `DOCKER_CONTEXT` and `BUILDX_BUILDER`. It leaves Docker's saved defaults unchanged and restores the previous job environment during cleanup. Setup also disables Buildx metadata provenance and Docker build-record uploads/summaries: the scoped gateway does not expose BuildKit content-record downloads. Image digest metadata is retained. Previous values are restored during cleanup. In explicit selection mode, set `BUILDX_METADATA_PROVENANCE=disabled`, `DOCKER_BUILD_RECORD_UPLOAD=false` and `DOCKER_BUILD_SUMMARY=false` on the build step yourself. @@ -42,6 +42,8 @@ The normal setup works with `docker/build-push-action`, `docker buildx build`, a Use one connection per job. Post-job cleanup disconnects the local proxy and removes its context, including after build failure. It keeps the remote builder and cache; organization idle policy sleeps compute. Runner termination can prevent cleanup. Disconnecting does not prove a remote operation has stopped: uncertain operations retain their slot until the service reconciles them or an administrator recovers the builder. -Each builder admits one operation at a time. Organization concurrency defaults to one across all its builders. Serialize jobs sharing a builder and provision separate caches for independent workloads; GitHub concurrency only coordinates workflows in the same repository. Use `queue: max` to retain pending builds (up to 100); without it GitHub replaces the previous pending build. Concurrent work outside it can still be refused. Remote builders are in preview; current VMs have 2 CPUs, 4 GiB RAM and an 8 GiB cache disk. Qualify image size, cold build memory and warm performance before switching production builds. +One builder can serve multiple native builds on the same daemon and retained cache. Organization concurrency defaults to one across all its builders; an admin can raise it with `com docker-builder settings --org acme --max-concurrency 4`. New VMs receive 2 CPUs and 4 GiB RAM per configured concurrent operation, with a 128 GiB cache disk. Changing the policy does not resize an already running VM: drain its builds, sleep it to checkpoint the cache, then wake it. Qualify image size, cold build memory and warm performance before switching production builds. + +Coordinate CI jobs to fit the organization's configured capacity. The single concurrency group in the example serializes jobs; multiple groups can admit more jobs when capacity has been increased. GitHub concurrency only coordinates workflows in the same repository, and concurrent work outside those groups can still be refused. Use `queue: max` to retain pending builds (up to 100); without it GitHub replaces the previous pending build. Use separate builders when workloads need independent caches. Remote builders are in preview. Run `node --test builder.test.mjs` to check the action locally. The integration qualification also exercises cache-mount reuse across real Buildx solves, a direct registry push, digest inspection and post-job cleanup. diff --git a/action.yml b/action.yml index 8d512db..e73ca77 100644 --- a/action.yml +++ b/action.yml @@ -14,6 +14,9 @@ inputs: builder: description: Existing organization builder slug or public ID. default: main + wake-timeout: + description: Maximum seconds to wait for the shared builder to become ready, before connecting Docker. + default: '900' use: description: Select this context and Buildx builder for subsequent job steps. Set false for explicit outputs only. default: 'true' diff --git a/builder.mjs b/builder.mjs index 1a175bd..96062a0 100644 --- a/builder.mjs +++ b/builder.mjs @@ -3,6 +3,7 @@ import { randomUUID } from 'node:crypto'; import { appendFile, mkdir } from 'node:fs/promises'; import { dirname, join } from 'node:path'; import { promisify } from 'node:util'; +import { setTimeout as delay } from 'node:timers/promises'; const exec = promisify(execFile); const selectedKeys = ['DOCKER_CONTEXT', 'BUILDX_BUILDER', 'BUILDX_METADATA_PROVENANCE', 'DOCKER_BUILD_RECORD_UPLOAD', 'DOCKER_BUILD_SUMMARY']; @@ -24,18 +25,23 @@ export async function command(executable, args, env, timeout = 120_000) { return stdout; } catch (error) { // Never include argv, environment, or stdout (which may contain credentials). - throw new Error(`${executable.split('/').at(-1)} ${args[0]} failed: ${error.stderr || error.code || 'command failed'}`); + const timedOut = error.killed && error.signal === 'SIGTERM' && error.code == null; + const details = [timedOut ? `timed out after ${timeout / 1000}s` : 'command failed', + `exit=${error.code ?? 'none'}`, `signal=${error.signal ?? 'none'}`, `killed=${Boolean(error.killed)}`]; + const operation = args.includes('version') ? 'version' : args.slice(0, 2).filter(a => /^[a-z-]+$/.test(a)).join(' '); + throw new Error(`${executable.split('/').at(-1)} ${operation}: ${details.join(', ')}${error.stderr ? `; ${String(error.stderr).slice(-4096)}` : ''}`); } } function withoutToken(env) { const copy = { ...env }; delete copy.INPUT_TOKEN; + delete copy.COMPOSAL_API_TOKEN; delete copy.VEX_API_TOKEN; return copy; } -export async function setup(env = process.env, run = command) { +export async function setup(env = process.env, run = command, wait = delay) { const token = input(env, 'token'); if (!token) throw new Error('token is required'); if (env.GITHUB_ACTIONS === 'true') console.log(`::add-mask::${escapeCommand(token)}`); @@ -48,8 +54,17 @@ export async function setup(env = process.env, run = command) { throw new Error('api-url must be an HTTPS origin'); } if (process.platform !== 'linux') throw new Error('This action requires a Linux runner'); + const startedAt = Date.now(); + let stageStartedAt = startedAt; + const stage = name => { + const now = Date.now(); + console.log(`${name} (${((now - stageStartedAt) / 1000).toFixed(1)}s; total ${((now - startedAt) / 1000).toFixed(1)}s)`); + stageStartedAt = now; + }; const install = input(env, 'install-cli', 'true'); if (!['true', 'false'].includes(install)) throw new Error('install-cli must be true or false'); + const wakeTimeout = Number(input(env, 'wake-timeout', '900')); + if (!Number.isInteger(wakeTimeout) || wakeTimeout < 30 || wakeTimeout > 3600) throw new Error('wake-timeout must be between 30 and 3600 seconds'); const use = input(env, 'use', 'true'); if (!['true', 'false'].includes(use)) throw new Error('use must be true or false'); const cleanEnv = withoutToken(env); @@ -65,18 +80,68 @@ export async function setup(env = process.env, run = command) { if (!env.GITHUB_PATH) throw new Error('GITHUB_PATH is required'); await appendFile(env.GITHUB_PATH, `${dirname(cli)}\n`); } + const cliVersion = (await run(cli, ['--version'], cleanEnv, 10_000)).trim(); + if (!/^com(?:posal)? [0-9]+\.[0-9]+\.[0-9]+/.test(cliVersion)) throw new Error('Could not identify the Composal CLI version'); + stage(`Composal CLI available: ${cliVersion}`); if (!env.RUNNER_TEMP) throw new Error('RUNNER_TEMP is required'); // The CLI keys supervisors by config home as well as builder identity. // A job must not reuse or disconnect a pre-existing local connection. const configHome = join(env.RUNNER_TEMP, `com-builder-state-${randomUUID()}`); await mkdir(configHome, { recursive: true, mode: 0o700 }); - const cliEnv = { ...cleanEnv, XDG_CONFIG_HOME: configHome, VEX_API_TOKEN: token, VEX_API_BASE_URL: api.origin }; + const cliEnv = { ...cleanEnv, XDG_CONFIG_HOME: configHome, COMPOSAL_API_TOKEN: token, VEX_API_TOKEN: token, VEX_API_BASE_URL: api.origin }; const options = ['--org', org, '--api-base-url', api.origin, '--json']; + const inspect = async (selector, deadline = Date.now() + 30_000) => { + for (let attempt = 0; ; attempt++) { + try { + return JSON.parse(await run(cli, ['docker-builder', 'status', selector, ...options], cliEnv, + Math.max(1, deadline - Date.now()))).docker_builder; + } catch (error) { + if (attempt >= 2 || Date.now() >= deadline || + !/\b50[0234]\b|timed out|connection reset|ECONNRESET|ETIMEDOUT/.test(error.message)) throw error; + await wait(Math.min(250 * (attempt + 1), deadline - Date.now())); + } + } + }; // CI never creates a builder accidentally: members use an admin-provisioned cache. - const resource = JSON.parse(await run(cli, ['docker-builder', 'status', builder, ...options], cliEnv)).docker_builder; + const resource = await inspect(builder); if (!/^org_[a-zA-Z0-9]+$/.test(resource?.organization_id) || !/^docker_builder_[a-zA-Z0-9]+$/.test(resource?.id)) { throw new Error('Composal CLI returned an unexpected builder identity'); } + stage(`Builder resolved: ${resource.status || 'unknown state'}`); + if (resource.status !== 'ready') { + // Complete the control-plane wake before Docker opens its first HTTP + // connection. Native ping/version clients can abandon that socket during + // a long wake even though the VM subsequently becomes healthy. + const gateway = new URL(resource.gateway_url); + if (gateway.protocol !== 'wss:' || gateway.username || gateway.password || gateway.search || gateway.hash || + gateway.pathname !== `/v1/organizations/${resource.organization_id}/docker`) { + throw new Error('Composal CLI returned an unexpected builder gateway'); + } + gateway.protocol = 'https:'; + gateway.pathname = '/healthz'; + await run('curl', ['-fsS', '--max-time', '5', gateway.href], cleanEnv); + const deadline = Date.now() + wakeTimeout * 1_000; + let nextProgress = Date.now() + 15_000; + let current = resource; + while (current.status !== 'ready') { + if (Date.now() >= deadline) throw new Error(`Remote builder wake timed out after ${wakeTimeout}s (state=${current.status}, generation=${current.generation ?? 'unknown'}); the shared wake continues independently of this job`); + if (current.status === 'sleeping') { + await run(cli, ['docker-builder', 'wake', resource.id, ...options], cliEnv, Math.max(1, deadline - Date.now())); + } else if (!['starting', 'stopping'].includes(current.status)) { + throw new Error('Composal CLI returned an unexpected builder state'); + } + if (Date.now() >= nextProgress) { + stage(`Waiting for builder: state=${current.status}, generation=${current.generation ?? 'unknown'}`); + nextProgress = Date.now() + 15_000; + } + await wait(Math.min(1_000, Math.max(1, deadline - Date.now()))); + current = await inspect(resource.id, deadline); + if (current?.id !== resource.id || current.organization_id !== resource.organization_id) { + throw new Error('Composal CLI returned an unexpected builder identity'); + } + } + stage('Remote builder wake completed'); + } const context = `com-ci-${randomUUID()}`; // Register cleanup before setup, including malformed responses or failed wake. for (const [name, value] of Object.entries({ context, cli, config_home: configHome, org: resource.organization_id, builder: resource.id, api: api.origin })) { @@ -88,9 +153,27 @@ export async function setup(env = process.env, run = command) { result.organization_id !== resource.organization_id || result.builder_id !== resource.id) { throw new Error('Composal CLI returned an unexpected builder connection'); } + stage('Local builder proxy configured'); + console.log('Checking remote Docker through the configured context.'); const dockerEnv = { ...cleanEnv, DOCKER_CONTEXT: context, BUILDX_BUILDER: context }; - const version = await run('docker', ['--context', context, 'version', '--format', '{{.Server.Version}}'], dockerEnv, 300_000); - if (!/^\d+\.\d+\.\d+/.test(version.trim())) throw new Error('Remote Docker daemon is not ready'); + // A fresh private proxy can reset its first connection while establishing + // the remote session. Retry this read only; never replay a build or push. + // The control-plane wake is complete. Bound each native probe separately + // so a stale connection cannot consume the entire readiness deadline. + const readinessDeadline = Date.now() + 90_000; + for (let attempt = 0; attempt < 5; attempt++) { + try { + const version = await run('docker', ['--context', context, 'version', '--format', '{{.Server.Version}}'], dockerEnv, + Math.max(1, Math.min(15_000, readinessDeadline - Date.now()))); + if (!/^\d+\.\d+\.\d+/.test(version.trim())) throw new Error('Remote Docker daemon is not ready'); + break; + } catch (error) { + const transient = /timed out after|connection reset by peer|connection refused|unexpected EOF|i\/o timeout|TLS handshake timeout|dial unix.*no such file or directory/.test(error.message); + if (!transient || attempt === 4 || Date.now() >= readinessDeadline) throw error; + await wait(Math.min(2_000, readinessDeadline - Date.now())); + } + } + stage('Remote Docker ready'); // Inspect alone can exit zero with a node error; version must succeed first. await run('docker', ['buildx', 'inspect', context, '--bootstrap'], dockerEnv, 300_000); if (use === 'true') { @@ -107,7 +190,7 @@ export async function setup(env = process.env, run = command) { for (const [name, value] of Object.entries({ context, builder: context, driver: 'docker' })) { await record(env.GITHUB_OUTPUT, name, value); } - console.log('Composal Docker builder is ready.'); + stage('Composal Docker builder is ready.'); } export async function cleanup(env = process.env, run = command) { @@ -122,7 +205,7 @@ export async function cleanup(env = process.env, run = command) { if (env.STATE_builder && env.STATE_org && env.STATE_cli && env.STATE_api) { try { await run(env.STATE_cli, ['docker-builder', 'disconnect', env.STATE_builder, '--org', env.STATE_org, '--api-base-url', env.STATE_api], - { ...cleanEnv, XDG_CONFIG_HOME: env.STATE_config_home, VEX_API_TOKEN: input(env, 'token'), VEX_API_BASE_URL: env.STATE_api }); + { ...cleanEnv, XDG_CONFIG_HOME: env.STATE_config_home, COMPOSAL_API_TOKEN: input(env, 'token'), VEX_API_TOKEN: input(env, 'token'), VEX_API_BASE_URL: env.STATE_api }); } catch (error) { failures.push(error.message); } } try { diff --git a/builder.test.mjs b/builder.test.mjs index cd5c10e..aacc7fe 100644 --- a/builder.test.mjs +++ b/builder.test.mjs @@ -3,7 +3,7 @@ import assert from 'node:assert/strict'; import { mkdtemp, readFile, rm } from 'node:fs/promises'; import { tmpdir } from 'node:os'; import { join } from 'node:path'; -import { cleanup, reportFailure, setup } from './builder.mjs'; +import { command, cleanup, reportFailure, setup } from './builder.mjs'; async function fixture(t, inputs = {}, failure = '') { const directory = await mkdtemp(join(tmpdir(), 'com-builder-test-')); @@ -16,7 +16,8 @@ async function fixture(t, inputs = {}, failure = '') { const run = async (executable, args, childEnv) => { calls.push({ executable, args, env: childEnv }); if (args.includes(failure)) throw new Error(`refused ${env.INPUT_TOKEN}`); - if (args[1] === 'status') return JSON.stringify({ docker_builder: { id: 'docker_builder_456', organization_id: 'org_123' } }); + if (args[0] === '--version') return 'com 1.17.88\n'; + if (args[1] === 'status') return JSON.stringify({ docker_builder: { id: 'docker_builder_456', organization_id: 'org_123', status: 'ready' } }); if (args[1] === 'setup') { const context = args[args.indexOf('--context') + 1]; return JSON.stringify({ context, buildx_builder: context, driver: 'docker', foreground: false, @@ -39,13 +40,14 @@ async function fixture(t, inputs = {}, failure = '') { test('setup returns a ready explicit builder and selects it for subsequent steps without persisting its token', async t => { const f = await fixture(t); await setup(f.env, f.run); - assert.deepEqual(f.calls.map(c => c.args.slice(0, 2)), [['docker-builder', 'status'], ['docker-builder', 'setup'], ['--context', f.calls[2].args[1]], ['buildx', 'inspect']]); - for (const call of f.calls.slice(0, 2)) { + assert.deepEqual(f.calls.map(c => c.args.slice(0, 2)), [['--version'], ['docker-builder', 'status'], ['docker-builder', 'setup'], ['--context', f.calls[3].args[1]], ['buildx', 'inspect']]); + for (const call of f.calls.slice(1, 3)) { + assert.equal(call.env.COMPOSAL_API_TOKEN, f.env.INPUT_TOKEN); assert.equal(call.env.VEX_API_TOKEN, f.env.INPUT_TOKEN); assert.ok(!call.args.join(' ').includes(f.env.INPUT_TOKEN)); } - assert.equal(f.calls[2].env.VEX_API_TOKEN, undefined); - assert.equal(f.calls[3].env.DOCKER_CONTEXT, f.calls[2].args[1]); + assert.equal(f.calls[3].env.COMPOSAL_API_TOKEN, undefined); + assert.equal(f.calls[4].env.DOCKER_CONTEXT, f.calls[3].args[1]); const environment = await readFile(f.env.GITHUB_ENV, 'utf8'); assert.ok(environment.includes('DOCKER_CONTEXT<<')); assert.ok(environment.includes('BUILDX_METADATA_PROVENANCE<<')); @@ -59,7 +61,77 @@ test('setup returns a ready explicit builder and selects it for subsequent steps await cleanup({ ...f.env, ...state }, f.run); assert.deepEqual(f.calls.slice(-2).map(c => c.args.slice(0, 2)), [['docker-builder', 'disconnect'], ['context', 'rm']]); assert.ok(!f.calls.some(c => c.args.includes('sleep') || c.args.includes('recover'))); - assert.equal(f.calls.at(-2).env.XDG_CONFIG_HOME, f.calls[0].env.XDG_CONFIG_HOME); + assert.equal(f.calls.at(-2).env.XDG_CONFIG_HOME, f.calls[1].env.XDG_CONFIG_HOME); +}); + +test('a cold builder reaches ready before any native Docker connection is configured', async t => { + const f = await fixture(t); + const states = ['sleeping', 'starting', 'ready']; + const waits = []; + let state; + const run = async (exe, args, env, timeout) => { + const result = await f.run(exe, args, env, timeout); + if (args[1] === 'status') { + state = states.shift(); + return JSON.stringify({ docker_builder: { id: 'docker_builder_456', organization_id: 'org_123', + status: state, gateway_url: 'wss://builder.example/v1/organizations/org_123/docker' } }); + } + if (args[1] === 'setup' || args.includes('version')) assert.equal(state, 'ready'); + return result; + }; + await setup(f.env, run, async ms => waits.push(ms)); + assert.deepEqual(waits, [1_000, 1_000]); + assert.equal(f.calls.filter(c => c.args[1] === 'wake').length, 1); + const health = f.calls.findIndex(c => c.executable === 'curl'); + const wake = f.calls.findIndex(c => c.args[1] === 'wake'); + assert.ok(health < wake); + assert.ok(f.calls[health].args.includes('https://builder.example/healthz')); + assert.equal(f.calls[health].env.COMPOSAL_API_TOKEN, undefined); +}); + +test('an unreachable gateway does not wake compute or configure Docker', async t => { + const f = await fixture(t); + const run = async (exe, args, env, timeout) => { + if (exe === 'curl') throw new Error('gateway unavailable'); + const result = await f.run(exe, args, env, timeout); + if (args[1] === 'status') return JSON.stringify({ docker_builder: { + id: 'docker_builder_456', organization_id: 'org_123', status: 'sleeping', + gateway_url: 'wss://builder.example/v1/organizations/org_123/docker', + } }); + return result; + }; + await assert.rejects(setup(f.env, run), /gateway unavailable/); + assert.ok(!f.calls.some(c => ['wake', 'setup'].includes(c.args[1]))); +}); + +test('transient status failures are retried before configuring the native proxy', async t => { + const f = await fixture(t); + const waits = []; + let statuses = 0; + const run = async (exe, args, env, timeout) => { + if (args[1] === 'status' && statuses++ === 0) throw new Error('500 Internal Server Error'); + return f.run(exe, args, env, timeout); + }; + await setup(f.env, run, async ms => waits.push(ms)); + assert.equal(statuses, 2); + assert.deepEqual(waits, [250]); +}); + +test('a stalled control-plane wake has one bounded deadline and publishes no context', async t => { + const f = await fixture(t); + let now = 0; + t.mock.method(Date, 'now', () => now); + const run = async (exe, args, env, timeout) => { + const result = await f.run(exe, args, env, timeout); + if (args[1] === 'status') return JSON.stringify({ docker_builder: { + id: 'docker_builder_456', organization_id: 'org_123', status: 'starting', + gateway_url: 'wss://builder.example/v1/organizations/org_123/docker', + } }); + return result; + }; + await assert.rejects(setup(f.env, run, async () => { now = 900_001; }), /wake timed out after 900s/); + assert.ok(!f.calls.some(c => c.args[1] === 'setup')); + await assert.rejects(readFile(f.env.GITHUB_OUTPUT), { code: 'ENOENT' }); }); test('bootstrap failure retains cleanup state and emits no usable builder', async t => { @@ -73,13 +145,14 @@ test('bootstrap failure retains cleanup state and emits no usable builder', asyn test('missing builder fails before setup; CI cannot silently create a misspelled cache', async t => { const f = await fixture(t, {}, 'status'); await assert.rejects(setup(f.env, f.run), /refused/); - assert.equal(f.calls.length, 1); + assert.equal(f.calls.length, 2); }); test('installer cannot inherit the API token and the installed CLI is retained for post cleanup', async t => { - const f = await fixture(t, { 'INPUT_INSTALL-CLI': 'true', VEX_API_TOKEN: 'unrelated-token' }); + const f = await fixture(t, { 'INPUT_INSTALL-CLI': 'true', COMPOSAL_API_TOKEN: 'unrelated-token', VEX_API_TOKEN: 'legacy-unrelated-token' }); await setup(f.env, f.run); for (const call of f.calls.slice(0, 2)) { + assert.equal(call.env.COMPOSAL_API_TOKEN, undefined); assert.equal(call.env.VEX_API_TOKEN, undefined); assert.equal(call.env.INPUT_TOKEN, undefined); } @@ -177,6 +250,113 @@ test('jobs sharing a builder have separate local supervisor identities', async t const second = await fixture(t, { XDG_CONFIG_HOME: '/previous-config' }); await setup(first.env, first.run); await setup(second.env, second.run); - assert.notEqual(first.calls[0].env.XDG_CONFIG_HOME, '/previous-config'); - assert.notEqual(first.calls[0].env.XDG_CONFIG_HOME, second.calls[0].env.XDG_CONFIG_HOME); + assert.notEqual(first.calls[1].env.XDG_CONFIG_HOME, '/previous-config'); + assert.notEqual(first.calls[1].env.XDG_CONFIG_HOME, second.calls[1].env.XDG_CONFIG_HOME); +}); + + +test('fresh proxy connection resets are retried before publishing a ready context', async t => { + const f = await fixture(t); + let probes = 0; + const waits = []; + const run = (exe, args, env, timeout) => { + if (args.includes('version') && ++probes < 3) throw new Error('read unix: connection reset by peer'); + return f.run(exe, args, env, timeout); + }; + await setup(f.env, run, async ms => waits.push(ms)); + assert.equal(probes, 3); + assert.deepEqual(waits, [2000, 2000]); + assert.ok((await readFile(f.env.GITHUB_OUTPUT, 'utf8')).includes('driver<<')); +}); + +test('persistent proxy failure exhausts bounded readiness probes without usable outputs', async t => { + const f = await fixture(t); + let probes = 0; + const waits = []; + const run = (exe, args, env, timeout) => { + if (args.includes('version')) { probes++; throw new Error('connection reset by peer'); } + return f.run(exe, args, env, timeout); + }; + await assert.rejects(setup(f.env, run, async ms => waits.push(ms)), /connection reset/); + assert.equal(probes, 5); + assert.deepEqual(waits, [2000, 2000, 2000, 2000]); + await assert.rejects(readFile(f.env.GITHUB_OUTPUT), { code: 'ENOENT' }); + await cleanup({ ...f.env, ...await f.state() }, f.run); +}); + +test('authorization failures are not retried as proxy startup races', async t => { + const f = await fixture(t); + let probes = 0; + const waits = []; + const run = (exe, args, env, timeout) => { + if (args.includes('version')) { probes++; throw new Error('unauthorized'); } + return f.run(exe, args, env, timeout); + }; + await assert.rejects(setup(f.env, run, async ms => waits.push(ms)), /unauthorized/); + assert.equal(probes, 1); + assert.deepEqual(waits, []); +}); + +test('a stalled native connection uses one readiness budget and cannot restart that budget on failure', async t => { + const f = await fixture(t); + let now = 0; + t.mock.method(Date, 'now', () => now); + const timeouts = []; + const waits = []; + const run = (exe, args, env, timeout) => { + if (args.includes('version')) { + timeouts.push(timeout); + now = 300_001; + throw new Error('connection reset by peer'); + } + return f.run(exe, args, env, timeout); + }; + await assert.rejects(setup(f.env, run, async ms => waits.push(ms)), /connection reset/); + assert.deepEqual(timeouts, [15_000]); + assert.deepEqual(waits, []); + await assert.rejects(readFile(f.env.GITHUB_OUTPUT), { code: 'ENOENT' }); +}); + + +test('a long shared wake completes before opening short-lived Docker probes', async t => { + const f = await fixture(t); + let now = 0; + t.mock.method(Date, 'now', () => now); + const run = async (exe, args, env, timeout) => { + const result = await f.run(exe, args, env, timeout); + if (args[1] === 'status') return JSON.stringify({ docker_builder: { + id: 'docker_builder_456', organization_id: 'org_123', status: now < 760_000 ? 'starting' : 'ready', + gateway_url: 'wss://builder.example/v1/organizations/org_123/docker', + } }); + if (args.includes('version')) { + assert.equal(now, 760_000); + assert.equal(timeout, 15_000); + } + return result; + }; + await setup(f.env, run, async () => { now = 760_000; }); + assert.ok((await readFile(f.env.GITHUB_OUTPUT, 'utf8')).includes('driver<<')); +}); + +test('native probe timeouts are retried without repeating setup or a build', async t => { + const f = await fixture(t); + let probes = 0; + const run = (exe, args, env, timeout) => { + if (args.includes('version') && probes++ === 0) throw new Error('timed out after 15s'); + return f.run(exe, args, env, timeout); + }; + await setup(f.env, run, async () => {}); + assert.equal(probes, 2); + assert.equal(f.calls.filter(c => c.args[1] === 'setup').length, 1); +}); + +test('command failures preserve timeout and signal details without stdout or environment', async () => { + await assert.rejects(command(process.execPath, ['-e', "console.log('private-stdout'); setInterval(() => {}, 1000)"], + { PRIVATE_TOKEN: 'private-env' }, 100), error => { + assert.match(error.message, /timed out after 0.1s/); + assert.match(error.message, /signal=SIGTERM, killed=true/); + assert.ok(!error.message.includes('private-stdout')); + assert.ok(!error.message.includes('private-env')); + return true; + }); });