[federerl]

#711 was merged into main as d1ccafe. This PR now targets main and precedes #714.

Closes #710

Summary

Adds POST /api/cli/cancelExp so the CLI (and agents using a CLI token) can cancel a running experiment. Web cancellation now goes through the same logic.

• Job first, status second. The experiment is marked CANCELLED only after the backend accepts the Job deletion. status and finished are set in one conditional update (status: RUNNING → CANCELLED, finished: true), so a runner that already finished is not overwritten and the two fields can't disagree.
• Authorization. The CLI can cancel only the caller's own experiments. Shared users get 403. Unrelated users get 404, so the response doesn't reveal whether the experiment exists. Admins cancel from the web UI only.
• cancelExperimentById auth fix (related to Security hardening: enforce server-side authorization and protect user data #708). This server action used to accept any caller. It now resolves the authenticated user and current role from the database and allows only the owner or a current admin.
• Audit. CLI cancellation goes through withAudit (action cancel_experiment, agent cli). If the audit write fails, the route returns 503 degraded and nothing is cancelled. Web cancellation is not audited.
• Backend. /cancelExperiment validates jobName (must start with runner-) and returns 404 instead of 500 when the Job doesn't exist.

Responses

Case	HTTP	Body
Cancel accepted	202	{success:true, eid, outcome:"cancel_requested", status:"CANCELLED", job_termination_confirmed:false}
Already cancelled	200	outcome:"already_cancelled"
Bad input	400	error:"Other"
Bad token	401	error:"auth_error"
Shared, not owner	403	error:"forbidden"
Missing / not visible	404	error:"not_found"
Not RUNNING (incl. CREATED)	409	error:"not_cancellable", status
Job deleted, DB write failed	500	error:"status_update_failed", job_deletion_requested:true. Retrying fixes it.
Backend refused	502	error:"backend_error"
Audit unavailable	503	error:"degraded"

Layout

• lib/experimentCancelCore.mjs: pure logic with injected deps, same pattern as agentAuditCore.mjs.
• lib/experimentCancelStore.ts: MongoDB and backend calls. These are kept out of experimentCancel.ts because the agent-audit static test forbids update/delete in any file that references the audit log.
• lib/experimentCancel.ts: entry points for the web and CLI. Not a 'use server' module.
• app/api/cli/cancelExp/route.ts: the route.

Behavior change

On the web, a CREATED experiment can no longer be cancelled. It used to be marked CANCELLED even though the Job could still start later. The cancel button in the experiment list and the Stop button on the admin page now follow CANCELLABLE_STATUSES, so they are hidden or disabled for CREATED.

cancelExperimentById returns expected refusals as { ok: false, message } instead of throwing, because Next.js replaces the message of errors thrown by a server action with a generic one in production. The UI shows the message in the toast.

Cancellation races

Job deletion remains asynchronous. Runner updates now include an atomic status: { $nin: ['CANCELLED', 'ARCHIVED'] } condition so a late completion cannot overwrite an accepted cancellation or an archived terminal experiment. Experiment IDs are canonicalized before constructing the Job name, including uppercase ObjectIds. Missing-Job retries remain supported.

Testing

• Local Node 20 full suite, 645 passed, including Enforce server-side ownership and session authorization (#708) #711 authorization tests and cancellation regressions.
• Type check passed. ESLint passed with 12 existing warnings.
• Backend MongoDB tests, 7 passed on Python 3.10.
• Fresh independent review identified and fixed the archived-state race. Regression tests cover archived CANCELLED, COMPLETED, and FAILED records and verify that active and legacy missing-status records remain writable. The previous guard fails the new regression; independent re-review found no remaining actionable issues.
• The combined Enforce server-side ownership and session authorization (#708) #711/Add authenticated cancel experiment API for the CLI #712/Isolate experiment runners and infrastructure credentials #714 candidate passed an independent integration review. PR workflow base filters include the dependency branches.
• Verified on a local Tilt/minikube cluster: cancelling a RUNNING experiment deletes the Job and marks it CANCELLED, CREATED experiments show no cancel button, and refusals are shown in the toast.

cc @rhit-changs (author of #710)

🤖 Generated with Claude Code

Collaborator review integration

Integrated #711 at d1faa36, including the shared submission allowlist, serialized SSE authorization/query handling, copy-error recovery, archive state validation, and shared server utilities. Preserved collaborator commit 54d2183, including the cancellation UI behavior and structured Server Action refusal messages. The archive/runner race fix is 99dc835.

Hosted CI on the latest commit

All applicable checks passed on 99dc835: frontend tests/type checks/lint passed with 645 tests, backend and Google authentication checks passed, and frontend/backend container builds passed. Runner/documentation builds were skipped by path filters.