[danii1]

Why

Task local-test-e2e-cover-scheduled-workspace-autom: test(e2e): cover scheduled workspace automations end to end

Scheduled workspace [[automations]] had no end-to-end coverage. The scheduler, occurrence materialization, open_pr policy, allow_no_changes, run attribution, and live config reconciliation were only covered by unit tests, so regressions could silently stop PR creation or mutate repositories for jobs meant to publish nothing.

What changed

• Added an optional automations list plus appendAutomation() to packages/e2e/helpers/worker.ts. The generated workspace.toml gains [[automations]] tables only when requested, so the default fixture is unchanged.
• Extended the helper additively with automation run records, schedule-cursor reads, occurrence-file listing, and waitForAutomationRun().
• Extended packages/e2e/helpers/fake-agent.ts so prompts containing E2E_NO_CHANGE select the existing no-change path while E2E_AGENT_MODE still overrides.
• Added packages/e2e/tests/local/worker-automations.test.ts, which starts one real worker daemon against a local bare remote and four same-minute automatons:
  • auto-pr-on: branch pushed, exactly one commit, only src/greeting.ts changed.
  • auto-pr-off: agent ran and the run succeeded, but --no-git suppressed publishing despite workspace --create-pr.
  • noop-allowed: succeeded with --allow-no-changes and no pushed commit.
  • noop-blocked: failed with Implementation was not committed.
  • Asserts runs.origin = "scheduled", automation_id, occurrence files, schedule rows, no unconfigured runs, and a live daemon.
• Added a fifth automation through live workspace.toml reload and asserted its next scheduled run.
• Updated packages/e2e/README.md, packages/e2e/docs/coverage-matrix.md (row moved to default), and backlog item 13.

Scope

Included: scheduled [[automations]] scheduling, no-op semantics, PR policy, attribution, occurrence files, and hot reload. Excluded: estimations, working windows, concurrency behavior assertions, injected worker add, Sentry, dashboard, and any production scheduler changes.

Tradeoffs

Hot reload was kept because the test completes in roughly 90–112 seconds in isolation, below the stated budget. It adds a second cron-minute wait to the test but the per-test timeout remains 180 seconds. No-op runs may still record a local branch before deciding nothing changed; assertions therefore verify nothing is committed or published, while noop-allowed is distinguished from noop-blocked by run status. The existing helper default does not include --auto-review, so the policy test proves override of the present workspace --create-pr rather than changing the shared default.

Blast Radius

Affected files are test-only: the worker fixture, fake agent, new local test, and e2e documentation. WorkerFixture.runRecords() now selects additional columns (origin, automation_id, started_at, finished_at); existing tests only read subsets or use toMatchObject, and the full suite passed. The default generated workspace and all production scheduler code are unchanged.

Verification

Ran from the monorepo root/package:

• bun run --filter @getdevintern/code build — passed.
• bun test tests/local/worker-automations.test.ts — passed, 45 assertions, ~91s.
• bun run --filter @devintern/e2e test — passed: 136 pass, 16 skip, 4 todo, 0 fail across 156 tests in 25 files (~172s).
• bun run --filter @devintern/e2e typecheck — passed.
• bun run --filter @devintern/e2e lint — passed.
• bun run --filter @devintern/e2e format:check — passed.
• No /tmp/devintern-e2e-worker-* fixture paths remained after the runs.

---

This PR was automatically created by @devintern/code

[danii1]

Artifacts & verification

Run: 51 · Commit: 7bf631b022a41642d871195fd8e1d4e77c123f63

Change explanation

Produced by pr-explainer-narrative, execution 1 at 7bf631b022a41642d871195fd8e1d4e77c123f63.

Why this change is needed

The task states that scheduled [[automations]] are unattended, user-configured work. The scheduler owns durable UTC cursors and leases, a per-automation open_pr policy that must override workspace worker_task_args, and allow_no_changes semantics that decide whether a no-op is a success or a failure. The task also states the migration to open_pr default-off makes regressions here silent: PRs can stop for every automation, or jobs can mutate repositories whose output lands outside the repo. No e2e test previously ran a real scheduled occurrence, and PRs 362/364/365 did not cover scheduled [[automations]].

What changed

This is a test-and-documentation change under packages/e2e. No production code is modified.

• Fixture support for scheduled automations. WorkerFixture now accepts an optional automations list and serializes each entry as a [[automations]] table in the generated workspace.toml (id, enabled, cron/interval, prompt, repo, open_pr, allow_no_changes). The default workspace is unchanged when the option is omitted. Workspace worker_task_args are now configurable so a test can isolate an automation's own open_pr opt-in.
• Live reload and state readers. The helper can append a new automation to a running workspace, read durable automation_schedules rows, list materialized occurrence markdown files, and query runs by automation_id. Run records now expose origin, automation_id, started_at, and finished_at.
• Deterministic no-change path. The fake agent recognizes an E2E_NO_CHANGE prompt marker and takes its existing no-change path. When DEVINTERN_AUTOMATION_ID is set, recorded invocations include the owning automation so tests can attribute each agent call.
• End-to-end coverage. A new tests/local/worker-automations.test.ts starts a real daemon against a temporary workspace and local bare remote, with four automations sharing one cron minute and max_concurrency = 2. It reads the queue DB and bare remote rather than CLI stdout.
  • auto-pr-on: run is scheduled with the expected automation_id and succeeds; the recorded branch is pushed and changes exactly src/greeting.ts in one commit with the expected content. PR creation itself is not covered because the bare remote exposes no code-host platform, so pr_url is null.
  • auto-pr-off: despite workspace defaults containing --create-pr, the per-automation policy suppresses the branch and push, while the agent still runs.
  • noop-allowed: succeeds, publishes no remote branch and no new base commit.
  • noop-blocked: fails with a reason containing Implementation was not committed and publishes no branch.
  • Occurrence files exist for each automation, automation_schedules has one row per configured automation, no unconfigured automation produces a run, and the daemon is still alive at the end.
  • Hot reload appends a fifth automation, waits for the reloader to reconcile it, and asserts its next scheduled occurrence.
• Second opt-in fixture. A second test omits --create-pr from workspace defaults and verifies an automation with open_pr = true still pushes its branch. pr_url remains null because no code-host platform is detectable.
• Documentation. The e2e README documents the fixture, cron-minute cost, and open_pr/allow_no_changes matrix. The coverage matrix row moves to default with the live-code-host PR gap named, and backlog item 13 is updated.

Validation actually performed

Observed in the pipeline evidence:

• The __verify-1 artifact reports the CLI build, the full local e2e suite (136 pass, 16 skip, 4 todo, 0 fail), e2e typecheck, lint, and format check all passing.
• The auto-review artifact reports the new test file passing twice (2 pass, 57 expects) at 127.0s and 171.6s, the existing worker suites passing (24 tests total), and tsc --noEmit plus oxlint clean.
• That same __verify-1 artifact is recorded as warn with approval withheld (approved=false) pending a high-priority finding: the noop-allowed run row records a local branch even though the task's literal requirement says "no branch". The test only proves nothing was pushed and does not assert branch absence or zero commits ahead of base.

No additional validation was run while writing this explanation.

Material risks

• Test-only blast radius. Observed from the diff: every changed file is under packages/e2e, so production behavior is not altered. The main regression risk is test flakiness, not runtime behavior.
• Timing sensitivity. The suite depends on cron boundaries and a fs.watch reload debounce. The recorded 171.6s run is close to the 180s timeout for the first test; on a slower CI host this could cross the limit. This is inference from the recorded duration.
• No-op contract gap. The task says noop-allowed should have no branch, but production records a local branch before halting and the test does not assert that the branch is absent or zero commits ahead. A regression that creates commits locally without pushing them could pass.
• Enabled filtering is not actually distinguished. All automations in the matrix are enabled, so the README claim about one schedule row per enabled automation cannot distinguish enabled filtering from no filtering. The auto-review records this as a low-priority finding.
• PR creation remains uncovered. This is documented in the README and coverage matrix rather than silently claimed; reviewers should treat it as a residual gap.
• Helper default compatibility. The helper is shared with existing suites. The diff intends the default workspace to stay unchanged, but that depends on the serialization preserving existing fields and ordering.

Where reviewers should focus

• packages/e2e/tests/local/worker-automations.test.ts — the assertion matrix, the noop-allowed semantics, and the timeout/timing assumptions.
• packages/e2e/helpers/worker.ts — workspace.toml generation, preservation of the default workspace, and the new queue queries.
• packages/e2e/helpers/fake-agent.ts — mode detection ordering (E2E_NO_CHANGE after existing modes) and per-automation invocation attribution.
• packages/e2e/README.md and packages/e2e/docs/coverage-matrix.md — claims should match what is actually asserted, especially around enabled filtering and PR creation.

Suggested checks (not performed here)

• Run the acceptance commands from the task, including the CLI build.
• Strengthen noop-allowed to assert the branch has zero commits ahead of base, matching the stated contract.
• Add a disabled automation entry so the enabled option and the README claim are exercised end to end.
• Re-run the new test on a slower CI runner to confirm the 180s budget is comfortable.

Visual explanation

Produced by pr-explainer-visual, execution 1 at 7bf631b022a41642d871195fd8e1d4e77c123f63.

Show artifact

Summary

This change adds end-to-end coverage for scheduled workspace [[automations]], a path that previously had no test running a real scheduled occurrence. It matters because the scheduler owns durable cursors, leases, a per-automation open_pr policy that must override workspace worker_task_args, and allow_no_changes semantics, so a regression could silently stop PRs or mutate repositories. The diff extends the e2e worker fixture and fake agent additively and adds worker-automations.test.ts with four same-minute automations plus a hot-reload check. No production code is changed.

Evidence

devintern worker (real daemon, temp workspace, local bare remote)
        |
        v
AutomationAcquirer --> automation_schedules (one row per enabled id)
        |
        +-- auto-pr-on    open_pr=true            implement   -> succeeded, branch pushed
        |                                                          (1 commit, src/greeting.ts)
        +-- auto-pr-off   open_pr=false           implement   -> succeeded, no branch/push
        +-- noop-allowed  allow_no_changes=true   no-change   -> succeeded, nothing published
        +-- noop-blocked  allow_no_changes unset  no-change   -> failed:
        |                                                          "Implementation was not committed"
        |
        +-- append [[automations]] -> reloader reconciles -> next scheduled run

• Test-only blast radius: the diff adds packages/e2e/tests/local/worker-automations.test.ts, extends helpers/worker.ts and helpers/fake-agent.ts, and updates README.md plus docs/coverage-matrix.md.
• Validation reported by __verify-1: CLI build, the full local e2e suite (136 pass, 16 skip, 4 todo, 0 fail), e2e typecheck, lint, and format check all pass.
• Validation reported by auto-review: the new test passed twice (2 pass, 57 expects) at 127.0s and 171.6s; existing worker suites passed (24 tests total); tsc --noEmit and oxlint were clean.
• Before/after execution results were not captured. The evidence contains only post-change runs.
• __verify-1 is recorded as warn with approved=false: the noop-allowed run row records a local branch, while the task literally asks for “no branch.” The test asserts no remote branch and no base commit, and the README states a local branch may still be recorded. PR creation is also not covered here because the bare remote exposes no code-host platform, so pr_url is null.

Merge Danger

This is reversible without production impact: every changed file is under packages/e2e, and no runtime package imports the e2e helpers. Reverting removes the new coverage and the additive fixture options; scheduled automations would again have no end-to-end test, and the coverage-matrix row would return to gap.

The shared helper changes do affect existing e2e suites: WorkerFixture still generates the same default workspace when automations is omitted, and the fake agent checks E2E_NO_CHANGE after the existing mode checks. The reported 24 passing existing worker tests support that, but those suites are the main dependency to watch.

The concrete shipping risk is e2e timing, not product behavior: the first test ran at 171.6s against a 180s timeout, and hot reload depends on the fs.watch debounce. A slower CI runner could cross the limit. The residual contract gap is that noop-allowed does not assert branch absence, and live PR creation remains uncovered. Safe to ship without a staged rollout because no production path changes, but expect possible CI flakiness and review the noop-allowed assertion before treating the gap as fully closed.

Verification report

Produced by __verify-1, execution 1 at 84982e106f6005e259a92964dca5c5192f82f3b2.

Show artifact

{
  "summary": "Implementation is substantially complete and verified: the CLI build, the full local e2e suite (136 pass, 16 skip, 4 todo, 0 fail), typecheck, lint, and format:check all pass. The new test exercises the real daemon, cron scheduling, open_pr override, allow_no_changes success/failure, run attribution, occurrence files, automation_schedules, and live config reload. One literal acceptance criterion is unmet: the noop-allowed run row records a local branch even though the task requires 'no branch'; the test only proves nothing was pushed and does not assert the branch is absent or has zero commits ahead. Minor robustness and documentation items follow. approved=false pending that clarification/fix.",
  "items": [
    {
      "priority": "high",
      "confidence": "high",
      "category": "testing",
      "file": "packages/e2e/tests/local/worker-automations.test.ts",
      "line": "115-124",
      "issue": "The task requires the noop-allowed run row to be 'succeeded with no branch and no new commit', but the test only asserts status succeeded, pr_url null, and no remote branch. Production records a local branch before the no-op halt, so the run row actually has branch='feature/<task-key>' and the test never checks that the branch has zero commits ahead of base.",
      "evidence": "Ran a temporary copy of the fixture with only noop-allowed: TMP_RUN_ROW {\"id\":1,\"origin\":\"scheduled\",\"automation_id\":\"noop-allowed\",\"task_key\":\"2026-11-04t20-34-00-048z\",\"status\":\"succeeded\",\"branch\":\"feature/2026-11-04t20-34-00-048z\",\"repo\":\"fixture\",\"pr_url\":null,\"outcome_reason\":null,\"attempt\":1}. Production records the branch at packages/code/src/lib/task/pipeline.ts:997-1003 before the no-op halt at packages/code/src/lib/agent/run-harness-finalize.ts:300-307. README lines 115-117 explicitly say 'a local branch may still be recorded'.",
      "suggestion": "Either assert the actual contract (status succeeded, no remote ref, and `git rev-list --count <base>..<branch>` === 0) and update the task/README to state that a local branch is recorded but nothing is committed or pushed, or, if 'no branch' is authoritative, change the no-op path to avoid recording a branch (a production change outside this test-only task). Add the zero-commits-ahead assertion in either case."
    },
    {
      "priority": "medium",
      "confidence": "high",
      "category": "documentation",
      "file": "packages/e2e/helpers/worker.ts",
      "line": "257",
      "issue": "The task says to keep `[defaults].worker_task_args` with `--create-pr --auto-review`, but the fixture default is `--create-pr --skip-clarity-check --skip-comments`; the new test therefore never exercises stripping `--auto-review` from the workspace flags.",
      "evidence": "packages/e2e/helpers/worker.ts:257 shows `worker_task_args = \"--create-pr --skip-clarity-check --skip-comments\"`; grep for `--auto-review` in packages/e2e finds only coverage-matrix gap rows, not the fixture or test. The test still proves `open_pr=false` strips `--create-pr` and adds `--no-git`, which is the functional intent.",
      "suggestion": "Either add `--auto-review` to the test-specific workspace args (without changing the shared default) if proving that flag is stripped matters, or document that the fixture default is `--create-pr --skip-clarity-check --skip-comments` and that `--create-pr` is the flag the scenario relies on."
    },
    {
      "priority": "low",
      "confidence": "medium",
      "category": "testing",
      "file": "packages/e2e/tests/local/worker-automations.test.ts",
      "line": "195-201",
      "issue": "`remoteBranchExists` returns false for any non-zero git exit code, so a git error (e.g., corrupt or missing bare repo, exit 128) would be read as 'branch absent' and could let the no-push assertions pass for the wrong reason.",
      "evidence": "Test lines 195-201: `return result.exitCode === 0;`. `git show-ref --verify --quiet` uses exit 1 for a missing ref but other codes for real errors.",
      "suggestion": "Treat exit code 1 as absent and throw or fail the assertion on any other non-zero code."
    },
    {
      "priority": "low",
      "confidence": "medium",
      "category": "code-quality",
      "file": "packages/e2e/helpers/worker.ts",
      "line": "79-83",
      "issue": "`automationTable` silently prefers `interval` when both `cron` and `interval` are supplied, even though the interface says exactly one; a caller mistake would produce a config that does not test the intended schedule.",
      "evidence": "packages/e2e/helpers/worker.ts:79-83 branches on `interval !== undefined` and otherwise writes `cron`.",
      "suggestion": "Validate that exactly one of cron/interval is set (or document the precedence) before writing the table."
    },
    {
      "priority": "low",
      "confidence": "medium",
      "category": "testing",
      "file": "packages/e2e/tests/local/worker-automations.test.ts",
      "line": "149-157",
      "issue": "The 'no run row for an automation that was not due/absent' check only inspects attributed ids after the first window; the fixture has no disabled or not-yet-due automation other than the later hot-reload entry, so disabled-entry suppression is not exercised.",
      "evidence": "Test lines 149-157; the fixture list at lines 12-21 has four enabled automations. The absent hot-reload entry is covered by the configured-set check before it is appended.",
      "suggestion": "Optionally add a disabled automation and assert it gets no run row (and, if desired, no active schedule), or state that the absent hot-reload entry is the intended 'absent' case."
    },
    {
      "priority": "low",
      "confidence": "medium",
      "category": "testing",
      "file": "packages/e2e/tests/local/worker-automations.test.ts",
      "line": "145-147",
      "issue": "The auto-pr-off 'agent invocation happened' assertion is indirect: it counts implement-mode invocations across the whole run, so it would not catch a scenario where auto-pr-off's run succeeded without invoking the agent but another implement run made the count >=2.",
      "evidence": "Test lines 145-147 count `mode === \"implement\"` across all invocations rather than correlating an invocation with the auto-pr-off run.",
      "suggestion": "Record invocation order or include a per-run marker/task key in the invocation log and assert the auto-pr-off occurrence specifically invoked the agent."
    },
    {
      "priority": "low",
      "confidence": "low",
      "category": "testing",
      "file": "packages/e2e/tests/local/worker-automations.test.ts",
      "line": "190-192",
      "issue": "Worst-case runtime headroom is thin: two cron boundaries (up to 120s), four sequential shared-base runs, fixture preparation, and up to 25s graceful shutdown can approach the 180s timeout; the observed run was 86.64s.",
      "evidence": "Observed 86.64s for the new test and 139.42s for the full local suite; worst-case arithmetic is unproven.",
      "suggestion": "If flakiness appears, reduce the stop/dispose grace or assert the reload before the first window's second occurrence; do not exceed the 180s cap."
    },
    {
      "priority": "info",
      "confidence": "high",
      "category": "code-quality",
      "file": "packages/e2e/helpers/worker.ts",
      "line": "265-268",
      "issue": "Blast-radius safety: the helper change touches every WorkerFixture consumer, but the default generated workspace is unchanged because `options.automations ?? []` appends zero lines; the added run-record columns are inert because existing consumers read named fields or row counts.",
      "evidence": "Full local suite: 136 pass, 16 skip, 4 todo, 0 fail across 25 files (139.42s). Existing runRecords/waitForSettled consumers use `.map`, `.toHaveLength`, or named fields, never deep equality. Fake-agent's new mode is last in the chain and gated on `E2E_NO_CHANGE`, which appears only in the new test/README; `E2E_AGENT_MODE` still wins at packages/e2e/helpers/fake-agent.ts:9.",
      "suggestion": "None; this is the safety fact for the blast radius."
    },
    {
      "priority": "info",
      "confidence": "high",
      "category": "testing",
      "file": "packages/e2e/tests/local/worker-automations.test.ts",
      "line": "25-193",
      "issue": "Acceptance verification passed: build, full local e2e suite, typecheck, lint, and format:check.",
      "evidence": "`bun run --filter @getdevintern/code build` exited 0; `bun test tests/local` -> 136 pass/16 skip/4 todo/0 fail; `bun run typecheck` -> tsc --noEmit exit 0; `bun run lint` -> oxlint exit 0; `bun run format:check` -> all files formatted.",
      "suggestion": "None."
    }
  ],
  "approved": false
}

Further artifacts omitted because they exceed the comment preview budget.