Repository navigation
[local-test-e2e-cover-sentry-error-monitor-auto] test(e2e): cover Sentry error-monitor auto-fix against an offline fake API - #363
Conversation
…(e2e): cover Sentry error-monitor auto-fix against an offline fake API
Fix MEDIUM testing finding in worker-sentry-monitor.test.ts: the test only asserted the generated error-fixes markdown, while the deterministic fake agent ignored prompt content, so dropping the Sentry payload from the agent prompt would still pass end-to-end. Seed a unique marker into the fake Sentry issue title and set E2E_EXPECT_DESCRIPTION_MARKER before prepare(). The fake agent now fails the run unless that marker arrives in its actual input, which run-harness.ts derives from the generated task markdown. Document the marker delivery in the e2e README.
Artifacts & verificationRun: 30 · Commit: Change explanationProduced by test(e2e): cover Sentry error-monitor auto-fix against an offline fake APIWhy this change is neededThe task states that Sentry error monitors are an external integration the repository does not control. The worker fetches unresolved issues over the Sentry API, gates on occurrence count and actionability, claims the issue for deduplication, runs the normal implementation pipeline from a generated markdown task, and optionally comments back on the Sentry issue. None of that path was covered, so a regression in the request path, response parsing, dedupe key, or comment endpoint could ship undetected. Because the production client already accepts a configurable What changed
Validation actually performedObserved evidence from the pipeline:
Also observed during review: a probe importing the fake showed that requests with no Not recorded: no live Sentry project, real token, or network-backed run was exercised. No manual product verification beyond the automated checks above. Material risks
Where reviewers should focus
Visual explanationProduced by Show artifactSummaryAdds the first end-to-end coverage of the Sentry error-monitor auto-fix path: [redacted] in-process fake Sentry API, a small opt-in hook to append an EvidenceValidation observed in the pipeline evidence:
Before/after execution results were not captured, so this explanation only reports the post-change runs above. Merge DangerReverting removes a test, the fake provider, and two additive helper changes, so no production behavior is lost — only the new coverage. The only shared-helper surface is Verification reportProduced by Show artifactAuto-review summaryProduced by Show artifact |
…e-cover-sentry-error-monitor-auto # Conflicts: # packages/e2e/docs/coverage-matrix.md
|
🔀 This branch had merge conflicts with |
Why
Task local-test-e2e-cover-sentry-error-monitor-auto: test(e2e): cover Sentry error-monitor auto-fix against an offline fake API
The worker’s Sentry error-monitor path was entirely untested: issue polling, occurrence gating, actionability validation, durable
issue:<id>claiming, generated markdown task handoff, run attribution, branch push, and comment reporting could regress without notice. The new test exercises that full path against an in-process fake, with no Sentry account, token, project, or network access.What changed
packages/e2e/providers/fake-sentry.ts: a strict127.0.0.1:0Bun server implementing the project-scoped and organization-wide issue-list routes plus the issue-comment route. It returns a seeded issue payload, records list parameters and comments, records unexpected requests, and returns diagnostic 404s for wrong routes/scopes.WorkerFixtureinpackages/e2e/helpers/worker.tswith an optionalworkspaceTomlExtraappended to the generated workspace config, leaving the default TOML byte-for-byte unchanged, and exposed theruns.origincolumn throughWorkerRunRecord.packages/e2e/tests/local/worker-sentry-monitor.test.ts: starts the real worker with an[[error_monitors]]source, waits for exactly one succeedederror_monitorrun, asserts the pushedfeature/branch contains only the greeting edit, validates the generatederror-fixes/*.mduntrusted block and issue text without credentials, asserts one success comment, and waits for two further polls to prove no duplicate run or comment.packages/e2e/README.mdwith an offline fake Sentry section and default command, andpackages/e2e/docs/coverage-matrix.mdwith both Sentry rows atdefault, the test filename as evidence, and backlog item 15 marked implemented.Scope
Included is only the Sentry error-monitor polling/auto-fix/dedupe path and its local test infrastructure. Production Sentry client, error-monitor acquirer, other providers, other e2e helpers, and unrelated tests were not changed. No commit, push, label, or pull request was created.
Tradeoffs
task_keyissentry-e2e-4501, not the bare Sentry id: production derives the workflow key from the generated markdown filename (<monitor-id>-<issue-short-id>). The test asserts that exact key and that it contains the fake issue id; making it equal the bare id would require a production change outside scope.Blast Radius
WorkerFixture.runRecords()now selects the existingorigincolumn, which is additive for current callers.workspaceTomlExtradefaults to undefined, so existing workspace fixtures generate identical TOML. The fake provider and new test are isolated to the e2e package. Documentation consumers are limited to the e2e README and coverage matrix.Verification
Ran in order from the repository root:
bun run --filter @getdevintern/code build— exit 0.bun run --filter @devintern/e2e test— 134 pass, 16 skip, 4 todo, 0 fail across 24 files (51.54s).bun run --filter @devintern/e2e typecheck— exit 0.bun run --filter @devintern/e2e lint— exit 0.bun run --filter @devintern/e2e format:check— all files correctly formatted.Also ran
bun test tests/local/worker-sentry-monitor.test.ts(1 pass, 2.20s) and with--rerun-each 3(3 pass, 6.55s) to check stability.This PR was automatically created by @devintern/code