Skip to content

feat: one-shot advisory cross-check - #537

Merged
juzser merged 4 commits into
mainfrom
feat/advisory-crosscheck
Nov 4, 2026
Merged

juzser merged 4 commits into
mainfrom
feat/advisory-crosscheck

Conversation

@juzser

@juzser juzser commented Nov 4, 2026

Copy link
Copy Markdown
Owner

Phase 1a, T5 of the lean-mode scope (docs/specs/lean-mode-scope.md §5, decisions O-2 and O-9 / Q-2 A).

The operator must decide before merge: the shipped crosscheck.yml has send_diff: false, and a project overlay accepts only providers:. With the shipped defaults, every run therefore skips. That is correct behaviour, and the skip reason says so. The question is how a project opts in.

Problem

Today's cross-check is a quorum vote that needs min_providers: 2. In the 2026-11-03 trial, only codex was configured, so every close held. Each close also called codex twice, and one refute misread a check label.

Change

New module advisoryCrosscheck.ts, with two verbs.

bs crosscheck run

Usage: bs crosscheck run --task --base --head [--criterion… | --criteria-file | --plan] [--test-file…]

It always exits 0. Its status is one of ok, skipped, failed or discarded.

  • Provider. The first provider in independent_finder.providers order that has mode: active. Only listed vendors ever receive source. If none qualifies, the run is skipped and records why. The operator is never asked.
  • Prompt. It asks the two §5 questions: real defects with a concrete input and output, and criteria that no test would catch. It carries the criteria, the diff and the test files, and no check labels.
  • Bounds. The diff is read with maxBuffer = max_diff_bytes + 1. An oversize diff gives skipped/diff-too-large. Test files draw on the same byte budget, and any that are dropped are listed in test_files_dropped.
  • Read-only and fingerprinted. It reuses runJudge, fingerprintWorktree and checkWorktreeImmutable. If the tree moves, the result is discarded and no findings are recorded.
  • Honest sent. Errors raised before any byte leaves give skipped/sent: false. These are missing-api-key, cli-unavailable, disabled, not-external and unknown, each traced to the line that raises it. Anything after the send gives failed/sent: true, including auth-failed on a 401/403.
  • Envelope. The verb pins its envelope at pre-flight; it is classified in envelopePreflightSweep.

bs crosscheck reproduce <finding-id> --evidence <…>

Blocking happens only on reproduction (Q-2 A).

  • The verb folds existing findings first, raises a finding only if it is absent, and confirms only from raised.
  • It is idempotent: a repeat returns already_reproduced: true.
  • A finding already closed (refuted, fix-verified or expired) is refused with crosscheck.finding-closed.
  • After a reproduce, the normal confirmed-finding path routes the work to the coder.

Isolation

Findings go to an advisory-crosscheck event as advisory: true. No judge-verdict is written, and nothing reaches finding-raised before a reproduce. Quorum, epic close, provider-agreement stats, report and the projection are unaffected. Tests pin this.

Review

A fresh-context review ran three rounds, using fake transports only.

  • Round 1:
    • a diff over 1 MiB crashed the verb;
    • test files were uncapped;
    • provider selection ignored mode and the source allow-list;
    • errors after the send were logged as skipped;
    • a reproduce retry could reset a finding;
    • the sweep check was a no-op.
  • Round 2:
    • auth-failed after a POST was recorded as sent: false;
    • providers were picked in file order, not list order;
    • reproduce could re-mark a closed finding.

All of these are fixed, each with a test written first and seen to fail.

Verification

  • Full suite under heavy run: 276 files, 10924 passed.
  • Targeted files, the sweep and the doc guards, run from the repo root: 168 passed.
  • typecheck, typecheck:test and biome at error level: clean.
  • No real provider was called in any test.

🤖 Generated with Claude Code

juzser and others added 4 commits November 4, 2026 12:22
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…indings

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@juzser
juzser merged commit e5e02b3 into main Nov 4, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant