One failed group no longer skips the recompute for every group; add --recompute-only (#348)

[NewGraphEnvironment]

Summary

Fixes #348. In study_area_run.sh, one failed group no longer costs every other group its post-consolidate recompute.

• Coverage splits the run's groups instead of exiting. run_coverage, in the new data-raw/study_area_finish.sh, reads the dispatcher's database after consolidate and sorts the run's groups (every bucket, less SKIPs) into four classes:

  • Rows from a build this run finished. The group's newest <schema>.log row is this run_uid's and closed (the rule schema_consolidate.R uses), or the driver's done / consolidated-bucket signals say so.
  • No rows.
  • Rows that are not this run's finished build: an earlier run's, a later rebuild's, or a partial build's.
  • Lost at consolidate.

  The classes are written to <TS>_coverage.tsv.

• The recompute and the compare run over the first class only. If that class is empty, neither runs and the report says why. Every other class makes the run exit non-zero, and the report names those groups and gives the recovery commands. It sends a kept host's groups to kept_hosts.tsv first. A failed compare no longer hides the report.

• --recompute-only --run-uid=<uid> [--wsgs=A,B] finishes a run once its missing groups are re-modelled under the same run_uid, which is what Provenanced province-wide default run: all 217 modelable WSGs into fresh_default #246 needed.

  • It runs only the run's tail: no droplet, no pre-flight, no discharge resolution, no consolidate.
  • It refuses a --config other than the one the run's log rows name.
  • It warns when --wsgs= is a subset of the logged groups, or when a kept_hosts.tsv names the run.
  • The driver refuses it alongside dispatch flags, and refuses --run-uid or --wsgs without it.

• The tail lives in the lib, so both paths share it and tests can reach it. The pool functions stay in the driver, because pool_probe.sh, recompute_parity.sh and recompute_sweep.sh extract them by pattern. They only moved up the file.

Related Issues

• Relates to NewGraphEnvironment/sred#24

Test plan

• tests/testthat/test-study_area_finish.R: 70 expectations, all passing. It runs on a scratch schema on the local fwapg, uses the driver's real pool functions, and stubs Rscript to record its calls. Acceptance cases:
  • with one missing group, the other groups alone are recomputed and compared, and the run exits 1 naming the missing group;
  • with no groups at all, nothing is recomputed or compared, and the run says why.
• Mutants, each turning its test red:
  • the old exit on a missing group;
  • dropping the newest-row term;
  • removing the empty-set skip;
  • a kept-host set built from the bucket.
• test-study_area_hosts.R: 120/120. pool_probe.sh data-raw/study_area_run.sh: 37/37.
• Live, read-only, on fresh_default for run_uid 20261017_004619-3f4c713b:
  • 241 groups expected (from the log), 241/241 in the recompute class, 0 missing;
  • --config=default passes, bcfishpass is refused.
  • No recompute was run: it would rewrite shipped tables (~154 min).
• Full suite: 3195 pass. Two tests fail and two error, all outside this change: test-lnk_pipeline_prepare.R (golden dams pin, recorded stale in CLAUDE.md: 8 pinned, cabd.dams gives 7) and test-lnk_wsg_resolve.R (closure). No R/ file changes on this branch.
• Links to issues checked (titles match)

Notes

• Review: a plan review and four /code-check rounds; findings are in planning/archive/2026-10-issue-348-recompute-after-failed-group/. The plan review's blocker was a forgotten --config recomputing 241 groups under bcfishpass. Rounds 1–3 each found a defect inside the previous round's fix of the kept-host recovery text. The common cause was advice derived from a set without the qualifier that gives it meaning. Round 3 fixed it at the root: the report now defers to kept_hosts.tsv. Round 4's 37-row enumeration of every printed instruction ended the loop.
• Documented in RUNBOOK §8d, not fixed:
  • --recompute-only takes no lock;
  • study_area_verify.sql stays red for a group whose failed attempt left an open log row on the dispatcher;
  • a group whose soft date_end UPDATE failed cannot be rescued by --recompute-only.
• Deferred to study_area_run.sh: a run that stops before its tail leaves no record of which groups it was running #358:
  • persisting the run's group list when the buckets resolve;
  • carrying on to coverage after a consolidate refusal.

🤖 Generated with Claude Code

https://claude.ai/code/session_01WJE2szSCkeV9R3qKhuSwyx