Skip to content

chore: trim obsolete tests, make the suite hermetic, and speed it up - #94

Merged
917Dhj merged 1 commit into
developfrom
chore/test-hygiene
Nov 7, 2026
Merged

917Dhj merged 1 commit into
developfrom
chore/test-hygiene

Conversation

@917Dhj

@917Dhj 917Dhj commented Nov 7, 2026 •

Copy link
Copy Markdown
Owner

Summary

The test suite was slow and carried tests that guarded code, schemas, and docs wording that no longer exist. This PR removes those, makes the suite independent of the network and of a local OCR install, and cuts the wall time.

  • Deleted 19 tests:
    • test_no_legacy_rewrite_helpers.py;
    • removed-router, legacy-heading, and README-wording pins in test_contracts_consistency.py;
    • old v1/v3 review and old evidence-field rejections;
    • the keyword-domain-rules pin;
    • five paper-glossary plan/design pins.
  • Loosened:
    • star history: structural asserts instead of exact SVG;
    • Codex adapter: key set plus a short prompt;
    • the figure-protocol doc check, trimmed to its single-owner strings;
    • the type count, now pinned only in test_contracts_consistency.py.
  • extract_evidence.py: --source-manifest is required, and the unused direct-PDF mode is gone, along with --max-pages and its 5 tests. run_pipeline and run_state already pass the manifest.
  • Removed skills/paper-glossary/references/optimization-implementation-plan.md: nothing referenced it.
  • Hermetic tests: an autouse fixture runs every test, and every subprocess, against an empty HTTP cassette and a shadow pytesseract that fails to import. One new OCR test runs only where tesseract is installed.
  • Lazy pytesseract: it loads on first OCR use instead of on every import of extract_pdf_assets.
  • test_visual_review_cli.py: plans once per module, copies the run per test, and calls main() in-process. Four tests still go through a real subprocess.
  • CI: superseded runs on the same ref are cancelled, there is a 15-minute job timeout, and compileall and ruff run on ubuntu only.

Evidence

before (#93) after
pytest -n auto 51 s 14 s
serial pytest 194 s 82 s
test_visual_review_cli.py 77 s 4.5 s
40-page asset test 13.4 s 1.2 s
  • python3 -m pytest -q -n auto: 1730 passed, 1 skipped (the OCR test, no tesseract here).
  • python3 -m ruff check skills tests: clean.
  • python3 -m pytest -q -m corpus: 368 passed, 21 xfailed.

Merge Danger

  • Low risk. The only production changes are the lazy import and the now-required --source-manifest.
  • Anyone calling extract_evidence.py by hand without --source-manifest now gets an argparse error. Both in-repo callers already pass it.
  • The OCR path is no longer exercised in CI, which has no tesseract. It runs locally wherever tesseract is installed.

🤖 Generated with Claude Code

- Delete tests that only pinned removed code, legacy schemas, or docs
  wording; loosen three literal pins to structural checks.
- extract_evidence: require --source-manifest and drop the direct-PDF
  mode no caller uses.
- Remove the unreferenced paper-glossary optimization plan.
- Tests run offline (empty HTTP cassette) and OCR-free (shadow
  pytesseract); one OCR test runs only where tesseract is installed.
- Import pytesseract lazily.
- visual_review_cli tests plan once per module and call main in-process.
- CI: cancel superseded runs, 15-minute timeout, lint and compile on
  ubuntu only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@917Dhj
917Dhj merged commit 9e2147b into develop Nov 7, 2026
3 checks passed
@917Dhj
917Dhj deleted the chore/test-hygiene branch November 7, 2026 20:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant