Add a conformance suite for HermitCrab, with its adequacy argument

.claude/skills/author-coverage-cell/SKILL.md

@@ -0,0 +1,18 @@
+---
+name: author-coverage-cell
+description: >-
+  Use when asked to cover, satisfy, witness, or close an obligation cell in the HermitCrab
+  conformance suite — anything of the form "cover cell X", "satisfy this dataflow obligation",
+  "add words so this chain is witnessed", "close a coverage gap", or when handed a cell id from
+  conformance/dataflow-obligations.tsv. Authors the minimal-pair words in an existing language
+  grammar and the claimed_cells entry that records the evidence, then verifies by severance rather
+  than by inspection. Does NOT author new grammars, new fixtures, or grammar.xml content — if the
+  construct is absent from every language grammar, this skill reports that and stops.
+---
+
+The authoritative instructions for this skill live with the fixtures, because they ship to whoever
+receives `conformance/`. Read and follow:
+
+`conformance/skills/author-coverage-cell/SKILL.md`
+
+Do not follow a summary of it from memory; open the file.

.claude/skills/measure-authoring-quality/SKILL.md

@@ -0,0 +1,16 @@
+---
+name: measure-authoring-quality
+description: >-
+  Use when asked to measure, score, or benchmark the conformance authoring harness — "how good is
+  the author skill", "what's our first-pass yield", "run the harness eval", "measure before and
+  after a skill revision". Runs a batch of obligation cells through author-coverage-cell without
+  intervention, scores first-pass yield against the production gate, and attributes every failure to
+  a class. Produces a run record; changes no skill and fixes no fixture.
+---
+
+The authoritative instructions for this skill live with the fixtures, because they ship to whoever
+receives `conformance/`. Read and follow:
+
+`conformance/skills/measure-authoring-quality/SKILL.md`
+
+Do not follow a summary of it from memory; open the file.

.claude/skills/review-coverage-claim/SKILL.md

@@ -0,0 +1,16 @@
+---
+name: review-coverage-claim
+description: >-
+  Use when asked to review, verify, certify, or check a coverage claim in the HermitCrab conformance
+  suite — "review this claimed_cells entry", "does this word really demonstrate that cell", "check
+  the role attribution", or when handed an evidence card from conformance/evidence-cards/. Judges
+  ONLY role attribution: whether the word occupies the cell it claims. Never sets a status, never
+  edits a claim, never edits a grammar or an expectation. Declining is a valid outcome.
+---
+
+The authoritative instructions for this skill live with the fixtures, because they ship to whoever
+receives `conformance/`. Read and follow:
+
+`conformance/skills/review-coverage-claim/SKILL.md`
+
+Do not follow a summary of it from memory; open the file.

.claude/skills/revise-coverage-harness/SKILL.md

@@ -0,0 +1,16 @@
+---
+name: revise-coverage-harness
+description: >-
+  Use when asked to improve, revise, or fix the conformance authoring harness after a measurement —
+  "first-pass yield is low, fix the skill", "most failures are not-minimal-pair, revise the
+  instructions", "improve the harness based on the run record". Consumes a measurement run record,
+  diagnoses which instruction produced the dominant failure class, and revises exactly one thing.
+  Never revises without a measurement, and never touches the golden set.
+---
+
+The authoritative instructions for this skill live with the fixtures, because they ship to whoever
+receives `conformance/`. Read and follow:
+
+`conformance/skills/revise-coverage-harness/SKILL.md`
+
+Do not follow a summary of it from memory; open the file.

.gitattributes

@@ -5,3 +5,14 @@
 *.yml eol=lf
 *.cfg eol=lf
 *.ipynb eol=lf
+
+# Frozen bytes: conformance/*/fieldworks/project.fwdata's exact SHA-256 is pinned in that
+# fixture's phonology-mutations.yaml (see FieldWorksWitnessDirectoriesAreShapedCorrectly). Without
+# this, a platform/config-dependent CRLF checkout of an otherwise-text XML file makes the hash
+# non-reproducible across machines.
+*.fwdata binary
+
+# Preserve the original bytes of these historical oracle captures and their input list.
+conformance/edge-cases/simultaneous-subrule-genuine-overlap/hc_stderr.txt -text
+conformance/edge-cases/simultaneous-subrule-genuine-overlap/output.tsv -text
+conformance/edge-cases/simultaneous-subrule-genuine-overlap/words.txt -text

.github/workflows/conformance.yml

@@ -0,0 +1,73 @@
+name: Conformance
+
+on:
+  push:
+    branches:
+      - "**"
+    paths:
+      - "conformance/**"
+      - "src/SIL.Machine.Morphology.HermitCrab*/**"
+      - "src/SIL.Machine/**"
+      - "tests/SIL.Machine.Morphology.HermitCrab.Tests/**"
+      - ".github/workflows/conformance.yml"
+  pull_request:
+    paths:
+      - "conformance/**"
+      - "src/SIL.Machine.Morphology.HermitCrab*/**"
+      - "src/SIL.Machine/**"
+      - "tests/SIL.Machine.Morphology.HermitCrab.Tests/**"
+      - ".github/workflows/conformance.yml"
+
+jobs:
+  conformance:
+    name: Conformance suite
+    runs-on: ubuntu-22.04
+
+    steps:
+      - uses: actions/checkout@v6
+
+      - name: Setup .NET
+        uses: actions/setup-dotnet@v5
+        with:
+          dotnet-version: |
+            10.0.303
+            10.0.x
+
+      - name: Restore dependencies
+        run: dotnet restore src/SIL.Machine.Morphology.HermitCrab.Conformance/SIL.Machine.Morphology.HermitCrab.Conformance.csproj
+
+      # Gate (a): every fixture's declared parses/rejections must match what the C# oracle
+      # actually produces (self-check mode runs the reference engine in-process against every
+      # words.yaml, including budget_ms pathological fixtures).
+      - name: Run conformance self-check
+        run: >-
+          dotnet run --no-restore --project src/SIL.Machine.Morphology.HermitCrab.Conformance --
+          --fixtures conformance --include-pathological
+
+      # Gate (b): --coverage-report (re)writes coverage.csv/rules.csv, which the next step checks for
+      # staleness, and exits 1 on an uncovered construct, on a dead rule (exercised by zero words)
+      # missing from conformance/dead-rules-allowed.tsv, or on an allowed rule some word now exercises.
+      # `if: always()` so it reports its own status even when the self-check step above failed.
+      - name: Run coverage report
+        if: always()
+        run: >-
+          dotnet run --no-restore --project src/SIL.Machine.Morphology.HermitCrab.Conformance --
+          --fixtures conformance --coverage-report
+
+      # Gate (c): the checked-in coverage.csv/rules.csv must already reflect the fixtures as they
+      # stand -- generated so nobody needs to remember to refresh them, but that only holds if
+      # forgetting to refresh them is a CI failure. Runs after the coverage-report step (which just
+      # regenerated both) and `if: always()` for the same reason as above: a dead rule and a stale
+      # index are two different failures and must not collapse into one.
+      #
+      # conformance/fixtures.csv is intentionally NOT checked here: on this branch nothing
+      # regenerates it (--coverage-report only writes coverage.csv/rules.csv), so a freshness check
+      # against it would never catch drift -- it is a one-time snapshot from when it was ported, not
+      # a generated artifact.
+      - name: Verify generated coverage files are fresh
+        if: always()
+        run: |
+          if ! git diff --exit-code -- conformance/coverage.csv conformance/rules.csv; then
+            echo "::error::conformance/coverage.csv or rules.csv is stale relative to the fixtures (diff above). Run 'dotnet run --project src/SIL.Machine.Morphology.HermitCrab.Conformance -- --fixtures conformance --coverage-report' locally and commit the result."
+            exit 1
+          fi

.github/workflows/counterfactual-coverage.yml

@@ -0,0 +1,36 @@
+# The counterfactual sweep re-parses every conformance fixture once per grammar-observable surface with
+# that surface neutralized, so it costs minutes rather than seconds. It runs on demand and weekly rather
+# than per push; the cheap coverage ledger and fixture gates stay in the main CI job.
+name: Counterfactual coverage
+
+on:
+  workflow_dispatch:
+  schedule:
+    - cron: "0 6 * * 1"
+
+jobs:
+  counterfactual:
+    name: Recompute counterfactual evidence
+    runs-on: ubuntu-22.04
+
+    steps:
+      - uses: actions/checkout@v6
+
+      - name: Setup .NET
+        uses: actions/setup-dotnet@v5
+        with:
+          dotnet-version: |
+            10.0.303
+            10.0.x
+
+      - name: Restore dependencies
+        run: dotnet restore
+
+      - name: Build
+        run: dotnet build --no-restore -c Release
+
+      - name: Recompute and compare the evidence ledger
+        run: dotnet run --no-build -c Release --project src/SIL.Machine.Morphology.HermitCrab.Conformance -- --counterfactual --repository-root .
+
+      - name: Run the explicit counterfactual tests
+        run: dotnet test tests/SIL.Machine.Morphology.HermitCrab.Tests/SIL.Machine.Morphology.HermitCrab.Tests.csproj -c Release --filter "TestCategory=Counterfactual"

.gitignore

@@ -56,5 +56,11 @@ tests/SIL.Machine.Tests/Corpora/TestData/project/*
 tests/SIL.Machine.Tests/Corpora/TestData/pretranslations.json
 .idea
 
+# Claude Code conversation exports
+/20??-??-??-*.txt
+
+# Agent scratch: test result dumps, captured logs, throwaway SDK/CLI state
+.tmp/
+
 # Local, generated review summaries
 .review/

Machine.sln

@@ -45,6 +45,8 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "SIL.Machine.Tokenization.Se
 EndProject
 Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "SIL.Machine.Tokenization.SentencePiece.Tests", "tests\SIL.Machine.Tokenization.SentencePiece.Tests\SIL.Machine.Tokenization.SentencePiece.Tests.csproj", "{AB5F75C1-64B7-4E0F-A4B5-B14EB16E6DDC}"
 EndProject
+Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "SIL.Machine.Morphology.HermitCrab.Conformance", "src\SIL.Machine.Morphology.HermitCrab.Conformance\SIL.Machine.Morphology.HermitCrab.Conformance.csproj", "{8E40C2C5-F4C5-4A41-B78A-10A43FCF0CE3}"
+EndProject
 Global
 	GlobalSection(SolutionConfigurationPlatforms) = preSolution
 		Debug|Any CPU = Debug|Any CPU
@@ -99,6 +101,10 @@ Global
 		{AB5F75C1-64B7-4E0F-A4B5-B14EB16E6DDC}.Debug|Any CPU.Build.0 = Debug|Any CPU
 		{AB5F75C1-64B7-4E0F-A4B5-B14EB16E6DDC}.Release|Any CPU.ActiveCfg = Release|Any CPU
 		{AB5F75C1-64B7-4E0F-A4B5-B14EB16E6DDC}.Release|Any CPU.Build.0 = Release|Any CPU
+		{8E40C2C5-F4C5-4A41-B78A-10A43FCF0CE3}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
+		{8E40C2C5-F4C5-4A41-B78A-10A43FCF0CE3}.Debug|Any CPU.Build.0 = Debug|Any CPU
+		{8E40C2C5-F4C5-4A41-B78A-10A43FCF0CE3}.Release|Any CPU.ActiveCfg = Release|Any CPU
+		{8E40C2C5-F4C5-4A41-B78A-10A43FCF0CE3}.Release|Any CPU.Build.0 = Release|Any CPU
 	EndGlobalSection
 	GlobalSection(SolutionProperties) = preSolution
 		HideSolutionNode = FALSE
@@ -116,6 +122,7 @@ Global
 		{0BBEED18-3CC1-4BA6-8E11-603A5D2DEDC9} = {31B4F5EB-E148-4A51-84B5-B90BB8F38AD3}
 		{09135223-6285-4268-BFA3-147BDB5B663B} = {31B4F5EB-E148-4A51-84B5-B90BB8F38AD3}
 		{AB5F75C1-64B7-4E0F-A4B5-B14EB16E6DDC} = {66246A1C-8D45-40FB-A660-C58577122CA7}
+		{8E40C2C5-F4C5-4A41-B78A-10A43FCF0CE3} = {31B4F5EB-E148-4A51-84B5-B90BB8F38AD3}
 	EndGlobalSection
 	GlobalSection(ExtensibilityGlobals) = postSolution
 		SolutionGuid = {9F18C25E-E140-43C3-B177-D562E1628370}