Skip to content

Remove HermitCrab's analysis-cascade memo - #522

Merged
johnml1135 merged 16 commits into
masterfrom
hc/remove-analysis-memo
Oct 22, 2026
Merged

johnml1135 merged 16 commits into
masterfrom
hc/remove-analysis-memo

Conversation

@johnml1135

@johnml1135 johnml1135 commented Sep 24, 2026 •

Copy link
Copy Markdown
Collaborator

Retargeted to master. The conformance suite (#480) is deferred until after this release, so this PR no longer stacks on it. Its suite-side commit (removing the harness's --no-memoization flag along with the memo) is held back and will be added to #480 after this merges.

Quick summary

Removes HermitCrab's opt-in analysis-cascade memo; the sequential rule cascade
and MergeEquivalentAnalyses still work exactly as before. Measured as a
package (#490), it cost 28-32% and ~29 GB on Sena for no net speedup on FLEx's
parallel path. Pruning (#474 landed; #491/#494/#519 open) is untouched and is
where the real parse-time wins come from.

Where to look

  • AnalysisMergeKey (AnalysisStratumRule.cs L229-318) still distinguishes non-head count and
    per-rule unapplication counts -- pinned by AnalysisMergeKeyTests.cs and MergeEquivalentAnalysesTests.cs.
  • FinalTemplateState is part of that key. A non-blocking stratum inherits it from the stratum above,
    so merging two analyses that differ only there can drop a parse; pinned by
    AffixTemplateTests.FinalTemplateInLowerStratum_ParsesWhenUpperAnalysesDifferOnlyInFinalTemplateState.
  • Memo plumbing is gone (AnalysisScope.cs, AnalysisStateKey.cs, MemoizedCombinationRuleCascade.cs
    deleted) -- maxDegreeOfParallelism and the sequential cascade are unchanged public API.
  • Review guidance dropped the memo-key re-audit language (docs/review/hermitcrab.md and others).
  • ParallelCombinationRuleCascade and ParallelPermutationRuleCascade (src/SIL.Machine/Rules) now count
    distinct results toward MaxAlternatives, as the sequential cascades do, so the parallelism setting no longer
    decides whether a parse throws MaxAlternativesExceededException; pinned by RuleCascadeTests.cs.

Validation

  • dotnet test tests/SIL.Machine.Morphology.HermitCrab.Tests -- 108 passed, 0 failed.
  • dotnet test tests/SIL.Machine.Tests -- 889 passed, 3 skipped, 0 failed.
  • dotnet csharpier check . -- 715 files checked, clean.
  • pwsh -NoProfile -File scripts/comment-hygiene.ps1 -BaseRef origin/master -- clean.
  • git diff origin/master..HEAD --stat -- 20 files, +474/-1478. Beyond the memo removal, production
    code adds FinalTemplateState to AnalysisMergeKey and fixes the parallel cascades' MaxAlternatives
    count; tests add the lost-parse test, merge-key tests, cascade-count tests and single-threaded-vs-parallel
    parity tests.
  • ./local_check.sh --agent-strict: comment hygiene clean; the Release build stops at the native SentencePiece
    download on this machine, so the test commands above were run directly.

Reading this a year from now

The memo only ran when maxDegreeOfParallelism == 1 and tracing was off. FLEx uses
the default parallelism, so production never executed it. #490's ledger measured
the optimization round as a package: no net speedup, and cross-word memo sharing
was 28-32% slower on Sena and cost ~29 GB of memory. The gains keep coming from
search-space pruning that doesn't need a replay cache: #474 (landed), and #491,
#494, and #519 (open at the time of writing). Those shrink the work instead of
caching it, and they apply on the parallel path FLEx actually uses. The memo was
~1,400 lines of subtle code (ReplayOnto grafting, arrival-count bookkeeping)
whose correctness depended on an unenforced key invariant (#490, "narrowing that
key breaks correctness with no test failure").

What stays, and why
  • The public Morpher(..., maxDegreeOfParallelism) parameter and the sequential
    cascade. They are API and still useful without the memo.
  • Fix bug in MergeEquivalentAnalyses #493's MergeEquivalentAnalyses fix. It keyed on AnalysisStateKey; that key
    now lives as an internal AnalysisMergeKey in AnalysisStratumRule, covering the
    same dimensions (shape, stratum, syntactic and realizational feature structures,
    non-head count, per-rule unapplication counts, and FinalTemplateState).
Prior measurements (not reproduced by this review round)

These are earlier author measurements from the memo-removal work, not commands run
as part of this pass's validation, and no reproducible artifact for them is checked
into the tree:

  • Spot check on 5 real grammars (Sena, Amharic, Indonesian, Mbugwe, Aweti), 5 words
    each, all with at least one parse: analysis multisets (gloss, allomorph identity,
    root index, feature structures) were reported identical between master memo-off
    and this branch in both default and sequential modes, 25/25 match.
  • On the 14 words where master memo-on also completed, it was reported no faster
    than this branch's sequential mode (e.g. 3,286 ms memo-on vs 3,122 ms without it;
    412 vs 350 ms; the rest within noise).
  • Full-solution run reported as 1,003 passed / 3 skipped / 0 failed.

🤖 Generated with Claude Code


This change is Reviewable

@codecov-commenter

codecov-commenter commented Sep 25, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 82.52427% with 18 lines in your changes missing coverage. Please review.
✅ Project coverage is 74.57%. Comparing base (2f70c04) to head (a00aaea).

Files with missing lines Patch % Lines
...chine.Morphology.HermitCrab/AnalysisStratumRule.cs 79.31% 8 Missing and 10 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #522      +/-   ##
==========================================
- Coverage   74.57%   74.57%   -0.01%     
==========================================
  Files         456      453       -3     
  Lines       38472    38314     -158     
  Branches     5297     5280      -17     
==========================================
- Hits        28692    28573     -119     
+ Misses       8617     8576      -41     
- Partials     1163     1165       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@johnml1135
johnml1135 force-pushed the hc/remove-analysis-memo branch from 4490f59 to a84dc6f Compare September 25, 2026 22:54
@johnml1135
johnml1135 force-pushed the hc/remove-analysis-memo branch from a84dc6f to 8313ba0 Compare October 14, 2026 21:13
@johnml1135
johnml1135 changed the base branch from master to integrate-conformance-framework October 14, 2026 21:13
Comment thread src/SIL.Machine.Morphology.HermitCrab/AnalysisStratumRule.cs
Comment thread tests/SIL.Machine.Morphology.HermitCrab.Tests/MorpherTests.cs
Comment thread src/SIL.Machine.Morphology.HermitCrab.Conformance/Program.cs Outdated
Comment thread src/SIL.Machine.Morphology.HermitCrab.Conformance/ConformanceMorpherFactory.cs Outdated
Comment thread src/SIL.Machine.Morphology.HermitCrab/AnalysisStratumRule.cs
@claude

This comment has been minimized.

@johnml1135
johnml1135 force-pushed the hc/remove-analysis-memo branch from 8313ba0 to f3258c0 Compare October 15, 2026 00:46
@johnml1135
johnml1135 force-pushed the integrate-conformance-framework branch from 3a63132 to df2cf21 Compare October 15, 2026 21:21
@johnml1135
johnml1135 force-pushed the hc/remove-analysis-memo branch from f3258c0 to e723ade Compare October 16, 2026 13:19
@johnml1135
johnml1135 changed the base branch from integrate-conformance-framework to master October 16, 2026 13:19
@johnml1135
johnml1135 force-pushed the hc/remove-analysis-memo branch from e723ade to 1650c10 Compare October 20, 2026 23:45
Comment thread tests/SIL.Machine.Morphology.HermitCrab.Tests/MorpherTests.cs
Comment thread tests/SIL.Machine.Morphology.HermitCrab.Tests/MorpherTests.cs Outdated
Comment thread docs/review/hermitcrab.md Outdated
Comment thread src/SIL.Machine.Morphology.HermitCrab/Morpher.cs Outdated
Comment thread tests/SIL.Machine.Morphology.HermitCrab.Tests/MergeEquivalentAnalysesTests.cs Outdated
@claude

This comment has been minimized.

johnml1135 added a commit that referenced this pull request Oct 21, 2026
…state loss

edge-cases/final-template-merge-key: a final template in a non-blocking lower
stratum, and in the last stratum a non-final template rule and a plain rule
that both add "s". "sagdss" has two analyses differing only in
FinalTemplateState; a merge key that omits it can keep the NonTemplate one and
lose the parse. Passes on #480 (master engine) and on #522 with the fix, fails
on #522 without commit 4f63793. Ledgers, manifest and pinned counts updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@johnml1135
johnml1135 force-pushed the hc/remove-analysis-memo branch from 1650c10 to 88fc8a1 Compare October 21, 2026 14:13
@johnml1135
johnml1135 requested a review from jtmaxwell3 October 21, 2026 14:14
johnml1135 and others added 13 commits October 22, 2026 15:47
The merge key dropped FinalTemplateState, which AnalysisStateKey carried.
Two analyses that differ only in it (unapplying a template rule last
leaves None; unapplying a non-template rule last leaves NonTemplate)
were merged, and the canonical's NonTemplate then blocked a final
template in a lower, non-blocking stratum that the merged-away None
alternative would have allowed, losing the parse.

Add a parse test that fails without the key change, and a key test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The sequential CombinationRuleCascade (maxDegreeOfParallelism 1) is still a
distinct code path, and only the compounding grammar compared it with the
parallel default. Add comparisons for affix templates, a Linear stratum, and
Simultaneous epenthesis (buibui parses to one analysis), without the memo
hit-counter assertions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Two words with the same multiset of unapplied rules reached in different
orders must produce equal keys and equal hashes; that is what the XOR
multiset combine exists for. Replacing it with a rolling combine now fails
this test, and previously passed every AnalysisMergeKey test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The constructor mutates its argument by freezing SyntacticFeatureStruct,
which Word.FreezeImpl leaves unfrozen; a plain constructor does not signal
that, so say so at the constructor.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… parses

The compounding test compared two empty results for pʰutdidat, and the Linear test compared an empty
digusagd; both now assert the words that really parse and say which word does not.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The hermitcrab checklist lists exactly the AnalysisMergeKey fields, MaxDegreeOfParallelism is documented again as a
results-neutral performance knob, and the merge-order test comment is stated in terms of AnalysisMergeKey.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…acuously

The parallelism-cap test compared capped and unbounded results only, so two empty results for pʰutdat would pass. It now asserts the unbounded parse of pʰutdat is non-empty, matching the compounding parity test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The doc said every value produces equal parse results. With
MaxAlternatives set the parallel cascade counts alternatives before
removing duplicates, so it can throw where a value of 1 parses. The
doc is a rule again, as on master, and names that exception.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…geKey

AnalysisStateKeyTests covered both behaviors; the port dropped them, so
removing either from AnalysisMergeKey passed every test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On a Linear stratum the di and gu prefixes still both apply, but only in
rule order, so gudisagd parses and digusagd does not. The test and the
fixture comment said neither nesting parsed; the test now asserts
gudisagd is non-empty and compares it across both cascades.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ParallelCombinationRuleCascade and ParallelPermutationRuleCascade added
every rule output to the count before removing duplicates, so with
MaxAlternatives set they could throw where the sequential cascades,
which count a set, do not. Collect output in a set and count only new
results. The MaxDegreeOfParallelism doc no longer needs the exception.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
#519 landed on master with tests that set Word.AnalysisScope, which this
branch removes along with the memo. The transform does not read it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The parallel cascades now build a ConcurrentDictionary from Comparer,
whose netstandard2.0 constructor rejects null; before, Distinct(null)
fell back to the default. ParallelCombinationRuleCascade also called
Comparer.Equals directly, so a null comparer already crashed there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@johnml1135
johnml1135 force-pushed the hc/remove-analysis-memo branch from c5376ac to a00aaea Compare October 22, 2026 20:18
@claude

claude Bot commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

Review: approve

  1. Verdict: approve. a00aaea fixes F15. The rebase onto Prune reduplication matches whose copies cannot agree #519 and Restore analysis of literal rewrites without phonological features #538 needed one test-only change, 9740b96, and adds no new findings.
  2. Most important: nothing blocks the merge. RuleCascade.cs:33 now defaults a null comparer, so the parallel cascades no longer depend on the runtime accepting null (F15).
  3. Counts: Critical 0, Important 0, Low 0 new.
  4. Ran: the compare from c5376ac came back diverged (rebased onto 2f70c04), so I reviewed the whole PR again with /code-review high. It raised six candidates. All of them were pre-existing on master or not supported by the code, so none is posted. dotnet test tests/SIL.Machine.Tests --filter RuleCascadeTests: 3 passed, 0 failed. gh pr checks 522: both Build jobs and the advisory comment-hygiene checks passed; Create NuGet package and claude-review are pending. The advisory check does not stand in for the strict scan.
  5. Not verified: the null-comparer path on a .NET Framework runtime, which this environment cannot run. I also did not verify sequential-path performance without the memo; that still rests on the HermitCrab optimization round record: what was tried, and a faster package that is not mergeable #490 measurements in the PR body.

API / framework / parity: no public signature change. RuleCascade (SIL.Machine, netstandard2.0) now replaces a null comparer with EqualityComparer<TData>.Default, so its Comparer property is never null. The parallel cascades' MaxAlternatives counting changed as described in the last round. Target frameworks, packages, and machine.py parity: none verified changed.

Findings: F1 addressed (4f63793); F2 addressed (8987126); F3 withdrawn; F4 addressed (e2579c6); F5 accepted, unverified; F6 addressed (1650c10); F7 addressed (c0fb7d7); F8 addressed (c0fb7d7); F9 addressed (72851b3); F10 addressed (72851b3); F11 addressed (72851b3); F12 addressed (1106397, c5376ac); F13 addressed (d900f0f); F14 addressed (27a22a9); F15 addressed (a00aaea).

Reviewed at a00aaea

johnml1135 added a commit that referenced this pull request Oct 22, 2026
…state loss

edge-cases/final-template-merge-key: a final template in a non-blocking lower
stratum, and in the last stratum a non-final template rule and a plain rule
that both add "s". "sagdss" has two analyses differing only in
FinalTemplateState; a merge key that omits it can keep the NonTemplate one and
lose the parse. Passes on #480 (master engine) and on #522 with the fix, fails
on #522 without commit 4f63793. Ledgers, manifest and pinned counts updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@johnml1135
johnml1135 merged commit 2599204 into master Oct 22, 2026
12 checks passed
@johnml1135
johnml1135 deleted the hc/remove-analysis-memo branch October 22, 2026 21:09
johnml1135 added a commit that referenced this pull request Oct 22, 2026
…state loss

edge-cases/final-template-merge-key: a final template in a non-blocking lower
stratum, and in the last stratum a non-final template rule and a plain rule
that both add "s". "sagdss" has two analyses differing only in
FinalTemplateState; a merge key that omits it can keep the NonTemplate one and
lose the parse. Passes on #480 (master engine) and on #522 with the fix, fails
on #522 without commit 4f63793. Ledgers, manifest and pinned counts updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
johnml1135 added a commit that referenced this pull request Oct 22, 2026
The analysis memo is going away (#522), so --no-memoization, the
useMemoization parameters and their tests go with it, along with the
design and plan that introduced them. Self-check now takes both the result
set and the trace from a single traced parse instead of an untraced parse
plus a second traced one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
johnml1135 added a commit that referenced this pull request Oct 23, 2026
PR #491 added FinalTemplateState to the key, and #522 then removed
AnalysisStateKey with the analysis memo. Name master's AnalysisMergeKey,
which carries the listed fields plus FinalTemplateState.

Also say that the two engine facts in section 4 concern the removed memo.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
johnml1135 added a commit that referenced this pull request Oct 23, 2026
#522 removed AnalysisStateKey along with the analysis-cascade memo and
kept the same identity as AnalysisStratumRule.AnalysisMergeKey, so the
shared-walk equivalence test now compares outputs under that key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
johnml1135 added a commit that referenced this pull request Oct 23, 2026
The analysis memo that MemoEntry belonged to was removed in #522 and will not return, so its arrival-count constraint no longer applies. The remaining order-invariance constraint becomes the section's only item.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
johnml1135 added a commit that referenced this pull request Oct 23, 2026
Section 4 kept two engine facts for any revival of the analysis memo that
#522 removed. The memo is not coming back, so both facts now describe code
that does not exist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants