E-values for TextIndex substring hits

[mikessh]

What

text_evalues and text_threshold_for_evalue: control-calibrated E-values for
TextIndex substring hits, the way evalues already does them for an Index.
docs/text-index.rst had asserted "no E-value" since the index was introduced;
that line is now wrong and is fixed here.

proteome = seqtree.TextIndex.build(records, "aa", k=5)
decoy    = seqtree.TextIndex.build(shuffled, "aa", k=5)
for peptide, r in zip(peptides, seqtree.text_evalues(proteome, decoy, peptides, max_subs=2)):
    if r["p_enrichment"] < 0.01:
        ...

Why these choices

The arithmetic is evalue.evalue_result unchanged. Two things are specific to
text search and both are forced by appendix/evalue.tex rather than chosen for
convenience.

Counts are per distinct record, not per offset. One query typically hits a
record at several overlapping offsets — a repeat, a low-complexity stretch, a
tandem domain — and those events are strongly dependent. Counting them
separately gives the over-dispersed compound-Poisson count that prop:collapse
removes by counting unique clonotypes rather than reads, and rem:seedfamily
says to count the union of a dependent family rather than correct for its size.
N and M are therefore num_refs.

The ball is max_subs, never TextResult.score. That score is a
similarity — 0 at an exact match, more negative for a worse one — the
opposite sign to the non-negative penalty thetas_from_scores inverts. Feeding
it in would silently invert the cutoff. n_subs is non-negative, integer and
monotone in the scope, so it nests the way lem:monotone requires and
thetas_from_scores is reused unchanged.

Two limits, both refusals rather than partial answers

• A search whose hits were capped by max_hits raises. A truncated count is
  a floor, and an E-value computed from a floor is simply wrong.
• There is no load_control for texts. The background must be composition-
  and length-matched to the target; that assumption is what the whole
  calibration rests on, nothing in the library can check it, so it stays the
  caller's explicit choice. docs/roadmap.rst item 3 records the gap.

Implementation notes

Pure Python — no C++ change, so no editable.rebuild surface and no Windows CI
risk. The CSR arrays are read through memoryview rather than materialising
TextHit objects, so a dense control ball costs a list of integers instead of
one Python object per offset, and seqtree keeps its zero runtime dependencies.

Testing

Nine tests on the calibration rather than the search, which the C++ suite
already proves complete: a motif planted in 40 of 400 records is significant
while a random octamer is not; a motif repeated three times inside one record
produces three hits but counts once; E is monotone in the radius;
exclude_exact drops only zero-mismatch records; empty control and negative
radius are rejected; the threshold round-trips to the E-value it was asked for
and returns -1 below the rule-of-three bound 3N/M.

Gates: pytest tests/python -q → 429 passed, 49 skipped (includes
test_doc_coverage.py, with text_evalue registered in SUBMODULES).
sphinx-build -W --keep-going → build succeeded.

Validated in use

Driving an unrelated antibody mass-spec project: 3,784 LC-MS/MS peptides against
33,052 antibody records, with a control built by permuting each record's
hypervariable core in place so framework is preserved. Of 304 peptides with any
match, 10 are enriched at p_enrichment < 0.01, and they are exactly the ones
spanning the hypervariable junction. The control is told nothing about junctions
and recovers them anyway.

🤖 Generated with Claude Code

[mikessh]

Superseded by the consolidated seqtree 1.1.0 release PR #13; its implementation and validation are retained there.