Skip to content

feat: implement exact seeded base-pair probabilities (#257) - #258

Open
Alexander-Mitrofanov wants to merge 23 commits into
masterfrom
docs/257-base-pair-probability-plan
Open

Alexander-Mitrofanov wants to merge 23 commits into
masterfrom
docs/257-base-pair-probability-plan

Conversation

@Alexander-Mitrofanov

@Alexander-Mitrofanov Alexander-Mitrofanov commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

Exact seeded --model=P --mode=M prediction now counts each complete interaction once, even when it contains several overlapping or disjoint seeds. --out=bpProb:FILE reports actual interior-pair probabilities over the union of searched disjoint regions, using the same raw partition denominator as prediction without pair output.

The implementation follows the reviewed plan on this existing branch in seven separate commits, one per step:

  1. 88f4369: Establish an independent exhaustive weighted-chain oracle and all ten unrounded theory fixtures.
  2. 62afc24: Compile geometric stack-seed capability, active occurrences, and eligible domains.
  3. 5c8907c: Validate and compare both kernel candidates and select the suffix kernel.
  4. 8bd8534: Integrate unique partition accounting, shared boundary coefficients, extension hooks, and exact-zero handling.
  5. 99919de: Accumulate raw pair masses in original coordinates across successful disjoint regions.
  6. 3f6d3d1: Expose the matrix writer and CLI output, including multiFASTA, gzip, and shared streams.
  7. 48efc54: Complete validation, documentation, performance measurements, installed consumer checks, and packaging.

Stack-seed admission is preserved, including accepted mixed-length and singleton explicit seeds. Existing bulged and heuristic prediction paths remain available; unsupported combinations with bpProb fail explicitly. The new matrix contains actual pairs; existing spot output retains its coverage interpretation. Successful empty searches produce NA entries. Numerical range errors fail explicitly without a scaling fallback, and failed calculations do not publish a partial probability matrix. Corrected partition accounting can change ensemble results; no golden output was regenerated to conceal differences.

Validation:

  • GCC 14.4 release and debug builds each pass all seven test suites, including 91,361 API assertions in 85 cases.
  • Both kernels agree with independent enumeration across ten theory fixtures and 100 randomized heterogeneous cases, plus native energy and accessibility checks.
  • CLI checks cover reporting invariance, disjoint regions, signed coordinates, GU admission, tiny partitions, singleton and mixed seeds, empty searches, unsupported modes, I/O failures, scientific notation, gzip, and serial/parallel multiFASTA output.
  • All 74 installed public headers compile independently. The installed predictor example links through pkg-config; the missing -leasylogging dependency is fixed and CI now runs this consumer.
  • make dist succeeds and includes all 23 new distributable files. Encoding and git diff --check pass.
  • The native OxyS–fhlA measurement takes a median 0.09 s without the matrix and 0.16 s with it over three runs, with identical raw partition totals. These figures compare output modes, not legacy performance.

See kernel comparison and validation report for reproducible commands and limitations. Apple Clang validation is left to the existing macOS CI because that toolchain is unavailable locally.

Addresses #257.

@martin-raden

Copy link
Copy Markdown
Member

@Alexander-Mitrofanov

thanks, implement the plan in multiple commits (at least one per step) to this branch

@Alexander-Mitrofanov Alexander-Mitrofanov changed the title docs: plan exact seeded base-pair probabilities (#257) feat: implement exact seeded base-pair probabilities (#257) Oct 8, 2026
@martin-raden

martin-raden commented Oct 9, 2026 •

Copy link
Copy Markdown
Member

@Alexander-Mitrofanov

thanks, please add the following steps:

(step 8) SVG dot plot output

general SVG requirements:

  • sequence names are x/y "axis" labels
  • sequence nts are provided as surrounding frame
  • 1-nt unpaired probabilities are provided as surrounding frame of filled rectangles to highlight local accessibility of sequences. the filling gradient encodes the unpaired probability
  • probability gradients are to be implemted as segmented gradient, e.g.
linear-gradient(90deg, 
  #FFFFFF 0%, #FFFFFF 25%, 
  #AEC6CF 25%, #AEC6CF 50%, 
  #4682B4 50%, #4682B4 75%, 
  #002060 75%, #002060 100%
);
  • for the gradient, use at least 6 color steps following the suggestion from above to enable a visual distinction of probability value (classes)

  • below the plot, add a color gradient legend for the respective probability value ranges

  • ensure the color gradient is css based and defined in a way that it can be easily adapted later

  • use the same color gradient for the base pair probability rectangles in the central matrix

  • add gray colored guiding lines

    • every 10th, 50th nt using the final sequence index, i.e. if a sequence starts at -15 put the first guide line between -10 and -11
    • since the index 0 is missing, add a guide line between +1 and -1 if both are present in the sequence
  • add sequence indices at the guiding lines

    • add the index beside "+1" or every 10th nucleotide wrt its final index
  • add hover information to each probability rectangle showing the respective index (combination), probability type and value

  • if the base pair probabilities are seed-restricted predictions, add dark orange rectangle boundaries to all probability cells that are part of a seed.

    • add a respective information to the legend if this annotation is added

(step 9) recursion depiction

  • add recursion depictions for ensemble based predictors and base pair probability computation in doc/recursions
  • no linking needed, the SVGs are for internal documentation only

@martin-raden martin-raden left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Alexander-Mitrofanov

thanks, please revise:

  • do a "git update" to get the recent changes to this branch
  • implement the following issues

Comment thread src/IntaRNA/StackSeedDomain.h Outdated
//! admitted occurrences keyed by local start
std::map<std::pair<size_t,size_t>, size_t> lengths;
//! shortest retained length per ending vertex for suffix acceptance
std::map<std::pair<size_t,size_t>, size_t> endingLengths;

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • when encoding base pair information, use IntaRNA::Interaction::BasePair instead of "std::pair" or "std::array"
    • provides already a hash function
  • for sparse data storage (typical for seed starts) use a hash(map) rather than a std::map.
    • replace this using "std::unordered_map<BasePair, size_t, BasePair::Hash>"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • update AGENTS.md with respective guidelines for similar upcoming changes

Comment thread src/IntaRNA/StackSeedDomain.cpp Outdated
Comment on lines +20 to +32
void StackSeedDomain::add(Occurrence s, size_t n, size_t m, size_t w1, size_t w2) {
w1=std::min(w1,n); w2=std::min(w2,m);
if (s.length==0 || s.i>=n || s.j>=m || s.length>n-s.i || s.length>m-s.j
|| s.length>w1 || s.length>w2) return;
auto [p,inserted]=lengths.emplace(std::make_pair(s.i,s.j),s.length);
if (!inserted && p->second!=s.length) throw std::invalid_argument("multiple retained seed lengths at one start");
auto [end, fresh]=endingLengths.emplace(std::make_pair(s.i+s.length-1,s.j+s.length-1),s.length);
if (!fresh) end->second=std::min(end->second,s.length);
maximum=std::max(maximum,s.length);
const size_t end1=s.i+s.length, end2=s.j+s.length;
boxes.push_back({end1>w1?end1-w1:0,s.i,end2>w2?end2-w2:0,s.j});
}
void StackSeedDomain::forEachStart(const std::function<void(size_t,size_t)> & visit) const {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • generally: extend documentation level of function implementations with inline comments summarizing subsequent steps similar to remaining code, to simplify human code and sanity checks

Comment thread src/IntaRNA/StackSeedDomain.cpp Outdated
size_t i=n, j=m;
while (handler.updateToNextSeed(i,j,0,n-1,0,m-1)) {
const size_t len=handler.getSeedLength1(i,j);
if (len != handler.getSeedLength2(i,j)) throw std::logic_error("stack seed has unequal strand lengths");

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • check internal invariants like this with assert or NDEBUG precompiler checks that are not checked in release code
  • only to release-code checks for input validation and checks that might fail during normal execution
  • extend the AGENTS.md with respective guidelines

Alexander-Mitrofanov and others added 4 commits October 13, 2026 12:35
Use Interaction::BasePair and hash maps for sparse seed lookup, make the stack-length invariant debug-only, and document probability computation steps and contributor conventions.

Validate overlapping seed endings, deterministic traversal, and input rejection. Release and debug make tests -j2 each pass all seven suites (100967 assertions in 98 API cases); installed headers, pkg-config consumer, assertion-mode probe, and before/after CLI output checks also pass.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants