Add lane-order invariance check to validation - #130
Merged
cnicholas merged 1 commit intoSep 19, 2026
Merged
Conversation
cnicholas
added a commit
that referenced
this pull request
Sep 19, 2026
…et they run on README.txt now lists short_series_sampling.py and short_series_bands.py (#119) and mr_permutation_invariance.py (#130), all contributed by the #114 reporter, and the synthetic 24-organisation x 4-year ACO file built with Tom Bishop for that issue, so the permutation check runs out of the box from the repo root.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the question raised in #114. Adds
validation/mr_permutation_invariance.py,a dependency-free script (csv + numpy) that holds every observation fixed and
varies only the lane order of the combined chart, then reports which chart
results move and which do not.
Why
With
by=[]the moving range runs across the whole concatenated sequence, so thestep from one organisation to the next contributes a range like any other. On the
24-ACO file that is 23 of 95 ranges. The script asks what depends on which
organisation happens to be adjacent to which.
What it reports, on
aco_per_capita_expenditure.csvTwo results, and they point in opposite directions:
orderings the X chart never returned zero signals. "These 24 are not one
process" survives every ordering tested.
shipped order it is 3, and ordering lanes by level gives none. The flagged
pairs differ entirely between orderings.
The script also records, descriptively, that a boundary moving range changes
organisation and resets time in one number — organisation i's last
observation against organisation j's first. On this rising series the mean
boundary range is 1.66x the mean adjacent pure-level gap. That is a property of
the lane-major sequence on trending data, not of the two organisations, and the
script says so rather than attributing a share to the time step.
Confirmed against the library
Per #114,
companion=Trueflags exactly the three transitions the script reportsfor the shipped order, at the first-year points of ACO-019, ACO-021 and ACO-024,
plus the same four X points. The script uses the classical
D4 = 3.267, whichmatches the library's mR limit.
What it does not test
combined chart exists to read.
organisation-level claims belong: the recentred R5
Xbarby ACO, and thephased view. ⚠ Note that R5 is not wholly free of traversal — its points are
order-free organisation means, but in ADS 2 its limits are set from R2, which
is built from the same lane-major sequence. See Series length (T) as an axis in design-state admissibility — and a saturation question #114. This script covers only
the combined chart.
Runtime
About one second. No dependencies beyond numpy.