Repository navigation
feat: add the main GPU normalization schemes and a random choice between them - #135
Open
Hendrik-code wants to merge 2 commits into
Open
Hendrik-code wants to merge 2 commits into
Hendrik-code wants to merge 2 commits into
Conversation
…een them ZscoreNormalizationGPU was the only normalization a GPU config could reach. The other schemes a medical imaging pipeline uses -- min-max, the CT-style percentile clip, a robust median/IQR scaling, divide-by-max, divide-by-mean -- existed only in nnU-Net's preprocessing, which runs once per dataset. All five land in a new gpu/normalization.py behind a shared base that owns the clone, the channel guard, the statistics mask, both degeneracy guards and the region selection, so each scheme is an affine map plus an optional clip. At their defaults MinMaxNormalizationGPU reproduces nnU-Net's RescaleTo01Normalization and PercentileNormalizationGPU, given a clip_range, mean and std, reproduces CTNormalization; test_normalization.py pins both against the installed nnU-Net. The group is new: AugType.NORM, because its members are alternatives rather than additions -- a min-max rescale on top of a z-score discards the z-score. The builder therefore puts them in a RandomChooseXTransformsGPU with num_transforms=1 as the last stage in every pipeline mode, which makes each scheme's p its weight (P(scheme) = p / n) and leaves the remainder as no normalization at all. A config naming a single normalization is untouched, so no shipped pipeline changes shape. ZscoreNormalizationGPU moves from GE to NORM, which is what puts it in the same choice as the rest. In random_order mode that takes it out of the shuffled GE bucket and makes it a trailing stage; every shipped config sets its p to 0.0, so nothing fires differently, but the GE bucket no longer spends a draw on it and random_order runs will not reproduce earlier volumes bit for bit under one seed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
6 tasks done
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this change?
Adds the five normalization schemes the GPU side was missing, and makes the choice
between them an augmentation.
New in
smauglab/transforms/gpu/normalization.py:MinMaxNormalizationGPUout_range, optionally after a percentile clipRescaleTo01Normalizationat defaultsPercentileNormalizationGPUclip_range, then standardizeCTNormalizationgivenclip_range,mean,stdRobustZscoreNormalizationGPUMaxScaleNormalizationGPUmax|x|, or by a high percentile of itMeanScaleNormalizationGPUAll five share a base class that owns the clone, the channel guard, the statistics mask
(
stats_from: all voxels, the labelled anatomy, or everything preprocessing did notzero), both degeneracy guards and the
in_seg/out_segregion selection, so each schemeis an affine map plus an optional clip.
The choice reuses
RandomChooseXTransformsGPUrather than adding a combinator. A newAugType.NORMgroup collects the normalizations, and the builder emits them as onenum_transforms=1bucket at the end of the pipeline in every pipeline mode. Becausethat bucket draws one child uniformly and then spends that child's own
p, each scheme'spbecomes its weight (P(scheme) = p / n) and the remainder is "no normalization atall" — so there is no no-op transform to register.
pipeline.random_choose.norm_probabilitygates the stage.
Why?
Every scheme other than z-score lived only in nnU-Net's preprocessing, which runs once per
dataset. Leaving the intensity standardization fixed is one more thing a model can come to
depend on, and it is the step that differs most between sites and pipelines, so it is worth
randomizing for the same reason the contrast and gamma augmentations here are randomized.
How was it tested?
pytestpasses locallypre-commit run --all-filespassesunit_tests/test_normalization.py(33 tests) covers each scheme's maths, exact parityagainst the installed nnU-Net's
RescaleTo01Normalization,CTNormalizationandZScoreNormalization, both guards, the masked statistics, the builder's staging in allthree pipeline modes, and the weight semantics as a 400-draw statistical test. The new
classes also join the registry-driven suites (
test_transforms_gpu.py,test_container_fast_path.py,test_no_inplace_mutation.py,test_channel_guard.py).Full suite,
nnunetv22.4.2 env: 9 failed, 501 passed — the same 9test_trainers.pysubtests that fail on
origin/mainin that env (order_resampling_data/border_val_seg,the documented
>=2.5pin). Verified against a cleanorigin/mainworktree: 9 failed, 468passed. Also ran the normalization, builder, channel-guard, transform and registry tests
under
nnunetv22.6.2; the only failures there are the two pre-existingbatchgeneratorsv2SpatialTransformones, confirmed identical on cleanmain.End to end on one GPU: 50 batches of
(2, 1, 64, 64, 64)through a pipeline whosenormalization is a six-way choice, shapes, finiteness and label integrity held, and
record_applicationsshows all six schemes firing in proportion to theirp.Anything reviewers should look at closely?
min_scaleis measured against the standard deviation, not the intensity range.The range was the first thing tried and is wrong for the robust schemes: one 1e4 voxel
puts the range four orders of magnitude above the IQR, so a range-relative floor
declined to run on exactly the volume
RobustZscoreNormalizationGPUexists for.pipeline.same_on_batchdefaults to True and the container forces it onto every stage it holds, this bucket
included — the same is already true of the
taandgebuckets.pipeline.same_on_batch: falsegives each sample its own scheme. Pinned inTestOneDrawPerBatchOrPerSample.normalized by preprocessing and inference re-applies only that, so a config turning this
on is deliberately training scale invariance rather than matching inference. There is no
compensating re-normalization at the end; that was a conscious choice, and it is written
up in the README section.
PercentileNormalizationGPUtakesmean/std/clip_rangeas parameters becausennU-Net reads them from the dataset fingerprint and nothing hands a transform that at
training time. A config that knows its dataset states them; left as None they are
measured per sample.
Checklist
Augmentation behaviour is unchanged, or the change is intentional and described above
One intentional change:
ZscoreNormalizationGPUand the CPUZscoreNormalizationmovefrom
AugType.GEtoAugType.NORM, which is what puts z-score into the same choice asthe new schemes. In
random_ordermode that takes it out of the shuffled GE bucket andmakes it a trailing stage. Every shipped config sets its
pto0.0, so nothing firesdifferently — but the GE bucket no longer spends a uniform draw on it, so
random_orderruns will not reproduce earlier volumes bit for bit under the same seed. A config naming
a single normalization is otherwise left exactly where it was, including under
pipeline.order: config.New transforms are reachable from a config JSON
smauglab/configs/all_augmentations.jsonand the README matrix are regenerated withsmauglab matrix --write.🤖 Generated with Claude Code