Subset cells and genes while unpacking IVCSR/IVCSC

[aarmey]

Implements #67 except for the bounded-memory slab reading (see below).

API

vsparse.load_packed(path, obs_idx=mask, var_idx=positions)

This returns exactly load_packed(path)[obs_idx, var_idx], with X, every metadata field and the uns["vsparse"] stale flag all matching. The tests check that X's four arrays (major_ptr, values, value_ptr, indices) are bit-identical. Each selection can be:

• a 1-D boolean mask (a pandas Series such as obs["tissue"] == "blood" works directly);
• an array or list of integer positions, where negative positions count from the end and unsorted or repeated positions come back in that order;
• a slice, which converts to positions for free.

Labels, floats and 2-D input raise TypeError; out-of-range positions and wrong-length masks raise IndexError. To select by label, convert first with obs.index.get_indexer(labels).

How it works (_ivcsc.unpack_selected)

1. The whole packed stream is still read, because nothing records where a slice's bytes start.
2. A parallel count of varint-terminator bytes splits the stream into spans, each starting on a varint boundary.
3. Each span walks forward to the selected slices that start inside it. Whole blocks are skipped with the same vectorized count.
4. Only the selected slices are decoded, in parallel. major_ptr/value_ptr for the subset come from the stored pointers without decoding anything.
5. A minor-axis selection renumbers indices through a lookup table during that same decode, dropping indices that map to -1 and any groups left empty. This needs a counting pass and a fill pass, both over the selected slices only.
6. Unsorted or repeated selections decode the sorted unique set, then reorder in memory with the existing _select_major/_select_minor.

For IVCSC the axes swap: a gene selection is the fast path, and a cell selection is a filter applied during a full decode.

Side effect: varint counting is now branch-free, one add per byte. That also speeds up the existing parallel full decode, from about 14 to 11 ms on a 40 MB synthetic stream.

VCSCAnnData.__getitem__'s metadata subsetting moved into _subset_fields with no change in behavior, so load_packed uses the same code path.

Measurements

lupus.ivcsr.h5ad re-saved with zstd (1.26M cells, 48 threads, warm cache), X only:

selection	time	vs. full X (0.50 s)
random 0.1% of cells	0.20 s	0.40
random 1%	0.21 s	0.42
random 10%	0.27 s	0.53
one batch_cov (4.5%)	0.24 s	0.47
1% of cells + 10% of genes	0.21 s	0.42

The 1% result matches #67's ~0.25 s estimate. What remains is reading the four arrays from disk.

Whole load_packed calls on this file take about 3 s either way. Reading obs/var through anndata costs about 2.7 s, far more than X does, so that is the next thing to attack if load time matters.

The new subset_load_vs_full_load case in --set slow covers 0.1/1/10% random selections, a clustered 1%, and cells plus genes on a zstd file. It's in the slow set because the ratios depend on disk and thread count.

Not in this PR

• Bounded memory (step 4 of the plan). Peak memory is still the whole packed stream plus the output; the full decoded matrix is never allocated. Reading the stream in slabs of zstd chunks with the scan state carried across slab boundaries needs a slab-wise version of _compression.read_array. It's a separate change and matters at Parse-10M scale.
• The load_and_normalize acceptance item. That function and _decode_selected_rows were removed in Remove load_and_normalize; load, subset, then normalize #66, so it no longer applies.

Checks

• pytest: 1652 passed. tests/test_load_subset.py adds 64 tests. They cover IVCSR and IVCSC, a single span and many forced spans, multi-byte varints, empty rows at both ends and in the middle, empty/full/unsorted/repeated/negative/slice selections, pandas masks, the error cases, and direct unpack_selected against a full decode.
• ruff, codespell, ty: clean. Fast benchmarks: no regressions.
• xenon: no new findings. vulture: nothing new. jscpd: one 6-line decode loop is shared between _count_kept and _fill_selected, which is under the 10-line extraction threshold.

Unrelated bug found along the way, not fixed here: VCSCAnnData(X=arr) without obs/var comes out with shape (0, 0).

https://claude.ai/code/session_01QBeTZy6BKKGKGJ7fwvkWgM