Skip to content

Preserve binary-vector index compatibility - #2633

Draft
nvzm123 wants to merge 3 commits into
NVIDIA:mainfrom
nvzm123:zackm_cuvs_issue_2617
Draft

nvzm123 wants to merge 3 commits into
NVIDIA:mainfrom
nvzm123:zackm_cuvs_issue_2617

Conversation

@nvzm123

@nvzm123 nvzm123 commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

Closes #2617.

Depends on #2627; this draft should merge after that prerequisite.

Marks new binary-quantized segments with an NVIDIA-owned layout attribute, writes their flat vectors with Lucene102 binary storage, and keeps released unmarked segments readable through the legacy Lucene99 path. Marker/file mismatches fail as corruption rather than guessing, and mixed legacy/current merges rewrite to the marked layout.

Builds accelerated graphs from the original float vectors before the binary delegate flush, including the three-layer path, while retaining CPU fallback. The existing padded CAGRA serialization behavior from #2483 is unchanged.

Adds a checksum-pinned index fixture produced by the released 26.08 artifact plus GPU/CPU, compound/non-compound, mixed-merge, corruption, aligned/unaligned, and three-layer coverage. Search checks include rank-one identity, unique hits, and tie-aware brute-force recall floors.

Testing

  • mvn spotless:check verify (362 passed, 30 skipped)
  • focused compatibility/lazy suite (20 passed)
  • three-layer GPU case repeated 3 times (all passed)
  • released 26.08 reader against new GPU and CPU indexes, compound and non-compound, at k=10 (4/4 passed)
  • API generation repeated twice with identical output hashes

The full GPU suite still emits existing cuVS graph-clamping, dataset-dimension, GDS-fallback, and JVM native-access/incubator warnings. The new three-layer compatibility case does not emit graph-parameter clamping warnings.

@copy-pr-bot

copy-pr-bot Bot commented Sep 16, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Lucene: preserve released binary-vector indexes while enabling binary storage

1 participant