Skip to content

fix(vela): preserve row ownership in decoder executor replay - #210

Draft
psv666 wants to merge 4 commits into
codex/vela195-4b-acceptance-basefrom
codex/vela195-4b-acceptance-20261120
Draft

psv666 wants to merge 4 commits into
codex/vela195-4b-acceptance-basefrom
codex/vela195-4b-acceptance-20261120

Conversation

@psv666

@psv666 psv666 commented Nov 20, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

The executor replay in #195 consumes reference calls in global execution order, while its single-predict recorder records actual row ownership. On the real 4B example, it compares a 23-word native row with another row's 27-word call and fails before its numerical gate. Match each call by unique, unconsumed row + kind + head, retain the legacy 0.8B path, and reject missing, duplicate or unused calls.

planned row + kind + head
  -> exactly one recorded owner
  -> original finite/shape/count assertions
  -> original 0.35 logit ceiling
  -> every call consumed once

Also restore finite FP32 tensor validation dropped by the member parameterization and correct the README's stale packed-forward/test-count claims. The recipe explains how to preserve complete reference JSON/tensor pairs.

This stacked supplement preserves the author's #195 head a8ecddd and accepted 0.8B snapshot df02f95 (accepted #176/#208/#205 plus current main f1e674f2). Its contributor-owned base is 4792f8c; the focused diff is 5 files, +147/-30, 177 total text lines, with 150 changed Rust lines / 140 nonblank, noncomment code lines. No production forward/scoring arithmetic, shared CUDA/ABI, numerical tolerance or committed expectation changes.

Required human contributor answers

The contributor explicitly confirmed the common intent and necessary-stage-draft decision on 2026-11-19, and the independent 4B stage on 2026-11-20; the traceable source remains in the contributor's task record.

  1. Outcome: Reproducible, reviewable model acceptance so supported models can be used reliably.
  2. Highest-leverage variable: Reliable model coverage and numerical consistency.
  3. How this moves it: Reuse existing implementations and close observed contract/evidence gaps with reproducible validation.
  4. User and need: Model, application and agent developers needing stable interfaces, correct outputs and reproducible deployment.

Is a PR needed? Yes. Necessary draft supplements preserving original authors' history are authorized. The observed loader and replay-ownership gaps are separate from the delivered 0.8B cap and #205 recipe/test-placement work.

Evidence

  • Before: The original 4B GPU test exits 101 on example-request: span word count, native 23 / reference 27. A weight-free regression with reordered, equal-shape calls also fails against the old selection. The inherited loader checks at base 4792f8c report 2 passed / 1 failed because NaN is accepted.
  • After: Six executor ownership/finite/shape checks pass, including legacy 0.8B order and missing/duplicate/wrong-head rejection. All three loader checks pass. The restored GPU campaign passes all four cases at the unchanged 0.35 ceiling: 0.1602 / 0.0211 / 0.0738 / 0.0254. The failed example was retried once; the other three cases executed for their first time. Original failures remain in the archive.

Fresh 4B reference JSON and hidden tensors were frozen together before native comparison, from one real _predict per case, including inputs/logits/calibrated answers/execution batches. Same-run SystemOne finishing consumes the captured triple without another forward. Original product fixtures stay unchanged; an isolated replay view holds the current pairs. Current reference long-window output has 82 spans versus the historical committed 76. That reference-run difference is reported separately; historical exact replay is unrun because its original paired tensors/hash assets were unavailable.

Personally run on L20 Result
Real reference HTTP test-client vs Python worker pipeline 9/9 exact status, body and key order
Token plans / real-bundle processing 5/5 exact; 2/2 processing tests
Current paired native heads 5/5, original 1e-4 / 8e-3 gates; max logit 3.624e-5, answer 1.162e-6
Positive long-window CPU finishing All 82 spans match count, text, labels and Unicode offsets
Native response + four raw CPU response captures Original 1e-2 gate and strict structures; max 6.024080344557703e-7; usage independently summed from plans
Real 4B final-row budget 16,383 / 16,384 admitted; next row whole-request 413 / too_long
Final Rust workspace 503 passed, 126 ignored; fmt, clippy and release build pass
Vela Python / strict documentation 168 passed / strict build passes

Test Plan

Head: 77b31fc2863c41713002c083140f15629a930395, tree 4929dc8e84c787ad2295e206ffa7305a373a76eb. Base 4792f8c7dfe53be00cffd7a6beea67dd09e903af. The final recipe-only descendant has the same Rust source as tested f7ed41f and was rechecked with strict MkDocs.

From repository root:

cargo test --release --locked -p omni-vela-native --lib heads::tests -- --nocapture
cargo test --locked -p omni-vela-native --test vela_native_executor
cargo fmt --all --check
cargo clippy --workspace --locked --all-targets --features omni-laya/serve -- -D warnings
cargo test --workspace --locked --features omni-laya/serve
cargo build --workspace --release --locked --features omni-laya/serve
PYTHONPATH=src python -m pytest tests/vela -q
mkdocs build --strict

Demo / evidence

Pinned 4B package: d692394127f6f87fc581d8634c9558fe4776efce; actual extended bundle anchor: 948160dd331442f1cb9739793e2d78406feab20de4831f2d96fc091f87f8bef9. Present assets match fresh pinned HF Git/LFS metadata; all 426 faithful/rounded-BF16 backbone tensors, 37 FP32 heads, tokenizer/calibration/config were independently checked. Reference: Python 3.12.13, torch 2.11.0+cu128, transformers 5.19.0, tokenizers 0.23.2, numpy 2.5.3, safetensors 0.8.0; BF16-autocast backbone / FP32 heads.

Native profile: one NVIDIA L20, sm_89, driver 580.178.04, Rust 1.100.0 / CUDA 12.8.61; uniform BF16 backbone, FP32 heads, individual fixed-M64 readouts, no Graph or other opt-ins. Library SHA256 63f3f5199420a22dec2976899939e6a749ef998202bad26fcf8928545b95235b was reused only after actual source/symbol/ABI/target/profile match to accepted #177. Process maps, running-binary hashes and GPU UUID sampling bind the executed profile.

Raw commands/exits, original failures, frozen pairs, old-vs-new deltas, probes and source/binary/package/library hashes are retained off-repository in vela195-4b-evidence.tar.gz, SHA256 ee5602ddf1aeb39864720c81740620554ebe5ac7bc357e8fb8a8a557ad7dd4d7. The archive contains 256 files; the contributor retains the artifact. No generated run dump or new binary fixture is added by this focused supplement. The integration base retains 15 legacy unlabeled 0.8B replay inputs from accepted #177 alongside #195's labeled inputs; all 51 original #195 fixture blobs are unchanged. Current member tests use labeled paths. Final stack integration should review legacy deduplication while preserving #177 reproducibility.

These GPU results qualify small-case logits. CPU native responses use frozen GPU-reference hidden inputs, not native GPU-backbone outputs. Complete GPU typed answers, native HTTP/readiness/frontend/lifecycle, native positive long-window GPU backbone, 9B and other hardware remain unrun. Video/performance A/B: N/A for this loader/test/docs supplement; no latency or throughput claim.

Merge Danger

Door: two-way.

Blast Radius: validation.

Valid finite-input arithmetic is unchanged. Non-finite head tensors now fail at load, and executor parity checks compare the correct recorded owners. Retarget/revalidate after the underlying stack lands; this narrow profile does not establish family-wide readiness.

Self-review

The focused diff, architecture boundaries, failure/pass evidence and artifact hygiene were checked. No remaining blocking finding in this supplement. The inherited branch-to-main stack still exceeds 3,000 authored-code lines and requires full contributor/maintainer review. Keep draft; no reviewer request or merge is implied. Main-only CI filters may not run against this stacked base; the results above are personally executed checks.

  • I have reviewed the full diff and addressed the issues I found.
  • I have checked that the change follows the architecture and stays focused.
  • I have run the appropriate checks and reported results and unverified checks.
  • I have checked that the description and claims match implementation and evidence.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant