[superme1one]

Summary

Batch the eight Decode Attention QK reduction synchronizations on Iluvatar, reducing 48 barriers to 6 while preserving the original FP32 addition tree. CTA64, tile8, split4, online Softmax, workspace and combine remain unchanged.

Current CoreX build and GPU verification

CoreX current-code validation completed on 2026-11-06 against InfiniLM
12a4492f70dd33235e3d142e333ad4a9629d3b7b and InfiniOps
1a68b261d19f0ed6f76bbabeab7a25f479f05e07. Both complete SDKs compiled
and imported. Both roles used the identical documented CoreX validation adapter
and ABI0, with automatic tuning disabled. The adapter includes existing CUDA
Argmax/Triu/Tril registrations, unchanged rejecting unavailable factories, a
copy-on-write read-only-view loader, and the existing paged-only FlashVarlen
provider's metadata gate. See the companion InfiniLM
docs/performance/corex-validation/README.md for exact scope.

The affected native GPU tests passed without skips: GEMV 4, causal Softmax 8,
Decode Attention 8, plus 7 common Argmax/mask checks. The submitted device tests
now explicitly bind implementation index 0. GEMV fallback uses the supported
no-C, alpha=0.5 contract; this backend rejects C/bias at its base operator.

Original tasks 554/172 ran on one TG-V200 in forward and reverse role order.
Each of the four model processes used one simultaneous two-request warmup wave
and one simultaneous two-request measured wave: 16 requests total, 8 measured.
All 16 answers were correct with natural EOS, zero errors/truncations, normal
scope return, final library maps, atexit and actual OS exit0. The original
512-token greedy budget, EOS=[2,73440], strict last-#### numeric scorer, inputs
and weights were unchanged. Two physical rows, full Prefill and graph off were
held fixed. No new independent quality questions were added.

Mean two-request measured-wave time was 14.976775177 ->
9.782192262 seconds (-34.6843%).
Completed request throughput was 0.133540096 -> 0.204453148
requests/s (+53.1024%). Published token throughput
was 20.631944884 -> 31.588011330 tokens/s and is reported separately.
The time denominator includes queue and text processing and excludes loading,
warmup and final shutdown. Both role orders reduced application time; this is
the combined four-optimization gain, not an individual-PR or component sum.

Order	Official wave, s	Combined wave, s	Time reduction
forward	14.973374084	9.782256734	34.6690%
reverse	14.980176271	9.782127790	34.6995%

Token-different task IDs across the two measured orders: none. Answers and token differences are recorded separately.
Raw clocks, output IDs, strict scores, native receipts and ordinary exits were independently recomputed. This two-question integration check is kept separate from the historical 100-question experiment below.

Reproduction and numerical contract: docs/performance/iluvatar-decode-qk-reduction.md. Current machine-readable combined evidence and the common adapter are in companion InfiniLM #610. Final publication commits add documentation only to the tested production code; native fixture commits are recorded in the machine-readable evidence.

Historical 100-question quality and performance evidence

Historical four-optimization TG-V200/CoreX result: 100 fixed new GSM8K targets, max_tokens=512, greedy EOS=[2,73440], unchanged strict last-#### numeric scoring. Official accuracy 82/100 → 84/100; all requests completed/parsed, zero errors and truncations. 34/100 token-different, including one correct→wrong (1097: 480→448, reference480) and three wrong→correct (672,1259,1203).

Sum of per-version same-card formal paired-wave makespans 749.203818 → 505.326616 s (-32.5515%); completed request throughput +48.2613%. Published output tokens 13894→14213, so token throughput +51.6653% is reported separately. Each target's two versions used the same physical card; first20 serial, remaining80 sharded over two cards. Load/warmup/final shutdown excluded; queue/text processing included. These are combined gains, not individual-PR gains or the parallel run's wall time. Eight demonstrations came from a disjoint test-corpus subset (internal paired evaluation).

Historical official InfiniLM baseline: 6e7f9537173cd33bb30539ebf8d92ab6d869660d. The current-code two-task result above is a separate experiment.

Notes for Reviewers

The shared CUDA template defaults to the original reduction; only the Iluvatar compile-time branch selects the specialization. Current CoreX compilation and GPU tests passed. Other device backends were not rebuilt in this validation.

The four PRs form the measured combination. This validation does not attribute the total gain to any one PR. The original historical individual regression remains disclosed above.

Platforms and checks

Check	Result
CoreX complete official/combined SDK build and import	Passed
Affected Iluvatar native GPU regressions	20/20 passed, no skips
Common Argmax/triangular-mask GPU checks	7/7 passed, no skips
Same-card original two-task application, forward/reverse order	Passed, all 16 requests correct
Ordinary scope/maps/atexit/OS exit	Passed
Other hardware backends	Not executed in this validation

Related optimization PRs

• perf(iluvatar): specialize BF16 Gate-Up and Down GEMV
• perf(iluvatar): fuse causal Softmax with FP32 accumulation
• perf(iluvatar): batch Decode Attention QK reductions
• perf(attention): add opt-in Iluvatar fused causal Prefill

Checklist

• Focused implementation and conservative fallback.
• Current CoreX build and native GPU checks.
• Original two-task same-card quality and application performance check.
• Original scoring, inputs and generation budget retained.
• Runtime exits and raw measurements independently verified.
• Documentation preserves current and historical evidence separately.
• Diff whitespace check passed; no credentials, weights or private server addresses in the PR.