[superme1one]

Summary

Use normal runtime operator submission for fused causal Softmax on eligible Iluvatar BF16 Prefill inputs. INFINILM_ILUVATAR_FUSED_CAUSAL_PREFILL=1 opts in; the default and all ineligible paths retain Fill/Triu/Tril/Add plus Softmax. Install the companion InfiniOps FP32 causal provider before enabling the flag.

Motivation

The private fusion removed mask allocations and four separate masking launches, reducing two-task completion time by 7.633–7.646% on top of Gate/Up+Down GEMV. This PR replaces its direct experimental CUDA entry point with normal execution::submit, preserving owner checks, stream/recording policy and runtime operator mapping. It adds no dataset or generation-policy changes.

Type of Change

• perf — opt-in Prefill fusion.
• Breaking change.

Test Results of Involved Models on Supported Platforms

Check	Current submission result
C++17 host flag/shape/fallback test	Passed, MSVC /W4 /WX, exit0
Official scripts/format.py --ref HEAD^ --check	Passed
Whitespace / structured evidence / public-file scan	Passed
Current-base Iluvatar build + two-task integration	Pending; draft PR
Other device behavior	Flag/device gate leaves existing path intact; full runtime build not rerun in this submission pass

c++ -std=c++17 -I. test/runtime/causal_prefill_policy_test.cpp -o /tmp/causal-prefill-policy
/tmp/causal-prefill-policy

Historical model evidence is the recorded TG-V200/CoreX GQA BF16 private build. No current-revision screenshot/build result is claimed. This submission pass kept the GPU experiment paused; hardware validation is needed before requesting readiness. The existing 100-question run is reused, with a two-task integration regression sufficient for the code-port check.

Benchmark / Performance Impact

Historical four-optimization TG-V200/CoreX result: 100 fixed new GSM8K targets, max_tokens=512, greedy EOS=[2,73440], unchanged strict last-#### numeric scoring. Official accuracy 82/100 → 84/100; all requests completed/parsed, zero errors and truncations. 34/100 token-different, including one correct→wrong (1097: 480→448, reference480) and three wrong→correct (672,1259,1203).

Sum of per-version same-card formal paired-wave makespans 749.203818 → 505.326616 s (-32.5515%); completed request throughput +48.2613%. Published output tokens 13894→14213, so token throughput +51.6653% is reported separately. Each target's two versions used the same physical card; first20 serial, remaining80 sharded over two cards. Load/warmup/final shutdown excluded; queue/text processing included. These are combined gains, not individual-PR gains or the parallel run's wall time. Eight demonstrations came from a disjoint test-corpus subset (internal paired evaluation).

Measured baseline: InfiniLM 6e7f9537173cd33bb30539ebf8d92ab6d869660d. Main later advanced to cc6d52ae04f4c9ff17229bc7fc13a9ad105aca6d with auto-tuning initialization. Current submissions use that InfiniLM main and InfiniOps 1a68b261d19f0ed6f76bbabeab7a25f479f05e07. Historical GPU evidence applies to the recorded private builds, not this final rebased revision.

Full table and regression IDs: docs/performance/iluvatar-four-optimizations.md. Machine-readable aggregate: docs/performance/iluvatar-four-optimizations-results.json.

Notes for Reviewers

The companion provider is required: older InfiniOps versions have the same compatibility operator but sum BF16-rounded exponents. This version matches the existing InfiniLM FP32 Softmax order. The two GEMVs share one InfiniOps PR; Prefill spans two repositories; Decode is independent. All four submissions are separate from historical R5 PRs #601/#1010.

Related optimization PRs

• perf(iluvatar): specialize BF16 Gate-Up and Down GEMV
• perf(iluvatar): fuse causal Softmax with FP32 accumulation
• perf(iluvatar): batch Decode Attention QK reductions
• perf(attention): add opt-in Iluvatar fused causal Prefill

CI / ChatOps

Current-base GPU build/smoke remains pending. No GPU workflow or benchmark was dispatched during this PR-preparation task.

Checklist

Title, Branch, and Commits

• Conventional Commit title, branch prefix and commit message.
• One focused commit on current main; no stray merges/fixups.
• N/A: legacy issue-format branch.

Scope and Design

• Minimal operator integration with default-off behavior and preserved fallback.
• No debug counters, dead code, unrelated formatting or public signature changes.

General Code Hygiene

• English comments; normal formatting; one LF terminator; no trailing whitespace/BOMs.
• No unrelated files, model weights, internal URLs or credentials.

C++ Specific

• Repository formatter passed; no raw allocation; original runtime submission semantics retained.
• No legacy-model references or constructor-order changes.

Python Specific

• N/A: no Python production changes.

Testing

• Host flag and eligibility boundaries executed.
• Affected-platform fresh build and smoke tests on final revision.
• Same two-task application regression on final revision.
• N/A: a larger dataset rerun; the existing 100-target evidence is retained with its exact baseline.

Build, CI, and Tooling

• Current-revision Iluvatar build and runtime CI.
• Formatting and diff checks.

Documentation

• Flag, companion dependency, numerical contract, historical baseline and complete quality tradeoffs documented.

Security and Safety

• Public-only source/results; no private paths, secrets, weights or hardware identifiers.
• Existing project kernel lineage; no new third-party dependency.
• Narrow shape/device gates; kernel bounds remain in the companion provider.