Skip to content

About

Requant Network research: ternary (BitNet b1.58-style) weight GEMM on SM120 Blackwell tensor cores — exact NVFP4/FP8/INT8 paths, throughput/accuracy trade-off

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

2 Commits

Folders and files

Repository files navigation

gpu-ternary-tensorcore

Ternary (BitNet b1.58-style) weight GEMM on SM120 (Blackwell) tensor cores.

Status: COMPLETE — bounded spike, all experiments run on the me4me RTX 5060 Ti (GPU 0, SM120, cc 12.0), CUDA 13.3, driver 610.43.03. Every headline path is a NATIVE SM120 tensor-core route (no CPU/emulated fallback). All exactness claims verified entry-by-entry against an independent pure-Python integer oracle.

Main result

Ternary {-1,0,+1} weights can use SM120 tensor cores losslessly, and the fastest exact route is NVFP4 (FP4 E2M1), not INT8:

  • Exactness: INT8 IMMA, NVFP4 E2M1, FP8 E4M3, and a custom SIMT 2-bit-packed kernel all reproduce the integer oracle exactly on every output entry — 8,912,896 exact comparisons across 2 shapes × 2 seeds × 4 paths, 0 mismatches.
  • Throughput (4096³, resident, median of 60 launches): NVFP4 163.7 TMAC/s = 2.32× INT8 IMMA (70.5), 1.81× FP8 (90.2), 6.79× FP16 (24.1), 242× the SIMT 2-bit reference (0.68).
  • Accuracy trade-off (activation quantization, weights stay exact): relative L2 error ≈ 1.1% (int8), 2.6% (FP8 E4M3), 15.3% (FP4 E2M1). FP4 buys throughput at a real activation-precision cost.
  • Negative result: MXFP4 (VEC32_UE8M0) is unsupported on SM120 (cuBLASLt heuristic INVALID_VALUE); the native FP4 route is NVFP4 (VEC16_UE4M3). Ternary weights give no INT8 tensor-core speedup over dense INT8 (1.008×) — the ternary win is storage/bandwidth, not INT8 compute.

What has actually run

Experiment Where Count Outcome
cuBLASLt narrow-precision support probes server, GPU 0 5 probe programs NVFP4/FP8/INT8 supported & exact; MXFP4 unsupported
Correctness vs integer oracle server, GPU 0 2 shapes (256³,1024³) × 2 seeds × 4 paths EXACT, 0/8,912,896 mismatches
Negative controls server, GPU 0 wrong FP4 scale + overflow-K guard wrong scale diverges (63070/65536); overflow rejected pre-launch
Activation-quantization error server, GPU 0 2 shapes × 2 seeds × 3 quant paths int8 1.1% / FP8 2.6% / FP4 15.3% rel L2
Resident timing server, GPU 0 1024³,2048³,4096³ × 6 routes × 60 launches FP4 fastest; tables in RESULTS.md
Transfer-inclusive offload server, GPU 0 1024³ × 60 launches 1.077 ms (57× slower than resident)
CPU oracle self-test + unit tests local & server, stdlib 9 tests PASS

Scope limitations (explicit)

  • Bounded spike, not a full BitNet inference study: GEMM only, no end-to-end model, no training, no non-linear layers.
  • Timing is resident repeated launches (operands reused on-device) — no streaming claim. A separate transfer-inclusive offload boundary is reported for one shape.
  • Exactness is for ternary weights; activation quantization (int8/FP8/FP4) is approximate and its error is measured, not hidden.
  • FP4 exactness bound requires 6·K ≤ 2^23 (FP32 accumulate); INT8/SIMT require K·127 < 2^31. Both checked before launch; study shapes are far inside.
  • Single GPU (RTX 5060 Ti 16 GB), single CUDA/driver version. No multi-GPU, no other architectures.
  • NVFP4 scale-tensor internal swizzle is characterized only for uniform scales (sufficient for ternary weights at scale 1.0); non-uniform per-block scales were not validated.

Related work (cited, NOT re-run)

  • requant-network/gpu-fp4-research — FP4/FP8 GEMM accuracy.
  • requant-network/gpu-exact-tensorcore — exact modular FP4/INT8 tensor-core arithmetic (balanced base-13 encoding). This study is different: ternary weights, not modular fields.

Layout

cuda/ternary_bench.cu     native SM120 driver (INT8 IMMA, NVFP4, FP8, SIMT 2-bit, FP16/INT8 baselines)
oracle.py                 independent stdlib integer oracle + data generator + verifier + quantization
run.py                    server orchestrator (gen + GPU runs + verify + finalize)
tests/test_oracle.py      CPU-only stdlib test suite (9 tests)
experiments/              cuBLASLt support probes + PROBE_FINDINGS.md
results/*.json            measured results (UTC timestamp, GPU/CUDA/driver, source SHA-256)
METHODS.md                mathematical contract + timing boundaries
RESULTS.md                tables of measured numbers
FINAL_REPORT.md           conclusions, negative results, next decisions

Reproduce

# CPU oracle self-test (stdlib python3, no GPU):
python3 oracle.py selftest
python3 tests/test_oracle.py

# On the server (build, then full pipeline):
/usr/local/cuda/bin/nvcc -std=c++17 -O3 -arch=sm_120 cuda/ternary_bench.cu -o ternary_bench -lcublasLt -lcublas
python3 run.py <driver_version>     # e.g. 610.43.03

Every GPU run is pinned to CUDA_VISIBLE_DEVICES=0 and serialized under flock /tmp/requant-new-studies-gpu.lock. See METHODS.md for the exactness contract.

About

Requant Network research: ternary (BitNet b1.58-style) weight GEMM on SM120 Blackwell tensor cores — exact NVFP4/FP8/INT8 paths, throughput/accuracy trade-off

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages