Skip to content

Perf/single operand collector - #479

Open
jel221 wants to merge 2 commits into
masterfrom
perf/single-operand-collector
Open

jel221 wants to merge 2 commits into
masterfrom
perf/single-operand-collector

Conversation

@jel221

@jel221 jel221 commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

Human comment: Operand collectors take up a huge amount of logic on the FPGA and area on ASIC at NT=32. But

  1. we only read at most 3 registers (so the operand "collector" doesn't really "collect" many things, unlike other ISAs that name lots of registers),
  2. and register bank conflicts are rare
  3. and backpressure at dispatch is also pretty rare because the scoreboard counts VX_pending_size for each functional unit. So we can reduce this number to 1 per ISSUE_WIDTH.

I also have an LLVM PR if you want to ensure register bank conflcits are rare by design.
vortexgpgpu/llvm#4

The AI-generated body section has evidence of these regression cases not causing any notable performance issue

AI-generated comment:

Summary

One operand collector per issue slice by default (VX_CFG_NUM_OPCS = 1; it was up(NUM_WARPS / (4 * ISSUE_WIDTH)), one per 4 warps of a slice, so 4 per slice at 64 warps).

The scoreboard already issues only when the target unit's dispatch queue has room (per-FU credits, VX_scoreboard.sv), so a collector never waits on a busy unit. The only thing extra collectors hide is a register bank conflict, and those stall 0–9% of collector cycles on the kernels below. Each collector holds three full-width operands and its own 4 register-file banks, so the extra collectors cost a lot of area for no cycles.

No RTL or SimX code changes: both already take any collector count (warp wis % NUM_OPCS, sim/simx/operands.cpp). Any config can still ask for more with -DVX_CFG_NUM_OPCS=N.

Cycles

rtlsim, XLEN 32, 1 core, 8 SIMT kernels, v3.02 toolchain. Retired instructions are identical in every pair.

64 warps, 4 issue slices, 32 threads (16 collectors → 4):

Kernel 4 per slice (master) 1 per slice Change
vecadd 62,195 62,445 +0.4%
relu 41,733 41,569 −0.4%
dotproduct 233,211 223,525 −4.2%
sgemv 188,066 185,430 −1.4%
conv3 16,203 15,962 −1.5%
stencil3d 431,289 396,110 −8.2%
jacobi 688,095 670,208 −2.6%
bfs 47,350 46,102 −2.6%
geomean −2.6%

With the previous toolchain the same sweep is −2.1% geomean, worst +0.8% (sgemv). 2 per slice matches 4 per slice to the cycle on every kernel with both toolchains.

64 warps, 1 issue slice, 8 threads (4 collectors → 1):

Kernel 4 collectors 1 collector Change
vecadd 279,031 270,874 −2.9%
relu 304,117 295,131 −3.0%
dotproduct 1,533,402 1,378,860 −10.1%
sgemv 624,769 611,135 −2.2%
conv3 52,663 52,282 −0.7%
stencil3d 1,599,800 1,509,506 −5.6%
jacobi 2,469,080 2,212,076 −10.4%
bfs 27,650 25,498 −7.8%
geomean −5.4%

dotproduct reports FAIL at 2048 threads under both settings: a float summation-order mismatch in the check (16339.037 vs 16339.059), not a timing effect. Its cycles are valid.

Why fewer collectors is faster is not root-caused; the likely cause is the changed issue order of instructions and memory requests. Nothing measured shows a slowdown that would argue for more than one per slice.

Area

Beacon FPGA core build (16 warps, 16 threads; 4 collectors → 1):

Before After Change
LUT 143,741 126,361 −12.1%
FF 163,874 142,023 −13.3%
BRAM 129 39 −70%
Logic levels 15 15 0
Predicted Fmax (advisory) 285.6 MHz 283.6 MHz −0.7%

Beacon ASIC core build (ASAP7, 8 warps, 8 threads; 2 collectors → 1):

Before After Change
Cells 769,936 748,197 −2.8%
FF 102,791 96,581 −6.0%
SRAM macros 16 12 −25%
Predicted Fmax 1052 MHz 1052 MHz 0

The other 13 builds of each Beacon flow pass unchanged.

Impact and evidence

See docs/review/code-review-policy.md.

Impact class: R3 (ci/change_impact.py: config-knobs, golden-values). No review agent pass has been run on this change.

Contracts touched:

  • config-knobs: pipeline.VX_CFG_NUM_OPCS default up(NUM_WARPS / (4 * ISSUE_WIDTH)) → 1. No profile or CI case sets the knob, and no doc names its default (docs/designs/mlp_optimizations.md describes the wis % NUM_OPCS mapping, which is unchanged). The cases that show the moved default are the perf_gate rows below.
  • golden-values: perf baselines (8 files), ci/baselines/synthesis/beacon/core.json and ci/baselines/synthesis/beacon_asic/core.json, re-recorded with the commands named in the commit messages:
    • pytest ci -m perf_gate --update-baselines
    • ci/logic_gate.py -b core --update-baseline and ci/logic_gate.py --tool beacon_asic -b core --update-baseline (Beacon fails on an improvement beyond its 5% band until re-recorded)

Perf baseline deltas. Only 4 of 49 cases moved cycles, all multi-core; the rest changed config_hash only.

Case Before After Change
core:sgemm-mc-nt16 479,787 474,193 −1.2%
core:sgemm-mc-nt4 1,679,928 1,680,553 +0.04%
tensor:fp16-mc-nt16 36,319 35,638 −1.9%
tensor:fp16-mc-nt4 219,890 218,111 −0.8%

The one value that moved toward a looser bar is core:sgemm-mc-nt4 (+625 cycles, +0.04%), from this commit.

evidence status detail
functional tests (simx / rtlsim / xrt) run rtlsim: the 8 kernels above at both shapes and both collector counts, all PASS but dotproduct (summation order, see above); the 49 perf_gate cases. xrt not run: no AFU or device change.
model_parity run In CI on this branch (all parity cases pass); SimX already models the collector count.
perf_gate run 49/49 pass against the re-recorded baselines (deltas above).
Beacon logic_gate (FPGA and ASIC) run 14/14 pass in each flow; core improved as above and is re-recorded in both.
synthesis (fpga_gate / asic_gate) not run Nightly on master only. The core baselines will move (smaller) on the first nightly after merge and need a human --update-baseline.
XLEN 64 build not run No XLEN-dependent change; CI covers XLEN 64.

A not run is not a pass. Say why it was not run.

  • No test silenced (known_issue:, wider tolerance, smaller args, removed case) without a root-cause plan.
  • Golden data changed only through its update command, named in the commit message.
  • Cycle-moving RTL changes carry the matching SimX change, or the parity case that shows none moved.
  • A toolchain pin moves only after the prebuilt tag exists.

Notes for review:

  • The policy says a human re-records golden data. These were re-recorded by an agent running the update commands at the author's direction; the deltas are listed above so they can be checked or re-run.
  • A compiler pass that assigns registers to avoid bank conflicts (LLVM fork, feature/register-bank-hints) gains about 1% at 2 or more collectors per slice and nothing at 1. It is not needed for this change and is not part of it.

…es (pytest ci -m perf_gate --update-baselines) and lock in Beacon core area (ci/logic_gate.py -b core --update-baseline)
…/logic_gate.py --tool beacon_asic -b core --update-baseline)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant