Skip to content

feat(plugin): add a compute sharing mode without HSA_CU_MASK - #265

Merged
moezdil merged 1 commit into
mainfrom
feat/compute-sharing-mode
Nov 18, 2026
Merged

moezdil merged 1 commit into
mainfrom
feat/compute-sharing-mode

Conversation

@moezdil

@moezdil moezdil commented Nov 17, 2026

Copy link
Copy Markdown
Contributor

What type of PR is this?

/kind feature

What this PR does / why we need it:

Adds the "AMD: support compute Sharing mode (not setting CU_MASK)" item from the HAMi v2.12 roadmap (Project-HAMi/HAMi#3442).

-compute_sharing (chart dp.computeSharing, default off) makes Allocate skip the CU slice: no HSA_CU_MASK, no reserved CUs, no hami.io/amd-cu-allocated annotation, and no CU rebuild from the node's pods. Every container on a GPU runs on all of its CUs and the hardware schedules their work. HIP_DEVICE_MEMORY_LIMIT_<i>, LD_AUDIT, dmem, ROCR_VISIBLE_DEVICES and HIP_VISIBLE_DEVICES are unchanged.

The mode is per node, read at plugin start. Mixing masked and unmasked pods on one GPU would let the unmasked ones run on a masked pod's CUs, so the docs say to drain the node before switching. The HAMi scheduler still counts amd.com/gpucores, so in this mode it limits how many pods share a card, not what each gets.

Which issue(s) this PR fixes:

Part of Project-HAMi/HAMi#3442

Special notes for your reviewer:

Tested on an RX 9070 XT (gfx1201, 64 CUs), RKE2 with HAMi 2.11.0. Two pods with amd.com/gpucores: 50, amd.com/gpumem: 4096 ran an FP32 FMA HIP kernel:

Plugin HSA_CU_MASK CU annotation One pod alone Both pods at once
0.0.4 0:0-31, 0:32-63 written ~6.8 TFLOPS ~6.8 each
this PR, -compute_sharing=true unset none ~13.3 TFLOPS ~6.8 each

HIP_DEVICE_MEMORY_LIMIT_0 and LD_AUDIT were set in both modes. A third gpucores: 50 pod stayed Pending with CardInsufficientCore.

TestAllocateComputeSharing covers a core slice, a memory-only slice and a whole GPU while another pod holds every CU. With the option forced off, all three cases fail. go vet ./... and go test -race ./... pass in the CI builder image; helm lint and both template renders pass.

Does this PR introduce a user-facing change?:

Yes, the new -compute_sharing flag and the dp.computeSharing chart value, off by default.

AI Disclosure:

Written with AI assistance and reviewed and tested by the author.

-compute_sharing (chart dp.computeSharing) lets every container on a GPU
run on all of its CUs. Allocate sets no HSA_CU_MASK, reserves no CUs and
writes no hami.io/amd-cu-allocated annotation. Memory limits, LD_AUDIT,
dmem and the device visibility variables are unchanged.

Signed-off-by: mesutoezdil <mesudozdil@gmail.com>
@hami-robot hami-robot Bot added the kind/feature Categorizes issue or PR as related to a new feature. label Nov 17, 2026
@coderabbitai

coderabbitai Bot commented Nov 17, 2026

Copy link
Copy Markdown

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 92035db0-e159-4827-8be1-f022f6b1ca53
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@hami-robot

hami-robot Bot commented Nov 17, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: moezdil

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@hami-robot hami-robot Bot added the approved label Nov 17, 2026
@moezdil
moezdil requested a review from FouoF November 17, 2026 12:16
@moezdil moezdil assigned moezdil and Copilot and unassigned Copilot Nov 17, 2026
@moezdil
moezdil merged commit d6494bf into main Nov 18, 2026
11 of 12 checks passed
@moezdil
moezdil deleted the feat/compute-sharing-mode branch November 18, 2026 20:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved kind/feature Categorizes issue or PR as related to a new feature.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants