OCI: offer the H100, H200, L40S and A100-v2 shapes

[fede-kamel]

Closes #4301.

SUPPORTED_SHAPE_FAMILIES in the OCI backend still lists only the GPU shapes that existed when Bare Metal support landed (#1325): GPU3, GPU4, A10. Every newer NVIDIA shape from the gpuhunt catalog was filtered out by _supported_instances, so dstack apply could never select an OCI H100, H200, L40S or A100-v2 even with quota. #2343 asked for H100; its blocker (Fabric Manager, #2348) was fixed in #2355 and the images now install open kernel modules and Fabric Manager.

This adds those four families. They are all x86-64 hosts with NVIDIA GPUs, which is what the published dstack-cuda-* Marketplace images run, and they are the newer NVIDIA shapes the gpuhunt catalog already carries today.

Deliberately excluded, with the reason recorded next to the list and in tests:

• AMD Instinct (BM.GPU.MI300X.*, BM.GPU.MI355X.*) - no ROCm image is published, and create_instance always picks the CUDA image.
• Grace superchips (BM.GPU.GB200.*, BM.GPU.GB300.*) - arm64 hosts, the images are x86-64 only.
• Blackwell (BM.GPU.B200.*, BM.GPU.B300.*, BM.GPU.RTXPRO.*) - x86-64 NVIDIA hosts that the images should run, but not tested on real hardware, so held back. See the testing note below.

It also documents why SUPPORTED_REGIONS is fixed: the Marketplace listings carrying the images are regional, so widening requires publishing images first.

Testing

On real hardware. Provisioned through this backend in us-phoenix-1 and ran a dstack task on each:

	offer	VM.GPU.A10.1	BM.GPU.A10.4
arch	x86	x86_64	x86_64
vCPUs	30 / 128	30	128
memory	240 / 1024 GB	235 GiB usable	1007 GiB usable
GPU	A10:24GB:1 / :4	1 x A10, 23028 MiB	4 x A10, 23028 MiB
torch	-	device_count()==1, CUDA tensor OK	device_count()==4, CUDA tensor OK

Driver on the published image: 580.167.08.

This exercises the offer -> provision -> run path, including the bare metal path these families use, but it does not cover the four shapes this PR adds - I do not have access to that hardware. If you would rather hold this until someone can run an H100 through it, that is fair.

Two bugs blocked the backend from being usable at all before any of this could be tested; they are separate PRs, not needed for this one:

• OCI: support profiles that authenticate with a security token #4308 - profiles that authenticate with a security token
• OCI: resolve availability domains from the configured compartment #4309 - availability domains for a compartment in another tenancy

Unit. src/tests/_internal/core/backends/oci/test_compute.py (10 supported / 13 unsupported shapes). OCI backend suite 43 passed, whole backends suite 473 passed, ruff check and ruff format clean.

Catalog side

In dstackai/gpuhunt#262. Without it this change already affects H100, H200, L40S and A100-v2, which the catalog carries today.

Known gap, not fixed here

BM.GPU.B4.8 (8 x A100 40GB, 128 OCPUs) is a distinct shape from BM.GPU4.8 and is returned by the Compute API, but it is absent from the Cost Estimator feed gpuhunt reads, so the catalog has no row and no price for it. Adding BM.GPU.B4. to this list alone would be dead code, so it needs the gpuhunt side first. Noted in #4301.

[jvstme]

@fede-kamel, thanks for both of the PRs, we appreciate the effort!

Have you had a chance to test each of these new shapes end-to-end against real hardware? If not, would you have a chance to do so?

The testing sequence I would use is:

• Provision each new shape through dstack and run a dstack dev environment.
• In the dev environment:
  • Verify that the GPU is detected and has the same specs as advertised in the dstack offer (use nvidia-smi).
  • Verify that the CPU and RAM specs match those advertised in the dstack offer.
  • Verify that the GPU works:

    pip install numpy torch
    python -c "import torch; print('CUDA available:', torch.cuda.is_available() and torch.cuda.get_device_name(0))"
    python -c "import torch; print('Random tensor:', torch.rand(10, device=torch.device('cuda')))"
    

We'd love to test them ourselves as well, just to make sure they'll work as expected for our users, but we don't currently have the right quotas.

[fede-kamel]

Thanks for the review, and for being specific about what you wanted tested - it turned out to be the right question.

Short answer: no, not for the shapes these PRs add. I do not have access to H100, H200, L40S, A100-v2, B200, B300 or RTX PRO hardware, and could not get any for this. I would rather say that plainly than imply coverage I do not have.

What I could test, I did. I ran your sequence on A10, on both a VM and a bare metal host in us-phoenix-1:

	dstack offer	VM.GPU.A10.1	BM.GPU.A10.4
arch	x86	x86_64	x86_64
vCPUs	30 / 128	30	128
memory	240 / 1024 GB	235 GiB usable	1007 GiB usable
GPU	A10:24GB:1 / :4	1 x A10, 23028 MiB	4 x A10, 23028 MiB
torch	-	device_count()==1, CUDA tensor OK	device_count()==4, CUDA tensor OK

Everything advertised matched the machine. The row I care about is arch: the Cost Estimator reports processorType: "arm" for these shapes and is wrong, and gpuhunt's is_arm_cpu() override is what keeps the catalog right - that is now confirmed on hardware rather than argued from a comment. Driver on the published image is 580.167.08.

That covers the bare metal path the new families use, but it is not evidence about the new families themselves.

So I have scoped this PR down. Dropped BM.GPU.B200.*, BM.GPU.B300.* and BM.GPU.RTXPRO.* and retitled it. What is left is H100, H200, L40S and A100-v2 - the newer NVIDIA shapes the catalog already carries. R580 supports Blackwell, so I think those three would work, but "I think" is not what you asked for and they can come back when someone can boot one.

Two bugs turned up before I could test anything, both of which stopped the backend from being usable at all against my account. Filed separately since they are independent of the shape list:

• OCI: support profiles that authenticate with a security token #4308 - profiles created by oci session authenticate cannot authenticate; clients are built without a SecurityTokenSigner, so creds_valid() returns False for a profile the OCI CLI accepts.
• OCI: resolve availability domains from the configured compartment #4309 - availability domains are resolved from the credentials' tenancy, so a compartment_id in another tenancy makes every list_shapes return 404 and no offers are ever produced.

Both have repro steps, unit tests, and 453 passing in the backends suite.

On testing these: if you would rather hold them until someone can run one independently, that is a reasonable call and I would not argue with it.

[peterschmidt85]

On your quota problem: I work at Oracle, on OCI. If it would help, I can try to get dstack access to the shapes you are missing so you can test these yourselves rather than take my word for anything - or run a sequence you specify on capacity here and post the raw output. Tell me which you would prefer, or if you would rather just hold these until you can verify independently, that is a reasonable call too.

@fede-kamel Either or both options would be actually really cool:

1. See if you can get such a quota to test it on your side (if possible)

2. See if you could help us get such quota for our account so we can test it (if that can be arranged without involving sales from OCI)

[fede-kamel]

Follow-up on the testing question, since I could not get hardware for these four.

I cross-checked what the catalog produces for them against Oracle's Compute Shapes documentation, which is an OCI source independent of the Cost Estimator the gpuhunt provider reads. Every field matches:

shape	catalog	documentation
BM.GPU.H100.8	224 vCPU, 2048 GB, 8 x H100 80 GB	112 OCPU, 2048 GB, 640 GB total
BM.GPU.H200.8	224 vCPU, 3072 GB, 8 x H200 141 GB	112 OCPU, 3072 GB, 1128 GB total
BM.GPU.L40S.4	224 vCPU, 1024 GB, 4 x L40S 48 GB	112 OCPU, 1024 GB, 192 GB total
BM.GPU.A100-v2.8	256 vCPU, 2048 GB, 8 x A100 80 GB	128 OCPU, 2048 GB, 640 GB total

The documentation also confirms all four shape names exactly as the allowlist matches them, which was the other thing that could silently be wrong - the Cost Estimator does decorate some names, BM.GPU.GB200.4 (NVL72) being the one that caught me out in the catalog PR.

This is not a substitute for running a job on one, and I am not presenting it as one. What it does rule out is the failure mode where the offer advertises specs that do not exist. Whether that is enough to land this, or whether you would rather wait for an actual run, is your call - I am happy either way, and the quota conversation is in your court too.

[fede-kamel]

One more thing that I should have made explicit earlier, because it bears on how much the missing hardware test actually costs us here.

The backend does not branch on GPU model. Tracing what this PR changes:

• _supported_instances() is a prefix match on the shape name. No GPU logic at all.
• create_instance() has exactly one GPU-dependent branch - requires_nvidia_proprietary_kernel_modules(gpu_name), which picks between the open-kernel-modules image and the proprietary one.
• Everything after that - cloud-init, security group, subnet, launch_instance - takes shape as an opaque string.

And that one branch resolves identically for the shape I tested and for the four this PR adds:

A10  -> False      H100 -> False
A100 -> False      H200 -> False
                   L40S -> False
V100, P100, P40, M60 -> True   (pre-Turing, different image)

So the BM.GPU.A10.4 run went through the same code path, the same dstack-cuda-* image and the same launch call that a BM.GPU.H100.8 run would. The interfaces are identical; the shape name and the catalog metadata are the only things that vary.

That narrows what is genuinely untested for these four shapes to two things, and both have now been checked against Oracle's published documentation:

1. the shape name string - confirmed, all four appear exactly as the allowlist matches them
2. the catalog metadata - confirmed, vCPU / RAM / per-GPU memory all match

What remains unverified is whether OCI's Hopper hosts have some host-level quirk the A10 hosts do not. That is a real residual, and it is why I am not claiming this is equivalent to a run. But it is a much smaller gap than "untested", and it does not grow with each shape added, since they all share one path.

The same reasoning applies to B200/B300/RTX PRO, which also resolve to False on that branch. I dropped them from this PR anyway, since Blackwell driver support is newer than Hopper's and I would rather be conservative there. Happy to restore them if you would rather the argument be applied consistently.