Repository navigation
Add --cpus and --memory resource limits - #25
Merged
Merged
Conversation
anoek
force-pushed
the
resource-limits
branch
from
October 31, 2026 13:31
4d0d757 to
8f0e1cd
Compare
Sandboxes can now be limited to a number of CPUs and an amount of memory using cgroup v2. --cpus accepts fractional values, and negative values mean all but that many CPUs, so -2 on a 32 CPU machine allows 30. --memory accepts sizes like 512M or 4G and disables swap for the sandbox. Both can also be set with SANDBOX_CPUS / SANDBOX_MEMORY or in config files. Sandboxes with limits get a cgroup at /sys/fs/cgroup/sandbox/<uid>-<name>. PID 1 is started in it with CLONE_INTO_CGROUP, which also makes it the root of the sandbox's cgroup namespace, and processes that join the sandbox move into it before entering its namespaces. Stopping the sandbox removes the cgroup. Limits are recorded in the sandbox settings, and changing them requires stopping the sandbox, like network settings.
Tests that check default, environment, or changed limits now skip config files, so limits set in ~/.config/sandbox/config.toml don't change the results.
A sandbox started inside another sandbox can't manage cgroups, so with limits in a config file nested sandboxes refused to start. They now warn and run without a cgroup of their own, which leaves them subject to the outer sandbox's limits. Outside a sandbox, failing to set up limits is still an error. --cpus and --memory (and SANDBOX_CPUS, SANDBOX_MEMORY, and config files) now accept "unlimited", which removes a limit set by a less specific source. Also read memory.events in the same command as the allocation in the memory enforcement test, so the result doesn't depend on which process the OOM killer picks.
delete stops sandboxes as uid 0, so looking for the cgroup by the name we'd create (<uid>-<name>) missed it and the cgroup was never removed. Find the cgroup from the sandbox's PID 1 instead, accepting any uid. Also stop writing cgroup.kill when removing a cgroup. By that point the only processes left in it are the `sandbox` commands waiting on the killed sandbox processes. Killing those cut off their exit status reporting (and truncated coverage profiles mid-write). They exit by themselves once their child is gone, which the removal already waits for.
The swap accounting test now skips config files, so a cpus limit in ~/.config/sandbox/config.toml can't turn it into a cpus+memory test and leave the memory-only path untested.
anoek
force-pushed
the
resource-limits
branch
from
October 31, 2026 14:14
8f0e1cd to
730b633
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
--cpusand--memoryto limit how much CPU and memory a sandbox can use, enforced with cgroup v2.--cpus=2,--cpus=1.5: limit total CPU time (cpu.max), not pinning to specific cores, sonprocinside still reports every CPU.--cpus=-2: negative values mean "all but N", so on a 32 CPU machine this allows 30. "Available" comes fromstd::thread::available_parallelism(), which equalsnprocunless already under a CPU quota.--memory=512M,--memory=4G: setsmemory.maxand setsmemory.swap.maxto 0 (when swap accounting exists) so the limit means what it says. Going over the limit OOM-kills a process in the sandbox.unlimitedremoves a limit set by a less specific source, e.g.--cpus=unlimitedto run once without thecpus = -2from your global config.SANDBOX_CPUS/SANDBOX_MEMORYandcpus = -2/memory = "8G"in config files, with the usual precedence.sandbox config cpus memoryshows the resolved values.How it works
/sys/fs/cgroup/sandbox/<uid>-<name>. The parentsandboxcgroup is created on first use, andcpu/memoryare enabled incgroup.subtree_controlas needed.CLONE_INTO_CGROUP(Linux 5.7+), which also makes that cgroup the root of the sandbox's existing cgroup namespace. Inside,/proc/self/cgroupis0::/and/sys/fs/cgroup/cpu.maxetc. show the sandbox's own limits.sandbox execmoves itself into the sandbox's cgroup beforesetns. Previously, joined processes stayed in the caller's cgroup (e.g. the terminal's systemd scope), since joining a cgroup namespace doesn't move a process between cgroups.stopanddeletefind the cgroup from the sandbox's PID 1 (sodelete, which runs as uid 0, finds it too), kill the sandbox as before, then remove the cgroup once the waitingsandboxcommands have exited. They aren't killed withcgroup.kill, so they still report their child's exit status.settings.json(withserde(default), so settings from older running sandboxes still load). Running an existing sandbox with different limits errors and asks you to stop it first, the same as network settings./sys/fs/cgroupis read-only inside a sandbox). Instead of failing, a nested sandbox warns and runs without its own cgroup, sharing the outer sandbox's limits. The warning names them, e.g.(cpus=2 memory=unlimited)./sys/fs/cgroup, or inside a container), starting a sandbox with limits fails with an explanation, before anything is mounted.Testing
make teston a host with writable cgroup v2, with limits set in the user config (cpus=-2,memory="128G"): 321 passed, 0 failed (3 ignored, which the suite skips by design). That run was on the stack's top branch (Add tests for logger levels and uncovered error paths #26), before a final test-only change; theai_test_resource_limitsfile (19 tests) passes again after that change.tests/ai_test_resource_limits.rscovers:unlimitedand invalid valuesstopanddelete, replacing a stale cgroup, a busy cgroup, and a name collision between two storage directoriessrc/config/limits.rs.Notes
CLONE_INTO_CGROUPisn't in the pinnedlibccrate yet, so it's defined insrc/util/clone3.rs.cpusetpinning (which would change whatnprocreports inside) isn't included.