fix: respect CPU affinity and cgroup CPU quota for default thread counts

[manunicholasjacob]

The default n_threads and n_threads_batch come from multiprocessing.cpu_count() in Llama.__init__ and in server/settings.py.
That count is every CPU on the host, so inside docker run --cpus=2 on a 20-thread machine the defaults are 10 and 20 threads.
llama.cpp's worker threads then exhaust the CFS quota early in each period and the container is throttled almost all the time.
The same happens with a reduced affinity mask (taskset, docker run --cpuset-cpus, Kubernetes static CPU manager).

This adds default_cpu_count() in llama_cpp/_utils.py and uses it at those call sites:

• start from len(os.sched_getaffinity(0)) where available, else multiprocessing.cpu_count();
• cap at ceil(quota / period) from cgroup v2 cpu.max, or cgroup v1 cpu.cfs_quota_us / cpu.cfs_period_us, taking the tightest limit on the process cgroup and its ancestors;
• no quota, a missing file, or a non-Linux host leaves the count unchanged.

Explicit n_threads / n_threads_batch values, and the -1 server option, keep working; -1 now resolves to the usable count instead of the host count.
On an unrestricted host the result is the same as before (20 on the test machine).

Tests

• tests/test_utils.py: fake /proc/self/cgroup and cgroup roots for v2 quota, v2 max, tightest ancestor, v1 quota, v1 -1, missing files; affinity capped by quota (including fractional quotas rounding up), affinity without quota, and no sched_getaffinity.
• tests/test_llama.py::test_llama_default_threads_use_usable_cpu_count: Llama picks n // 2 and n from the helper.
• make lint (ruff check + format check) passes.
• In a python:3.11-slim container with a CPU wheel built from main (1652066) and the patched files: 22 tests pass. The remaining 18 need Hugging Face model downloads, which this offline run skipped; they fail the same way on unpatched main.

Measurements

Validation environment: i7-12700H (20 logical CPUs), WSL2 + Docker, python:3.11-slim, CPU wheel built from main with GGML_NATIVE=ON, Qwen2.5-0.5B-Instruct Q4_K_M, 256-token prompt then 64 greedy tokens, n_ctx=1024.
Each run is a fresh container. Arms were interleaved, alternating order each rep, with throttling read from the container's cpu.stat around each phase.
Results are the median [min, max] in tokens/s.

docker run --cpus=2 (6 reps per arm):

arm	threads (gen/batch)	prompt tok/s	gen tok/s	throttled periods
main	10 / 20	1.74 [1.29, 1.82]	3.00 [1.12, 9.15]	10801 / 10803
this PR	1 / 2	78.15 [48.88, 81.87]	14.91 [6.23, 14.94]	22 / 535
explicit n_threads=2, n_threads_batch=2	2 / 2	80.42 [73.42, 82.91]	25.50 [23.40, 26.63]	33 / 345

docker run --cpuset-cpus=0-3 (5 reps per arm, no quota):

arm	threads (gen/batch)	prompt tok/s	gen tok/s
main	10 / 20	135.14 [120.32, 147.94]	5.20 [4.63, 5.64]
this PR	2 / 4	178.05 [163.94, 188.84]	35.53 [33.76, 36.34]
explicit 2 / 2	2 / 2	113.01 [112.11, 119.16]	36.14 [34.03, 36.47]

Under --cpus=2 the PR's generation default is one thread because the existing // 2 heuristic halves the usable count.
I left that heuristic alone to keep this change narrow; it is why the explicit 2-thread arm decodes faster in that row.
The machine had other background load during these runs, which accounts for the low minimums.

AI disclosure: this change, its tests and the benchmark harness were written with Claude Code (Anthropic), and the measurements above were run with it on my machine.

Generated with Claude Code