[earakely-scale]

Two things a prime-rl GPU run on GridSim hit, through verifiers' NeMo Gym taskset and serve: rollouts were masked while the bundle server only needed time, and abandoned sessions held its episodes for 10 minutes.

Sessions wait out a busy env server

The resources server opens one session on the bundle's OpenEnv server per rollout. reopened() already waits for room when the server refuses with "at capacity", retrying for patience (the server's reset_timeout, 600 s). But it gave up on the first other connection error.

Under prime-rl's load, the bundle server answered handshakes after the OpenEnv client's 10 s connect timeout: "couldn't open a session on the server at http://127.0.0.1:8200: ConnectionError: Failed to connect to ws://127.0.0.1:8200/ws: timed out". Those rollouts were masked at once while the server only needed time. On the run's final eval that was 34% of rollouts.

reopened() now retries a connection that timed out or was dropped within the same patience: TimeoutError, resets, EOF, and httpx timeouts and read errors anywhere in the cause chain. A refused connection (nothing listening) or a wrong answer still fails at once. Checked on real errors: a socket that accepts but never answers the handshake gives a chain ending in TimeoutError after 10 s and is retried; a closed port gives ConnectionRefusedError in 0.0 s and fails at once.

serve --idle-timeout

verifiers' NeMo Gym taskset never calls /close_session; the run's log shows 0 calls. When prime-rl cancels in-flight rollouts (64 at the end of training), their sessions stay open on our server. Their episodes then hold the bundle server's slots until the idle timeout (600 s): 56 GridSim containers were still up 10 minutes after the run, which starved the final eval.

serve --idle-timeout SECONDS overrides the config's, so such a client's abandoned episodes are reclaimed sooner. It's unset by default: Gym and NeMo RL close or verify every session.

Tests

• tst/unit/training/sessions_test.py: a session open that times out is retried until it opens, and a refused one fails after one attempt. The busy case fails without the fix.
• tst/nemo_gym: serve --idle-timeout 1 ends the episode of a session its client abandons: the next tool call is refused, the server logs it, and the episode is torn down.
• tst/nemo_gym, tst/verifiers, tst/unit/training and the dependency guard pass locally with nemo-gym (54) and in a dev-only venv without it (37).

GPU run on this branch

prime-rl v0.10.0 with its pinned verifiers, on 4× L40S. GRPO on GridSim from the SFT warm start: 40 steps, 32 rollouts a step, evals on the 200 eval days. serve --masked-as-error --idle-timeout 120.

	Before (same config, main)	This branch
/verify answered 503 (masked)	90	0
Final eval rollouts failed	34%	0%
Abandoned sessions reclaimed	after 600 s (56 containers still up 10 min after the run)	55, within 120 s each

Eval reward: 0.565 → 0.580 → 0.566 (steps 0, 20, 40). Train reward by 10-step blocks: 0.63 → 0.69 → 0.74 → 0.71, with no failed training rollouts. verifiers still made 0 /close_session calls; prime-rl cancelled 64 in-flight rollouts at the end, and --idle-timeout reclaimed them.

🤖 Generated with Claude Code