fix(sandbox): reclaim socket descriptors before exhaustion

[pimlock]

Summary

Keep socket-heavy workloads from exhausting the sandbox broker's file descriptors and taking down the sandbox control connection.

Problem

The broker retains a file descriptor for each socket in a pre-connect state. When the workload closes one, the broker does not learn about it immediately. It removes the stale descriptor during cleanup.

The socket registry allows 4,096 metadata entries, but the broker may have a soft RLIMIT_NOFILE of 1,024. Previously, cleanup normally started only when the registry filled or socket(2) returned EMFILE.

For example:

1. A workload repeatedly creates and closes unconnected sockets.
2. Stale broker descriptors accumulate until the process reaches its soft limit.
3. The broker may no longer have a descriptor available to scan /proc or serve the control path.
4. The sandbox remains marked ready, but exec and relay requests stop responding.

Connected sockets are different. The broker releases their source descriptors but keeps their metadata for policy mediation, so the metadata limit must remain separate from the descriptor limit.

Fix

Keep the 4,096-entry metadata limit and add a separate retained-descriptor budget. The budget uses the inherited soft RLIMIT_NOFILE, subtracts descriptors already open at startup, and reserves 64 more for control-path work.

With a soft limit of 1,024 and 24 descriptors already open, the broker can retain up to 936 pre-connect descriptors. Connected socket metadata can still use the full registry because those entries no longer retain broker descriptors.

Before opening another socket, the broker runs cleanup if either limit is full. If cleanup cannot free capacity, the workload receives EMFILE while the broker keeps enough descriptors to scan /proc and serve control traffic.

Related Issue

No issue required. This is a localized reliability bug found while investigating a sandbox that stopped responding under socket churn.

Changes

• Keep separate bounds for socket metadata and retained descriptors.
• Account for descriptors already open when the broker starts.
• Reclaim stale sockets before descriptor exhaustion.
• Test both retained pre-connect sockets and connected metadata.

Testing

• mise run pre-commit
• mise run ci
• mise run e2e

Checklist

• Commit messages follow Conventional Commits.
• Commits include DCO sign-offs.
• Tests cover the new behavior.
• Full repository CI passes.

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[pimlock]

/ok to test b74e9d3

[github-actions]

Label test:e2e applied, but pull-request/3532 is at b74e9d3 while the PR head is 4dbc2cd. A maintainer needs to comment /ok to test 4dbc2cd80bb690a95884d19ee402e37ec27ff199 to refresh the mirror. Once the mirror catches up, re-run Branch E2E Checks from the Actions tab.