Repository navigation
feat(usage): show each pooled key's state by slot name - #96
Conversation
ANTHROPIC_API_KEY_1, _2, ... join ANTHROPIC_API_KEY in the credential pool. Requests use the first key that is not resting, so the spare only carries traffic after the primary is refused. The Claude API reports an empty credit balance and an owner-set spend limit as 400 invalid_request_error, and a tier spend cap as a 429 with enforced_spend_limit_reached. Raw dispatch now reads that small JSON error body, rests the key for an hour and marks the response. Unified routes, native /anthropic/v1/* routes and the auto:intelligence chain then send the request once on the next key. Other 400 errors still go straight back to the caller. The Worker forwards ANTHROPIC_API_KEY_<n> to the Container. A body read for classification is replayed from the cache, so relays still forward Anthropic's error.
…wn flags Rerun the route failover tests with scoped cooldowns, reset scheduling and prompt-cache affinity enabled, as production runs them, and add a streamed chat whose first key ran out of credit. The shared refusal check is now public, since the native route imports it.
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars.
 |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Claude API keys now form a credential pool, so the intelligence transport reads them with get_api_keys. The fixture that marks the primary unconfigured patches that list too.
GET /admin/providers/usage now adds credential_slots for providers with a key pool, such as the Claude API and Codex Everywhere. Each configured key appears under its environment-variable name, never its value. Entries show whether the key is active or resting, plus this process's dispatch, error and refusal counts and its last refusal reason. Operators can see which Anthropic key is serving traffic and why a key moved aside. The counters are process-local and restart with the Container. The Claude Console stays the record of each key's spend.
5724bd5 to
a6a0192
Compare
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a6a0192ede
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| active = next((row for row in rows if not row["resting_seconds"]), None) | ||
| if active is not None: | ||
| active["active"] = True |
There was a problem hiding this comment.
With POOL_RESET_SCHEDULING_ENABLED=true, the first key without a legacy rest is not necessarily the key used for new conversations. CredentialPool._scheduled_available() also excludes keys with exhausted account-wide usage windows and reorders eligible keys by reset time, but this selection reads only _resting. Consequently, the usage endpoint can mark an exhausted key as active while requests use its spare, or report an active key when the whole pool is unavailable. The existing reset-order and account-exhaustion cases in tests/test_pool_reset_scheduling.py demonstrate these selection paths. Derive the active slot from the scheduler-aware eligible order, accounting for account-wide observations even if model-specific rests remain intentionally omitted.
Useful? React with 👍 / 👎.
GitHub Actions
Failure
Stacked on #95.
What
GET /admin/providers/usagenow addscredential_slotsfor providers with a key pool (the Claude API and the Codex Everywhere pools). There is one entry per configured key, in selection order. Each key is named by its environment variable (ANTHROPIC_API_KEY,ANTHROPIC_API_KEY_1), never by its value, and shows:active: the first key that is not resting, which new conversations useresting_secondsrequests,errors,refusalsandlast_used_atlast_refusal(for exampleanthropic_credit_exhausted) andlast_refusal_atWhy
With two Claude API keys failing over to each other, operators need to see which key is serving traffic and why a key moved aside.
Limits
The counters live in the Container process and reset when it restarts (every deploy). They count dispatches, not dollars. The Claude Console's Usage and Cost pages, filtered by API key, remain the record of each key's credit. A rate-limit rest scoped to one model doesn't appear in
resting_seconds. Recording slot names in the durable usage ledger would need a migration and row-schema change, so it is left for a follow-up.Tests
test_provider_usage_service.py, the pool, cooldown and reset-scheduling suites, and the outbound inventory all pass.