Skip to content

fix(telemetry): gate the prompt cache like the model report, and keep the first 20 miss causes - #623

Open
JDProfresh wants to merge 2 commits into
Ark0N:masterfrom
JDProfresh:fix/prompt-cache-report-bounds
Open

JDProfresh wants to merge 2 commits into
Ark0N:masterfrom
JDProfresh:fix/prompt-cache-report-bounds

Conversation

@JDProfresh

Copy link
Copy Markdown

These are the two small follow-ups you flagged on #607, one commit each: promptCache was stored for any session mode even though the model report from the same payload is capability-gated, and more than 20 miss causes 400'd the whole statusline post instead of truncating the list.

Store the prompt cache only for CLIs whose statusline reports it

POST /api/status-telemetry gated the model report (noteReportedModel) on capabilities.statusLineTelemetry but stored promptCache unconditionally. The prompt_cache object rides the exact same statusline payload the exporter POSTs, so it is the same question: does this CLI's statusline speak for it. I reused statusLineTelemetry (the capability that also decides whether the exporter is injected at all) rather than add a new field, resolved once per request into a local and applied to both the model report and the prompt-cache store. No CLI-id branch. A non-capable session (codex) now stores no prompt cache; the rest of the post (plan usage) still lands.

Keep the first 20 miss causes instead of refusing the post

last_miss_cause.causes was z.array(z.unknown()).max(20), so an over-long array failed validation and lost the entire post: the footer text the route answers with, plan usage, the model report and the cache readout. I replaced the .max(20) with a z.preprocess that slices the array to 20 before the array schema validates, so an over-long array costs one bounded slice rather than per-element validation across every entry, and the element bound (z.unknown()) stays intact on the 20 kept. parsePromptCache already keeps the string ones among those.

Tests

  • A codex (no statusline exporter) post with a valid prompt_cache stores nothing on the session and still broadcasts plan usage.
  • A 25-cause post is accepted (200, not 400) and keeps the first 20 causes in order.

Removed each guard and watched its test go red, then restored.

Verified

On this branch, off today's master (1.41.1, a8a8bb71):

  • The full npm test gate passes: 11,260 passed, 12 skipped, 0 failed. typecheck, lint, format:check, check:frontend-syntax, check:browser-excludes and prettier on the touched test pass.
  • On a throwaway instance built from this branch (removed after), one claude and one codex session, the same statusline post to each with a warm prompt_cache carrying 25 miss causes: both answered 200 with the footer text, the claude session stored the cache with the first 20 causes in order, and the codex session stored neither the cache nor a model. On master that post is a 400 for both.

… reports it

The status-telemetry route gated the model report on
capabilities.statusLineTelemetry but stored the prompt-cache report for
any session mode, even though both ride the same statusline payload the
exporter POSTs. Gate the prompt-cache store on the same capability so a
CLI with no statusline exporter no longer carries prompt-cache state it
never actually reported.
… post

last_miss_cause.causes was z.array(z.unknown()).max(20), so a statusline
payload carrying more than 20 causes failed validation and lost the
whole post: plan usage, model, cache and the footer with it. Truncate
to the first 20 with a preprocess that slices before the array
validates, so an over-long array costs one bounded slice rather than
per-element work across every entry.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant