Skip to content

feat(gateway): keep the last response on a Responses WebSocket - #56

Merged
fylorn merged 1 commit into
devfrom
feat/responses-ws-connection-state
Sep 24, 2026
Merged

fylorn merged 1 commit into
devfrom
feat/responses-ws-connection-state

Conversation

@fylorn

@fylorn fylorn commented Sep 24, 2026

Copy link
Copy Markdown
Contributor

Closes the previous_response_id gap in the Responses WebSocket bridge (#43).

The gap, and the options

OpenAI's socket mode keeps the connection's most recent response in memory, so a turn can continue from it with previous_response_id even with store: false — which is how Codex runs. The desktop gateway gets that for free: it pipes the whole socket to one upstream socket, and the upstream keeps the state. This bridge runs each response.create as its own request through the HTTP pipeline, so the upstream had no connection and a store: false continuation failed.

Options considered:

  1. Pipe the socket to the upstream, like the desktop. Gives the upstream's own state, but only works when the route's upstream speaks Responses over a WebSocket, and a piped socket bypasses per-turn limits, model access, conversion, inspection, billing and audit — the reason feat(gateway): accept Gemini clients, and the Responses API over a WebSocket #43 bridged turns instead. Also pins the whole connection to one route with no failover.
  2. Keep the state on the connection (chosen). Works for every upstream and format and keeps every per-turn guarantee.

What it does

The connection keeps its most recent response as the conversation so far: the input the turn went upstream with, plus the output items of its response.completed (with the upstream's store id/status removed — they would be looked up and not found; call_id stays, it ties tool results to calls). A turn whose previous_response_id names it goes upstream with that history in input and no previous_response_id, which any upstream accepts and which is billed for what it actually sends. Like OpenAI, only the most recent response is kept; a previous_response_id naming anything else goes upstream as sent (for an upstream that stored it). A failed or cancelled turn leaves the chain where it was.

Tests

  • Unit: history built from input + output items, store ids dropped, unknown ids passed through.
  • Integration: two turns with store: false, the second naming the first's id, against a Chat upstream: the second upstream request carries user "one", the assistant's answer, user "two". Fails on the old code, passes now.
  • Local: fmt, clippy (--all-targets, --lib), 690 unit tests, the WebSocket integration file.

🤖 Generated with Claude Code

OpenAI's socket mode keeps the connection's most recent response, so a
turn can name it in previous_response_id with store: false, which is how
Codex runs. Each turn here is a separate upstream request, possibly to an
upstream in another format with no such store, so that failed.

The connection now keeps its most recent response as the conversation so
far: the turn's full input plus the output items of its
response.completed (without their store ids). A turn naming it goes
upstream with that history in input and no previous_response_id, valid
against any upstream and billed as what it is. A previous_response_id
naming anything else goes upstream as sent. A failed turn leaves the
chain where it was.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@fylorn
fylorn merged commit 443de8d into dev Sep 24, 2026
6 checks passed
@fylorn
fylorn deleted the feat/responses-ws-connection-state branch September 24, 2026 12:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant