Skip to content

feat(sep-1036): URL mode elicitation conformance scenarios - #558

Open
nbarbettini wants to merge 1 commit into
modelcontextprotocol:mainfrom
nbarbettini:claude/conformance-url-elicitation-d5484c
Open

nbarbettini wants to merge 1 commit into
modelcontextprotocol:mainfrom
nbarbettini:claude/conformance-url-elicitation-d5484c

Conversation

@nbarbettini

@nbarbettini nbarbettini commented Oct 15, 2026 •

Copy link
Copy Markdown
Contributor

Closes #542.

TL;DR

Adds conformance coverage for URL mode elicitation (SEP-1036), targeting the 2026-07-28 revision. Until now the suite had no URL elicitation checks.

  • Client scenario (elicitation-sep1036-client-url-mode), with eight checks:
    • the client answers every new URL-mode request across several rounds, and echoes the newest requestState each time;
    • it handles a URL request and a form request sent together;
    • it treats a request with no mode as form mode;
    • it never fetches a URL the user declined;
    • it waits for a retry the server holds open, both as plain JSON and on an SSE stream.
  • Server scenario (elicitation-sep1036-url-mode), with seven checks:
    • the URL-mode request is well formed and its url is valid;
    • decline and cancel get a proper answer;
    • a tool that needs URL mode returns -32021 (HTTP 400) to a client that didn't declare it, and never sends a mode the client didn't declare;
    • the removed -32042 error never appears.
  • Includes reference fixtures in the everything-client and everything-server, a deliberately broken server, and negative tests for every graded check.
  • No tier impact: both scenarios were added after the frozen 2026-07-28 requirement set.

Background

URL elicitation (SEP-1036) has been in the protocol since 2025-11-25, and got a major update with MRTR in 2026-07-28. However, it hasn't been represented in the conformance suite until now.

This PR adds scenarios to help both client and server authors stress-test their implementations against the expected spec behavior for URL elicitation.

Both scenarios target 2026-07-28 only. That revision changed how elicitation works: a server now asks for input by returning an InputRequiredResult (MRTR, SEP-2322), and the client answers by retrying the original request. This PR doesn't test the 2025-11-25 shape.

Client scenario: elicitation-sep1036-client-url-mode

Here the harness plays the MCP server, and the SDK's conformance client is the thing under test. The server lists six tools. Each one answers a tools/call with an InputRequiredResult, and the scenario watches how the client answers it and retries.

The client follows a small contract, written out in the scenario description:

  • Declare elicitation: { "form": {}, "url": {} } on every request.
  • Call every listed tool once, answering every input request and retrying with the requestState echoed back.
  • Decline a URL-mode request whose message starts with DECLINE:, and accept the rest.
  • Never fetch or open a URL.
  • Answer form-mode requests with { "confirmed": true }.

The DECLINE: rule is how the scenario can tell that the client asked its handler (standing in for the user) about each request, rather than reusing an earlier answer.

A client that doesn't support URL mode can declare form mode only. The mock server then answers its URL tools with -32021, the way a correct server should, and the URL checks report SKIPPED rather than failing.

sep-1036-client-supports-declared-mode. The spec says a client that declares the elicitation capability must support at least one mode, form or URL. For backward compatibility, an empty elicitation: {} counts as form mode only, so it passes. The only way to fail this check is to declare elicitation with neither mode named, like { "foo": {} }. It's a narrow check, and that's on purpose: it tests exactly what the sentence says and nothing more.

sep-2322-client-constructs-inputs. This is the main client check. The server sends three URL-mode requests in a row, each with a new key and a new requestState. The first two should be accepted. The third reuses the first round's URL but asks for a decline. Under MRTR, a client that receives input requests has to construct answers to them before it retries, and each round is a new request, so the client should go back to its handler every time. The check catches three kinds of bug:

  • a client that resends the previous round's answers, which leaves the new key unanswered;
  • a client that answers a new request from an earlier answer instead of asking its handler, which accepts round three instead of declining;
  • a client that gives up after a fixed number of rounds.

Prompting the user again each time is correct behavior here. The server is the one deciding what it needs.

sep-2322-client-request-state-echoed. Same flow as above. On every retry, the client has to send back the newest requestState exactly as it received it. A client that echoes an older round's state fails, and the error message says which round it came from.

sep-2322-client-constructs-inputs-mixed. One InputRequiredResult carries a URL-mode request and a form-mode request together. The client has to answer both: an accept for the URL, and an accept with content for the form. This has its own check ID because a client can get the round-by-round flow right and still drop one of two requests sent at once. It's skipped for clients that only declare URL mode, since the server can't send them a form request.

sep-1036-client-missing-mode-is-form. The spec says a request without a mode field is form mode. The test request deliberately carries a stray url field alongside its requestedSchema, to catch a client that decides the mode by looking for a URL. A correct client accepts it with form content. A client that answers it like a URL request, with an accept and no content, fails.

sep-1036-client-no-url-prefetch. Clients must not automatically pre-fetch a URL-mode URL or any of its metadata. The scenario serves each declined URL from a separate local HTTP server, so any request that reaches it (a favicon, a HEAD, a link preview) shows up. The check is only judged on the path the user declined, because that's the only path where a fetch is unambiguous:

  • On the accept path, the spec's URL Mode Flow has the client open the URL before it sends the accept, so a consented open looks exactly like a prefetch.
  • Cancel is ambiguous too, since the spec's own cancel examples include "browser failed to load".

sep-1036-client-held-json-retry-complete and sep-1036-client-held-sse-retry-complete. After the client accepts, the server holds its retry open for five seconds before answering. The spec allows that; the URL Mode Flow notes the server "may need to block until the request is fulfilled". One tool holds the response as plain JSON, with no bytes sent until it's ready. The other holds it on an SSE stream with keep-alive comments, plus progress notifications if the client sent a progressToken. A client that closes the request before the answer arrives fails. Five seconds is well inside any reasonable client timeout; the TypeScript SDK defaults to 60.

These two have no requirement rows in sep-1036.yaml. The spec gives servers permission to hold the retry, but no MUST or SHOULD says clients have to wait for it. They're flow checks, like the existing sep-2322-multi-round-r* checks.

Server scenario: elicitation-sep1036-url-mode

Here the harness plays the client, and the SDK's server is under test. The server needs one new test tool, test_url_elicitation, which can only do its job through URL mode:

  • If the client hasn't declared elicitation.url, it returns a -32021 MissingRequiredClientCapabilityError, with HTTP 400 and data.requiredCapabilities: { "elicitation": { "url": {} } }.
  • Otherwise it returns an InputRequiredResult holding one URL-mode request.
  • When retried with an answer, it returns a result. Decline and cancel need to get a well-formed answer too, for example a tool result with isError: true.

The scenario calls this tool several times, declaring different elicitation modes each time. The calls that declare both modes come first, so a server that remembers capabilities from an earlier request gets caught by the form-only calls that follow.

sep-1036-url-request-params. The URL-mode request has to say mode: "url" and carry a string message and a string url. The check also flags a request that has a url but forgot the mode, because clients are required to treat a request without mode as form mode.

sep-1036-url-valid. The url has to be a valid URL. "Valid" here means exactly what the spec's JSON schema means by format: uri: an absolute RFC 3986 URI, checked by the same ajv-formats validator the wire-schema check uses, so the two can never disagree. Relative URLs and strings with spaces fail. The spec doesn't restrict the scheme beyond a SHOULD about HTTPS outside development, so the check doesn't either.

sep-2322-server-handles-decline-cancel. The spec says servers must handle the user declining or cancelling. The scenario runs two separate flows: one where it retries with decline, and one where it retries with cancel. Any well-formed answer counts as handling it: a tool result (including isError: true), another InputRequiredResult offering an alternative, or a deliberate JSON-RPC error. What fails is an internal error (-32603), a timeout, or a dropped connection, which are the signs of a server that only coded the accept path. This follows the existing MRTR checks: sep-2322-reject-tampered-state already treats a deliberate JSON-RPC error as correct handling. The requirement row lives in sep-2322.yaml, because the sentence was added to the elicitation page by the MRTR PR, not by SEP-1036.

sep-1036-server-no-unsupported-mode. Servers must not send elicitation requests in modes the client didn't declare. The scenario calls the URL tool twice while declaring form mode only. Once it uses elicitation: {}, which means form mode under the backward-compatibility rule; this is the one most likely to catch a server that just checks "did the client declare elicitation?". Once it uses { "form": {} }. In both cases the response must not contain a URL-mode request.

It also checks the reverse direction: a client that declares URL mode only calls the existing SEP-2322 form tool, test_input_required_result_elicitation, and must not get a form request back. If a server doesn't have that tool, the reverse probe is recorded as not exercised and the check still grades the two form-only calls. This complements sep-2322-respect-client-capabilities, which only covers a client that declares no elicitation at all.

sep-2575-server-rejects-undeclared-capability and sep-2575-missing-capability-http-400. These are existing SEP-2575 check IDs, exercised here for the URL mode case. The 2026-07-28 base protocol says that when a request needs a capability the client didn't declare, the server must return -32021, list the missing capability in data.requiredCapabilities, and use HTTP 400. Since test_url_elicitation can't work without URL mode, the form-only calls should get exactly that. The response must also name elicitation.url specifically. A bare { "elicitation": {} } would be inaccurate, because the client did declare elicitation; it's only the URL mode that's missing.

sep-1036-server-no-url-elicitation-required-error. The 2026-07-28 base protocol lists error codes from earlier revisions that implementations must not emit any more, and -32042 (URLElicitationRequiredError) is one of them. Every response in this scenario is checked for it. A server ported from 2025-11-25 is likely to still return that error where it should return an InputRequiredResult. That sentence came from spec PR #2907 (error code allocation), which isn't a SEP, so there's no natural SEP file for it. The row is in sep-1036.yaml because -32042 was SEP-1036's error code, with a comment saying where the sentence really came from.

A server that doesn't have test_url_elicitation at all gets every check reported as untestable (FAILURE with details.untestable, per #248). That includes the checks that would otherwise pass trivially because a missing tool never sends anything.

Requirement tracking

  • src/seps/sep-1036.yaml is new. It has 7 check rows and 29 excluded rows. The exclusions are client UI requirements (consent, showing the full URL, highlighting the domain, Punycode warnings) and server requirements that happen out of band or inside the server (binding state to the user, phishing protections, third-party credentials). None of those can be seen on the wire.
  • src/seps/sep-2322.yaml gains three rows: sep-2322-client-constructs-inputs, its mixed-modes variant, and sep-2322-server-handles-decline-cancel.
  • traceability.json isn't regenerated here; the traceability workflow normally does that.

Fixtures and tests

  • everything-client: gains a handler for the client scenario. Its shared stateless request helper can now read SSE responses as well as JSON, since clients must support both.
  • everything-server: gains test_url_elicitation. Its existing MRTR form tool now answers -32021 to a client that declared URL mode only, instead of sending form mode anyway. All 14 MRTR scenarios still pass.
  • Broken server: examples/servers/typescript/sep-1036-url-elicitation-broken-server.ts breaks each server requirement on purpose.
  • Client tests (11): the everything-client passes every check, small inline clients each break one requirement and fail the matching check, and a form-only client is skipped rather than failed.
  • Server tests (8): the scenario runs against the broken server, and against a server without the tool to cover the untestable path. The positive run is the existing all-scenarios.test.ts against the everything-server.
  • Full suite: npm test (661 tests), npm run lint, and npm run typecheck pass. CLI runs of both scenarios against the in-repo fixtures pass, including wire-schema-valid.

What SDKs need to do

Neither scenario affects tier scoring. Both were added after the frozen 2026-07-28 requirement set, and both currently live in the draft suite.

When an SDK bumps to a conformance release that includes them:

  • Client side: it needs a handler that follows the contract above. For the TypeScript SDK v2, that's about 25 lines in test/conformance/src/everythingClient.ts, since its Client already drives MRTR rounds through the registered elicitation/create handler. I ran it against typescript-sdk main (324299b2) and it passes all eight checks plus wire-schema-valid.
  • Server side: it needs test_url_elicitation, plus capability gating on its MRTR form tool.

Deliberately not graded

  • How a server says "not done yet" after an accept. It can hold the retry open, send another InputRequiredResult, or do something else; the spec allows several ways and defines no "pending" signal. This PR only checks that clients cope with a server that holds the request open. It doesn't grade servers on which way they choose.
  • What a client does with an elicitation mode it never declared. The spec doesn't say, so there's nothing to grade against yet.
  • Client UI requirements like consent prompts and URL display. A headless harness can't see them.

🤖 Generated with Claude Code
📝 Edited by Nate, the human

Add client and server scenarios for URL mode elicitation on the
2026-07-28 revision, where elicitation requests travel inside an
InputRequiredResult (SEP-2322).

- elicitation-sep1036-client-url-mode: rounds of URL-mode requests,
  mixed URL and form inputs, requests without mode, no prefetch of a
  declined URL, and retries the server holds open (JSON and SSE).
- elicitation-sep1036-url-mode: URL-mode request shape and url
  validity, decline/cancel handling, no unsupported modes, -32021 for
  clients without URL mode, and no -32042.
- sep-1036.yaml requirement rows; sep-2322.yaml gains client
  constructs-inputs rows and the decline/cancel row.
- everything-client handler (plus SSE response parsing),
  everything-server test_url_elicitation tool and form-tool gating, a
  broken server, and negative tests.

Closes modelcontextprotocol#542.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@pkg-pr-new

pkg-pr-new Bot commented Oct 15, 2026

Copy link
Copy Markdown

Open in StackBlitz

npx https://pkg.pr.new/@modelcontextprotocol/conformance@558

commit: c8fd5b4

@nbarbettini
nbarbettini marked this pull request as ready for review October 15, 2026 15:41

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

SEP-1036 (URL elicitation): conformance checks

1 participant