Skip to content

feat: add version negotiation conformance tests (closes #102)#224

Open
alexdoroshevich wants to merge 2 commits intomodelcontextprotocol:mainfrom
alexdoroshevich:feat/version-negotiation-102
Open

feat: add version negotiation conformance tests (closes #102)#224
alexdoroshevich wants to merge 2 commits intomodelcontextprotocol:mainfrom
alexdoroshevich:feat/version-negotiation-102

Conversation

@alexdoroshevich
Copy link
Copy Markdown

@alexdoroshevich alexdoroshevich commented Apr 13, 2026

What

Issue #102 identified three uncovered MUST requirements from the MCP lifecycle spec
and HTTP transport spec. The existing server-initialize scenario only verifies that
initialization succeeds; it does not check which version is echoed back or how the
server handles an unsupported version.

Spec text (2025-11-25):

Sec.3.1 Version Negotiation

"If the server supports the requested protocol version, it MUST respond with the same version."
"Otherwise, the server MUST respond with another protocol version it supports."

Sec.2.3 Transport / Protocol Version Header

"If using HTTP, the client MUST include the MCP-Protocol-Version: HTTP header on all subsequent requests to the MCP server."

What This PR Tests

One ClientScenario (server-version-negotiation) with three checks:

Check ID Spec keyword What it tests Failure mode caught
version-echo MUST Client sends 2025-11-25 -- server must echo same version Server responds with wrong/missing version
version-negotiate MUST Client sends 1999-01-01 -- server must respond with a supported version, not an error Server returns {"error": ...} instead of negotiating
http-protocol-version-header MUST Subsequent ping with MCP-Protocol-Version: <negotiated> header -- server must not reject it Server returns 4xx/5xx on a compliant request

specVersions is set to ['2025-11-25'] only. All three checks send/expect 2025-11-25; listing older versions would produce false FAILUREs on servers that implement only those versions.

Design Decisions

Why raw fetch() instead of connectToServer()

The TypeScript SDK's Client hard-codes the protocol version in every initialize
request and provides no public API to override it. Version negotiation checks
must control protocolVersion directly, so raw fetch() is the only option.

Why a streaming SSE reader

The MCP StreamableHTTP transport responds with Content-Type: text/event-stream
for all POST requests -- including initialize and ping -- and keeps the
connection open indefinitely. Calling response.text() on such a body blocks
forever. readFirstSSEMessage() reads incrementally, extracts the first
data: {...} event, and calls reader.cancel() in a finally block to close
the connection cleanly.

The timeout sentinel is created once before the read loop (not per-chunk) so a
server that trickles data cannot extend the deadline indefinitely.

Three-tier version validation in version-negotiate

The check uses KNOWN_SPEC_VERSIONS (the three published releases) plus a year floor to distinguish three outcomes:

  • FAILURE -- version does not match YYYY-MM-DD format, or year < 2025 (catches echo-back of 1999-01-01)
  • WARNING -- valid date format and year >= 2025, but not in KNOWN_SPEC_VERSIONS (server may be running a newer spec release the harness doesn't know about yet -- update KNOWN_SPEC_VERSIONS to verify fully)
  • SUCCESS -- version is a known published MCP spec release

This avoids both false FAILUREs on servers running a new spec and false SUCCESS on servers that blindly echo back whatever the client sends.

MCP-Protocol-Version header uses the negotiated version

Check 3 sends the version the server actually returned in check 1 (captured in negotiatedVersion), not the hardcoded CURRENT_PROTOCOL_VERSION constant. This ensures the header probe is honest even when check 1 fails with a version mismatch.

Session cleanup

Sessions opened by each check are collected in sessionsToClear and deleted via
DELETE /mcp in an outer finally block to avoid orphaned sessions in the test server.

Files Changed

src/scenarios/server/version-negotiation.ts       <- new (ClientScenario, 3 checks)
src/scenarios/server/version-negotiation.test.ts  <- new (13 unit tests)
src/scenarios/index.ts                            <- +1 import, +1 entry in allClientScenariosList

No changes to everything-server.ts -- the SDK's McpServer already implements
version negotiation correctly, so the scenario passes against the reference server.

How Has This Been Tested?

Manual run against the everything-server (passing case):

$ node dist/index.js server --url http://localhost:3001/mcp --scenario server-version-negotiation --verbose
[
  { "id": "version-echo",   "status": "SUCCESS", "details": { "sentVersion": "2025-11-25", "receivedVersion": "2025-11-25" } },
  { "id": "version-negotiate", "status": "SUCCESS", "details": { "sentVersion": "1999-01-01", "receivedVersion": "2025-11-25" } },
  { "id": "http-protocol-version-header", "status": "SUCCESS", "details": { "sentHeaderValue": "2025-11-25", "httpStatus": 200 } }
]
Passed: 3/3, 0 failed, 0 warnings

Unit tests proving failure paths (version-negotiation.test.ts, 13 tests):

The unit tests stub fetch() and prove the scenario catches broken servers -- not just that it passes on a well-behaved one:

  • version-echo FAILURE: wrong version echoed, JSON-RPC error returned, protocolVersion missing
  • version-negotiate FAILURE: JSON-RPC error returned instead of negotiating, 1999-01-01 echoed back (year < 2025)
  • version-negotiate WARNING: future unknown version 2026-03-15 (valid format, year >= 2025, not in KNOWN_SPEC_VERSIONS)
  • http-protocol-version-header: FAILURE on HTTP 4xx, SKIPPED when server unreachable, SUCCESS on -32601 (ping optional), WARNING on unexpected errors
  • Regression: check 3 sends the server-returned version as MCP-Protocol-Version header, not the hardcoded constant

Full suite: npm test -- 106/106 passed.

Lint/format: npm run lint -- clean. No non-ASCII characters in either new file.

Pre-push hook: both Test and Code Formatting hooks passed.

Breaking Changes

None.

Types of Changes

  • New feature (adds test coverage for existing spec requirements)
  • Bug fix
  • Breaking change
  • Documentation update

Checklist

  • Read the MCP spec sections referenced in the checks
  • Followed scenario design rules: one scenario, multiple checks, same check ID for SUCCESS/FAILURE
  • specReferences included on every check
  • npm run build passes
  • npm run typecheck passes
  • npm test passes (106/106)
  • npm run lint passes
  • Scenario registered in allClientScenariosList under lifecycle scenarios
  • Sessions opened during testing are cleaned up with DELETE /mcp

Additional Context

The version-negotiate check targets the most commonly mis-implemented requirement
in this section. Many servers reject an initialize with an unsupported version via
a JSON-RPC error, whereas the spec requires responding with a supported version instead.
This check catches that pattern in any SDK that gets version negotiation wrong.

@alexdoroshevich alexdoroshevich force-pushed the feat/version-negotiation-102 branch 2 times, most recently from 2e571ae to c408eca Compare April 14, 2026 00:13
Alexander Doroshevich added 2 commits April 13, 2026 20:28
…xtprotocol#102)

Adds ServerVersionNegotiationScenario with three independent checks:

- version-echo: client sends 2025-11-25, server must echo the same version
- version-negotiate: client sends 1999-01-01, server must respond with a
  supported version (not a JSON-RPC error). Reports WARNING for unrecognised
  future versions to avoid false failures on servers running newer spec releases.
- http-protocol-version-header: subsequent ping carrying the negotiated
  MCP-Protocol-Version header must be accepted (2xx or -32601).

Why raw fetch() instead of connectToServer(): the SDK Client hard-codes the
protocol version with no public override API.

Why a streaming SSE reader: StreamableHTTP keeps the connection open
indefinitely; response.text() blocks forever. readFirstSSEMessage() reads
incrementally, extracts the first data: event, and calls reader.cancel() in
a finally block to close the connection cleanly. The timeout sentinel is
created once before the read loop so a slow server cannot extend the deadline.

check1Threw gates check 3 as SKIPPED (not FAILURE) when the server is
unreachable. negotiatedVersion captures the server-returned version from
check 1 for use as the MCP-Protocol-Version header in check 3.

specVersions is restricted to ['2025-11-25']: all checks send/expect that
version; listing older versions would cause false FAILUREs on servers that
implement only those versions.
13 tests covering all check outcomes without a real server (fetch stubbed
with vi.stubGlobal):

version-echo: SUCCESS (happy path), FAILURE (wrong version, JSON-RPC error,
  missing protocolVersion field)
version-negotiate: FAILURE (JSON-RPC error, 1999-01-01 echo-back),
  WARNING (2026-03-15 -- valid format but not a known spec release)
http-protocol-version-header: SUCCESS (-32601 method not found),
  FAILURE (HTTP 4xx), SKIPPED (transport error, fetch called exactly twice),
  WARNING (unexpected JSON-RPC error, unparseable 2xx body)
Regression: check 3 sends the version returned by check 1 as the
  MCP-Protocol-Version header, not the hardcoded CURRENT_PROTOCOL_VERSION
  constant.
@alexdoroshevich alexdoroshevich force-pushed the feat/version-negotiation-102 branch from 8a87a66 to eaffb31 Compare April 14, 2026 03:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant