Conformance Tests for MCP (github.com)

The repository provides a conformance test framework for MCP (Model Context Protocol) client implementations. It’s designed to validate client behavior against the MCP spec by launching a test server for a chosen scenario, running your client against that server, capturing protocol interactions, and executing a suite of automated conformance checks. This helps implementers catch interoperability and spec-compliance bugs early, accelerate debugging with reproducible traces, and drive consistent client implementations across the ecosystem. Note: the project is a work in progress and unstable — join the #conformance-testing-wg on the MCP Contributors Discord to follow development. Technically, the framework is an npm project (npm install) that runs with commands like npm run start -- --command "<client-command>" --scenario <scenario-name> (the framework appends the server URL as the final CLI argument). You can also run a test server-only mode (npm run test:server -- --server-url <url> --all). Tests spawn a scenario-specific server, execute your client, capture stdout/stderr and the MCP transcript, then produce results in results/<scenario>-<timestamp>/ including checks.json (array of pass/fail checks), stdout.txt, and stderr.txt. Key code organization: Runner (src/runner) orchestrates execution, Scenarios (src/scenarios) host server behaviors and getChecks(), Checks (src/checks.ts) implement validation logic, and Types (src/types.ts) share types. Examples include a valid TypeScript client (passes) and a broken one (fails). To add scenarios, implement the Scenario interface (start/stop/getChecks) in src/scenarios and register it in src/scenarios/index.ts; see src/runner/DESIGN.md for architecture details.