{"_id":"@cahlan/mcp-test","_rev":"2-21c62639f920ca44554fcd54ed2900df","name":"@cahlan/mcp-test","dist-tags":{"latest":"0.2.0"},"versions":{"0.1.0":{"name":"@cahlan/mcp-test","version":"0.1.0","keywords":["mcp","model-context-protocol","testing","compliance","protocol","cli"],"license":"MIT","_id":"@cahlan/mcp-test@0.1.0","maintainers":[{"name":"cahlan","email":"cahlan@gmail.com"}],"homepage":"https://github.com/cahlan/mcp-test","bugs":{"url":"https://github.com/cahlan/mcp-test/issues"},"bin":{"mcp-test":"dist/index.js"},"dist":{"shasum":"b35334cbd2c144bd2a8efa8ba37e853b2ac25c97","tarball":"https://registry.npmjs.org/@cahlan/mcp-test/-/mcp-test-0.1.0.tgz","fileCount":48,"integrity":"sha512-vv83uvyvW25j/WrVcHgwPKqenkucp7GIiPDA4YPEDJXLd8mofTxiNH24nH5F7Ohoc0iDExD+ErRVZGkUAlbqLw==","signatures":[{"sig":"MEQCIGoBMbmRJ2Ha6ZmM2A/fKi15oYjvZ9IOHODLBRm9umf+AiB8LSp3VWxVv3JSEBQT65emQdzDMPrEN0oLoTkHlsnSFA==","keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U"}],"unpackedSize":197260},"main":"./dist/index.js","type":"module","types":"./dist/index.d.ts","engines":{"node":">=18"},"gitHead":"08c163d60ffd0551a13efc966244fc59f04058ae","scripts":{"dev":"tsx src/index.ts","test":"vitest","build":"tsc","prepublishOnly":"npm run build"},"_npmUser":{"name":"cahlan","email":"cahlan@gmail.com"},"repository":{"url":"git+https://github.com/cahlan/mcp-test.git","type":"git"},"_npmVersion":"10.9.4","description":"Protocol compliance testing CLI for MCP servers","directories":{},"_nodeVersion":"22.22.0","dependencies":{"zod":"^3.24.2","chalk":"^5.4.1","js-yaml":"^4.1.0","commander":"^12.1.0","@modelcontextprotocol/sdk":"^1.12.1"},"_hasShrinkwrap":false,"devDependencies":{"tsx":"^4.19.2","vitest":"^2.1.8","typescript":"^5.7.3","@types/node":"^22.13.4","@types/js-yaml":"^4.0.9"},"_npmOperationalInternal":{"tmp":"tmp/mcp-test_0.1.0_1771511905340_0.6621410078679624","host":"s3://npm-registry-packages-npm-production"}},"0.2.0":{"name":"@cahlan/mcp-test","version":"0.2.0","description":"Protocol compliance testing CLI for MCP servers","main":"./dist/index.js","bin":{"mcp-test":"dist/index.js"},"type":"module","scripts":{"build":"tsc","dev":"tsx src/index.ts","test":"vitest","prepublishOnly":"npm run build"},"keywords":["mcp","model-context-protocol","testing","compliance","protocol","cli"],"license":"MIT","repository":{"type":"git","url":"git+https://github.com/cahlan/mcp-test.git"},"homepage":"https://github.com/cahlan/mcp-test","bugs":{"url":"https://github.com/cahlan/mcp-test/issues"},"dependencies":{"@modelcontextprotocol/sdk":"^1.12.1","chalk":"^5.4.1","commander":"^12.1.0","js-yaml":"^4.1.0","zod":"^3.24.2"},"devDependencies":{"@types/js-yaml":"^4.0.9","@types/node":"^22.13.4","tsx":"^4.19.2","typescript":"^5.7.3","vitest":"^2.1.8"},"engines":{"node":">=18"},"_id":"@cahlan/mcp-test@0.2.0","gitHead":"0ae46ec9c2329dca502beed7f1e5fa0a227f8536","types":"./dist/index.d.ts","_nodeVersion":"22.22.0","_npmVersion":"10.9.4","dist":{"integrity":"sha512-i9HHEjs4r+YIDaXMEJ7M+9Mruh30VQ2mCNQst8BcSt3TgwcEm3qTu37rvy7Qi48zWQzcU92s1M55Lomgs9tOhw==","shasum":"9bd31a36776922386c90554dbfbfbd95c290b58e","tarball":"https://registry.npmjs.org/@cahlan/mcp-test/-/mcp-test-0.2.0.tgz","fileCount":52,"unpackedSize":230257,"signatures":[{"keyid":"SHA256:DhQ8wR5APBvFHLF/+Tc+AYvPOdTpcIDqOhxsBHRwC7U","sig":"MEUCIHr9lmcatsBeZbdpNkXcUSefNZgg7ZzdeiJqsjFk1dLJAiEAhFyF1XEi5eCY9FrE3CNupo93PX1cOgArH8TCVWtqloM="}]},"_npmUser":{"name":"cahlan","email":"cahlan@gmail.com"},"directories":{},"maintainers":[{"name":"cahlan","email":"cahlan@gmail.com"}],"_npmOperationalInternal":{"host":"s3://npm-registry-packages-npm-production","tmp":"tmp/mcp-test_0.2.0_1771602662014_0.207600011260185"},"_hasShrinkwrap":false}},"time":{"created":"2026-02-19T14:38:25.288Z","modified":"2026-02-20T15:51:02.312Z","0.1.0":"2026-02-19T14:38:25.495Z","0.2.0":"2026-02-20T15:51:02.166Z"},"bugs":{"url":"https://github.com/cahlan/mcp-test/issues"},"license":"MIT","homepage":"https://github.com/cahlan/mcp-test","keywords":["mcp","model-context-protocol","testing","compliance","protocol","cli"],"repository":{"type":"git","url":"git+https://github.com/cahlan/mcp-test.git"},"description":"Protocol compliance testing CLI for MCP servers","maintainers":[{"name":"cahlan","email":"cahlan@gmail.com"}],"readme":"# mcp-test\n\n[![MCP Compliance](https://img.shields.io/badge/MCP-compliant-green)](https://github.com/cahlan/mcp-test)\n[![npm version](https://img.shields.io/npm/v/mcp-test)](https://www.npmjs.com/package/mcp-test)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n\n**Protocol compliance testing for MCP servers.**\n\nThe MCP ecosystem has hundreds of servers. No one knows if they actually follow the spec. Until now.\n\n`mcp-test` runs 24 automated compliance checks against any MCP server, validating the initialization handshake, version negotiation, tool/resource/prompt schemas, error handling, and JSON-RPC correctness — all against the [official MCP specification](https://modelcontextprotocol.io/specification/2025-03-26). Supports both **stdio** and **Streamable HTTP** transports.\n\n## Quick Start\n\n```bash\n# Test a stdio server\nnpx @cahlan/mcp-test run --server \"node my-server.js\"\n\n# Test an HTTP server (Streamable HTTP transport)\nnpx @cahlan/mcp-test run --url http://localhost:3000\n```\n\nThat's it. One command. Works with any MCP server — stdio or HTTP.\n\n## What It Looks Like\n\n```\nmcp-test v0.1.0 — Protocol Compliance Suite\nTesting: node my-server.js\n────────────────────────────────────────────────────────────\n\n  Lifecycle\n    ✓ lifecycle-001    Initialize handshake completes successfully\n    ✓ lifecycle-002    Server accepts initialized notification\n    ✓ lifecycle-003    Server response contains valid serverInfo\n    ✓ lifecycle-004    Server responds to ping after initialization\n\n  Versioning\n    ✓ version-001      Server returns a valid protocol version\n    ✓ version-002      Server capabilities object is well-formed\n\n  Tools\n    ✓ tools-001        tools/list returns valid tool array\n    ✓ tools-002        Tool inputSchema is valid JSON Schema\n    ✓ tools-003        tools/call returns valid content result\n    ✓ tools-004        tools/call with unknown tool returns error\n    ─ tools-005        tools/list supports cursor-based pagination  skipped\n\n  ...\n\n────────────────────────────────────────────────────────────\nResults: 20 passed · 4 skipped\nCompliance Score: 100% (critical: 100% · major: 100%)\n```\n\nBy default, only failures are shown. Use `-v` for the full view above.\n\n## Installation\n\n```bash\n# Run directly with npx (no install needed)\nnpx @cahlan/mcp-test run --server \"node my-server.js\"\nnpx @cahlan/mcp-test run --url http://localhost:3000\n\n# Or install globally\nnpm install -g mcp-test\nmcp-test run --server \"python my_server.py\"\nmcp-test run --url http://localhost:3000\n```\n\n## Usage\n\n### Run compliance tests\n\n```bash\n# Basic usage — stdio transport\nmcp-test run --server \"node dist/server.js\"\n\n# HTTP transport (Streamable HTTP)\nmcp-test run --url http://localhost:3000\n\n# Show all tests (not just failures)\nmcp-test run --server \"node dist/server.js\" --verbose\n\n# TAP output for CI systems\nmcp-test run --server \"node dist/server.js\" --output tap\n\n# JSON output for programmatic use\nmcp-test run --server \"node dist/server.js\" --output json\n\n# Only test specific categories\nmcp-test run --server \"node dist/server.js\" --filter tools\nmcp-test run --server \"node dist/server.js\" --filter lifecycle\n\n# Fail CI if any major or higher test fails\nmcp-test run --server \"node dist/server.js\" --fail-on major\n\n# Custom timeout per test\nmcp-test run --server \"node dist/server.js\" --timeout 10000\n```\n\n### List available tests\n\n```bash\nmcp-test list\nmcp-test list --filter errors\n```\n\n## What It Checks\n\n24 tests across 7 categories:\n\n| Category | Tests | What it validates |\n|----------|-------|-------------------|\n| **Lifecycle** | 4 | Initialize handshake, `initialized` notification, serverInfo structure, ping |\n| **Versioning** | 2 | Protocol version format (YYYY-MM-DD), capabilities object structure |\n| **Tools** | 5 | `tools/list` array, inputSchema validation, `tools/call` content, unknown tool error, pagination |\n| **Resources** | 5 | `resources/list` array, `resources/read` contents, invalid URI error, pagination, templates |\n| **Prompts** | 3 | `prompts/list` array, `prompts/get` messages, pagination |\n| **Errors** | 3 | Unknown method error, malformed request handling, error response structure |\n| **JSON-RPC** | 2 | Response ID matching, concurrent request handling |\n\n### Severity Levels\n\n- 🔴 **Critical** — MUST requirements. Server is broken without these.\n- 🟡 **Major** — MUST/SHOULD requirements. Important for interoperability.\n- 🟢 **Minor** — SHOULD/MAY requirements. Nice to have.\n\n### Smart Skipping\n\nTests are automatically skipped when they don't apply:\n- Tool tests skip if the server doesn't declare `tools` capability\n- Resource tests skip if the server doesn't declare `resources` capability\n- Pagination tests skip if the server doesn't return `nextCursor`\n\n## Transports\n\n`mcp-test` supports two MCP transport mechanisms:\n\n### stdio (default)\n\nSpawns the server as a subprocess and communicates over stdin/stdout:\n\n```bash\nnpx @cahlan/mcp-test run --server \"node my-server.js\"\nnpx @cahlan/mcp-test run --server \"python my_server.py\"\n```\n\n### Streamable HTTP\n\nConnects to a running MCP HTTP server. Start your server first, then point `mcp-test` at it:\n\n```bash\nnpx @cahlan/mcp-test run --url http://localhost:3000\n```\n\nThe tool will POST JSON-RPC requests to `{url}/mcp` following the [MCP Streamable HTTP transport spec](https://spec.modelcontextprotocol.io/specification/basic/transports/#streamable-http). Features:\n\n- Automatic SSE and plain JSON response handling\n- Session management via `mcp-session-id` header\n- Session termination via HTTP DELETE on close\n\n> **Note:** `--server` and `--url` are mutually exclusive — use one or the other.\n\n## CI Integration\n\n### GitHub Actions\n\nAdd this to your MCP server repo:\n\n```yaml\n# .github/workflows/mcp-compliance.yml\nname: MCP Protocol Compliance\n\non:\n  push:\n    branches: [main]\n  pull_request:\n\njobs:\n  compliance:\n    name: MCP Compliance Check\n    runs-on: ubuntu-latest\n    steps:\n      - uses: actions/checkout@v4\n      - uses: actions/setup-node@v4\n        with:\n          node-version: '20'\n      - run: npm ci\n      - run: npm run build\n      - name: Run MCP compliance tests\n        run: npx @cahlan/mcp-test@latest run --server \"node dist/server.js\" --output tap --fail-on critical\n```\n\n### TAP Output\n\nUse `--output tap` for [TAP](https://testanything.org/) format, supported natively by most CI systems:\n\n```\nTAP version 14\n1..24\nok 1 - lifecycle-001 Initialize handshake completes successfully\nok 2 - lifecycle-002 Server accepts initialized notification\nnot ok 3 - tools-001 tools/list returns valid tool array\n  ---\n  message: tools/list result must contain a \"tools\" array\n  severity: critical\n  ...\n# Tests: 24, Passed: 20, Failed: 1, Errors: 0, Skipped: 3\n# Compliance Score: 95%\n```\n\n### Exit Codes\n\n| Code | Meaning |\n|------|---------|\n| `0` | All tests passed (at the configured `--fail-on` severity level) |\n| `1` | One or more tests failed at or above the `--fail-on` threshold |\n| `2` | Fatal error (server wouldn't start, bad arguments, etc.) |\n\n## Compliance Score\n\nThe compliance score is the percentage of **critical + major** tests that pass (skipped tests don't count). A server can score 100% even if minor tests are skipped.\n\n```\nCompliance Score: 92% (critical: 100% · major: 83%)\n```\n\n## Found a Bug in Your Server?\n\nCheck the [MCP specification](https://modelcontextprotocol.io/specification/2025-03-26) for the authoritative requirements. Each test links to the relevant spec section.\n\nCommon issues we've found:\n- **tools/list returns object instead of array** — `{ tools: { myTool: ... } }` instead of `{ tools: [...] }`\n- **Error codes are strings** — JSON-RPC requires integer error codes (`-32601`, not `\"NOT_FOUND\"`)\n- **Missing `jsonrpc: \"2.0\"` in responses** — every JSON-RPC message must include this field\n- **Ping response ID mismatch** — response ID must match the request ID exactly\n\n## Badge\n\nShow your server is compliant:\n\n```markdown\n[![MCP Compliance](https://img.shields.io/badge/MCP-compliant-green)](https://github.com/cahlan/mcp-test)\n```\n\n## Contributing\n\nFound a spec requirement we're not testing? [Open an issue](https://github.com/cahlan/mcp-test/issues).\n\n## License\n\nMIT\n","readmeFilename":"README.md"}