Skip to content

Python SDK — Tier 1 Assessment #2304

Description

@felixweinberger

Python SDK — Tier 1 Assessment

SDK: modelcontextprotocol/python-sdk
Version: 1.26.0 (branch: v1.x)
Assessed: 2026-02-23
Requested tier: Tier 1
Tool: @modelcontextprotocol/conformance tier-check

Result

All Tier 1 requirements are met.

# Requirement Tier 1 Standard Value Status
1 Server Conformance 100% pass rate 30/30 (100%) ✅
2 Client Conformance 100% pass rate 20/20 (100%) ✅
3 Issue Triage Within 2 business days 96.8% (241/249) ✅
4 P0 Resolution Within 7 days 0 open ✅
5 Stable Release Required + clear versioning 1.26.0 ✅
6 Spec Tracking Before spec release 7-day gap (evidence) ✅
7 Documentation Comprehensive w/ examples 48/48 features ✅
8 Dependency Policy Published update policy DEPENDENCY_POLICY.md ✅
9 Roadmap Published roadmap ROADMAP.md ✅
10 Versioning Policy Documented breaking changes VERSIONING.md ✅

Requirements Detail

1. Server Conformance — ✅ 30/30 (100%)

Tested against spec versions 2025-06-18 and 2025-11-25 using the conformance test suite. CI runs continuously via the conformance workflow.

Evidence: Full assessment report · CLI output · How to run it yourself

All 30 server scenarios
Scenario Status Checks Spec Versions
server-server-initialize PASS 1/1 2025-06-18, 2025-11-25
server-tools-list PASS 1/1 2025-06-18, 2025-11-25
server-tools-call-simple-text PASS 1/1 2025-06-18, 2025-11-25
server-tools-call-image PASS 1/1 2025-06-18, 2025-11-25
server-tools-call-audio PASS 1/1 2025-06-18, 2025-11-25
server-tools-call-embedded-resource PASS 1/1 2025-06-18, 2025-11-25
server-tools-call-mixed-content PASS 1/1 2025-06-18, 2025-11-25
server-tools-call-error PASS 1/1 2025-06-18, 2025-11-25
server-tools-call-with-logging PASS 1/1 2025-06-18, 2025-11-25
server-tools-call-with-progress PASS 1/1 2025-06-18, 2025-11-25
server-tools-call-sampling PASS 1/1 2025-06-18, 2025-11-25
server-tools-call-elicitation PASS 1/1 2025-06-18, 2025-11-25
server-server-sse-multiple-streams PASS 2/2 2025-11-25
server-resources-list PASS 1/1 2025-06-18, 2025-11-25
server-resources-read-text PASS 1/1 2025-06-18, 2025-11-25
server-resources-read-binary PASS 1/1 2025-06-18, 2025-11-25
server-resources-templates-read PASS 1/1 2025-06-18, 2025-11-25
server-resources-subscribe PASS 1/1 2025-06-18, 2025-11-25
server-resources-unsubscribe PASS 1/1 2025-06-18, 2025-11-25
server-prompts-list PASS 1/1 2025-06-18, 2025-11-25
server-prompts-get-simple PASS 1/1 2025-06-18, 2025-11-25
server-prompts-get-with-args PASS 1/1 2025-06-18, 2025-11-25
server-prompts-get-embedded-resource PASS 1/1 2025-06-18, 2025-11-25
server-prompts-get-with-image PASS 1/1 2025-06-18, 2025-11-25
server-ping PASS 1/1 2025-06-18, 2025-11-25
server-logging-set-level PASS 1/1 2025-06-18, 2025-11-25
server-completion-complete PASS 1/1 2025-06-18, 2025-11-25
server-elicitation-sep1034-defaults PASS 5/5 2025-11-25
server-elicitation-sep1330-enums PASS 5/5 2025-11-25
server-dns-rebinding-protection PASS 2/2 2025-11-25

2. Client Conformance — ✅ 20/20 (100%)

Core: 4/4, Auth: 16/16 scored. Two informational scenarios fail (auth/resource-mismatch on draft, auth/cross-app-access-complete-flow on extension) — not scored for tier per conformance testing rules.

Evidence: Full assessment report · CLI output · How to run it yourself

All 24 client scenarios (20 scored + 4 informational)
Scenario Status Checks Spec Versions Notes
initialize PASS 1/1 2025-06-18, 2025-11-25
tools_call PASS 1/1 2025-06-18, 2025-11-25
sse-retry PASS 3/3 2025-11-25
elicitation-sep1034-client-defaults PASS 5/5 2025-11-25
auth/basic-cimd PASS 13/13 2025-11-25
auth/pre-registration PASS 13/13 2025-11-25
auth/metadata-default PASS 13/13 2025-11-25
auth/metadata-var1 PASS 13/13 2025-11-25
auth/metadata-var2 PASS 13/13 2025-11-25
auth/metadata-var3 PASS 13/13 2025-11-25
auth/token-endpoint-auth-post PASS 18/18 2025-06-18, 2025-11-25
auth/token-endpoint-auth-basic PASS 18/18 2025-06-18, 2025-11-25
auth/token-endpoint-auth-none PASS 18/18 2025-06-18, 2025-11-25
auth/scope-from-scopes-supported PASS 14/14 2025-11-25
auth/scope-from-www-authenticate PASS 14/14 2025-11-25
auth/scope-omitted-when-undefined PASS 14/14 2025-11-25
auth/scope-step-up PASS 20/20 2025-11-25
auth/scope-retry-limit PASS 10/10 2025-11-25
auth/2025-03-26-oauth-metadata-backcompat PASS 12/12 2025-03-26
auth/2025-03-26-oauth-endpoint-fallback PASS 7/7 2025-03-26
auth/resource-mismatch FAIL 13/14 draft Informational
auth/client-credentials-basic PASS 8/8 extension Informational
auth/client-credentials-jwt PASS 8/8 extension Informational
auth/cross-app-access-complete-flow FAIL 8/10 extension Informational — not scored

3. Issue Triage — ✅ 96.8% (241/249)

241 of 249 issues triaged within 2 business days. All 12 required labels are present.

Evidence: Open issues without any triage label

4. P0 Resolution — ✅ 0 open

No open P0 issues. 1 total P0 was closed within 7 days.

Evidence: Open P0 issues (should show 0 results)

5. Stable Release — ✅ 1.26.0

Published on PyPI and GitHub Releases. Breaking change policy documented in VERSIONING.md — SemVer 2.0.0 with enumerated breaking change categories.

6. Spec Tracking — ✅ 7-day gap

SDK v1.23.0 released 2025-12-02 13:28 UTC, 7 days after the spec 2025-11-25 tag (21:17 UTC).

7. Documentation — ✅ 48/48 features

All 48 non-experimental MCP features documented with code examples across docs/:

Full feature coverage table (48/48)
# Feature Location Examples
1 Tools - listing docs/server.md 2
2 Tools - calling docs/server.md, docs/client.md 2
3 Tools - text results docs/server.md, docs/client.md 2
4 Tools - image results docs/server.md 1
5 Tools - audio results docs/server.md 1
6 Tools - embedded resources docs/server.md, docs/client.md 2
7 Tools - error handling docs/server.md 3
8 Tools - change notifications docs/server.md 1
9 Resources - listing docs/server.md, docs/client.md 2
10 Resources - reading text docs/server.md, docs/client.md 2
11 Resources - reading binary docs/server.md 1
12 Resources - templates docs/server.md 1
13 Resources - template reading docs/server.md 1
14 Resources - subscribing docs/server.md 1
15 Resources - unsubscribing docs/server.md 1
16 Resources - change notifications docs/server.md 2
17 Prompts - listing docs/server.md, docs/client.md 2
18 Prompts - getting simple docs/server.md, docs/client.md 2
19 Prompts - getting with arguments docs/server.md, docs/client.md 2
20 Prompts - embedded resources docs/server.md 1
21 Prompts - image content docs/server.md 1
22 Prompts - change notifications docs/server.md 1
23 Sampling - creating messages docs/server.md, docs/client.md 2
24 Elicitation - form mode docs/server.md 1
25 Elicitation - URL mode docs/server.md 1
26 Elicitation - schema validation docs/server.md 1
27 Elicitation - default values docs/server.md 1
28 Elicitation - enum values docs/server.md 1
29 Elicitation - complete notification docs/server.md 1
30 Roots - listing docs/client.md 1
31 Roots - change notifications docs/client.md 1
32 Logging - sending log messages docs/server.md, docs/client.md 2
33 Logging - setting level docs/server.md, docs/client.md 2
34 Completions - resource argument docs/server.md 1
35 Completions - prompt argument docs/server.md 1
36 Ping docs/protocol.md, docs/client.md 2
37 Streamable HTTP transport (client) docs/client.md 1
38 Streamable HTTP transport (server) docs/server.md 3
39 SSE transport - legacy (client) docs/client.md 1
40 SSE transport - legacy (server) docs/server.md 1
41 stdio transport (client) docs/client.md 1
42 stdio transport (server) docs/server.md, docs/low-level-server.md 2
43 Progress notifications docs/server.md 1
44 Cancellation docs/protocol.md 1
45 Pagination docs/protocol.md, docs/low-level-server.md 2
46 Capability negotiation docs/protocol.md 1
47 Protocol version negotiation docs/protocol.md 1
48 JSON Schema 2020-12 support docs/protocol.md 1

8. Dependency Policy — ✅

DEPENDENCY_POLICY.md — Conservative update philosophy: dependencies updated only for security vulnerabilities, bugs, or needed features. GitHub security alerts enabled. Dependabot configured for automated updates.

9. Roadmap — ✅

ROADMAP.md — Spec implementation tracked via per-revision GitHub Project boards. v2 SDK milestones documented with target dates.

10. Versioning Policy — ✅

VERSIONING.md — SemVer 2.0.0 adherence. Enumerates breaking and non-breaking change categories. Communication via changelog, deprecation warnings, migration guides, and PR labels.


Raw Assessment Output


Reproduce This Assessment

Step-by-step instructions

Prerequisites

  • Node.js >= 20
  • Python >= 3.10 with uv
  • GitHub CLI (gh) authenticated (gh auth login)
  • Git

Quick check (repository health only, ~30 seconds)

git clone https://github.com/modelcontextprotocol/conformance.git
cd conformance
npm install
npm run build
npm run --silent tier-check -- --repo modelcontextprotocol/python-sdk --branch v1.x --skip-conformance

Full check (with conformance tests, ~5 minutes)

# Terminal 1: Clone and start the SDK's conformance server
git clone https://github.com/modelcontextprotocol/python-sdk.git
cd python-sdk
git checkout v1.x
uv sync --frozen --package mcp-everything-server
uv run --frozen mcp-everything-server --port 3001
# Server starts on http://localhost:3001/mcp

# Terminal 2: Run the full tier-check
cd conformance
npm run --silent tier-check -- \
  --repo modelcontextprotocol/python-sdk \
  --branch v1.x \
  --conformance-server-url http://localhost:3001/mcp \
  --client-cmd 'uv run --frozen python ../python-sdk/.github/actions/conformance/client.py'

Output formats

# Terminal output (default)
npm run --silent tier-check -- --repo modelcontextprotocol/python-sdk --skip-conformance

# Markdown report
npm run --silent tier-check -- --repo modelcontextprotocol/python-sdk --skip-conformance --output markdown

# Machine-readable JSON
npm run --silent tier-check -- --repo modelcontextprotocol/python-sdk --skip-conformance --output json

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions