fix(mcp): fail closed on missing upstream credentials by joshua-berri · Pull Request #41364 · BerriAI/litellm

joshua-berri · 2026-09-16T03:16:10Z

TLDR

Problem this solves:

Incomplete OBO settings can bypass the required token exchange
Static-auth requests can proceed without a usable credential

How it solves it:

Require OBO token exchange whenever the server selects OBO
Incomplete OBO client credentials error instead of falling back to another auth mode
Validate effective credentials before protected requests reach upstream
Native MCP checks its prepared request, OpenAPI checks the merged headers before HTTP
Placeholder values like Bearer Bearer or a bare Basic fail validation
Preserve existing header precedence and supported caller credentials

User Flow

Before: selecting OBO can still send an unauthenticated tool request

Register an HTTP MCP server with POST http://localhost:4000/v1/mcp/server and auth_type none
Request GET http://localhost:4000/mcp-rest/tools/list?server_id=ID and see echo listed
Update it with PUT http://localhost:4000/v1/mcp/server to auth_type oauth2_token_exchange, supplying client_id but no client_secret
Repeat the list request and call echo twice with POST http://localhost:4000/mcp-rest/tools/call; all three return 200 and the upstream executes without authentication

After: selecting OBO requires authentication before upstream discovery or execution

Register the same HTTP MCP server with POST http://localhost:4000/v1/mcp/server and auth_type none
Request GET http://localhost:4000/mcp-rest/tools/list?server_id=ID and see echo listed
Update it with PUT http://localhost:4000/v1/mcp/server to auth_type oauth2_token_exchange, supplying client_id but no client_secret
Repeat the list request and call echo twice with POST http://localhost:4000/mcp-rest/tools/call; all three return 401 and no request reaches upstream

Relevant issues

Affected release

Linear ticket

Resolves LIT-4501

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

I have added meaningful tests
The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
My PR's scope is as isolated as possible; it only solves 1 specific problem
I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Two DB-backed proxies, one per commit, each one process started with --num_workers 2 and proxy_config_reload_interval_seconds: 5 so both workers pick up registry changes from the database (SYNC_WAIT=8 below waits 8s after every registry write). Before is the merge base 9cd7873 on 127.0.0.1:25002 with database litellm_lit7914_before, After is this PR's head 70ef8b2 on 127.0.0.1:45138 with database litellm_lit7914_after. The upstream is a local FastMCP echo server per leg (127.0.0.1:51521 for Before, 127.0.0.1:23920 for After) whose one addition is a middleware that appends the auth headers of every request it receives to a JSONL file, the same thing the upstream owner would read off their access log. The admin key drives the management routes and is also the caller on the MCP protocol probe, so the incomplete OBO case carries no user subject to exchange and lands on the 401 branch. The author's earlier run at this head covered the other branch with a Keycloak user JWT: incomplete OBO returned 500 token_exchange requires client_id and client_secret with zero upstream requests where the merge base returned 200

Each case registers the server with auth_type: none, lists once so the tool is known, updates the server to the auth under test, waits for both workers to reload, then lists, calls echo twice over /mcp-rest, and finally lists and calls through the gateway's /mcp endpoint with the mcp SDK client. After every step the echo server's log is diffed to show what actually reached upstream

Proxy config for both legs

general_settings:
  store_model_in_db: true
  proxy_config_reload_interval_seconds: 5

Upstream echo MCP server, python echo_mcp_server.py PORT LOG (mcp 1.28.1)

import json
import sys
import time
from pathlib import Path

import uvicorn
from mcp.server.fastmcp import FastMCP
from starlette.middleware.base import BaseHTTPMiddleware

PORT = int(sys.argv[1])
LOG = Path(sys.argv[2])
WATCHED = ("authorization", "x-api-key", "x-custom")

server = FastMCP("echo", host="127.0.0.1", port=PORT, stateless_http=True, json_response=True)


@server.tool()
def echo(message: str) -> str:
    return f"echo: {message}"


class RecordAuthHeaders(BaseHTTPMiddleware):
    async def dispatch(self, request, call_next):
        body = await request.body()
        method = None
        try:
            method = json.loads(body).get("method") if body else None
        except ValueError:
            method = None
        entry = {
            "t": round(time.time(), 3),
            "path": request.url.path,
            "mcp_method": method,
            "headers": {name: request.headers.get(name) for name in WATCHED if name in request.headers},
        }
        with LOG.open("a") as f:
            f.write(json.dumps(entry) + "\n")
        return await call_next(request)


app = server.streamable_http_app()
app.add_middleware(RecordAuthHeaders)
uvicorn.run(app, host="127.0.0.1", port=PORT, log_level="warning")

MCP protocol client used at the end of each case, python mcp_client_probe.py BASE_URL ALIAS (mcp 1.28.1)

import asyncio
import json
import os
import sys

from mcp import ClientSession
from mcp.client.streamable_http import streamablehttp_client

URL = sys.argv[1] + "/mcp"
ALIAS = sys.argv[2]
HEADERS = {"Authorization": f"Bearer {os.environ['LITELLM_MASTER_KEY']}", "x-mcp-servers": ALIAS}


def describe(exc: BaseException) -> str:
    inner = getattr(exc, "exceptions", None)
    if inner:
        return "; ".join(describe(e) for e in inner)
    return f"{type(exc).__name__}: {str(exc)[:300]}"


async def main() -> None:
    try:
        async with streamablehttp_client(URL, headers=HEADERS) as (read, write, _):
            async with ClientSession(read, write) as session:
                await session.initialize()
                listed = await session.list_tools()
                names = [t.name for t in listed.tools]
                print("mcp client tools/list:", names)
                echo = [n for n in names if n.endswith("echo")]
                if not echo:
                    return
                result = await session.call_tool(echo[0], {"message": f"{ALIAS}_mcp_client"})
                print("mcp client tools/call isError:", result.isError, "|", json.dumps([c.model_dump() for c in result.content])[:300])
    except BaseException as exc:
        print("mcp client error:", describe(exc))


asyncio.run(main())

Commands (zsh, with PORT, ECHO_PORT, ECHO_LOG, SYNC_WAIT=8 and LITELLM_MASTER_KEY set per leg)

BASE_URL=http://127.0.0.1:$PORT
HEADERS=(-H "Authorization: Bearer $LITELLM_MASTER_KEY" -H 'Content-Type: application/json')
MARK=0
mark() { MARK=$(wc -l < $ECHO_LOG 2>/dev/null | tr -d ' '); MARK=${MARK:-0} }
upstream() {
  tail -n +$((MARK+1)) $ECHO_LOG 2>/dev/null > new.jsonl
  echo "upstream requests seen by the echo server during this step: $(wc -l < new.jsonl | tr -d ' ')"
  python - new.jsonl <<'PYEOF'
import collections, json, sys
rows = [json.loads(l) for l in open(sys.argv[1]) if l.strip()]
counts = collections.Counter((r["mcp_method"], ", ".join(f"{k}: {v}" for k, v in r["headers"].items()) or "no auth header") for r in rows)
for (method, headers), n in counts.items():
    print(f"  {n}x {method}: {headers}")
PYEOF
}
show() { jq -c 'if type=="array" then map(.name) else . end' 2>/dev/null | head -c 400; echo }
probe() {
  local NAME=$1 AUTH_JSON=$2
  local ALIAS="${NAME}_srv" MARKER="${NAME}_echo"
  echo; echo "== probe $NAME: $AUTH_JSON"
  REGISTER_JSON=$(jq -nc --arg name "$NAME" --arg alias "$ALIAS" --arg url "http://127.0.0.1:$ECHO_PORT/mcp" \
    '{server_name:$name,alias:$alias,url:$url,transport:"http",auth_type:"none"}')
  curl -sS "${HEADERS[@]}" -X POST "$BASE_URL/v1/mcp/server" --data "$REGISTER_JSON" -o register.json -w 'register %{http_code}\n'
  SERVER_ID=$(jq -er '.server_id' register.json)
  sleep $SYNC_WAIT
  curl -sS "${HEADERS[@]}" "$BASE_URL/mcp-rest/tools/list?server_id=$SERVER_ID" -o warm.json -w 'warm list %{http_code}: '; show < warm.json
  UPDATE_JSON=$(jq -nc --argjson server "$REGISTER_JSON" --arg id "$SERVER_ID" --argjson auth "$AUTH_JSON" '$server + {server_id:$id} + $auth')
  curl -sS "${HEADERS[@]}" -X PUT "$BASE_URL/v1/mcp/server" --data "$UPDATE_JSON" -o update.json -w 'update %{http_code}: '; jq -c '{auth_type, credentials: (.credentials // null | if . == null then null else keys end)}' update.json 2>/dev/null || head -c 300 update.json
  sleep $SYNC_WAIT
  mark
  curl -sS "${HEADERS[@]}" "$BASE_URL/mcp-rest/tools/list?server_id=$SERVER_ID" -o list.json -w 'list %{http_code}: '; show < list.json
  CALL_JSON=$(jq -nc --arg id "$SERVER_ID" --arg marker "$MARKER" '{server_id:$id,name:"echo",arguments:{message:$marker}}')
  for attempt in 1 2; do
    curl -sS "${HEADERS[@]}" -X POST "$BASE_URL/mcp-rest/tools/call" --data "$CALL_JSON" -o call$attempt.json -w "call $attempt %{http_code}: "; head -c 300 call$attempt.json; echo
  done
  upstream
  echo "-- MCP protocol client (mcp SDK streamable HTTP) against $BASE_URL/mcp with x-mcp-servers: $ALIAS"
  mark
  python mcp_client_probe.py $BASE_URL $ALIAS
  upstream
}
OBO=$(jq -nc --arg ep "http://127.0.0.1:$ECHO_PORT/token" '{auth_type:"oauth2_token_exchange",token_exchange_endpoint:$ep,credentials:{client_id:"lit4501-m2m"}}')
NOCRED='{"auth_type":"api_key","credentials":{}}'
WITHCRED='{"auth_type":"api_key","credentials":{"auth_value":"lit7914-fixture-key"}}'
OPEN='{"auth_type":"none"}'
probe obo "$OBO"
probe nocred "$NOCRED"
probe withcred "$WITHCRED"
probe open "$OPEN"

Before (`9cd7873`)

### leg before | commit 9cd787386ea43aa9d6b18d8f31d7528a020a0622
litellm resolves to: /private/tmp/claude-501/-Users-mateo-Development-litellm/eee1f170-5a06-486c-8c5c-4f35e7531475/scratchpad/drivers/LIT-7914/wt-base/litellm/__init__.py
topology: one proxy process on 127.0.0.1:25002 with --num_workers 2, DB litellm_lit7914_before, upstream echo MCP server on 127.0.0.1:51521/mcp
liveliness: "I'm alive!"

== probe obo: {"auth_type":"oauth2_token_exchange","token_exchange_endpoint":"http://127.0.0.1:51521/token","credentials":{"client_id":"lit4501-m2m"}}
register 201
warm list 200: {"tools":[{"name":"echo","title":null,"description":"","inputSchema":{"properties":{"message":{"title":"Message","type":"string"}},"required":["message"],"title":"echoArguments","type":"object"},"outputSchema":null,"icons":null,"annotations":null,"_meta":null,"execution":null,"mcp_info":{"server_name":"obo","server_id":"26129325-f669-42e7-8033-fa365cb4530e","alias":"obo_srv"}}],"error":null,"messa
update 202: {"auth_type":"oauth2_token_exchange","credentials":null}
list 200: {"tools":[{"name":"echo","title":null,"description":"","inputSchema":{"properties":{"message":{"title":"Message","type":"string"}},"required":["message"],"title":"echoArguments","type":"object"},"outputSchema":null,"icons":null,"annotations":null,"_meta":null,"execution":null,"mcp_info":{"server_name":"obo","server_id":"26129325-f669-42e7-8033-fa365cb4530e","alias":"obo_srv"}}],"error":null,"messa
call 1 200: {"_meta":null,"content":[{"type":"text","text":"echo: obo_echo","annotations":null,"_meta":null}],"structuredContent":{"result":"echo: obo_echo"},"isError":false}
call 2 200: {"_meta":null,"content":[{"type":"text","text":"echo: obo_echo","annotations":null,"_meta":null}],"structuredContent":{"result":"echo: obo_echo"},"isError":false}
upstream requests seen by the echo server during this step: 11
  3x initialize: no auth header
  3x notifications/initialized: no auth header
  3x tools/list: no auth header
  2x tools/call: no auth header
-- MCP protocol client (mcp SDK streamable HTTP) against http://127.0.0.1:25002/mcp with x-mcp-servers: obo_srv
mcp client tools/list: ['obo_srv-echo']
mcp client tools/call isError: False | [{"type": "text", "text": "echo: obo_srv_mcp_client", "annotations": null, "meta": null}]
upstream requests seen by the echo server during this step: 7
  2x initialize: no auth header
  2x notifications/initialized: no auth header
  2x tools/list: no auth header
  1x tools/call: no auth header

== probe nocred: {"auth_type":"api_key","credentials":{}}
register 201
warm list 200: {"tools":[{"name":"echo","title":null,"description":"","inputSchema":{"properties":{"message":{"title":"Message","type":"string"}},"required":["message"],"title":"echoArguments","type":"object"},"outputSchema":null,"icons":null,"annotations":null,"_meta":null,"execution":null,"mcp_info":{"server_name":"nocred","server_id":"3f9b805f-3dd7-414a-ae36-0f6e48dd4842","alias":"nocred_srv"}}],"error":null,
update 202: {"auth_type":"api_key","credentials":null}
list 200: {"tools":[{"name":"echo","title":null,"description":"","inputSchema":{"properties":{"message":{"title":"Message","type":"string"}},"required":["message"],"title":"echoArguments","type":"object"},"outputSchema":null,"icons":null,"annotations":null,"_meta":null,"execution":null,"mcp_info":{"server_name":"nocred","server_id":"3f9b805f-3dd7-414a-ae36-0f6e48dd4842","alias":"nocred_srv"}}],"error":null,
call 1 200: {"_meta":null,"content":[{"type":"text","text":"echo: nocred_echo","annotations":null,"_meta":null}],"structuredContent":{"result":"echo: nocred_echo"},"isError":false}
call 2 200: {"_meta":null,"content":[{"type":"text","text":"echo: nocred_echo","annotations":null,"_meta":null}],"structuredContent":{"result":"echo: nocred_echo"},"isError":false}
upstream requests seen by the echo server during this step: 11
  3x initialize: no auth header
  3x notifications/initialized: no auth header
  3x tools/list: no auth header
  2x tools/call: no auth header
-- MCP protocol client (mcp SDK streamable HTTP) against http://127.0.0.1:25002/mcp with x-mcp-servers: nocred_srv
mcp client tools/list: ['nocred_srv-echo']
mcp client tools/call isError: False | [{"type": "text", "text": "echo: nocred_srv_mcp_client", "annotations": null, "meta": null}]
upstream requests seen by the echo server during this step: 7
  2x initialize: no auth header
  2x notifications/initialized: no auth header
  2x tools/list: no auth header
  1x tools/call: no auth header

== probe withcred: {"auth_type":"api_key","credentials":{"auth_value":"lit7914-fixture-key"}}
register 201
warm list 200: {"tools":[{"name":"echo","title":null,"description":"","inputSchema":{"properties":{"message":{"title":"Message","type":"string"}},"required":["message"],"title":"echoArguments","type":"object"},"outputSchema":null,"icons":null,"annotations":null,"_meta":null,"execution":null,"mcp_info":{"server_name":"withcred","server_id":"9cf7bb42-7779-4622-ab6a-fcd7beae1ae4","alias":"withcred_srv"}}],"error":n
update 202: {"auth_type":"api_key","credentials":null}
list 200: {"tools":[{"name":"echo","title":null,"description":"","inputSchema":{"properties":{"message":{"title":"Message","type":"string"}},"required":["message"],"title":"echoArguments","type":"object"},"outputSchema":null,"icons":null,"annotations":null,"_meta":null,"execution":null,"mcp_info":{"server_name":"withcred","server_id":"9cf7bb42-7779-4622-ab6a-fcd7beae1ae4","alias":"withcred_srv"}}],"error":n
call 1 200: {"_meta":null,"content":[{"type":"text","text":"echo: withcred_echo","annotations":null,"_meta":null}],"structuredContent":{"result":"echo: withcred_echo"},"isError":false}
call 2 200: {"_meta":null,"content":[{"type":"text","text":"echo: withcred_echo","annotations":null,"_meta":null}],"structuredContent":{"result":"echo: withcred_echo"},"isError":false}
upstream requests seen by the echo server during this step: 11
  3x initialize: x-api-key: lit7914-fixture-key
  3x notifications/initialized: x-api-key: lit7914-fixture-key
  3x tools/list: x-api-key: lit7914-fixture-key
  2x tools/call: x-api-key: lit7914-fixture-key
-- MCP protocol client (mcp SDK streamable HTTP) against http://127.0.0.1:25002/mcp with x-mcp-servers: withcred_srv
mcp client tools/list: ['withcred_srv-echo']
mcp client tools/call isError: False | [{"type": "text", "text": "echo: withcred_srv_mcp_client", "annotations": null, "meta": null}]
upstream requests seen by the echo server during this step: 12
  5x initialize: x-api-key: lit7914-fixture-key
  3x notifications/initialized: x-api-key: lit7914-fixture-key
  3x tools/list: x-api-key: lit7914-fixture-key
  1x tools/call: x-api-key: lit7914-fixture-key

== probe open: {"auth_type":"none"}
register 201
warm list 200: {"tools":[{"name":"echo","title":null,"description":"","inputSchema":{"properties":{"message":{"title":"Message","type":"string"}},"required":["message"],"title":"echoArguments","type":"object"},"outputSchema":null,"icons":null,"annotations":null,"_meta":null,"execution":null,"mcp_info":{"server_name":"open","server_id":"22e0179c-9ff5-4e14-970b-32614799cb2a","alias":"open_srv"}}],"error":null,"mes
update 202: {"auth_type":"none","credentials":null}
list 200: {"tools":[{"name":"echo","title":null,"description":"","inputSchema":{"properties":{"message":{"title":"Message","type":"string"}},"required":["message"],"title":"echoArguments","type":"object"},"outputSchema":null,"icons":null,"annotations":null,"_meta":null,"execution":null,"mcp_info":{"server_name":"open","server_id":"22e0179c-9ff5-4e14-970b-32614799cb2a","alias":"open_srv"}}],"error":null,"mes
call 1 500: {"detail":{"error":"internal_server_error","message":"An unexpected error occurred: Tool echo not found"}}
call 2 200: {"_meta":null,"content":[{"type":"text","text":"echo: open_echo","annotations":null,"_meta":null}],"structuredContent":{"result":"echo: open_echo"},"isError":false}
upstream requests seen by the echo server during this step: 7
  2x initialize: no auth header
  2x notifications/initialized: no auth header
  2x tools/list: no auth header
  1x tools/call: no auth header
-- MCP protocol client (mcp SDK streamable HTTP) against http://127.0.0.1:25002/mcp with x-mcp-servers: open_srv
mcp client tools/list: ['open_srv-echo']
mcp client tools/call isError: False | [{"type": "text", "text": "echo: open_srv_mcp_client", "annotations": null, "meta": null}]
upstream requests seen by the echo server during this step: 9
  4x initialize: no auth header
  2x notifications/initialized: no auth header
  2x tools/list: no auth header
  1x tools/call: no auth header

### LEG before DONE

After (`70ef8b2`)

### leg after | commit 70ef8b24b6675faa14e7e3ec006d79debe032609
litellm resolves to: /private/tmp/claude-501/-Users-mateo-Development-litellm/eee1f170-5a06-486c-8c5c-4f35e7531475/scratchpad/drivers/LIT-7914/wt/litellm/__init__.py
topology: one proxy process on 127.0.0.1:45138 with --num_workers 2, DB litellm_lit7914_after, upstream echo MCP server on 127.0.0.1:23920/mcp
liveliness: "I'm alive!"

== probe obo: {"auth_type":"oauth2_token_exchange","token_exchange_endpoint":"http://127.0.0.1:23920/token","credentials":{"client_id":"lit4501-m2m"}}
register 201
warm list 200: {"tools":[{"name":"echo","title":null,"description":"","inputSchema":{"properties":{"message":{"title":"Message","type":"string"}},"required":["message"],"title":"echoArguments","type":"object"},"outputSchema":null,"icons":null,"annotations":null,"_meta":null,"execution":null,"mcp_info":{"server_name":"obo","server_id":"493136ef-c43d-4aaa-a749-6481d3195e78","alias":"obo_srv"}}],"error":null,"messa
update 202: {"auth_type":"oauth2_token_exchange","credentials":null}
list 401: {"detail":"Unauthorized"}

call 1 401: {"detail":"Unauthorized"}
call 2 401: {"detail":"Unauthorized"}
upstream requests seen by the echo server during this step: 0
-- MCP protocol client (mcp SDK streamable HTTP) against http://127.0.0.1:45138/mcp with x-mcp-servers: obo_srv
mcp client error: HTTPStatusError: Client error '401 Unauthorized' for url 'http://127.0.0.1:45138/mcp'
upstream requests seen by the echo server during this step: 0

== probe nocred: {"auth_type":"api_key","credentials":{}}
register 201
warm list 200: {"tools":[{"name":"echo","title":null,"description":"","inputSchema":{"properties":{"message":{"title":"Message","type":"string"}},"required":["message"],"title":"echoArguments","type":"object"},"outputSchema":null,"icons":null,"annotations":null,"_meta":null,"execution":null,"mcp_info":{"server_name":"nocred","server_id":"2ac47653-71f0-4e26-a508-4472d56610c4","alias":"nocred_srv"}}],"error":null,
update 202: {"auth_type":"api_key","credentials":null}
list 500: {"detail":{"error":"internal","message":"Failed to list tools from server nocred_srv"}}

call 1 500: {"detail":{"error":"internal_server_error","message":"An unexpected error occurred: Tool echo not found"}}
call 2 500: {"detail":{"error":"internal_server_error","message":"An unexpected error occurred: Tool echo not found"}}
upstream requests seen by the echo server during this step: 0
-- MCP protocol client (mcp SDK streamable HTTP) against http://127.0.0.1:45138/mcp with x-mcp-servers: nocred_srv
mcp client tools/list: []
upstream requests seen by the echo server during this step: 0

== probe withcred: {"auth_type":"api_key","credentials":{"auth_value":"lit7914-fixture-key"}}
register 201
warm list 200: {"tools":[{"name":"echo","title":null,"description":"","inputSchema":{"properties":{"message":{"title":"Message","type":"string"}},"required":["message"],"title":"echoArguments","type":"object"},"outputSchema":null,"icons":null,"annotations":null,"_meta":null,"execution":null,"mcp_info":{"server_name":"withcred","server_id":"e22e2715-aaf2-4ee3-a6ae-1a5e18c4af00","alias":"withcred_srv"}}],"error":n
update 202: {"auth_type":"api_key","credentials":null}
list 200: {"tools":[{"name":"echo","title":null,"description":"","inputSchema":{"properties":{"message":{"title":"Message","type":"string"}},"required":["message"],"title":"echoArguments","type":"object"},"outputSchema":null,"icons":null,"annotations":null,"_meta":null,"execution":null,"mcp_info":{"server_name":"withcred","server_id":"e22e2715-aaf2-4ee3-a6ae-1a5e18c4af00","alias":"withcred_srv"}}],"error":n
call 1 200: {"_meta":null,"content":[{"type":"text","text":"echo: withcred_echo","annotations":null,"_meta":null}],"structuredContent":{"result":"echo: withcred_echo"},"isError":false}
call 2 200: {"_meta":null,"content":[{"type":"text","text":"echo: withcred_echo","annotations":null,"_meta":null}],"structuredContent":{"result":"echo: withcred_echo"},"isError":false}
upstream requests seen by the echo server during this step: 11
  3x initialize: x-api-key: lit7914-fixture-key
  3x notifications/initialized: x-api-key: lit7914-fixture-key
  3x tools/list: x-api-key: lit7914-fixture-key
  2x tools/call: x-api-key: lit7914-fixture-key
-- MCP protocol client (mcp SDK streamable HTTP) against http://127.0.0.1:45138/mcp with x-mcp-servers: withcred_srv
mcp client tools/list: ['withcred_srv-echo']
mcp client tools/call isError: False | [{"type": "text", "text": "echo: withcred_srv_mcp_client", "annotations": null, "meta": null}]
upstream requests seen by the echo server during this step: 12
  5x initialize: x-api-key: lit7914-fixture-key
  3x notifications/initialized: x-api-key: lit7914-fixture-key
  3x tools/list: x-api-key: lit7914-fixture-key
  1x tools/call: x-api-key: lit7914-fixture-key

== probe open: {"auth_type":"none"}
register 201
warm list 200: {"tools":[{"name":"echo","title":null,"description":"","inputSchema":{"properties":{"message":{"title":"Message","type":"string"}},"required":["message"],"title":"echoArguments","type":"object"},"outputSchema":null,"icons":null,"annotations":null,"_meta":null,"execution":null,"mcp_info":{"server_name":"open","server_id":"f388d670-cdfe-424a-b34d-86e14fcb33a7","alias":"open_srv"}}],"error":null,"mes
update 202: {"auth_type":"none","credentials":null}
list 200: {"tools":[{"name":"echo","title":null,"description":"","inputSchema":{"properties":{"message":{"title":"Message","type":"string"}},"required":["message"],"title":"echoArguments","type":"object"},"outputSchema":null,"icons":null,"annotations":null,"_meta":null,"execution":null,"mcp_info":{"server_name":"open","server_id":"f388d670-cdfe-424a-b34d-86e14fcb33a7","alias":"open_srv"}}],"error":null,"mes
call 1 200: {"_meta":null,"content":[{"type":"text","text":"echo: open_echo","annotations":null,"_meta":null}],"structuredContent":{"result":"echo: open_echo"},"isError":false}
call 2 200: {"_meta":null,"content":[{"type":"text","text":"echo: open_echo","annotations":null,"_meta":null}],"structuredContent":{"result":"echo: open_echo"},"isError":false}
upstream requests seen by the echo server during this step: 11
  3x initialize: no auth header
  3x notifications/initialized: no auth header
  3x tools/list: no auth header
  2x tools/call: no auth header
-- MCP protocol client (mcp SDK streamable HTTP) against http://127.0.0.1:45138/mcp with x-mcp-servers: open_srv
mcp client tools/list: ['open_srv-echo']
mcp client tools/call isError: False | [{"type": "text", "text": "echo: open_srv_mcp_client", "annotations": null, "meta": null}]
upstream requests seen by the echo server during this step: 9
  4x initialize: no auth header
  2x notifications/initialized: no auth header
  2x tools/list: no auth header
  1x tools/call: no auth header

### LEG after DONE

Case	Before `9cd7873`	After `70ef8b2`
OBO, client_id only	200, 18 anonymous upstream requests	401, 0 upstream requests
api_key, empty value	200, 18 anonymous upstream requests	500, 0 upstream requests
api_key with value	200, x-api-key forwarded	200, x-api-key forwarded
auth none	200, anonymous by design	200, anonymous by design

Observations from the run

Empty api_key value: generic 500s, no misconfigured reason; PR causes
Empty api_key value: MCP client sees empty tool list; PR causes
Incomplete OBO under admin key: 401, not 500; PR causes
Auth none: one transient tool-not-found at base; PR leaves alone

Type

🐛 Bug Fix

Caveats (if any)

Severe

Misconfigured protected servers now reject requests before upstream dispatch

Medium

Real Linear browser consent was unavailable
Successful OBO token exchange was outside the agreed verification scope

Final Attestation

The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Note

High Risk
Changes authentication and credential resolution for MCP upstream calls; misconfigured protected servers now error before connect, and OBO routing behavior shifts so incomplete configs no longer fall back to unauthenticated v1 paths.

Overview
Fail-closed upstream auth for MCP so misconfigured or empty credentials cannot reach the upstream on discovery, native MCP, or OpenAPI tool calls.

Declared oauth2_token_exchange (OBO) servers always map to the v2 token-exchange resolver (including incomplete client_secret and BYOK), instead of deferring to v1 and allowing static-token fallback or anonymous connects. Static auth modes (api_key, bearer, basic, token, raw authorization) are checked via new validate_static_credential / prepare_mcp_client: native clients preview egress headers with MCPClient.prepare_request_auth() before connect; OpenAPI tools validate merged headers right before HTTP. Placeholder values (bare schemes, duplicated scheme tokens, invalid Basic) are rejected with HTTP errors and no upstream request.

discovery_auth_fingerprint now hashes the same preview path used for validation.

^{Reviewed by Cursor Bugbot for commit 70ef8b2. Bugbot is set up for automated code reviews on this repo. Configure here.}

joshua-berri · 2026-09-16T03:17:08Z

@cursor review Please review the current commit for missing-credential enforcement, caller-header compatibility, and any actionable regressions before readiness is declared

codspeed · 2026-09-16T03:18:37Z

Merging this PR will not alter performance

✅ 31 untouched benchmarks

_{Comparing litellm_fix_mcp_auth_fail_closed_4501 (70ef8b2) with main (4e99640)}

greptile-apps · 2026-09-16T03:19:51Z

Greptile Summary

This PR hardens MCP outbound credential handling while preserving established credential precedence and supported authentication modes.

Keeps declared token-exchange configurations within the existing resolver, including incomplete configurations.
Validates effective static credentials before native MCP or OpenAPI upstream dispatch.
Adds regression coverage for missing, malformed, forwarded, caller-supplied, and custom-header credentials.
The previously reported raw-Authorization concern was withdrawn; complete raw values remain supported while incomplete scheme-only values are rejected.

Confidence Score: 5/5

The current tip appears safe to merge with no remaining actionable review concerns.

No new changes were made since the previous review, both previous findings are resolved, and the raw-Authorization finding was explicitly withdrawn after confirming that complete raw values remain supported.

Important Files Changed

Filename	Overview
litellm/proxy/_experimental/mcp_server/outbound_credentials/adapter.py	Adds centralized effective-credential validation and keeps declared token exchange on the resolver path.
litellm/experimental_mcp_client/client.py	Extracts authenticated request preparation so validation and discovery fingerprinting use the same effective headers.
litellm/proxy/_experimental/mcp_server/mcp_server_manager.py	Validates prepared native MCP clients and propagates authentication metadata into generated OpenAPI tools.
litellm/proxy/_experimental/mcp_server/openapi_to_mcp_generator.py	Validates merged effective headers immediately before OpenAPI upstream requests.
tests/test_litellm/proxy/_experimental/mcp_server/test_mcp_server_manager.py	Adds broad regression coverage for protected credential preparation, precedence, and zero-dispatch rejection.
tests/test_litellm/proxy/_experimental/mcp_server/test_openapi_to_mcp_generator.py	Covers effective OpenAPI header validation, custom credential slots, and supported non-static modes.

_{Reviews (7): Last reviewed commit: "fix(mcp): reject bare schemes in raw aut..." | Re-trigger Greptile}

joshua-berri · 2026-09-16T03:19:57Z

@greptileai Please review the current commit for credential validation, preserved caller-header behavior, and actionable regressions before this PR is considered ready

cursor

Stale Bugbot comment from a previous run.

codecov · 2026-09-16T03:25:09Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

joshua-berri · 2026-09-16T03:33:41Z

@greptileai @cursor review @veria-ai Please re-review commit 258176d for the Basic-separator and rendered-scheme repairs, including existing caller-header compatibility controls

joshua-berri · 2026-09-16T03:38:37Z

@cursor review Please re-review commit 258176d and confirm the rendered scheme-only credential finding is resolved without changing supported caller credentials

cursor

Stale Bugbot comment from a previous run.

joshua-berri · 2026-09-16T15:08:50Z

@greptileai @cursor review @veria-ai Please review fdb8e35 for credential validation, unchanged OpenAPI header precedence, OAuth compatibility, and resolution of prior findings

joshua-berri · 2026-09-16T15:14:08Z

@cursor review Please review commit fdb8e35, focusing on effective credentials, preserved header precedence, OAuth compatibility, and previously resolved findings

cursor

Stale Bugbot comment from a previous run.

joshua-berri · 2026-09-16T16:01:12Z

@greptileai @cursor review @veria-ai Please review 6c517bf for the API-key Authorization correction, supported credential compatibility, and resolution of prior findings

veria-ai · 2026-09-16T16:16:33Z

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

joshua-berri · 2026-09-16T16:30:28Z

@cursor review Please verify 6c517bf resolves the prior API-key Authorization finding and report any remaining correctness or credential compatibility issues

joshua-berri · 2026-09-16T16:30:59Z

@veria-ai Please reassess the raw Authorization finding against the existing opaque-value contract and measured compatibility regression described in the inline reply

cursor

Stale Bugbot comment from a previous run.

joshua-berri · 2026-09-16T17:44:50Z

@cursor review Please review 70ef8b2 for raw Authorization bare-scheme rejection, valid credential compatibility, and any remaining issues in the final diff

joshua-berri · 2026-09-16T17:44:51Z

@greptileai @veria-ai Please review 70ef8b2, including the adopted raw Authorization correction, meaningful compatibility controls, and confirmation that all prior concerns are resolved

cursor

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit 70ef8b2. Configure here.}

joshua-berri · 2026-09-16T17:50:26Z

@greptileai Please reassess against the approved fail-closed requirement: reject known bare schemes, preserve complete raw credentials, and avoid introducing an opt-in bypass

joshua-berri · 2026-09-16T17:52:33Z

@greptileai Please refresh the current-tip review summary to reflect your withdrawn raw-authorization finding and confirm any remaining actionable concerns

mateo-berri

LGTM. Thanks!

^- is this a backwards incompat change that we need to communicate on? do we need to get misbah on it? or no

joshua-berri · 2026-09-16T23:06:46Z

@mateo-berri This corrects broken OBO fallback and missing-credential handling. Configurations relying on that behavior now fail; a brief release note should suffice

joshua-berri added 4 commits September 15, 2026 18:06

fix(mcp): reject missing upstream authentication credentials

1355347

fix(mcp): reject scheme-only Basic credentials

8a3add3

fix(mcp): preserve usable alternate header credentials

84e1478

fix(mcp): preserve explicit caller authorization credentials

be50693

joshua-berri requested a review from a team September 16, 2026 03:16

greptile-apps Bot reviewed Sep 16, 2026

View reviewed changes

Comment thread litellm/proxy/_experimental/mcp_server/upstream.py Outdated

github-advanced-security Bot found potential problems Sep 16, 2026

View reviewed changes

Comment thread litellm/proxy/_experimental/mcp_server/upstream.py Fixed