feat(proxy): add paginated GET /public/v1/model_hub - #38636

TLDR

Problem this solves:

• Public Model Hub loads every public model group at once
• 300 groups is a 311 KB single response today
• A few thousand groups makes the page stop responding
• Health checks are read for every group, not the visible ones

How it solves it:

• Adds paginated GET /public/v1/model_hub, first route on /public/v1
• Sorting, search and filters come from the shared list framework
• Health is resolved after the page slice, not before
• /public/model_hub is untouched, so the shipped UI keeps working

User Flow

Before: someone browsing a proxy's public model catalogue waits on a single huge response, and on a large proxy the page stops responding

1. They open the public Model Hub page in a browser
2. The page requests GET https://litellm-domain/public/model_hub, with no way to ask for less
3. Every published model group comes back in one response: 311 KB for 300 groups, growing to megabytes for a few thousand
4. Every row is drawn at once, and on a large catalogue the tab stops responding before anything is readable
5. There is no way to ask for page 2, to sort by price, or to narrow to one provider, so the only option is to wait for all of it

After: the same person asks for one page at a time, and gets an answer sized to what they can actually see

1. They open the public Model Hub page in a browser
2. The page requests GET https://litellm-domain/public/v1/model_hub?page=1&page_size=25
3. 25 model groups come back in 24 KB, alphabetical, wrapped in {data, meta, links}; meta says total_count: 300 and total_pages: 12
4. links.next is a ready-made URL for page 2, and links.last for page 12, so paging forward is one request
5. They narrow with ?q=claude, sort with ?sort=-input_cost_per_token, or filter with ?filter[providers][contains]=anthropic, and each answer is one page
6. Asking for something the endpoint does not offer, say ?limit=10, comes back as a 400 that names every parameter it does accept, instead of being silently ignored
7. Nothing changes for anyone still calling GET https://litellm-domain/public/model_hub: same path, same bare array, same 311 KB

Relevant issues

Linear ticket

Resolves LIT-5410

Pre-Submission checklist

• I have added meaningful tests
• The handful of test files covering my change pass locally: uv run pytest tests/test_litellm/proxy/public_endpoints/public_v1/test_model_hub.py tests/test_litellm/proxy/list_api/ -v
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Shared setup, used identically for Before and After: a config publishing 300 public model groups across 5 providers and 3 modes, a real Postgres with one health-check row per group, and the proxy booted with

uv run python litellm/proxy/proxy_cli.py --config proof_config.yaml --port 4000

Postgres statement logging is on, so the health-check read each endpoint issues is quoted from the database's own log.

Before (39e5b0c)

Case 1: there is no paginated public model hub endpoint

1. curl -s -o /dev/null -w "%{http_code}\n" "http://localhost:4000/public/v1/model_hub?page=1&page_size=25"

404

2. curl -s "http://localhost:4000/public/v1/model_hub?page=1&page_size=25"

{"detail":"Not Found"}

Case 2: the only public model hub endpoint returns every group in one response

1. curl -s http://localhost:4000/public/model_hub | wc -c

311440

2. curl -s http://localhost:4000/public/model_hub | jq length

300

Case 3: it reads every health check row, not just the ones it shows

1. Serve one request, then read the health-check query out of the Postgres log:

FROM "public"."LiteLLM_HealthCheckTable" WHERE 1=1

Case 4: sort, search, filter and the error contract

1. Not applicable, the endpoint does not exist yet.

Case 5: a repeated sort key

1. Not applicable, the endpoint does not exist yet.

After (04cbe79)

Case 1: the paginated endpoint serves one page

1. curl -s "http://localhost:4000/public/v1/model_hub?page=1&page_size=25" | jq -c '{meta, links, rows: (.data|length)}'

{"meta":{"total_count":300,"page":1,"page_size":25,"total_pages":12},"links":{"self":"/public/v1/model_hub?page_size=25&page=1","first":"/public/v1/model_hub?page_size=25&page=1","prev":null,"next":"/public/v1/model_hub?page_size=25&page=2","last":"/public/v1/model_hub?page_size=25&page=12"},"rows":25}

Case 2: the same 300 groups, one page instead of the whole list

1. curl -s http://localhost:4000/public/model_hub | wc -c

311440

2. curl -s "http://localhost:4000/public/v1/model_hub?page=1&page_size=25" | wc -c

24070

3. curl -s http://localhost:4000/public/model_hub | jq -c '{type: type, length: length}' (the old endpoint still answers with its bare array)

{"type":"array","length":300}

Case 3: it reads health only for the rows it shows

1. Serve "http://localhost:4000/public/v1/model_hub?page=1&page_size=10", then read the health-check query out of the Postgres log:

FROM "public"."LiteLLM_HealthCheckTable" WHERE "public"."LiteLLM_HealthCheckTable"."model_name" IN ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10)
parameters: $1 = 'anthropic-model-001', $2 = 'anthropic-model-006', $3 = 'anthropic-model-011', $4 = 'anthropic-model-016', $5 = 'anthropic-model-021', $6 = 'anthropic-model-026', $7 = 'anthropic-model-031', $8 = 'anthropic-model-036', $9 = 'anthropic-model-041', $10 = 'anthropic-model-046'

2. Serve http://localhost:4000/public/model_hub on the same build and read its query, to confirm the old endpoint is unchanged:

FROM "public"."LiteLLM_HealthCheckTable" WHERE 1=1

3. curl -s "http://localhost:4000/public/v1/model_hub?page_size=2" | jq -c '[.data[]|{model_group,health_status,health_response_time}]'

[{"model_group":"anthropic-model-001","health_status":"healthy","health_response_time":101.0},{"model_group":"anthropic-model-006","health_status":"unhealthy","health_response_time":106.0}]

Case 4: sort, search, filter and the error contract

1. curl -s "http://localhost:4000/public/v1/model_hub?sort=-model_group&page_size=5" | jq -c '[.data[].model_group]'

["openai-model-295","openai-model-290","openai-model-285","openai-model-280","openai-model-275"]

2. curl -s --globoff "http://localhost:4000/public/v1/model_hub?filter[providers][contains]=mistral&page_size=3" | jq -c '{total_count:.meta.total_count, rows:[.data[]|{model_group,providers}]}'

{"total_count":60,"rows":[{"model_group":"mistral-model-002","providers":["mistral"]},{"model_group":"mistral-model-007","providers":["mistral"]},{"model_group":"mistral-model-012","providers":["mistral"]}]}

3. curl -s --globoff "http://localhost:4000/public/v1/model_hub?filter[mode][in]=embedding,image_generation&page_size=3" | jq -c '{total_count:.meta.total_count, rows:[.data[]|{model_group,mode}]}'

{"total_count":60,"rows":[{"model_group":"deepseek-model-009","mode":"image_generation"},{"model_group":"deepseek-model-019","mode":"image_generation"},{"model_group":"deepseek-model-029","mode":"image_generation"}]}

4. curl -s "http://localhost:4000/public/v1/model_hub?q=model-04&page_size=5" | jq -c '{total_count:.meta.total_count, rows:[.data[].model_group]}'

{"total_count":10,"rows":["anthropic-model-041","anthropic-model-046","deepseek-model-044","deepseek-model-049","groq-model-043"]}

5. curl -s "http://localhost:4000/public/v1/model_hub?limit=10" (unknown parameter, HTTP 400, application/problem+json)

{"type":"urn:litellm:error:unknown-query-parameter","title":"Unknown query parameter","status":400,"detail":"Unrecognized query parameter(s): limit.","allowed":["filter[mode]","filter[mode][in]","filter[providers][contains]","page","page_size","q","sort"]}

6. curl -s "http://localhost:4000/public/v1/model_hub?sort=providers" (undeclared sort field, HTTP 400)

{"type":"urn:litellm:error:invalid-sort-field","title":"Invalid sort field","status":400,"detail":"Cannot sort model groups by: 'providers'.","allowed":["input_cost_per_token","max_input_tokens","max_output_tokens","mode","model_group","output_cost_per_token"]}

7. curl -s -o /dev/null -w '%{http_code}\n' -H 'Authorization: Bearer sk-not-a-real-key' "http://localhost:4000/public/v1/model_hub?page_size=2" (a wrong key must not make a public route 401)

200

Case 5: a repeated sort key is refused rather than sorted twice

1. curl -s "http://localhost:4000/public/v1/model_hub?sort=model_group,model_group" (HTTP 400, application/problem+json)

{"type":"urn:litellm:error:duplicate-sort-field","title":"Duplicate sort field","status":400,"detail":"Sort field(s) named more than once: model_group. Each may appear once.","allowed":["input_cost_per_token","max_input_tokens","max_output_tokens","mode","model_group","output_cost_per_token"]}

2. The same field repeated 5000 times, which previously bought 5000 sort passes over every published group:

curl -s -o /dev/null -w '%{http_code} in %{time_total}s\n' "http://localhost:4000/public/v1/model_hub?sort=$(python3 -c "print(','.join(['model_group']*5000))")"
400 in 0.253914s

3. All six distinct sortable fields at once, the legitimate maximum, still served:

curl -s -o /dev/null -w '%{http_code} in %{time_total}s\n' "http://localhost:4000/public/v1/model_hub?sort=model_group,mode,max_input_tokens,max_output_tokens,input_cost_per_token,output_cost_per_token&page_size=3"
200 in 0.078176s

What the adversarial review sweep covered beyond the committed tests

Five review passes ran over the branch before it was opened. Beyond what is pinned in the test files, they exercised and found clean:

• A 2000-trial randomized sort fuzz against an independent ORDER BY ... NULLS LAST comparator: zero mismatches across mixed ascending and descending keys, nulls, string, int and float fields, and tiebreaker ties
• Walking all 300 rows by links.next alone at page sizes 7, 13, 17 and 50: 300 unique rows, no gaps and no duplicates on any walk, with the correct remainder on every last page
• A sorted diff of all 300 rows from /public/model_hub against a full walk of the new endpoint: byte-identical
• q=model_001 matching nothing while q=model-001 matches one, proving _ and % are escaped as literals rather than treated as wildcards; a quote-and-semicolon search string returns nothing and leaves the table intact
• page_size of 101 and 100000 clamping to 100; page and page_size given as negatives, floats, hex, text and a 24-digit integer all answered rather than crashing
• No authorization header, a malformed header, a wrong bearer key and a wrong x-api-key all returning 200, never 401
• No remaining importer of the framework's old module path anywhere in the repo, and no route registered under /public that could shadow the new one

Type

🆕 New Feature

Caveats (if any)

Medium

• filter[providers][contains] matches a substring, not a whole provider name
• Duplicate sort fields now 400 on /management/v1/budgets too
• They were a no-op there, so no ordering changes

Low

• ui/litellm-dashboard/ diff is the generated schema.d.ts lines only
• Regenerating it is required by the OpenAPI sync check
• No dashboard code changes; the UI still calls /public/model_hub
• Health is read per request, as on the endpoint this supersedes
• Now bounded to the page and indexed, so strictly less work
• Caching it is a reasonable follow-up, not done here
• No feature filter yet: supports_* are booleans and FilterSpec has no boolean type
• ManagementProblem keeps its name in the new neutral package
• Renaming it would touch the app-wide handler and every call site
• The other three public hubs still return unpaginated arrays
• pyproject.toml's mutmut scope still lists only management_endpoints/
• It no longer covers the moved framework or the new package
• Left alone to keep this PR's pyproject.toml diff empty

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR