fix(moonshot, together_ai): send the reasoning effort Kimi K3 accepts - #38611

TLDR

Problem this solves:

• Moonshot 400s on any explicit reasoning_effort
• Together silently folds Kimi K3 max down to high
• Both providers document and accept low, high, max

How it solves it:

• Moonshot offers reasoning_effort when the registry says the model reasons
• Together sends a level the map entry declares, unchanged
• Undeclared levels keep the existing per-model clamp
• A bridge-derived {effort, summary} object is unwrapped to its level string

User Flow

Before: a developer routing Kimi K3 through the gateway cannot ask for max thinking, on either provider

1. They send POST https://litellm-domain/v1/chat/completions with "model": "kimi-k3-moonshot" and "reasoning_effort": "max"
2. It comes back 400 with moonshot does not support parameters: ['reasoning_effort'], and the request never left the gateway
3. They drop the parameter to get a 200, and every call now runs at whatever effort the provider defaults to
4. They try the same thing against Together with "model": "kimi-k3-together" and get a 200, so it looks fine
5. Together bills and answers at high, because the gateway rewrote max on the way out, and nothing in the response says so
6. Their Anthropic-format client sends POST https://litellm-domain/v1/messages with a thinking block and gets the same 400

After: both providers take the level the caller asked for

1. They send the same POST with "model": "kimi-k3-moonshot" and "reasoning_effort": "max"
2. It comes back 200 with a completion
3. They send the same POST with "model": "kimi-k3-together" and "reasoning_effort": "max"
4. It comes back 200, and Together receives "reasoning_effort": "max" rather than high
5. Asking for low or high on either provider does what it says, and both are reachable for the first time on Moonshot
6. The same POST https://litellm-domain/v1/messages with a thinking block comes back 200 with a thinking content block

Relevant issues

• Moonshot never listed reasoning_effort among its supported params, so an explicit level raised before the request left the proxy
• Together clamps max to high for every model except one prefix, which caught Kimi K3 after the map declared it takes max
• Only Kimi K3 on Together changes value; every other Together model keeps its current clamp exactly
• The /v1/messages and /v1/responses bridges already derived reasoning_effort for non-Claude targets, so the same 400 blocked both surfaces; when a thinking summary is in play the derived value is an object like {"effort": "high", "summary": "detailed"}, which Moonshot rejects with "expected type string" even once the param is allowed, so the config unwraps it to the level string

Linear ticket

Resolves LIT-6330

Pre-Submission checklist

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Shared setup, a proxy with both providers registered on their real APIs, launched with LITELLM_LOCAL_MODEL_COST_MAP=true so the bundled map's reasoning_effort_levels entry is loaded (see Caveats):

model_list:
  - model_name: kimi-k3-moonshot
    litellm_params:
      model: moonshot/kimi-k3
      api_key: os.environ/MOONSHOT_API_KEY
  - model_name: kimi-k3-together
    litellm_params:
      model: together_ai/moonshotai/Kimi-K3
      api_key: os.environ/TOGETHERAI_API_KEY

Outgoing bodies are captured by pointing one extra deployment per provider at a local listener, since a downgrade is invisible from the client side.

Before (74050e0)

/v1/chat/completions, Moonshot rejects the parameter

1. curl -s -X POST http://127.0.0.1:16422/v1/chat/completions -H "Authorization: Bearer sk-1234" -d '{"model":"kimi-k3-moonshot","messages":[{"role":"user","content":"Reply ok."}],"reasoning_effort":"max","max_tokens":2048}'
2. status=400, body litellm.UnsupportedParamsError: moonshot does not support parameters: ['reasoning_effort'], for model=kimi-k3

/v1/chat/completions, Together downgrades max on the wire

1. Same request against the wire-capture deployment for each of low, high, max, minimal, xhigh

2. What Together received:

   asked	sent
   low	low
   high	high
   max	high
   minimal	low
   xhigh	high

/v1/messages, thinking never reaches Moonshot

1. curl -s -X POST http://127.0.0.1:16422/v1/messages -H "Authorization: Bearer sk-1234" -d '{"model":"kimi-k3-moonshot","max_tokens":4096,"messages":[{"role":"user","content":"Reply ok."}],"thinking":{"type":"enabled","budget_tokens":4096}}'
2. status=400, same UnsupportedParamsError for ['reasoning_effort']

/v1/responses, reasoning effort never reaches Moonshot

1. curl -s -X POST http://127.0.0.1:16422/v1/responses -H "Authorization: Bearer sk-1234" -d '{"model":"kimi-k3-moonshot","input":"Reply ok.","reasoning":{"effort":"max"}}'
2. status=400, same UnsupportedParamsError for ['reasoning_effort']

After (7fd136d)

/v1/chat/completions, Moonshot rejects the parameter

1. curl -s -X POST http://127.0.0.1:16431/v1/chat/completions -H "Authorization: Bearer sk-1234" -d '{"model":"kimi-k3-moonshot","messages":[{"role":"user","content":"Reply ok."}],"reasoning_effort":"max","max_tokens":2048}'
2. status=200, model=kimi-k3-moonshot completion_tokens=46

/v1/chat/completions, Together downgrades max on the wire

1. Same five requests against the wire-capture deployment

2. What Together received:

   asked	before	after
   low	low	low
   high	high	high
   max	high	max
   minimal	low	low
   xhigh	high	high

3. A real call to kimi-k3-together at max returns status=200

/v1/messages, thinking never reaches Moonshot

1. Same request as before, status=200, and the wire shows reasoning_effort: "high" derived from the 4096 budget
2. With "thinking":{"type":"enabled","budget_tokens":8192,"summary":"detailed"} the wire still shows the plain string "high" rather than the {"effort","summary"} object, and the real call returns status=200 with content types ['thinking', 'text']; before the unwrap commit Moonshot answered that object with 400 the reasoning_effort field (expected type string) is illegal

/v1/responses, reasoning effort never reaches Moonshot

1. Same request as before, status=200, wire shows reasoning_effort: "max"

Why max is worth having

Measured directly against both provider APIs on one prompt, so the downgrade is not cosmetic:

Type

🐛 Bug Fix

Caveats (if any)

Medium

• azure_ai also rejects the param, left out because Foundry is unverified
• LIT-6330 covers azure_ai too, so it stays open
• Together passthrough reads reasoning_effort_levels off the loaded map
  • the public remote map carries the key on no entry yet
  • a default proxy therefore keeps the max to high clamp until it ships
  • LITELLM_LOCAL_MODEL_COST_MAP=true or a model_info declaration activates it today

Low

• Bedrock Moonshot skips this config's param list, so it is unaffected
• 5 of 23 Moonshot entries gain the param, all kimi reasoning models
• Together tests copy the clamp tables' literals rather than importing them
• A bridge thinking summary has no Moonshot equivalent and is dropped

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR