feat(policy_engine): explicit priority for policy attachment execution order - #41571

TLDR

Problem this solves:

• Attachment tier order (global, teams, keys, tags, models) is fixed
• No way to run a model-scoped policy before a tag-scoped one

How it solves it:

• Optional integer priority on a policy attachment (config, API, DB, Admin UI)
• Prioritised attachments run first, lowest value first
• Attachments without priority keep today's tier order
• priority is bounded to a signed 32-bit integer, so bad values fail with 422

User Flow

Before: an admin wants the model-scoped policy to run first but the tag-scoped policy always wins

1. They attach tag-policy with tags: [production] and model-policy with models: [gpt-4o-mini] in config.yaml
2. They send POST https://litellm-domain/policies/resolve with {"model": "gpt-4o-mini", "tags": ["production"]}
3. matched_policies comes back as tag-policy then model-policy, and there is no field to change that
4. A request to POST https://litellm-domain/v1/chat/completions with that model and tag returns x-litellm-applied-policies: tag-policy,model-policy
5. On https://litellm-domain/ui/?page=policies the Attachments tab has no priority column or field

After: the admin sets priority and the order follows it

1. They add priority: 1 to the models attachment and priority: 2 to the tags attachment
2. They send the same POST https://litellm-domain/policies/resolve
3. matched_policies comes back as model-policy then tag-policy
4. The same POST https://litellm-domain/v1/chat/completions returns x-litellm-applied-policies: model-policy,tag-policy
5. On https://litellm-domain/ui/?page=policies the Attachments tab shows a sortable Priority column, Add New Attachment has an optional Priority input, and the Policy Simulator lists matches in the new order

Relevant issues

Affected release

Linear ticket

Resolves LIT-7979

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Live audit (no mocks) of the merge base 5ef40a6 and the PR tip 8164189, each in its own worktree with its own Postgres database, two uvicorn workers, LITELLM_DISABLE_NO_REDIS_WARNING=true, launched with python litellm/proxy/proxy_cli.py --config audit_config.yaml --port <4101 base | 4100 head> --num_workers 2 --use_v2_migration_resolver. Real OpenAI (openai/gpt-5.6) and Anthropic (anthropic/claude-sonnet-5) calls. Every LLM cell was also checked in LiteLLM_SpendLogs by request_id = <response id> (/v1/responses streams fall back to the x-litellm-call-id stored in metadata, pre-existing). The same config ran on both legs; the base leg silently ignores priority in it. Screenshots and the annotated recording are in this comment, the full matrix (SDK sync and async cells, streaming, sad paths, edge cases, chaos) is attached to the Devin session as audit_report.md

model_list:
  - model_name: audit-openai
    litellm_params: {model: openai/gpt-5.6, api_key: os.environ/OPENAI_API_KEY}
  - model_name: audit-anthropic
    litellm_params: {model: anthropic/claude-sonnet-5, api_key: os.environ/ANTHROPIC_API_KEY}
  - model_name: audit-ordered
    litellm_params: {model: openai/gpt-5.6, api_key: os.environ/OPENAI_API_KEY}

guardrails:
  - {guardrail_name: tag-guardrail, litellm_params: {guardrail: tool_permission, mode: pre_call, rules: []}}
  - {guardrail_name: model-guardrail, litellm_params: {guardrail: tool_permission, mode: pre_call, rules: []}}
  - {guardrail_name: team-guardrail, litellm_params: {guardrail: tool_permission, mode: pre_call, rules: []}}
  - {guardrail_name: ordered-model-guardrail, litellm_params: {guardrail: tool_permission, mode: pre_call, rules: []}}
  - {guardrail_name: ordered-tag-guardrail, litellm_params: {guardrail: tool_permission, mode: pre_call, rules: []}}

policies:
  tag-policy: {guardrails: {add: [tag-guardrail]}}
  model-policy: {guardrails: {add: [model-guardrail]}}
  team-policy: {guardrails: {add: [team-guardrail]}}
  block-model-policy:
    guardrails: {add: [ordered-model-guardrail]}
    pipeline: {mode: pre_call, steps: [{guardrail: ordered-model-guardrail, on_pass: block, on_fail: block}]}
  block-tag-policy:
    guardrails: {add: [ordered-tag-guardrail]}
    pipeline: {mode: pre_call, steps: [{guardrail: ordered-tag-guardrail, on_pass: block, on_fail: block}]}

policy_attachments:
  - {policy: tag-policy, tags: [production], priority: 2}
  - {policy: model-policy, models: [audit-openai, audit-anthropic, audit-ordered], priority: 1}
  - {policy: team-policy, teams: [platform]}
  - {policy: block-model-policy, models: [audit-ordered], priority: 10}
  - {policy: block-tag-policy, tags: [ordered], priority: 20}

general_settings:
  master_key: sk-1234
  database_url: os.environ/DATABASE_URL

db-policy (no guardrails) was created on both legs with POST /policies so DB attachments could be added through the API and the Admin UI

Before (5ef40a6)

Resolve order

1. curl -s -X POST http://localhost:4101/policies/resolve -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"model":"audit-ordered","tags":["ordered"]}'

   {"matched_policies":[{"policy_name":"block-tag-policy","matched_via":"tag:ordered"},{"policy_name":"model-policy","matched_via":"model:audit-ordered"},{"policy_name":"block-model-policy","matched_via":"model:audit-ordered"}]}

   Tag tier runs before model tier no matter what the config asked for

Blocking pipeline order on a real request

1. curl -s -D - http://localhost:4101/v1/chat/completions -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -H 'x-litellm-tags: ordered' -d '{"model":"audit-ordered","messages":[{"role":"user","content":"Reply with the single word ok"}],"max_tokens":5}'

   HTTP/1.1 400 Bad Request
   x-litellm-applied-policies: block-tag-policy,model-policy,block-model-policy
   {"error":{"message":"Content blocked by guardrail pipeline 'block-tag-policy'", ...}}
   

   The tag-scoped pipeline blocks first, so the model-scoped one never gets to run

/v1/chat/completions

1. curl -s -D - http://localhost:4101/v1/chat/completions -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -H 'x-litellm-tags: production' -d '{"model":"audit-openai","messages":[{"role":"user","content":"Reply with the single word ok"}],"max_completion_tokens":16}'

   HTTP/1.1 200 OK
   x-litellm-applied-policies: tag-policy,model-policy
   x-litellm-policy-sources: tag-policy=tag:production; model-policy=model:audit-openai
   {"id":"chatcmpl-EPDgUqMmWIloGzl3zOa6eVxHobKpB", ... "content":"ok" ...}
   

2. psql "$DATABASE_URL" -c "select request_id from \"LiteLLM_SpendLogs\" where request_id='chatcmpl-EPDgUqMmWIloGzl3zOa6eVxHobKpB'" returns 1 row

/v1/messages

1. curl -s -D - http://localhost:4101/v1/messages -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -H 'x-litellm-tags: production' -d '{"model":"audit-anthropic","max_tokens":16,"messages":[{"role":"user","content":"Reply with the single word ok"}]}'

   HTTP/1.1 200 OK
   x-litellm-applied-policies: tag-policy,model-policy
   {"id":"msg_011Cf9hYXdfuyiGYG8mHPnVK", ... "text":"ok" ...}
   

2. LiteLLM_SpendLogs has 1 row for msg_011Cf9hYXdfuyiGYG8mHPnVK

/v1/responses

1. curl -s -D - http://localhost:4101/v1/responses -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -H 'x-litellm-tags: production' -d '{"model":"audit-openai","input":"Reply with the single word ok","max_output_tokens":16}'

   HTTP/1.1 404 Not Found
   x-litellm-applied-policies: tag-policy,model-policy
   {"error":{"message":"litellm.NotFoundError: NotFoundError: OpenAIException - . Received Model Group=audit-openai ...","code":"404"}}
   

   The merge base returns a provider 404 for this model on the Responses API (reproduced twice, unrelated to policies; the policy headers still show the old tag-first order). The same request returns 200 on the PR tip below

Attachment create and list

1. curl -s -X POST http://localhost:4101/policies/attachments -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"policy_name":"db-policy","tags":["api-created"],"priority":5}'

   {"attachment_id":"ccbfbcca-e303-467b-a9d6-8f67c7e67068","policy_name":"db-policy","tags":["api-created"],"created_at":"2026-09-17T21:15:46.142000Z","definition_location":"db"}

   priority is silently dropped, the response and GET /policies/attachments/list have no such field

Out of range priority

1. Not applicable, the base has no priority field to validate

Admin UI

1. Open http://localhost:4101/ui/policies/, click the Attachments tab
2. The table shows Attachment ID, Policy, Scope, Teams, Keys, Models, Tags, Created At and no Priority column; Add New Attachment has no Priority field (screenshot in the evidence comment)

Chaos

1. Base proxy on its own Postgres cluster (port 4104), Postgres stopped 1 s into a 24 request mixed burst across the three endpoints with streaming, restarted after 23 s: 24 of 24 requests returned 200, 14 of 24 spend logs landed, the rest were dropped after Spend tracking - transient DB error writing spend logs, retry 3/3

Guardrail modes, callback modes, Claude Code

Same commands as the After section below, run against port 4101. Per-request, key attached and team attached guardrails were all added on top of the policy guardrails, the generic_api callback fired for YAML, key level and team level success and failure exactly as on head, and the Claude Code TUI answered priority audit. Only x-litellm-applied-policies differs: the base runs the fixed tier order (tag-policy,model-policy and, with the team key, team-policy,tag-policy,model-policy)

After (8164189)

Resolve order

1. curl -s -X POST http://localhost:4100/policies/resolve -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"model":"audit-ordered","tags":["ordered"]}'

   {"matched_policies":[{"policy_name":"model-policy","matched_via":"model:audit-ordered"},{"policy_name":"block-model-policy","matched_via":"model:audit-ordered"},{"policy_name":"block-tag-policy","matched_via":"tag:ordered"}]}

   Priority 1, then 10, then 20: the model-scoped pipeline now runs before the tag-scoped one

Blocking pipeline order on a real request

1. Same curl against port 4100

   HTTP/1.1 400 Bad Request
   x-litellm-applied-policies: model-policy,block-model-policy,block-tag-policy
   {"error":{"message":"Content blocked by guardrail pipeline 'block-model-policy'", ...}}
   

   The model-scoped pipeline blocks first, which is what LIT-7979 asks for. Same result on /v1/messages and /v1/responses with the same body shape

/v1/chat/completions

1. Same curl against port 4100

   HTTP/1.1 200 OK
   x-litellm-applied-policies: model-policy,tag-policy
   x-litellm-policy-sources: model-policy=model:audit-openai; tag-policy=tag:production
   {"id":"chatcmpl-EPDgKg5RY4vYP1zRLXaNGqMB7Ch8K", ... "content":"ok" ...}
   

2. LiteLLM_SpendLogs has 1 row for chatcmpl-EPDgKg5RY4vYP1zRLXaNGqMB7Ch8K. Streaming curl (chatcmpl-EPDgLaZtWx6rukY5DBnOm79CvTBCL) and the openai SDK sync, sync stream, async and async stream cells all returned the same headers and one spend log row each

/v1/messages

1. Same curl against port 4100

   HTTP/1.1 200 OK
   x-litellm-applied-policies: model-policy,tag-policy
   {"id":"msg_011Cf9hXuZVyLSGcABNTyuoT", ... "text":"ok" ...}
   

2. 1 spend log row for msg_011Cf9hXuZVyLSGcABNTyuoT; streaming curl and the anthropic SDK sync, sync stream, async and async stream cells match

/v1/responses

1. Same curl against port 4100

   HTTP/1.1 200 OK
   x-litellm-applied-policies: model-policy,tag-policy
   {"id":"resp_z7JWj1VIT3GDhZOjq7xcSiyizNwH5067_...", ... "object":"response" ...}
   

2. 1 spend log row for that id; streaming curl and the openai SDK sync, sync stream, async and async stream cells match

Attachment create and list

1. curl -s -X POST http://localhost:4100/policies/attachments -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"policy_name":"db-policy","tags":["api-created"],"priority":5}'

   {"attachment_id":"291fc75c-8c5f-4a53-b534-3083b822842f","policy_name":"db-policy","tags":["api-created"],"priority":5,"definition_location":"db"}

2. curl -s http://localhost:4100/policies/attachments/list -H 'Authorization: Bearer sk-1234' lists priority on every attachment: config 2, 1, null, 10, 20 and the DB row 5

3. curl -s -X POST http://localhost:4100/policies/resolve ... -d '{"model":"audit-openai","tags":["production","api-created"],"team_alias":"platform"}' returns model-policy, tag-policy, db-policy, team-policy: priorities 1, 2, 5, then the unprioritised team attachment last

4. Omitted priority and explicit "priority": null both store SQL NULL and sort as unprioritised; 0, -5, -2147483648 and 2147483647 round-trip exactly through the DB and sort numerically (psql rows in the audit report)

Out of range priority

1. curl -s -w '\nHTTP %{http_code}' -X POST http://localhost:4100/policies/attachments -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' -d '{"policy_name":"db-policy","tags":["sad"],"priority":2147483648}'

   {"detail":[{"type":"less_than_equal","loc":["body","priority"],"msg":"Input should be less than or equal to 2147483647","input":2147483648,"ctx":{"le":2147483647}}]}
   HTTP 422
   

2. -2147483649 gives 422 greater_than_equal, "high" gives 422 int_parsing, 1.5 gives 422 int_from_float, no row is written. priority: high or priority: 2.5 in config.yaml stops the proxy at boot with the pydantic message instead of being silently ignored

Admin UI

1. Open http://localhost:4100/ui/policies/, click the Attachments tab: the table has a sortable Priority column showing 1, 2, 10, 20, the DB rows -5, 0, -2147483648, 2147483647, and blank for unset
2. Click Add New Attachment, pick db-policy, tag ui-created, Priority 3, Create Attachment: the row appears with Priority 3 and GET /policies/attachments/list returns "priority": 3
3. Type 9999999999 in Priority: the form shows Priority must be at most 2147483647 and does not submit
4. Policy Simulator tab, model audit-ordered, tag ordered, Simulate: model-policy, block-model-policy, block-tag-policy in that order
5. Logs page lists chatcmpl-EPDgKg5RY4vYP1zRLXaNGqMB7Ch8K and the audited /v1/messages and /v1/responses rows as successful requests
6. Screenshots and the annotated recording are in this comment. The dashboard dev server reached the audit proxy through a scratch CORS forwarder (localhost:4110 -> 4100) that only adds CORS headers; request and response bodies pass through unchanged

Chaos

1. Head proxy on the same owned cluster (port 4102), Postgres stopped 1 s into a 30 request mixed burst: every request returned 200 with the correct policy order (the attachment registry is in memory), readiness reported db: disconnected while down and db: connected after restart. 28 of 30 spend logs landed exactly once, 2 were dropped after retry 3/3, the same pre-existing loss as the base leg above. The PR does not touch the spend writer, so this is follow-up material, not a regression
2. Postgres paused for 53 s: no deadlock, burst finished, no duplicate spend rows, 8 of 24 dropped the same way on head
3. Proxy restarted mid burst: 24 of 24 returned 200 and landed once, the restarted proxy reloaded the DB attachment with its priority
4. One worker killed with kill -9 mid burst: the other worker kept serving, a new worker came up within 3 s, in-flight requests on the dead worker were lost as expected

Guardrail modes, callback modes, Claude Code (outside the diff, expected unchanged from base)

1. Per-request guardrail plus prioritised policies:

   curl -s -D - http://localhost:4100/v1/chat/completions -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' \
     -d '{"model":"audit-openai","messages":[{"role":"user","content":"Reply with the single word ok"}],"max_tokens":5,"guardrails":["team-guardrail"],"metadata":{"tags":["production"]}}'

   HTTP/1.1 200 OK
   x-litellm-applied-guardrails: tag-guardrail,model-guardrail,team-guardrail
   x-litellm-applied-policies: model-policy,tag-policy
   x-litellm-policy-sources: model-policy=model:audit-openai; tag-policy=tag:production
   

2. Key attached guardrail (/key/generate with metadata.guardrails: [team-guardrail]) on /v1/messages, and team attached guardrail (/team/new alias platform with metadata.guardrails: [tag-guardrail], key in that team) on /v1/responses and on streaming /v1/chat/completions: all 200, the attached guardrail appears in x-litellm-applied-guardrails, and the team key adds team-policy=team:platform to x-litellm-policy-sources behind the two prioritised policies (x-litellm-applied-policies: model-policy,tag-policy,team-policy)
3. Callback modes against a local generic_api HTTP sink (GENERIC_LOGGER_ENDPOINT): YAML success_callback and failure_callback on a second head proxy, key level metadata.logging: [{callback_name: generic_api, callback_type: success}], team level POST /team/{team_id}/callback with callback_type: failure, and a key with no callbacks. Success events arrived only where a success callback was configured, failure events (unknown model, 400 returned to the caller) only where a failure callback was, the uncallbacked key produced none, and POST /key/health reported {"callbacks":["generic_api"],"status":"healthy"}. Every request has exactly one LiteLLM_SpendLogs row by call id
4. Claude Code 2.1.275 run interactively in a terminal with ANTHROPIC_BASE_URL=http://localhost:4100, ANTHROPIC_MODEL=audit-anthropic and a virtual key tagged production: the TUI answered priority audit, and both requests it made landed in LiteLLM_SpendLogs with applied_guardrails from the tag and model policies

Type

🆕 New Feature

Caveats (if any)

Medium

• New nullable column priority on LiteLLM_PolicyAttachmentTable, additive migration, no row rewrites

Low

• When one policy is attached at several matching scopes, the first sorted attachment (now possibly a prioritised narrow one) is the one reported in matched_via
• effective_guardrails on /policies/resolve stays alphabetically sorted; guardrails in the flat list run in the proxy's callback order (the resolver unions them into a set, unchanged from main), so priority orders pipelines, the matched_policies list and the x-litellm-applied-policies header, not the flat list or x-litellm-applied-guardrails
• The proxy has no PUT/PATCH /policies/attachments/{id} on main, so a priority is changed by deleting and recreating the attachment; SpendLogs metadata carries applied_guardrails only, so policy names and order are visible in the response headers and the simulator, not the Logs page. Both are follow-ups outside this ticket
• Spend logs are dropped after three retries while Postgres is down or paused; reproduces at the merge base, follow-up outside this PR
• Docs update is in docs(policies): document attachment priority for policy execution order litellm-docs#1516

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

ran /live-pr-risk and found no regressions/backward incompatible risks

Link to Devin session: https://app.devin.ai/sessions/2357786e068841ec904e342e4e160b2f
Open in Devin Desktop: https://app.devin.ai/desktop/session/2357786e068841ec904e342e4e160b2f?variant=devin
Requested by: @yucheng-berri