localmodelproxy

Local OpenAI-compatible proxy for routing model requests across configurable backends while handling credentials and tracking token usage.

localmodelproxy gives OpenAI-compatible clients one local endpoint (http://127.0.0.1:8080/v1) while it routes chat-completion requests to configured local or hosted model backends. It monitors token usage per request and model, with optional cost tracking, while keeping upstream credentials in its YAML configuration and allowing local model names to map to upstream names.

The proxy is designed to run on demand and listens only on loopback addresses. Its supported API surface is deliberately small:

• GET /healthz
• GET /v1/models
• POST /v1/chat/completions

Configure each backend once, then point your clients at one local endpoint. The proxy chooses the backend from the requested model, supplies its configured credentials, and tracks usage across all requests.

Create ~/.localmodelproxy:

backends:
  # Pass through any otherwise-unmatched model to a local OpenAI-compatible server.
  - name: local
    type: openai_compatible
    base_url: http://127.0.0.1:11434/v1
    auth:
      type: none
    models: all

  - name: openai
    type: openai_compatible
    base_url: https://api.openai.com/v1
    auth:
      type: bearer
      token: ${OPENAI_API_KEY}
    models:
      - id: gpt-5.6-terra

  # Uses credentials automatically provided by `gcloud auth application-default login`.
  - name: google
    type: gcp_openai
    project: your-google-cloud-project
    location: global
    auth:
      type: google_adc
    models:
      - id: gemini-3.6-flash
        upstream_id: google/gemini-3.6-flash

Set OPENAI_API_KEY to your OpenAI API key, run localmodelproxy --headless, then configure your client to use http://127.0.0.1:8080/v1. See the installation guide, usage reference, and configuration examples for backend setup, authentication, aliases, and usage tracking.