Skip to content

Config: unify api_url + lm_proxy_url into a single origin URL + upstream protocol #46

Closed

Description

@diskerror

Problem

Today there are two URL knobs pointing at the same upstream:

```ini
[inference]
api_url = http://192.168.0.107:1234/v1 # used by internal inference client (chat, summarize)
lm_proxy_url = http://192.168.0.107:1234 # used by /v1/* pass-through proxy
```

They disagree on whether `/v1` belongs in the URL (see #45), they duplicate the same host:port, and they can silently drift out of sync. The right model is one URL per endpoint, and let Ragger decide what path suffix to tack on based on the upstream protocol it's speaking.

Target config

```ini
[inference]
url = http://192.168.0.107:1234 # origin only — no /v1, no /api/v1
api_key = lmstudio-local
protocol = openai # openai | anthropic (later: lmstudio-stateful)
model = devstral-small-2-24b-instruct-2512@q8_k_xl
```

  • One `url` serves both the internal inference client and the LM pass-through proxy. If proxy is enabled, it forwards to the same upstream. No more `lm_proxy_url`.
  • `protocol` drives which version-paths Ragger appends:
  • Version bump (v2) is a protocol concern, not a config concern. User doesn't change `url`.

Downstream (what Ragger exposes to clients)

Always OpenAI-compat for now. OC and other clients hit `/v1/chat/completions`, `/v1/models` on Ragger. Ragger translates to the upstream protocol internally.

Future optional: expose Anthropic-compat `/v1/messages` downstream too, for clients that prefer that shape. Not required for this issue.

Upstream (what Ragger calls)

Driven by `protocol`:

protocol GET models POST chat format file today
openai `/v1/models` `/v1/chat/completions` `formats/openai.json`
anthropic — (no list endpoint) `/v1/messages` `formats/anthropic.json`

When downstream is OpenAI-compat and upstream is Anthropic, Ragger transforms the request/response shape using the existing format infrastructure. That already exists for the internal inference path (`formats/anthropic.json`) — extending it to proxy means running the same transformer pipeline.

Subsumes / relates to

Migration

  • On config load, if `api_url` or `lm_proxy_url` is present:
    • Strip any trailing `/v1`, `/v1/`, `/api/v1`, `/api/v1/` to get origin.
    • Log a deprecation notice suggesting the new `url` + `protocol` form.
    • Treat the combined value as `url`. Default `protocol = openai`.
  • Remove both old keys in a later release once users have migrated.

Why this matters for the product

The user-facing vision: a single conversation accessible from the Ragger web GUI, Telegram, and any OC-style client, all backed by the same memory store. That requires Ragger to be the sole mediator to the upstream, regardless of which client the user is chatting from. Today's two-URL setup makes it too easy to mis-wire one path and have "Ragger chat" and "OC chat" silently hit different upstreams or miss capture.

Acceptance

  • `settings.ini` template + docs show only `url` + `protocol`.
  • Old `api_url` / `lm_proxy_url` still accepted with a deprecation warning for at least one release.
  • `protocol = anthropic` with appropriate `url` + `api_key` works for both internal inference and the `/v1/*` proxy (OpenAI-shaped requests from OC → Anthropic-shaped upstream calls).
  • No `/v1` or `/api/v1` anywhere in user-facing config.

Activity

  1. diskerror commented on Apr 24, 2026

    @diskerror
    OwnerAuthor

    Format expansion

    Existing `formats/` covers OpenAI and Anthropic — that's the minimum to make `protocol =` meaningful. The directory needs to grow:

    Each adds a JSON file describing path + request/response extractors, and — if request body shape is genuinely different — a named C++ transformer (`request_transform` field). The transform name is the only thing requiring a rebuild; JSON-only providers can be added without one.

    Also worth generalizing

    • Downstream format. Right now Ragger exposes OpenAI-compat only. Once the upstream matrix is broader, it costs little to expose Anthropic-compat `/v1/messages` downstream as well for clients that prefer it. Same format infrastructure, run in reverse.
    • Per-endpoint override. A user might have both a local LM Studio stateless endpoint and a cloud Anthropic endpoint active; the `[inference.]` section already supports this, just needs `protocol` / `format` wired per endpoint.

    Not blocking the core of this issue — worth tracking so the format system isn't designed around just two examples.

  2. added
    chatRagger Chat (Role 2) / proxy (Role 3)
    on Sep 5, 2026
  3. diskerror commented on Sep 5, 2026

    @diskerror
    OwnerAuthor

    Status note (2026-09-04): current direction is to solve this class of problem without Ragger proxying connections where possible. This issue depends on proxy/chat infrastructure that either doesn't exist yet or was removed (the old LM proxy was replaced by capture_turn — see commit 2b55ee9). Not closing — proxying remains a possible fallback if the non-proxy approach doesn't pan out — but deprioritized pending that exploration. Tagged chat (Ragger Chat / proxy) to track this cluster.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    chatRagger Chat (Role 2) / proxy (Role 3)

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions