Skip to content

FM Phase 6: Context-window auto-detection #27

Open

Description

@diskerror

On startup / first /chat, probe the inference endpoint for context size.

  • Ollama /api/show, llama.cpp /props, vLLM /v1/models
  • Cache in config / in-memory, log the detected value
  • Ship known-model lookup table as fallback
  • Unknown → conservative defaults, user can override
  • [inference] max_context in config is authoritative when set

Files: inference.cpp, config.cpp

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    backendServer / API workfading-memoryFading memory / chat manager design

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions