Skip to content

feat: Persist AI settings to TOML in the CLI - #653

Merged
ilyabo merged 8 commits into
mainfrom
codex/locate-cli-model-settings
May 27, 2026
Merged

ilyabo merged 8 commits into
mainfrom
codex/locate-cli-model-settings

Conversation

@ilyabo

@ilyabo ilyabo commented May 21, 2026 •

Copy link
Copy Markdown
Collaborator

Summary by CodeRabbit

  • New Features

    • Configure custom AI models, multiple providers, selectable default provider/model, and model parameters (max steps, additional instruction).
    • Edit and persist AI settings from the web UI with server-backed save and autosave (respects writable config) and optional WebSocket auth token.
    • Improved model availability/fallbacks; UI shows a centered “New session” button when no session; deleting the last session is allowed.
  • Bug Fixes

    • Autosave failures surface a user toast; invalid model parameters are rejected.
  • Documentation

    • CLI README updated with examples for custom models and model parameters.

Review Change Stack

@coderabbitai

coderabbitai Bot commented May 21, 2026 •

Copy link
Copy Markdown
Contributor

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e55da05f-f15c-404b-931c-82a3da882ade

📥 Commits

Reviewing files that changed from the base of the PR and between 8e86768 and 35f8d89.

📒 Files selected for processing (2)
  • packages/ai-core/__tests__/AiSlice.modelSelection.test.ts
  • packages/ai-core/src/AiSlice.ts

📝 Walkthrough

Walkthrough

Adds editable AI settings: backend parses and exposes providers, custom models, and model parameters; exposes PUT /api/ai/settings to persist changes to the TOML config; frontend store wires autosave (debounced, deduped) to save AI settings when the runtime config is writable.

Changes

AI Settings Persistence: Config Load, Edit, and Autosave

Layer / File(s) Summary
Type and contract definitions
apps/sqlrooms-cli-ui/src/runtimeConfig.ts, apps/sqlrooms-cli-ui/src/serverApi.ts
Extended RuntimeConfig with wsAuthToken, configWritable, and an aiSettings object (providers, customModels, modelParameters); added a type-only AiSettingsSliceConfig import.
Backend config parsing and defaults
python/sqlrooms-cli/sqlrooms/cli.py
_load_ai_runtime_config parses/normalizes ai.custom_models and ai.model_parameters, validates fields, extends return tuple, and supports default_provider == "custom".
Server initialization & runtime exposure
python/sqlrooms-cli/sqlrooms/cli.py, python/sqlrooms-cli/sqlrooms/web/launcher.py
main passes parsed AI custom models/parameters into SqlroomsHttpServer; server stores them and includes aiSettings and configWritable in _runtime_config().
TOML write and API endpoint
python/sqlrooms-cli/sqlrooms/web/launcher.py
Implements _write_ai_settings_to_toml to merge/persist AI settings (preserving api_key_env), and adds authenticated PUT /api/ai/settings that validates payload, writes TOML, updates in-memory defaults, and returns the config path.
Client API helper
apps/sqlrooms-cli-ui/src/serverApi.ts
Adds getApiHeaders (JSON + optional X-SQLRooms-Token) and saveAiSettingsToServer which PUTs AI settings to /api/ai/settings and surfaces errors.
Frontend store integration & autosave
apps/sqlrooms-cli-ui/src/store.ts
Store prefers runtimeConfig.aiSettings.providers, adds customModels and modelParameters into AI slice, conditionally uses a disabled CRDT when sync is off, and implements debounced, deduped autosave to persist AI settings when configWritable is true (shows toast on failure).
UI behavior & component tweaks
apps/sqlrooms-cli-ui/src/components/*, packages/ai-core/src/components/*
Session-driven rendering changes (new-session button, guarded composer), panel/layout/tailwind adjustments, and component behavior updates for model/provider selection and placeholders.
ai-core model selection & tests
packages/ai-core/src/AiSlice.ts, packages/ai-core/__tests__/*
Adds optional getAvailableModels, improved model resolution for new sessions, validates persisted currentSessionId, allows deleting last session, and adds unit tests for selection and deletion edge cases.
ai-settings UI updates
packages/ai-settings/src/components/*
Add showTitle?: boolean props and restructure provider rows/baseUrl inputs into grid layouts; optionally hide section headers when embedded.
Tests & docs
python/sqlrooms-cli/README.md, python/sqlrooms-cli/tests/*
README documents persistence; tests updated/added to validate loader parsing, TOML write behavior (including api_key_env preservation), PUT endpoint success and rejection cases.

Sequence Diagram(s)

sequenceDiagram
  participant UI as Frontend (saveAiSettingsToServer)
  participant Server as SqlroomsHttpServer
  participant Writer as _write_ai_settings_to_toml
  participant FS as Config file

  UI->>Server: PUT /api/ai/settings {settings, defaultProvider?, defaultModel?} (X-SQLRooms-Token)
  Server->>Writer: validate payload and call writer with config_path
  Writer->>FS: merge and write updated config.toml (preserve api_key_env)
  Writer-->>Server: return config_path
  Server-->>UI: 200 OK {config_path}
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Poem

🐰 I hop through TOML, nibble keys and strings,
Saving providers, models, and tiny settings things.
From UI to disk and back I bound,
Debounced and tidy — no dupes found.
A toast for failures, a nibble, then rebound.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 12.90% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately and concisely describes the main objective of the pull request: adding persistence of AI settings to TOML configuration in the CLI.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/locate-cli-model-settings

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

packages/ai-core/__tests__/AiSlice.modelSelection.test.ts

ESLint skipped: missing config or dependency (missing-dependency). The ESLint configuration references a package that is not available in the sandbox.

packages/ai-core/src/AiSlice.ts

ESLint skipped: the ESLint configuration for this file references a package that is not available in the sandbox.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (1)
python/sqlrooms-cli/tests/test_launcher.py (1)

440-485: ⚡ Quick win

Expand persistence assertions for /api/ai/settings.

The payload includes defaultModel and modelParameters, but this test does not assert those persisted/runtime outcomes. Adding them will catch partial-write regressions.

Suggested assertion additions
     assert doc["ai"]["default_provider"] == "anthropic"
+    assert doc["ai"]["default_model"] == "claude-4-sonnet"
     assert doc["ai"]["providers"][0]["api_key"] == "anthropic-key"
+    assert doc["ai"]["model_parameters"]["max_steps"] == 8
+    assert doc["ai"]["model_parameters"]["additional_instruction"] == ""
     assert server.llm_provider == "anthropic"
+    assert server.llm_model == "claude-4-sonnet"
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@python/sqlrooms-cli/tests/test_launcher.py` around lines 440 - 485, Expand
assertions in test_api_put_ai_settings_writes_config to verify that the sent
defaultModel and modelParameters were persisted and applied at runtime: after
loading the toml (doc) assert doc["ai"]["default_model"] == "claude-4-sonnet"
and that doc["ai"]["model_parameters"]["max_steps"] == 8 and
doc["ai"]["model_parameters"]["additional_instruction"] == "" (and/or equivalent
keys), and assert the server runtime fields were updated (e.g. server.llm_model
== "claude-4-sonnet" and server.model_parameters["max_steps"] == 8) so the test
catches partial-write regressions; locate assertions near the existing tomllib
load and server.llm_provider checks in test_api_put_ai_settings_writes_config
within SqlroomsHttpServer test setup.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/sqlrooms-cli-ui/src/store.ts`:
- Around line 715-726: The autosave updates lastSnapshot before the async PUT
completes, which prevents retries on failure; change the logic in the save timer
handler (the setTimeout block that calls saveAiSettingsToServer and references
getAiSettingsTomlPayload) so that lastSnapshot is only assigned after
saveAiSettingsToServer resolves successfully (i.e., in the .then or after
await), and leave lastSnapshot unchanged on .catch so the pending snapshot can
be retried; ensure you still clear saveTimer appropriately and keep the same
payload parsing behavior.

In `@python/sqlrooms-cli/README.md`:
- Around line 49-50: Update the README sentence "AI settings changed in the CLI
UI are saved back to this file automatically when config loading is enabled" to
also state that saving occurs only if the config file is writable; specifically
mention both conditions (config loading enabled and file writable) so users know
the CLI will not persist changes when the config file lacks write permissions.

In `@python/sqlrooms-cli/sqlrooms/web/launcher.py`:
- Around line 603-610: The current except block around
_write_ai_settings_to_toml maps client validation errors to 500; update error
handling to catch validation/client errors (e.g., ValueError) separately and
return JSONResponse({"error": str(exc)}, status_code=400) for those cases, while
preserving the general except Exception as exc branch to log via
logger.error("Failed to write AI settings to %s: %s", self.config_path, exc) and
return 500 for unexpected server errors; locate the try/except that calls
_write_ai_settings_to_toml in launcher.py and implement the specific ValueError
(or a small tuple of expected validation exceptions) except before the generic
one.
- Around line 158-160: The code currently accepts floats for max_steps and
silently truncates them via int(max_steps); change the validation around
max_steps so only integer values are accepted (e.g., check isinstance(max_steps,
int)), and if a non-integer numeric (or non-int) value is provided raise a clear
exception (TypeError or ValueError) with a message indicating maxSteps must be
an integer; update the branch that currently does params_table.add("max_steps",
int(max_steps)) to only run for ints and leave additionalInstruction handling
(model_parameters.get("additionalInstruction")) unchanged.

---

Nitpick comments:
In `@python/sqlrooms-cli/tests/test_launcher.py`:
- Around line 440-485: Expand assertions in
test_api_put_ai_settings_writes_config to verify that the sent defaultModel and
modelParameters were persisted and applied at runtime: after loading the toml
(doc) assert doc["ai"]["default_model"] == "claude-4-sonnet" and that
doc["ai"]["model_parameters"]["max_steps"] == 8 and
doc["ai"]["model_parameters"]["additional_instruction"] == "" (and/or equivalent
keys), and assert the server runtime fields were updated (e.g. server.llm_model
== "claude-4-sonnet" and server.model_parameters["max_steps"] == 8) so the test
catches partial-write regressions; locate assertions near the existing tomllib
load and server.llm_provider checks in test_api_put_ai_settings_writes_config
within SqlroomsHttpServer test setup.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: cbc355c4-30c7-414b-a894-250c8d3b3478

📥 Commits

Reviewing files that changed from the base of the PR and between f3f0b9e and f8381f8.

📒 Files selected for processing (8)
  • apps/sqlrooms-cli-ui/src/runtimeConfig.ts
  • apps/sqlrooms-cli-ui/src/serverApi.ts
  • apps/sqlrooms-cli-ui/src/store.ts
  • python/sqlrooms-cli/README.md
  • python/sqlrooms-cli/sqlrooms/cli.py
  • python/sqlrooms-cli/sqlrooms/web/launcher.py
  • python/sqlrooms-cli/tests/test_cli.py
  • python/sqlrooms-cli/tests/test_launcher.py

Comment thread apps/sqlrooms-cli-ui/src/store.ts Outdated
Comment thread python/sqlrooms-cli/README.md Outdated
Comment thread python/sqlrooms-cli/sqlrooms/web/launcher.py Outdated
Comment thread python/sqlrooms-cli/sqlrooms/web/launcher.py
models: Array<{modelName: string}>;
}
>;
customModels?: Array<{

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd use [] instead of Array

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
apps/sqlrooms-cli-ui/src/store.ts (1)

629-645: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Ignore stale autosave responses.

Older PUTs can finish after newer ones. In that case, Line 636 can rewind lastSnapshot, and Lines 640-644 can show a failure toast even though a later snapshot already saved successfully. Gate the .then/.catch side effects behind a monotonically increasing request id so only the latest in-flight save updates local autosave state.

💡 Minimal fix
 function startAiSettingsTomlAutosave() {
   if (!runtimeConfig.configWritable) return;
 
   let saveTimer: ReturnType<typeof setTimeout> | null = null;
+  let latestSaveRequestId = 0;
   let lastSnapshot = JSON.stringify(
     getAiSettingsTomlPayload(roomStore.getState()),
   );
@@
     }
     saveTimer = setTimeout(() => {
       saveTimer = null;
+      const requestId = ++latestSaveRequestId;
       void saveAiSettingsToServer(
         runtimeConfig,
         JSON.parse(snapshot) as ReturnType<typeof getAiSettingsTomlPayload>,
       )
         .then(() => {
+          if (requestId !== latestSaveRequestId) return;
           lastSnapshot = snapshot;
         })
         .catch((error) => {
+          if (requestId !== latestSaveRequestId) return;
           console.warn('Failed to save AI settings to SQLRooms config', error);
           toast.error('Failed to save AI settings', {
             id: AI_SETTINGS_SAVE_FAILED_TOAST_ID,
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/sqlrooms-cli-ui/src/store.ts` around lines 629 - 645, The autosave can
apply out-of-order results: capture a monotonically increasing save request id
when starting the save and only run the .then/.catch side effects if the
finished request id matches the latest id; implement a counter (e.g.,
saveRequestCounter) incremented before calling saveAiSettingsToServer and store
the current id in a local variable, then in the promise handlers check id ===
latestSaveRequestId before mutating lastSnapshot or showing toast.error
(references: saveAiSettingsToServer, lastSnapshot, saveTimer,
AI_SETTINGS_SAVE_FAILED_TOAST_ID).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@apps/sqlrooms-cli-ui/src/store.ts`:
- Around line 629-645: The autosave can apply out-of-order results: capture a
monotonically increasing save request id when starting the save and only run the
.then/.catch side effects if the finished request id matches the latest id;
implement a counter (e.g., saveRequestCounter) incremented before calling
saveAiSettingsToServer and store the current id in a local variable, then in the
promise handlers check id === latestSaveRequestId before mutating lastSnapshot
or showing toast.error (references: saveAiSettingsToServer, lastSnapshot,
saveTimer, AI_SETTINGS_SAVE_FAILED_TOAST_ID).

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e8b45aa3-6257-4719-8a96-9e62d16d1665

📥 Commits

Reviewing files that changed from the base of the PR and between b2f8f27 and f652307.

📒 Files selected for processing (1)
  • apps/sqlrooms-cli-ui/src/store.ts

@ilyabo ilyabo closed this May 27, 2026
@ilyabo ilyabo reopened this May 27, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/ai-core/src/AiSlice.ts`:
- Around line 398-403: When you repair an invalid baseConfig.currentSessionId
(in the block that checks sessionIdSet), also update baseConfig.openSessionTabs
so the repaired currentSessionId is present; either prepend/append the
reassigned id to openSessionTabs or reset openSessionTabs to
[baseConfig.currentSessionId] if it would otherwise be empty/invalid. Locate the
check that uses sessionIdSet and baseConfig.sessions and ensure after setting
baseConfig.currentSessionId you modify baseConfig.openSessionTabs to include
that id (reference symbols: baseConfig.currentSessionId, baseConfig.sessions,
sessionIdSet, baseConfig.openSessionTabs).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 0fab2be8-7530-4d58-855f-77a2d4d2a44d

📥 Commits

Reviewing files that changed from the base of the PR and between f652307 and 8e86768.

📒 Files selected for processing (15)
  • apps/sqlrooms-cli-ui/src/components/AssistantChatContainer.tsx
  • apps/sqlrooms-cli-ui/src/components/AssistantPanel.tsx
  • apps/sqlrooms-cli-ui/src/components/AssistantSettingsDialog.tsx
  • apps/sqlrooms-cli-ui/src/store.ts
  • packages/ai-core/README.md
  • packages/ai-core/__tests__/AiSlice.modelSelection.test.ts
  • packages/ai-core/src/AiSlice.ts
  • packages/ai-core/src/components/InlineApiKeyInput.tsx
  • packages/ai-core/src/components/ModelSelector.tsx
  • packages/ai-core/src/components/QueryControls.tsx
  • packages/ai-core/src/components/SessionControls.tsx
  • packages/ai-core/src/components/session/DeleteSessionButton.tsx
  • packages/ai-settings/src/components/AiModelParameters.tsx
  • packages/ai-settings/src/components/AiModelsSettings.tsx
  • packages/ai-settings/src/components/AiProvidersSettings.tsx
✅ Files skipped from review due to trivial changes (1)
  • packages/ai-core/README.md
🚧 Files skipped from review as they are similar to previous changes (1)
  • apps/sqlrooms-cli-ui/src/store.ts

Comment thread packages/ai-core/src/AiSlice.ts
@ilyabo
ilyabo merged commit 1589d5d into main May 27, 2026
6 checks passed
@ilyabo
ilyabo deleted the codex/locate-cli-model-settings branch May 27, 2026 17:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants