Skip to content

feat: CLI: Native DuckDB file loading - #409

Merged
ilyabo merged 1 commit into
mainfrom
cursor/native-duckdb-file-loading-5164
Mar 6, 2026
Merged

ilyabo merged 1 commit into
mainfrom
cursor/native-duckdb-file-loading-5164

Conversation

@ilyabo

@ilyabo ilyabo commented Mar 1, 2026 •

Copy link
Copy Markdown
Collaborator

Switch CLI UI file ingestion to direct DuckDB filesystem loading to bypass browser uploads in native mode.

cli_dropzone_native_filesystem_load_demo.mp4

Open in Web Open in Cursor 

Summary by CodeRabbit

  • Bug Fixes

    • Files are now loaded directly from your local filesystem by DuckDB instead of being uploaded to a server, improving performance and data privacy.
    • Updated user guidance messages to reflect the file loading process.
  • Documentation

    • Updated README documentation to reflect current file handling behavior and API endpoints.

@cursor

cursor Bot commented Mar 1, 2026

Copy link
Copy Markdown

Cursor Agent can help with this pull request. Just @cursor in comments and I'll start working on changes in this branch.
Learn more about Cursor Agents

@coderabbitai

coderabbitai Bot commented Mar 1, 2026 •

Copy link
Copy Markdown
Contributor
📝 Walkthrough

Walkthrough

The pull request removes server-side file upload functionality and transitions to loading files directly from the local filesystem using native file paths. DuckDB now references files directly rather than through server-uploaded copies, with updated documentation and user-facing messaging.

Changes

Cohort / File(s) Summary
User-facing messaging
apps/sqlrooms-cli-ui/src/components/DataSourcesPanel.tsx
Updated dropzone guidance text to reflect direct DuckDB filesystem loading instead of browser-local persistence.
Server API removal
apps/sqlrooms-cli-ui/src/serverApi.ts
Removed getApiBaseUrl helper, uploadFileToServer function, and RuntimeConfig dependency; eliminates server upload capability.
File loading refactoring
apps/sqlrooms-cli-ui/src/store.ts
Added FileWithNativePath type and getCliFileReference helper; replaced upload-to-server flow with direct filesystem path reference resolution in file loading.
Documentation updates
python/sqlrooms-cli/README.md
Updated configuration and API documentation to reflect direct filesystem loading; removed references to upload folder and /api/upload endpoint.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰 Files hop straight from your disk to the warren,
No server detours to labor and carry on,
DuckDB reads them native and true,
Faster than ever—the CLI's made new! ✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'Native DuckDB file loading' accurately describes the main change: switching the CLI UI to direct DuckDB filesystem loading instead of browser uploads.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
  • 📝 Generate docstrings (stacked PR)
  • 📝 Generate docstrings (commit on current branch)
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch cursor/native-duckdb-file-loading-5164

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@ilyabo
ilyabo marked this pull request as ready for review March 1, 2026 08:46
@ilyabo ilyabo changed the title Native DuckDB file loading feat: Native DuckDB file loading Mar 1, 2026
@ilyabo ilyabo changed the title feat: Native DuckDB file loading feat: CLI: Native DuckDB file loading Mar 1, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
apps/sqlrooms-cli-ui/src/store.ts (1)

114-130: Optional: Add defensive warning when native file path is missing.

In the CLI environment, the file.path property should consistently be available from native file drops. However, adding a console warning when the fallback to file.name is used would help surface any unexpected scenarios during development.

💡 Suggested defensive check
 function getCliFileReference(file: File): string {
   const nativePath = (file as FileWithNativePath).path;
   if (typeof nativePath === 'string' && nativePath.trim()) {
     return nativePath;
   }
+  console.warn(
+    `Native file path not available for "${file.name}". Falling back to filename only.`
+  );
   return file.name;
 }
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@apps/sqlrooms-cli-ui/src/store.ts` around lines 114 - 130, Add a defensive
warning when we fall back from a missing native path to file.name: inside
getCliFileReference (which uses type FileWithNativePath) detect when nativePath
is not a non-empty string and before returning file.name call console.warn (or
the app logger) indicating the missing native path for the given file (include
file.name and any available metadata). Then ensure connector.loadFile (which
wraps baseLoadFile) continues to call baseLoadFile with the returned reference;
keep the function names getCliFileReference, FileWithNativePath,
connector.loadFile and baseLoadFile intact so callers remain unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@apps/sqlrooms-cli-ui/src/store.ts`:
- Around line 114-130: Add a defensive warning when we fall back from a missing
native path to file.name: inside getCliFileReference (which uses type
FileWithNativePath) detect when nativePath is not a non-empty string and before
returning file.name call console.warn (or the app logger) indicating the missing
native path for the given file (include file.name and any available metadata).
Then ensure connector.loadFile (which wraps baseLoadFile) continues to call
baseLoadFile with the returned reference; keep the function names
getCliFileReference, FileWithNativePath, connector.loadFile and baseLoadFile
intact so callers remain unchanged.

ℹ️ Review info

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1ccbe61 and 9b5f4ff.

📒 Files selected for processing (4)
  • apps/sqlrooms-cli-ui/src/components/DataSourcesPanel.tsx
  • apps/sqlrooms-cli-ui/src/serverApi.ts
  • apps/sqlrooms-cli-ui/src/store.ts
  • python/sqlrooms-cli/README.md
💤 Files with no reviewable changes (1)
  • apps/sqlrooms-cli-ui/src/serverApi.ts

@ilyabo
ilyabo merged commit bc32e25 into main Mar 6, 2026
10 of 12 checks passed
@ilyabo
ilyabo deleted the cursor/native-duckdb-file-loading-5164 branch March 6, 2026 13:47
dmitriy-kostianetskiy pushed a commit that referenced this pull request Mar 9, 2026
ilyabo added a commit that referenced this pull request Mar 15, 2026
This reverts commit bc32e25.

Signed-off-by: Ilya Boyandin <[email protected]>
ilyabo added a commit that referenced this pull request Mar 16, 2026
bdjulbic pushed a commit that referenced this pull request Mar 16, 2026
Co-authored-by: Cursor Agent <[email protected]>
Signed-off-by: bdjulbic <[email protected]>
dmitriy-kostianetskiy pushed a commit that referenced this pull request Mar 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants