Skip to content

feat: Adding a datasource from a non main schema using fully qualified table name - #671

Merged
bdjulbic merged 7 commits into
mainfrom
feat/non-main-schema-support
Jun 8, 2026
Merged

bdjulbic merged 7 commits into
mainfrom
feat/non-main-schema-support

Conversation

@bdjulbic

@bdjulbic bdjulbic commented Jun 1, 2026 •

Copy link
Copy Markdown
Collaborator

Use fully qualified table name when adding data to the map.
This allows adding data to a map from a non-main schema.

@coderabbitai

coderabbitai Bot commented Jun 1, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds a quote‑aware getUnqualifiedSqlIdentifier in duckdb-core, re-exports it through duckdb and cells, updates tests, adds the dependency to Kepler, uses it for dataset labels, and makes Kepler dataset sync log per-dataset failures without aborting.

Changes

Unqualified Identifier Extraction and Integration

Layer / File(s) Summary
Core utility and tests
packages/duckdb-core/src/duckdb-utils.ts, packages/duckdb-core/__tests__/getUnqualifiedSqlIdentifier.test.ts, packages/duckdb-core/src/index.ts
Adds getUnqualifiedSqlIdentifier implementation in duckdb-core, updates tests to import it, and re-exports it from duckdb-core.
Propagate re-exports through duckdb and cells
packages/duckdb/src/index.ts, packages/cells/src/helpers.ts, packages/cells/src/index.ts
Re-exports the new utility from @sqlrooms/duckdb-core via packages/duckdb, updates cells/helpers to forward the symbol from @sqlrooms/duckdb, and exposes it in the cells package entrypoint.
Kepler integration and dataset synchronization improvements
packages/kepler/package.json, packages/kepler/src/KeplerSlice.ts
Adds @sqlrooms/duckdb-core to Kepler dependencies; imports getUnqualifiedSqlIdentifier to compute human-readable dataset labels in addTableToMap; changes syncKeplerDatasets to call addTableToMap per missing dataset and log per-dataset errors without aborting the loop.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • sqlrooms/sqlrooms#533: Also modifies packages/kepler/src/KeplerSlice.ts to adjust dataset/table sync behavior and referenced-data syncing.

Suggested reviewers

  • ilyabo
  • igorDykhta

Poem

🐰 I hop through quotes and dots with care,
I pluck the last name from a qualified lair.
Labels now bloom where table names hide,
Sync keeps moving, even when some slips slide.
Hooray — a helper tidy, small, and fair.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 75.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: enabling support for adding datasources from non-main schemas using fully qualified table names, which aligns with all the modifications across the codebase.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/non-main-schema-support

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
packages/duckdb-core/__tests__/duckdb-utils.test.ts (1)

3-27: ⚡ Quick win

Consider adding test coverage for table names containing dots.

The current test suite covers common cases well, but doesn't include the edge case where a table name itself contains a dot (e.g., "my.table"), which requires quoting in DuckDB.

📝 Suggested additional test cases
   it('returns the last segment when given a non-main schema', () => {
     expect(getTableNameFromQualified('analytics.events')).toBe('events');
   });
+
+  it('preserves dots within a quoted table name', () => {
+    expect(getTableNameFromQualified('"my.table"')).toBe('my.table');
+  });
+
+  it('preserves dots within a quoted table name in qualified identifier', () => {
+    expect(getTableNameFromQualified('schema."my.table"')).toBe('my.table');
+    expect(getTableNameFromQualified('"db"."schema"."my.table"')).toBe('my.table');
+  });
 });
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/duckdb-core/__tests__/duckdb-utils.test.ts` around lines 3 - 27, Add
tests to cover qualified identifiers where the final table name contains a dot
and is quoted (e.g., '"my.table"' or '"schema"."my.table"') to ensure
getTableNameFromQualified correctly returns the full table name including the
dot; update the test suite around getTableNameFromQualified to include cases for
a single quoted identifier with a dot and a multi-segment qualified identifier
where only the last segment is quoted and contains a dot, asserting the function
returns 'my.table'.
packages/kepler/src/KeplerSlice.ts (1)

575-575: 💤 Low value

String() conversion is unnecessary.

The tableName parameter is already typed as string (line 552), so the String() conversion is redundant.

♻️ Simplify by removing unnecessary conversion
-            const label = getTableNameFromQualified(String(tableName));
+            const label = getTableNameFromQualified(tableName);
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/kepler/src/KeplerSlice.ts` at line 575, The call wraps an
already-typed string in String() when computing the label; remove the
unnecessary conversion and pass the tableName variable directly into
getTableNameFromQualified (i.e., replace
getTableNameFromQualified(String(tableName)) with
getTableNameFromQualified(tableName)); ensure there are no other similar
redundant String() uses around the getTableNameFromQualified call or in the
surrounding function (KeplerSlice methods referencing tableName) and run type
checks.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/duckdb-core/src/duckdb-utils.ts`:
- Around line 51-52: getTableNameFromQualified currently strips all quotes then
splits on '.' so quoted identifiers containing dots (e.g. "my.table") are
broken; update getTableNameFromQualified to parse the string
character-by-character respecting quoted identifiers and escaped quotes (treat
'.' outside quotes as separator), return the last segment unwrapped and with
doubled quotes unescaped (similar logic to getUnqualifiedSqlIdentifier), and add
unit tests covering: unquoted names, quoted last segment with dots, and quoted
last segment containing escaped quotes ("" -> ") to ensure correct labels in
KeplerSlice.

---

Nitpick comments:
In `@packages/duckdb-core/__tests__/duckdb-utils.test.ts`:
- Around line 3-27: Add tests to cover qualified identifiers where the final
table name contains a dot and is quoted (e.g., '"my.table"' or
'"schema"."my.table"') to ensure getTableNameFromQualified correctly returns the
full table name including the dot; update the test suite around
getTableNameFromQualified to include cases for a single quoted identifier with a
dot and a multi-segment qualified identifier where only the last segment is
quoted and contains a dot, asserting the function returns 'my.table'.

In `@packages/kepler/src/KeplerSlice.ts`:
- Line 575: The call wraps an already-typed string in String() when computing
the label; remove the unnecessary conversion and pass the tableName variable
directly into getTableNameFromQualified (i.e., replace
getTableNameFromQualified(String(tableName)) with
getTableNameFromQualified(tableName)); ensure there are no other similar
redundant String() uses around the getTableNameFromQualified call or in the
surrounding function (KeplerSlice methods referencing tableName) and run type
checks.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 9795700c-36b1-4f80-8ecd-917f16fd4730

📥 Commits

Reviewing files that changed from the base of the PR and between 72c628a and a79769c.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (6)
  • packages/duckdb-core/__tests__/duckdb-utils.test.ts
  • packages/duckdb-core/src/duckdb-utils.ts
  • packages/duckdb-core/src/index.ts
  • packages/duckdb/src/index.ts
  • packages/kepler/package.json
  • packages/kepler/src/KeplerSlice.ts

Comment thread packages/duckdb-core/src/duckdb-utils.ts Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
packages/cells/src/helpers.ts (1)

104-104: ⚡ Quick win

Re-export getUnqualifiedSqlIdentifier from @sqlrooms/duckdb-core

@sqlrooms/duckdb-core already exports getUnqualifiedSqlIdentifier, and using it here keeps this pure helper sourced from the lowest-level package. This won’t materially change the @sqlrooms/cells -> @sqlrooms/duckdb`` dependency, since @sqlrooms/cells imports multiple symbols from `@sqlrooms/duckdb` elsewhere.

♻️ Proposed change
-export {getUnqualifiedSqlIdentifier} from '`@sqlrooms/duckdb`';
+export {getUnqualifiedSqlIdentifier} from '`@sqlrooms/duckdb-core`';
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/cells/src/helpers.ts` at line 104, The current re-export pulls
getUnqualifiedSqlIdentifier from '`@sqlrooms/duckdb`' but it should come from the
lower-level package '`@sqlrooms/duckdb-core`'; change the export so
getUnqualifiedSqlIdentifier is re-exported from '`@sqlrooms/duckdb-core`' (update
the export statement in helpers.ts to reference '`@sqlrooms/duckdb-core`') to keep
the helper sourced from the lowest-level package while preserving existing
imports of other symbols from '`@sqlrooms/duckdb`'.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@packages/cells/src/helpers.ts`:
- Line 104: The current re-export pulls getUnqualifiedSqlIdentifier from
'`@sqlrooms/duckdb`' but it should come from the lower-level package
'`@sqlrooms/duckdb-core`'; change the export so getUnqualifiedSqlIdentifier is
re-exported from '`@sqlrooms/duckdb-core`' (update the export statement in
helpers.ts to reference '`@sqlrooms/duckdb-core`') to keep the helper sourced from
the lowest-level package while preserving existing imports of other symbols from
'`@sqlrooms/duckdb`'.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 258fca73-19a2-4fba-835d-7f03c4e0d350

📥 Commits

Reviewing files that changed from the base of the PR and between f1d287a and 30c9a24.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (7)
  • packages/cells/src/helpers.ts
  • packages/duckdb-core/__tests__/getUnqualifiedSqlIdentifier.test.ts
  • packages/duckdb-core/src/duckdb-utils.ts
  • packages/duckdb-core/src/index.ts
  • packages/duckdb/src/index.ts
  • packages/kepler/package.json
  • packages/kepler/src/KeplerSlice.ts
💤 Files with no reviewable changes (1)
  • packages/kepler/package.json
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/kepler/src/KeplerSlice.ts

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends table handling to better support non-main schemas by (a) separating “dataset id” (still the table name) from the user-facing label and (b) moving a shared “unqualified identifier” utility into duckdb-core and re-exporting it through higher-level packages.

Changes:

  • Add getUnqualifiedSqlIdentifier to @sqlrooms/duckdb-core, re-export via @sqlrooms/duckdb, and surface via @sqlrooms/cells.
  • Update Kepler dataset labeling to display unqualified table names while keeping fully-qualified IDs for internal referencing.
  • Make Kepler dataset sync resilient to per-dataset load failures (non-blocking error logging).

Reviewed changes

Copilot reviewed 8 out of 9 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
pnpm-lock.yaml Adds workspace link entry for @sqlrooms/duckdb-core under the Kepler importer.
packages/kepler/src/KeplerSlice.ts Uses unqualified identifiers for dataset labels; makes sync continue past dataset load failures.
packages/kepler/package.json Adds @sqlrooms/duckdb-core dependency for shared SQL identifier utilities.
packages/duckdb/src/index.ts Re-exports getUnqualifiedSqlIdentifier from @sqlrooms/duckdb-core.
packages/duckdb-core/src/index.ts Exposes getUnqualifiedSqlIdentifier as part of the duckdb-core public API.
packages/duckdb-core/src/duckdb-utils.ts Implements the quote-aware getUnqualifiedSqlIdentifier helper.
packages/duckdb-core/tests/getUnqualifiedSqlIdentifier.test.ts Updates test import to the new source location for the helper.
packages/cells/src/index.ts Exports getUnqualifiedSqlIdentifier from cells public API surface.
packages/cells/src/helpers.ts Removes local implementation and re-exports getUnqualifiedSqlIdentifier from @sqlrooms/duckdb.
Files not reviewed (1)
  • pnpm-lock.yaml: Language not supported

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread packages/kepler/src/KeplerSlice.ts
* - `db.schema.table` -> `table`
* - `schema."my.funny.table"` -> `my.funny.table`
*/
export function getUnqualifiedSqlIdentifier(

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was moved to packages/duckdb-core/src/duckdb-utils.ts

@bdjulbic bdjulbic changed the title feat: non main schema support feat: Add datasource from a non main schema Jun 5, 2026
@bdjulbic bdjulbic changed the title feat: Add datasource from a non main schema feat: Adding a datasource from a non main schema using fully qualified table name Jun 5, 2026
@bdjulbic
bdjulbic merged commit 364ac78 into main Jun 8, 2026
3 checks passed
@bdjulbic
bdjulbic deleted the feat/non-main-schema-support branch June 8, 2026 06:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants