Repository navigation
feat: Adding a datasource from a non main schema using fully qualified table name - #671
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds a quote‑aware getUnqualifiedSqlIdentifier in duckdb-core, re-exports it through duckdb and cells, updates tests, adds the dependency to Kepler, uses it for dataset labels, and makes Kepler dataset sync log per-dataset failures without aborting. ChangesUnqualified Identifier Extraction and Integration
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
packages/duckdb-core/__tests__/duckdb-utils.test.ts (1)
3-27: ⚡ Quick winConsider adding test coverage for table names containing dots.
The current test suite covers common cases well, but doesn't include the edge case where a table name itself contains a dot (e.g.,
"my.table"), which requires quoting in DuckDB.📝 Suggested additional test cases
it('returns the last segment when given a non-main schema', () => { expect(getTableNameFromQualified('analytics.events')).toBe('events'); }); + + it('preserves dots within a quoted table name', () => { + expect(getTableNameFromQualified('"my.table"')).toBe('my.table'); + }); + + it('preserves dots within a quoted table name in qualified identifier', () => { + expect(getTableNameFromQualified('schema."my.table"')).toBe('my.table'); + expect(getTableNameFromQualified('"db"."schema"."my.table"')).toBe('my.table'); + }); });🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/duckdb-core/__tests__/duckdb-utils.test.ts` around lines 3 - 27, Add tests to cover qualified identifiers where the final table name contains a dot and is quoted (e.g., '"my.table"' or '"schema"."my.table"') to ensure getTableNameFromQualified correctly returns the full table name including the dot; update the test suite around getTableNameFromQualified to include cases for a single quoted identifier with a dot and a multi-segment qualified identifier where only the last segment is quoted and contains a dot, asserting the function returns 'my.table'.packages/kepler/src/KeplerSlice.ts (1)
575-575: 💤 Low value
String()conversion is unnecessary.The
tableNameparameter is already typed asstring(line 552), so theString()conversion is redundant.♻️ Simplify by removing unnecessary conversion
- const label = getTableNameFromQualified(String(tableName)); + const label = getTableNameFromQualified(tableName);🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/kepler/src/KeplerSlice.ts` at line 575, The call wraps an already-typed string in String() when computing the label; remove the unnecessary conversion and pass the tableName variable directly into getTableNameFromQualified (i.e., replace getTableNameFromQualified(String(tableName)) with getTableNameFromQualified(tableName)); ensure there are no other similar redundant String() uses around the getTableNameFromQualified call or in the surrounding function (KeplerSlice methods referencing tableName) and run type checks.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/duckdb-core/src/duckdb-utils.ts`:
- Around line 51-52: getTableNameFromQualified currently strips all quotes then
splits on '.' so quoted identifiers containing dots (e.g. "my.table") are
broken; update getTableNameFromQualified to parse the string
character-by-character respecting quoted identifiers and escaped quotes (treat
'.' outside quotes as separator), return the last segment unwrapped and with
doubled quotes unescaped (similar logic to getUnqualifiedSqlIdentifier), and add
unit tests covering: unquoted names, quoted last segment with dots, and quoted
last segment containing escaped quotes ("" -> ") to ensure correct labels in
KeplerSlice.
---
Nitpick comments:
In `@packages/duckdb-core/__tests__/duckdb-utils.test.ts`:
- Around line 3-27: Add tests to cover qualified identifiers where the final
table name contains a dot and is quoted (e.g., '"my.table"' or
'"schema"."my.table"') to ensure getTableNameFromQualified correctly returns the
full table name including the dot; update the test suite around
getTableNameFromQualified to include cases for a single quoted identifier with a
dot and a multi-segment qualified identifier where only the last segment is
quoted and contains a dot, asserting the function returns 'my.table'.
In `@packages/kepler/src/KeplerSlice.ts`:
- Line 575: The call wraps an already-typed string in String() when computing
the label; remove the unnecessary conversion and pass the tableName variable
directly into getTableNameFromQualified (i.e., replace
getTableNameFromQualified(String(tableName)) with
getTableNameFromQualified(tableName)); ensure there are no other similar
redundant String() uses around the getTableNameFromQualified call or in the
surrounding function (KeplerSlice methods referencing tableName) and run type
checks.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 9795700c-36b1-4f80-8ecd-917f16fd4730
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (6)
packages/duckdb-core/__tests__/duckdb-utils.test.tspackages/duckdb-core/src/duckdb-utils.tspackages/duckdb-core/src/index.tspackages/duckdb/src/index.tspackages/kepler/package.jsonpackages/kepler/src/KeplerSlice.ts
There was a problem hiding this comment.
🧹 Nitpick comments (1)
packages/cells/src/helpers.ts (1)
104-104: ⚡ Quick winRe-export
getUnqualifiedSqlIdentifierfrom@sqlrooms/duckdb-core
@sqlrooms/duckdb-corealready exportsgetUnqualifiedSqlIdentifier, and using it here keeps this pure helper sourced from the lowest-level package. This won’t materially change the@sqlrooms/cells ->@sqlrooms/duckdb`` dependency, since@sqlrooms/cellsimports multiple symbols from `@sqlrooms/duckdb` elsewhere.♻️ Proposed change
-export {getUnqualifiedSqlIdentifier} from '`@sqlrooms/duckdb`'; +export {getUnqualifiedSqlIdentifier} from '`@sqlrooms/duckdb-core`';🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/cells/src/helpers.ts` at line 104, The current re-export pulls getUnqualifiedSqlIdentifier from '`@sqlrooms/duckdb`' but it should come from the lower-level package '`@sqlrooms/duckdb-core`'; change the export so getUnqualifiedSqlIdentifier is re-exported from '`@sqlrooms/duckdb-core`' (update the export statement in helpers.ts to reference '`@sqlrooms/duckdb-core`') to keep the helper sourced from the lowest-level package while preserving existing imports of other symbols from '`@sqlrooms/duckdb`'.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@packages/cells/src/helpers.ts`:
- Line 104: The current re-export pulls getUnqualifiedSqlIdentifier from
'`@sqlrooms/duckdb`' but it should come from the lower-level package
'`@sqlrooms/duckdb-core`'; change the export so getUnqualifiedSqlIdentifier is
re-exported from '`@sqlrooms/duckdb-core`' (update the export statement in
helpers.ts to reference '`@sqlrooms/duckdb-core`') to keep the helper sourced from
the lowest-level package while preserving existing imports of other symbols from
'`@sqlrooms/duckdb`'.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 258fca73-19a2-4fba-835d-7f03c4e0d350
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (7)
packages/cells/src/helpers.tspackages/duckdb-core/__tests__/getUnqualifiedSqlIdentifier.test.tspackages/duckdb-core/src/duckdb-utils.tspackages/duckdb-core/src/index.tspackages/duckdb/src/index.tspackages/kepler/package.jsonpackages/kepler/src/KeplerSlice.ts
💤 Files with no reviewable changes (1)
- packages/kepler/package.json
🚧 Files skipped from review as they are similar to previous changes (1)
- packages/kepler/src/KeplerSlice.ts
There was a problem hiding this comment.
Pull request overview
This PR extends table handling to better support non-main schemas by (a) separating “dataset id” (still the table name) from the user-facing label and (b) moving a shared “unqualified identifier” utility into duckdb-core and re-exporting it through higher-level packages.
Changes:
- Add
getUnqualifiedSqlIdentifierto@sqlrooms/duckdb-core, re-export via@sqlrooms/duckdb, and surface via@sqlrooms/cells. - Update Kepler dataset labeling to display unqualified table names while keeping fully-qualified IDs for internal referencing.
- Make Kepler dataset sync resilient to per-dataset load failures (non-blocking error logging).
Reviewed changes
Copilot reviewed 8 out of 9 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| pnpm-lock.yaml | Adds workspace link entry for @sqlrooms/duckdb-core under the Kepler importer. |
| packages/kepler/src/KeplerSlice.ts | Uses unqualified identifiers for dataset labels; makes sync continue past dataset load failures. |
| packages/kepler/package.json | Adds @sqlrooms/duckdb-core dependency for shared SQL identifier utilities. |
| packages/duckdb/src/index.ts | Re-exports getUnqualifiedSqlIdentifier from @sqlrooms/duckdb-core. |
| packages/duckdb-core/src/index.ts | Exposes getUnqualifiedSqlIdentifier as part of the duckdb-core public API. |
| packages/duckdb-core/src/duckdb-utils.ts | Implements the quote-aware getUnqualifiedSqlIdentifier helper. |
| packages/duckdb-core/tests/getUnqualifiedSqlIdentifier.test.ts | Updates test import to the new source location for the helper. |
| packages/cells/src/index.ts | Exports getUnqualifiedSqlIdentifier from cells public API surface. |
| packages/cells/src/helpers.ts | Removes local implementation and re-exports getUnqualifiedSqlIdentifier from @sqlrooms/duckdb. |
Files not reviewed (1)
- pnpm-lock.yaml: Language not supported
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| * - `db.schema.table` -> `table` | ||
| * - `schema."my.funny.table"` -> `my.funny.table` | ||
| */ | ||
| export function getUnqualifiedSqlIdentifier( |
There was a problem hiding this comment.
this was moved to packages/duckdb-core/src/duckdb-utils.ts
Use fully qualified table name when adding data to the map.
This allows adding data to a map from a non-main schema.