Skip to content

fix: only sync tables used by kepler - #533

Merged
bdjulbic merged 4 commits into
mainfrom
532-feat-synckeplerdatasets-should-only-add-tables-to-kepler-which-are-used-in-some-visible-layer
Apr 14, 2026
Merged

bdjulbic merged 4 commits into
mainfrom
532-feat-synckeplerdatasets-should-only-add-tables-to-kepler-which-are-used-in-some-visible-layer

Conversation

@lixun910

@lixun910 lixun910 commented Apr 13, 2026 •

Copy link
Copy Markdown
Collaborator

Summary by CodeRabbit

  • New Features

    • Replaced the add-layer control with a local add-layer button and searchable dropdown to pick and add database tables; if only one option exists it adds immediately.
  • Performance

    • Improved dataset synchronization to skip irrelevant tables and only load datasets actively referenced by each map, reducing map load work.

@coderabbitai

coderabbitai Bot commented Apr 13, 2026 •

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: c781203d-19ef-4c73-9dc2-193ffd6cb0fc

📥 Commits

Reviewing files that changed from the base of the PR and between 1521c0a and fe55e2d.

📒 Files selected for processing (1)
  • packages/kepler/src/components/CustomLayerManager.tsx
🚧 Files skipped from review as they are similar to previous changes (1)
  • packages/kepler/src/components/CustomLayerManager.tsx

📝 Walkthrough

Walkthrough

Reorders Kepler init/config flow to call dataset syncing before map config updates; limits dataset syncing to only dataIds referenced by map visState; replaces the injected Kepler AddLayerButton with a local AddTableLayerButton that reads DuckDB tables and either syncs a dataset or adds a layer.

Changes

Cohort / File(s) Summary
Kepler sync logic
packages/kepler/src/KeplerSlice.ts
Calls syncKeplerDatasets() before updateMapConfigs(); syncKeplerDatasets now gathers referenced dataIds from visState.layers[*].config.dataId and visState.filters[*].dataId, limits candidates to db.tables with schema === 'main', and only invokes addTableToMap(dataId, ...) for referenced, available, and not-yet-synced datasets (uses keplerDatasets?.[dataId] existence check).
Layer management UI
packages/kepler/src/components/CustomLayerManager.tsx
Removes AddLayerButtonFactory usage and adds AddTableLayerButton that reads db.tables via useStoreWithKepler, shows a Typeahead dropdown when multiple tables exist, and on selection either calls addTableToMap(mapId, tableName, {autoCreateLayers: true, centerMap: true}) if dataset missing or addLayer(undefined, tableName) if present. Adds local React state/hooks, imports, and dropdown styles.
Package metadata ordering
packages/mosaic/package.json
Reordered dependencies entries (zustand and zod) without version or behavioral changes.

Sequence Diagram

sequenceDiagram
    participant User
    participant CustomLayerManager
    participant ReduxStore
    participant addTableToMap
    participant addLayer

    User->>CustomLayerManager: Click "Add layer"
    CustomLayerManager->>ReduxStore: read db.tables, keplerState.visState.datasets
    ReduxStore-->>CustomLayerManager: available tables + dataset status

    alt single table available
        CustomLayerManager->>ReduxStore: is dataset present?
        ReduxStore-->>CustomLayerManager: present / missing
        alt missing
            CustomLayerManager->>addTableToMap: addTableToMap(mapId, tableName, {autoCreateLayers: true, centerMap: true})
        else present
            CustomLayerManager->>addLayer: addLayer(undefined, tableName)
        end
    else multiple tables available
        CustomLayerManager->>User: show Typeahead dropdown
        User->>CustomLayerManager: select tableName
        CustomLayerManager->>ReduxStore: is dataset present?
        ReduxStore-->>CustomLayerManager: present / missing
        alt missing
            CustomLayerManager->>addTableToMap: addTableToMap(mapId, tableName, {autoCreateLayers: true, centerMap: true})
        else present
            CustomLayerManager->>addLayer: addLayer(undefined, tableName)
        end
    end
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related issues

Possibly related PRs

Suggested reviewers

  • ilyabo

Poem

🐰
I hop through maps and datasets bright,
I pull just what the layers write.
A dropdown twitches, choices near,
I bring the table — layers cheer. ✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: optimizing syncKeplerDatasets to only sync tables that are actively used by Kepler layers and filters, rather than all tables.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch 532-feat-synckeplerdatasets-should-only-add-tables-to-kepler-which-are-used-in-some-visible-layer

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
packages/kepler/src/components/CustomLayerManager.tsx (2)

266-279: Consider adding error handling for the async operation.

The addTableToMap call is awaited but errors are not caught. If the DuckDB query or data processing fails, the error will propagate unhandled, potentially leaving the UI in an inconsistent state (e.g., user expects a layer but nothing appears).

🛡️ Proposed error handling
 const onAddTableLayer = useCallback(
   async (tableName: string) => {
     const keplerDatasets = keplerState?.visState.datasets;
     if (!keplerDatasets?.[tableName]) {
-      await addTableToMap(mapId, tableName, {
-        autoCreateLayers: true,
-        centerMap: true,
-      });
+      try {
+        await addTableToMap(mapId, tableName, {
+          autoCreateLayers: true,
+          centerMap: true,
+        });
+      } catch (error) {
+        console.error(`Failed to add table "${tableName}" to map:`, error);
+        // Optionally: show user-facing error notification
+      }
     } else {
       addLayer(undefined, tableName);
     }
   },
   [addLayer, addTableToMap, keplerState?.visState.datasets, mapId],
 );
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/kepler/src/components/CustomLayerManager.tsx` around lines 266 -
279, The onAddTableLayer callback awaits addTableToMap but lacks error handling;
wrap the await call in a try/catch inside onAddTableLayer (referencing
onAddTableLayer and the addTableToMap call) and on failure log the error (use
console.error or processLogger) and surface a user-friendly notification or
fallback (e.g., call a passed-in onError handler or show toast) so the UI
remains consistent; ensure the else branch still calls addLayer(tableName) only
when addTableToMap succeeds.

38-39: Hardcoded color may not adapt to theme.

The color #2563EB is hardcoded instead of using a theme variable. If the application supports multiple themes, this blue may not provide sufficient contrast in all modes.

Consider using a theme variable
   .add-layer-button {
     background-color: ${(props) => props.theme.sidePanelBg || props.theme.panelBackground};
-    color: `#2563EB`;
+    color: ${(props) => props.theme.activeColor || props.theme.linkBtnColor || '#2563EB'};
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@packages/kepler/src/components/CustomLayerManager.tsx` around lines 38 - 39,
Replace the hardcoded color "#2563EB" in the styled rules inside
CustomLayerManager with a theme-driven value—use something like ${(props) =>
props.theme.primary || props.theme.linkColor || '#2563EB'} instead of the
literal to ensure the component uses the active theme; update the CSS rule that
currently reads "color: `#2563EB`" to reference props.theme (matching the existing
background-color usage) and keep a sensible fallback.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@packages/kepler/src/components/CustomLayerManager.tsx`:
- Around line 266-279: The onAddTableLayer callback awaits addTableToMap but
lacks error handling; wrap the await call in a try/catch inside onAddTableLayer
(referencing onAddTableLayer and the addTableToMap call) and on failure log the
error (use console.error or processLogger) and surface a user-friendly
notification or fallback (e.g., call a passed-in onError handler or show toast)
so the UI remains consistent; ensure the else branch still calls
addLayer(tableName) only when addTableToMap succeeds.
- Around line 38-39: Replace the hardcoded color "#2563EB" in the styled rules
inside CustomLayerManager with a theme-driven value—use something like ${(props)
=> props.theme.primary || props.theme.linkColor || '#2563EB'} instead of the
literal to ensure the component uses the active theme; update the CSS rule that
currently reads "color: `#2563EB`" to reference props.theme (matching the existing
background-color usage) and keep a sensible fallback.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 9aace18b-c506-4b0f-ad7e-f631107c658c

📥 Commits

Reviewing files that changed from the base of the PR and between 7f3afac and 4cb160d.

📒 Files selected for processing (2)
  • packages/kepler/src/KeplerSlice.ts
  • packages/kepler/src/components/CustomLayerManager.tsx

@lixun910
lixun910 requested a review from ilyabo April 14, 2026 01:02

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@packages/kepler/src/components/CustomLayerManager.tsx`:
- Around line 146-152: The options array currently built in CustomLayerManager
(useMemo producing options from dbTables.filter(...).map(...)) only includes DB
tables; change it to compute the union of state.db.tables (filtered by schema
=== 'main') and existing dataset IDs from keplerState.visState.datasets so the
dropdown exposes both DB tables and already-loaded datasets (the handler for
adding layers already supports "dataset is already loaded" via addTileSetToMap).
Locate the useMemo that defines options and update its source to merge unique
names/IDs from dbTables and keplerState.visState.datasets, producing TableOption
{label, value} entries; also apply the same union logic to the other similar
options construction referenced near the second handler to ensure both menus
include datasets not present in db.tables.
- Around line 155-157: The quick fix is to await and handle promise rejections
from layer-add calls so the UI only closes on success: make the handler
(handleSelect) async, await the result of onAdd(options[0].value) (and any other
call site that invokes onAddTableLayer/addTableToMap), wrap the await in
try/catch, log or surface the error in the catch, and only proceed to close the
menu/return on successful completion; apply the same await+try/catch change to
the other branch that calls onAddTableLayer/addTableToMap.
- Around line 195-200: The add-layer button is currently removed from keyboard
navigation because tabIndex={-1}; make it keyboard-focusable by removing
tabIndex or setting it to 0 on the Button referenced by buttonRef, and update
the open flow in handleClick to programmatically move focus into the typeahead
input after opening (use buttonRef to detect open and focus the typeahead
component). Ensure the disabled logic (disabled || !options.length) remains
unchanged and only adjust focus behavior so keyboard users can tab to the
Button, activate it, and have the typeahead receive focus.

In `@packages/kepler/src/KeplerSlice.ts`:
- Around line 435-436: The code calls get().kepler.syncKeplerDatasets() before
updateMapConfigs(), causing syncKeplerDatasets() to compute referencedDataIds
from empty visState; swap the order so updateMapConfigs() runs first to hydrate
visState/layers/filters and then call get().kepler.syncKeplerDatasets(); apply
the same swap for the other occurrence around the second call site (the block at
the other occurrence referenced in the comment). Ensure the calls remain
adjacent and preserve any surrounding logic in initialize()/setConfig() flows.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 34dac182-d074-47ec-b6f5-b6d8cf6c8962

📥 Commits

Reviewing files that changed from the base of the PR and between 4cb160d and 1521c0a.

📒 Files selected for processing (3)
  • packages/kepler/src/KeplerSlice.ts
  • packages/kepler/src/components/CustomLayerManager.tsx
  • packages/mosaic/package.json
✅ Files skipped from review due to trivial changes (1)
  • packages/mosaic/package.json

Comment thread packages/kepler/src/components/CustomLayerManager.tsx Outdated
Comment thread packages/kepler/src/components/CustomLayerManager.tsx
Comment thread packages/kepler/src/components/CustomLayerManager.tsx Outdated
Comment thread packages/kepler/src/KeplerSlice.ts
@lixun910
lixun910 requested a review from bdjulbic April 14, 2026 01:48
@bdjulbic
bdjulbic merged commit b269675 into main Apr 14, 2026
4 checks passed
@bdjulbic
bdjulbic deleted the 532-feat-synckeplerdatasets-should-only-add-tables-to-kepler-which-are-used-in-some-visible-layer branch April 14, 2026 10:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: syncKeplerDatasets should only add tables to kepler which are used in some (visible) layer

2 participants