Repository navigation
feat: Deck.gl + geoarrow layers integration - #549
Conversation
Signed-off-by: Ilya Boyandin <[email protected]>
Signed-off-by: Ilya Boyandin <[email protected]>
Signed-off-by: Ilya Boyandin <[email protected]>
Signed-off-by: Ilya Boyandin <[email protected]>
Signed-off-by: Ilya Boyandin <[email protected]>
|
Warning Rate limit exceeded
Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 35 minutes and 44 seconds. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (6)
📝 WalkthroughWalkthroughAdds two new packages: Changes
Sequence DiagramsequenceDiagram
participant Client as Client Code
participant DeckJsonMap as DeckJsonMap
participant PreparedStore as PreparedDatasetStore
participant Preparer as prepareDeckDataset
participant Detect as detectGeometryColumn
participant JsonConfig as createDeckJsonConfiguration
participant ColorScale as compileColorScale
participant DeckJSON as `@deck.gl/json`
participant Renderer as Deck Renderer
Client->>DeckJsonMap: provide spec + datasets
DeckJsonMap->>DeckJsonMap: validate spec (Zod)
DeckJsonMap->>PreparedStore: syncDatasetsForConsumer
PreparedStore->>Preparer: ensureEntry -> resolveArrowTable / executeSql
Preparer->>Detect: detectGeometryColumn(table,...)
Detect-->>Preparer: ResolvedGeometryColumn
Preparer->>Preparer: build GeoArrow / GeoJSON payloads (cached)
DeckJsonMap->>JsonConfig: create config (preProcessClassProps)
JsonConfig->>ColorScale: compileColorScale(table, colorScaleSpec)
ColorScale-->>JsonConfig: accessor + legend
JsonConfig->>DeckJSON: convert spec -> deck props
DeckJSON->>Renderer: render layers
Renderer->>Client: visible map
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related issues
Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Signed-off-by: Ilya Boyandin <[email protected]>
Signed-off-by: Ilya Boyandin <[email protected]>
Signed-off-by: Ilya Boyandin <[email protected]>
Signed-off-by: Ilya Boyandin <[email protected]>
Signed-off-by: Ilya Boyandin <[email protected]>
Signed-off-by: Ilya Boyandin <[email protected]>
Signed-off-by: Ilya Boyandin <[email protected]>
Signed-off-by: Ilya Boyandin <[email protected]>
There was a problem hiding this comment.
I think we could speed up the cross filtering between the charts and the table by building MosaicProfiler to instantiate vg.table() via the mosaic package. Similar for the summaries in the table, we could use the vg.chart().
Signed-off-by: Ilya Boyandin <[email protected]>
Signed-off-by: Ilya Boyandin <[email protected]>
Signed-off-by: Ilya Boyandin <[email protected]>
Signed-off-by: Ilya Boyandin <[email protected]>
Signed-off-by: Ilya Boyandin <[email protected]>
Signed-off-by: Ilya Boyandin <[email protected]>
Signed-off-by: Ilya Boyandin <[email protected]>
Signed-off-by: Ilya Boyandin <[email protected]>
Signed-off-by: Ilya Boyandin <[email protected]>
Signed-off-by: Ilya Boyandin <[email protected]>
Signed-off-by: Ilya Boyandin <[email protected]>
Signed-off-by: Ilya Boyandin <[email protected]>
Signed-off-by: Ilya Boyandin <[email protected]>
Signed-off-by: Ilya Boyandin <[email protected]>
There was a problem hiding this comment.
Actionable comments posted: 15
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
packages/ui/src/theme/theme-provider.tsx (1)
139-150:⚠️ Potential issue | 🟡 MinorUpdate
resolvedThemewhen the system theme changes.When
themeis"system",resolvedThemeonly changes on React re-render; OS dark/light changes won’t update the context or root class.Proposed fix
const [theme, setTheme] = useState<Theme>(() => getThemePreference({defaultTheme, storageKey}), ); + const [systemTheme, setSystemTheme] = useState<ResolvedTheme>(() => + getResolvedTheme('system'), + ); + const resolvedTheme = theme === 'system' ? systemTheme : theme; // Apply theme class before paint to avoid a light-theme flash on initial load. // (useLayoutEffect on the client, fall back to useEffect in non-DOM environments) const useIsomorphicLayoutEffect = typeof window !== 'undefined' ? useLayoutEffect : useEffect; + useIsomorphicLayoutEffect(() => { + if (typeof window === 'undefined') return; + if (typeof window.matchMedia !== 'function') return; + + const mediaQuery = window.matchMedia('(prefers-color-scheme: dark)'); + const updateSystemTheme = () => + setSystemTheme(mediaQuery.matches ? 'dark' : 'light'); + + updateSystemTheme(); + mediaQuery.addEventListener('change', updateSystemTheme); + return () => mediaQuery.removeEventListener('change', updateSystemTheme); + }, []); + useIsomorphicLayoutEffect(() => { if (typeof window === 'undefined') return; const root = window.document.documentElement; root.classList.remove('light', 'dark'); - root.classList.add(getResolvedTheme(theme)); - }, [theme]); + root.classList.add(resolvedTheme); + }, [resolvedTheme]); const value = { theme, - resolvedTheme: getResolvedTheme(theme), + resolvedTheme,🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/ui/src/theme/theme-provider.tsx` around lines 139 - 150, The current useIsomorphicLayoutEffect only runs when the React prop `theme` changes, so when `theme === 'system'` OS color-scheme changes don't update `resolvedTheme` or the root class; modify the effect that uses `useIsomorphicLayoutEffect` to, when window is present and `theme === 'system'`, subscribe to window.matchMedia('(prefers-color-scheme: dark)') change events, and in the listener re-compute `getResolvedTheme(theme)`, update the documentElement classes and cause the context value's `resolvedTheme` to update (e.g., via state or forcing an update), and clean up the listener on unmount; ensure existing behavior remains for non-'system' themes.packages/mosaic/src/useMosaicClient.ts (1)
55-75:⚠️ Potential issue | 🟡 MinorRemove redundant conversion in
onQueryResultcallback.The
onQueryResultcallback inensureClient(MosaicSlice.ts line 292) already converts the result to Arrow and passes anArrowTableto the callback. ApplyingtoArrowClientResultagain in useMosaicClient line 74 is redundant—whiletoArrowClientResultreturns the value unchanged if already an Arrow table, the extra type-check overhead is unnecessary.The memo at line 60 correctly converts
clientState?.datasince the store holds raw data. But line 74 should simply pass the result directly:onQueryResult: (result) => { queryResultRef.current?.(result); },🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/mosaic/src/useMosaicClient.ts` around lines 55 - 75, The onQueryResult callback passed into ensureClient is redundantly calling toArrowClientResult before forwarding the value to queryResultRef.current even though ensureClient (see MosaicSlice.ensureClient) already provides an ArrowTable to its callback; remove the extra conversion and forward the result directly by changing the onQueryResult handler in useMosaicClient to call queryResultRef.current?.(result) instead of queryResultRef.current?.(toArrowClientResult(result)), keeping the rest of the ensureClient call and related symbols (stableId, selectionName, directSelection, queryRef) intact.packages/mosaic/src/MosaicSlice.ts (1)
189-317:⚠️ Potential issue | 🟡 Minor
queryResultCallbackis wrapped increateClientbut stored raw inensureClient.In
createClient(Line 235–238) the storedqueryResultCallbackis a wrapper that runstoArrowClientResultbefore invoking the user callback. InensureClient(Line 313) it's assigned directly tooptions.onQueryResult. Today the actual delivery path runs through the localwrappedQueryResultin both methods so this is latent, but sinceTrackedClient.queryResultCallbackis typed(result: ArrowTable) => void, a future caller that reaches intotracked.queryResultCallback(rawData)will silently get unconverted data fromensureClient-created clients. Align the two methods so the stored callback has identical semantics (either both wrapped, or both raw plus a doc comment), or drop the field if it's only used for identity.♻️ Suggested alignment (wrap in both methods)
set((state) => produce(state, (draft) => { draft.mosaic.clients[options.id] = { id: options.id, client, createdAt: Date.now(), isLoading: true, data: null, selection, - queryResultCallback: options.onQueryResult, + queryResultCallback: options.onQueryResult + ? (result: unknown) => + options.onQueryResult!(toArrowClientResult(result)) + : undefined, }; }), );🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/mosaic/src/MosaicSlice.ts` around lines 189 - 317, The stored queryResultCallback semantics are inconsistent: createClient stores a wrapper that converts raw data via toArrowClientResult before calling the user callback, but ensureClient stores options.onQueryResult raw, which can lead to callers receiving unconverted data; update ensureClient so the stored draft.mosaic.clients[options.id].queryResultCallback is the same wrapped form used in createClient (i.e., if options.onQueryResult is provided, store a function that takes the raw result, runs toArrowClientResult(result) and then calls options.onQueryResult with that ArrowTable), and ensure the TrackedClient.queryResultCallback type remains (result: ArrowTable) => void.
🧹 Nitpick comments (15)
packages/deck/src/json/colorScaleFunction.ts (1)
22-30:isColorScaleMarkerdoesn't validate thecolorScalepayload.The guard only asserts the symbol property is truthy, so any object that happens to carry the symbol (even without
colorScale, or with a non-config value) passes. Downstream,getColorScalethen returnsvalue.colorScalecast toColorScaleConfigwithout verification. Since these markers are created internally viacreateColorScaleMarker, practical risk is low, but a small additional check would make the guard self-consistent with the declared type.🛡️ Suggested tightening
export function isColorScaleMarker(value: unknown): value is ColorScaleMarker { return Boolean( value && typeof value === 'object' && (value as {[SQLROOMS_COLOR_SCALE_MARKER]?: boolean})[ SQLROOMS_COLOR_SCALE_MARKER - ], + ] && + 'colorScale' in (value as object), ); }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/deck/src/json/colorScaleFunction.ts` around lines 22 - 30, isColorScaleMarker currently only checks for the SQLROOMS_COLOR_SCALE_MARKER symbol, so objects with that symbol but missing or invalid payload pass; update isColorScaleMarker to also verify the payload by asserting the object has a colorScale property and that colorScale matches the expected shape (e.g., non-null object or expected array/structure of ColorScaleConfig) before returning true. Locate isColorScaleMarker and extend its Boolean condition to check typeof (value as any).colorScale and/or Array.isArray((value as any).colorScale) according to the intended ColorScaleConfig shape; ensure this aligns with how createColorScaleMarker constructs the marker and how getColorScale casts/uses value.colorScale.packages/color-scales/src/config.ts (1)
16-28: Optional: simplify RGBA union withz.tuple(...).rest()or length-aware tuple.The explicit 3‑tuple + 4‑tuple union works, but it duplicates the channel constraints four times. In Zod 4 you can define a single reusable channel and compose, which makes future edits safer.
♻️ Optional refactor
-export const RGBAColor = z.union([ - z.tuple([ - z.number().int().min(0).max(255), - z.number().int().min(0).max(255), - z.number().int().min(0).max(255), - ]), - z.tuple([ - z.number().int().min(0).max(255), - z.number().int().min(0).max(255), - z.number().int().min(0).max(255), - z.number().int().min(0).max(255), - ]), -]); +const channel = z.number().int().min(0).max(255); +export const RGBAColor = z.union([ + z.tuple([channel, channel, channel]), + z.tuple([channel, channel, channel, channel]), +]);🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/color-scales/src/config.ts` around lines 16 - 28, Refactor RGBAColor to avoid repeating the per-channel constraint: create a reusable constant (e.g., channel = z.number().int().min(0).max(255)) and then build RGBAColor using that symbol (for example z.tuple([channel, channel, channel]).or(z.tuple([channel, channel, channel, channel])) or a length-aware tuple/rest approach), replacing the current explicit duplicated numeric constraints inside RGBAColor so future edits only change the channel definition once.packages/deck/src/DeckJsonMapSpec.ts (1)
7-13: Consider exporting individualColorScaleConfigvariants from@sqlrooms/color-scalesto enable extending them with'@@function'before reassembling the union.Currently,
z.intersectionoverColorScaleConfig(az.discriminatedUnion) loses the discriminator fast-path optimization and produces less precise inferred types. Extending each variant with'@@function': z.literal('colorScale')and reassembling the union would preserve discriminator performance and improve type clarity. However, this requires the individual variants (SequentialColorScaleConfig,DivergingColorScaleConfig, etc.) to be exported from the color-scales package. If the package's public API is intentionally minimal, the current approach is reasonable. Non-blocking — keep as is if maintaining a smaller public API surface is preferred.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/deck/src/DeckJsonMapSpec.ts` around lines 7 - 13, The ColorScaleFunction currently intersects '@@function' with the discriminated union ColorScaleConfig which prevents Zod's discriminator fast-path and yields less precise types; to fix, export the individual variant schemas (e.g., SequentialColorScaleConfig, DivergingColorScaleConfig, etc.) from `@sqlrooms/color-scales`, then extend each variant with '@@function': z.literal('colorScale') and reassemble them into a discriminated union before exporting ColorScaleFunction so the discriminator stays intact and inference improves (update references to ColorScaleFunction and ColorScaleConfig accordingly).packages/color-scales/jest.config.js (1)
1-5: Optional: add JSDoc type annotation for parity withpackages/deck/jest.config.js.The sibling
packages/deck/jest.config.jsincludes/**@type{import('ts-jest').JestConfigWithTsJest} */. Adding the same here improves editor IntelliSense and keeps configs consistent.♻️ Proposed change
import nodeConfig from '@sqlrooms/preset-jest/node.js'; +/** `@type` {import('ts-jest').JestConfigWithTsJest} */ export default { ...nodeConfig, };🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/color-scales/jest.config.js` around lines 1 - 5, Add a JSDoc type annotation above the export to match the sibling config: place /** `@type` {import('ts-jest').JestConfigWithTsJest} */ immediately before the export default block (which currently spreads ...nodeConfig imported from '@sqlrooms/preset-jest/node.js') so editors get the same IntelliSense and parity with packages/deck/jest.config.js.packages/deck/__tests__/DeckSpec.test.ts (1)
35-52: Consider tightening the negative test.
.toThrow()will pass on any thrown error, including unrelated failures (e.g., a typo in a required field added later). Consider.toThrow(/scheme/i)or asserting against a Zod issue path to ensure the test specifically catches invalid-scheme rejection.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/deck/__tests__/DeckSpec.test.ts` around lines 35 - 52, The test in DeckSpec.test.ts currently uses a broad expect(...).toThrow() which can mask unrelated errors; update the assertion on DeckJsonMapSpec.parse(...) to specifically assert that the rejection is due to the invalid color scheme (e.g., use expect(() => DeckJsonMapSpec.parse(...)).toThrow(/scheme/i) or parse in a try/catch and assert the ZodError contains an issue with the getFillColor.scheme path), referencing the DeckJsonMapSpec.parse call and the getFillColor.scheme field so the test only passes for the intended invalid-scheme error.packages/color-scales/__tests__/schemas.test.ts (1)
1-20: Schema tests look fine.Consider also adding a negative test (e.g., invalid
schemeor missingtype) to lock in rejection behavior ofColorScaleConfig.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/color-scales/__tests__/schemas.test.ts` around lines 1 - 20, Add a negative test to assert ColorScaleConfig rejection for invalid input: create a new it block that calls ColorScaleConfig.parse (or safer: use try/catch or zod's safeParse) with an invalid payload (e.g., missing type or invalid scheme like scheme: 'BadScheme') and assert it throws or returns an error; reference ColorScaleConfig.parse in the test and mirror the existing test style used for sequential scales so the suite verifies rejection behavior as well.packages/color-scales/__tests__/scale.test.ts (1)
30-52: Strengthen legend assertions.The
if (legend?.type === 'categorical')guard means a regression wherelegendis undefined would silently skip theitems.lengthcheck and still pass. Consider assertinglegendis defined (e.g.,expect(legend).toBeDefined()) before the discriminated branch, so the length assertion is guaranteed to run.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/color-scales/__tests__/scale.test.ts` around lines 30 - 52, The tests currently use a conditional guard (if (legend?.type === ...)) that can hide a missing legend; before branching on legend.type in both the categorical and quantile/stepped tests, add an explicit assertion that legend is defined (e.g., expect(legend).toBeDefined()) and then assert the discriminant (expect(legend.type).toBe('categorical') / 'stepped') so the subsequent items length checks on buildColorScaleLegend's returned legend always run; update the two test blocks around the legend variable to remove optional chaining for the type and items checks after the presence assertion.packages/deck/src/prepare/toGeoJsonBinary.ts (1)
15-36: Hoist Arrow column lookups out of the row loop.
table.getChild(fieldName)is invokedrowCount × propertyCounttimes. For large earthquake-size datasets (50k+ rows × many columns) that's a measurable amount of schema hash-lookup work on a hot path. Resolve the vectors once and reuse:⚡ Proposed fix
- const propertyColumns = table.schema.fields - .map((field) => field.name) - .filter((fieldName) => fieldName !== geometryColumnName); + const propertyVectors = table.schema.fields + .map((field) => field.name) + .filter((fieldName) => fieldName !== geometryColumnName) + .map((fieldName) => ({fieldName, vector: table.getChild(fieldName)})); @@ - const properties: Record<string, unknown> = {}; - for (const fieldName of propertyColumns) { - properties[fieldName] = table.getChild(fieldName)?.get(rowIndex); - } + const properties: Record<string, unknown> = {}; + for (const {fieldName, vector} of propertyVectors) { + properties[fieldName] = vector?.get(rowIndex); + }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/deck/src/prepare/toGeoJsonBinary.ts` around lines 15 - 36, The loop calls table.getChild(fieldName) for every row×column which is expensive; hoist those lookups by resolving the child vectors once into an array before iterating rows (e.g. build an array like children = propertyColumns.map(name => table.getChild(name))) and then inside the for loop use children[idx]?.get(rowIndex) to populate properties; update references to propertyColumns/table.getChild/getGeometryAt/features/table.numRows accordingly and ensure you handle any undefined children when reading values.packages/deck/src/datasets/usePreparedDatasetStates.ts (1)
10-33: Module-global counter is fine for uniqueness, but consider resilience to HMR/Strict Mode.
nextPreparedDatasetConsumerIdis module-global; in dev under React 19 Strict Mode, components mount/unmount twice, which is handled correctly here since each instance grabs its own id via lazyuseState. Under HMR/module re-exec the counter resets, but ids still collide only across consumers from different module instances — typically not an issue in a single app bundle.createId()(already used inuseMosaicClient) orReact.useId()would sidestep the module-state concern if you want consistency with the rest of the codebase.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/deck/src/datasets/usePreparedDatasetStates.ts` around lines 10 - 33, The module-global nextPreparedDatasetConsumerId used in usePreparedDatasetStates can reset across HMR/module re-exec and may collide; replace the manual counter with a stable id generator already used elsewhere (e.g., call createId() as used by useMosaicClient or use React.useId()) so each consumer gets a robust unique id; update the consumer id initialization in usePreparedDatasetStates to use the chosen id generator instead of incrementing nextPreparedDatasetConsumerId and remove or deprecate the module-global counter.packages/color-scales/src/ColorScaleLegend.tsx (1)
31-43: Tick alignment relies on hard-coded offsets of 50/100.Tick labels are laid out with
justify-betweenand only re-aligned atoffset === 50or100. For any legend with >3 ticks or non-standard offsets, labels won't align with their gradient positions. The existing TODO on L9 already flags this, so it's just a heads up—consider using absolute positioning (left: ${offset}%) for more accurate placement.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/color-scales/src/ColorScaleLegend.tsx` around lines 31 - 43, The tick alignment currently assumes only offsets 50 and 100 inside the legend.ticks map in ColorScaleLegend.tsx, causing misaligned labels for other offsets; change the rendering of ticks in the map (where tick.offset and tick.label are used and className is built) to position labels absolutely relative to the gradient container using left: `${tick.offset}%` (or equivalent CSS style) and adjust text alignment (e.g., translateX or text-align) based on offset to avoid overlap; ensure the gradient container is position: relative and each tick span is position: absolute so labels align with their gradient stop instead of relying on justify-between.examples/deckgl-mosaic/src/components/map/MapView.tsx (2)
207-222: Replaceanycoercions with properly typed callback signatures from@deck.gl/core.The three
as anycoercions on deck callbacks (onHover as any,({viewState: next}: any), and{object?: any}) mask type safety.@deck.gl/coreexports proper callback types:
onViewStateChangereceives{viewState, oldViewState?, interactionState?}onHoverandgetTooltipreceivePickingInfowith object metadataWhile the destructuring
{viewState: next}correctly extracts the needed field, usinganyprevents TypeScript from catching future mistakes. Import the callback types from@deck.gl/coreto replace these coercions.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@examples/deckgl-mosaic/src/components/map/MapView.tsx` around lines 207 - 222, The deckProps callbacks in MapView use unsafe "any" coercions (onViewStateChange, onHover, getTooltip) which disables type checking; replace these with the correct types from `@deck.gl/core`: import and use the OnViewStateChange callback shape for onViewStateChange (accepting {viewState, oldViewState?, interactionState?}) and use PickingInfo for the onHover and getTooltip parameters (e.g., onHover: (info: PickingInfo) => void and getTooltip: (info: PickingInfo) => string | null), update the signatures in the deckProps object (references: deckProps, onViewStateChange, onHover, getTooltip) and remove the "as any" casts so TypeScript enforces correct payload shapes.
59-63: Remove the manualgetSelection('brush')effect;useMosaicClientalready handles selection creation.The effect on lines 59–63 is redundant. The
useMosaicClienthook below (line 69) already callsensureClientwithselectionName: 'brush', which internally invokesgetSelection('brush')and updates the store. Having both initialization paths adds unnecessary complexity and a stale selector dependency ongetSelection.The manual effect also ignores the mosaic connection status, while
useMosaicClientcorrectly gates initialization onconnectionStatus === 'ready'. Removing the manual effect simplifies the component and lets the established hook pattern handle lifecycle.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@examples/deckgl-mosaic/src/components/map/MapView.tsx` around lines 59 - 63, The useEffect that calls getSelection('brush') is redundant because useMosaicClient already ensures the 'brush' selection via ensureClient and respects connectionStatus; remove the entire useEffect block (the effect referencing brush and getSelection) to avoid duplicate initialization and the stale getSelection dependency, and rely on useMosaicClient to create/update the selection (look for useEffect, getSelection('brush'), useMosaicClient, ensureClient, and the selectionName 'brush' to locate the code to delete).packages/mosaic/src/MosaicColorLegend.tsx (1)
46-61: Add an exhaustive default to future-proof the scale-kind switch.If a new
ColorScaleKindis added to@sqlrooms/color-scales, this switch silently returnsundefinedand the downstreamvg.colorScale(undefined)failure will be hard to trace. Adefaultthat performs an exhaustive-never check would surface the gap at compile time.♻️ Proposed refactor
function getVgColorScaleType(colorScale: ColorScaleConfig) { switch (colorScale.type) { case 'sequential': return 'linear'; case 'diverging': return 'diverging'; case 'quantize': return 'quantize'; case 'quantile': return 'quantile'; case 'threshold': return 'threshold'; case 'categorical': return 'categorical'; + default: { + const _exhaustive: never = colorScale; + throw new Error( + `Unsupported colorScale.type: ${(_exhaustive as {type: string}).type}`, + ); + } } }🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/mosaic/src/MosaicColorLegend.tsx` around lines 46 - 61, The switch in getVgColorScaleType(colorScale: ColorScaleConfig) is non-exhaustive and can return undefined if a new ColorScaleKind is added; update the switch to include a default case that does an exhaustive-never check (e.g., accept a never-typed value or throw with a clear message) so the compiler will error for unknown ColorScale kinds and avoid passing undefined into vg.colorScale; reference getVgColorScaleType and ColorScaleConfig (and the ColorScaleKind union) when applying the fix.packages/deck/src/datasets/helpers.ts (1)
8-21: Minor: prefer a single-declaration export and consider whether stable identities need to survive HMR.
- The
function nextAccessTimestamp()+ separateexport {...}(Line 20–21) can just beexport function nextAccessTimestamp()for consistency with the other helpers in this file.- Module-level
nextTableIdentity/nextSqlSourceIdentitycounters paired with module-scopedWeakMaps are fine for runtime, but note that any test assertion that depends on the literal id (e.g.table:1) will be sensitive to test ordering within a Jest worker. The newpreparedDatasetStore.test.tssuite should avoid matching identity strings directly.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/deck/src/datasets/helpers.ts` around lines 8 - 21, Change the standalone declaration and export of nextAccessTimestamp to an inline exported function (export function nextAccessTimestamp()) to match the other helpers; leave the module-scoped counters nextTableIdentity/nextSqlSourceIdentity and WeakMaps tableIdentities/sqlSourceIdentities as-is for runtime behavior, but update any tests (e.g., preparedDatasetStore.test.ts) that assert on literal identity strings like "table:1" to avoid depending on module-level counter values across Jest workers/HMR (use pattern matching or compare semantic properties instead).packages/deck/src/json/compileColorScale.ts (1)
80-127: Materialized column accessed twice; GeoArrow batch lookup is undocumented internal structure.Two observations worth addressing:
getColumnValues(vector)materializes the entire vector into an array. When bothcompileColorScaleandbuildColorScaleLegendare invoked for the same dataset during spec conversion, this full column is materialized twice. For multi-million-row Arrow tables this is avoidable overhead. Consider memoizing per(table, field)pair or computing the materialized array once and passing it through both functions.The GeoArrow fast path (lines 86–99) accesses
value.data?.dataas a{data: {get: (i: number) => unknown}}structure. This pattern also appears incompileGeoArrowAccessor.tsbut relies on internal structure from@geoarrow/[email protected]rather than a documented public contract. Document which GeoArrow version this shape was verified against and note that it may shift in future releases.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/deck/src/json/compileColorScale.ts` around lines 80 - 127, compileColorScale currently materializes the full column via getColumnValues(vector) every time and also implements an undocumented GeoArrow fast-path that reads value.data?.data; to fix this, memoize the materialized array per unique (table, field) pair (compute once where getColumn(table, colorScale.field) is called and pass the resulting values array into both compileColorScale and buildColorScaleLegend instead of calling getColumnValues twice) and update getGeoArrowOrRowValue to avoid relying on internal shapes (or at minimum annotate the function with a clear comment citing the exact `@geoarrow/deck.gl-layers` version and the expected structure being accessed), referencing the functions getColumn, compileColorScale, buildColorScaleLegend, getColumnValues, and getGeoArrowOrRowValue so the change is easy to locate.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@examples/deckgl-mosaic/README.md`:
- Line 15: Replace the typographic right single quote in the phrase "Mosaic’s"
with a straight ASCII apostrophe to match the repo's documentation style; search
for the string "Mosaic’s" in the README where the sentence mentions the global
Coordinator and DeckJsonMap and change it to "Mosaic's" so the prose is
consistent with other docs.
In `@packages/color-scales/README.md`:
- Around line 253-264: The README documents the wrong deck JSON function
discriminator (`sqlroomsColorScale`) causing integration breakage; update the
text to state the actual deck function discriminator `@@function: 'colorScale'`
(while keeping references to ColorScaleConfig, getFillColor/getLineColor and
MosaicColorLegend) so examples and compatibility guidance use
`{"@@function":"colorScale", ...}` instead of `sqlroomsColorScale`.
In `@packages/color-scales/src/scale.ts`:
- Around line 399-404: The current guard returns createNullColorAccessor when
numericValues is empty, which incorrectly disables explicit-domain
sequential/diverging/quantize and threshold scales; instead, only treat empty
samples as fatal for scales that require sampling (auto-domain or quantile).
Change the logic around
getNumericValues/createNullColorAccessor/getNumericScale: compute numericValues,
then if numericValues is empty check colorScale.type and colorScale.domainMode
(or the equivalent properties used to denote 'auto' vs explicit and 'quantile'
vs other kinds) and only return createNullColorAccessor(nullColor) for
auto-domain or quantile scales; otherwise proceed to call
getNumericScale(colorScale, numericValues) and build the accessor. Apply the
same conditional change to the other identical block referenced (the section
around lines 448-453 that also uses numericValues/getNumericScale).
- Around line 44-46: The bigint branch in coerceFiniteNumber currently returns
Number(value) which can yield Infinity for very large BigInts; change the bigint
handling to convert to a Number, then validate with Number.isFinite (same
pattern as the number and string branches) and return undefined if not finite so
coerceFiniteNumber preserves its finite-number-or-undefined contract. Ensure you
update the bigint branch inside coerceFiniteNumber to mirror the Number.isFinite
check used for the number and string cases.
In `@packages/db/package.json`:
- Around line 47-50: The package.json lists "@jest/globals": "^30.3.0" while
"jest" is still "^30.1.3" causing a minor-version skew with the lockfile; update
the "jest" dependency in package.json to "^30.3.0" (matching "@jest/globals")
and then regenerate the lockfile (run npm install or yarn install) so the
resolved versions align; target the "jest" entry in package.json to make this
change.
In `@packages/deck/__tests__/compileColorScale.test.ts`:
- Around line 127-162: The tests dereference legend (from buildColorScaleLegend)
without guarding for null even though buildColorScaleLegend returns
ResolvedColorLegend | null; update the tests to handle nullable returns by
either asserting non-null with a `!` when you are certain the input yields a
legend or adding an explicit null check (e.g., expect(legend).not.toBeNull() or
if (legend) { ... }) before accessing legend.type, legend.title,
legend.gradient, legend.ticks, or legend.items so the assertions are type-safe
under strict null checks and clearly reference the buildColorScaleLegend result.
In `@packages/deck/package.json`:
- Line 8: The package.json currently sets "sideEffects": false which causes CSS
side-effect imports used by DeckJsonMap to be dropped; update the package
metadata so CSS is preserved by replacing "sideEffects": false with an array
that includes the CSS files imported by DeckJsonMap (for example a glob like
["**/*.css"] or the exact MapLibre CSS path that DeckJsonMap imports) so
bundlers will retain those side-effectful CSS imports.
In `@packages/deck/src/datasets/normalizeDatasets.ts`:
- Around line 10-27: normalizeDatasetEntry currently treats any non-SQL input as
Arrow, causing invalid objects (e.g., {geometryColumn: 'g'}) to become
{arrowTable: undefined}; update normalizeDatasetEntry to explicitly check
isArrowTableDatasetInput(input) for the Arrow branch and if neither
isSqlDatasetInput(input) nor isArrowTableDatasetInput(input) throw a descriptive
Error (include the dataset id and problematic input) so bad inputs fail loudly;
reference normalizeDatasetEntry, isSqlDatasetInput, and isArrowTableDatasetInput
when making the change.
In `@packages/deck/src/datasets/PreparedDatasetStore.ts`:
- Around line 349-361: The bug is that get().syncConsumer(consumerId, cacheKeys)
runs before ensureEntry creates missing entries so new entries are created with
empty consumers and can be evicted; fix by calling get().ensureEntry(...) for
every descriptor first (use the same descriptors/cacheKey logic in
PreparedDatasetStore where ensureEntry is called) and only after the loop invoke
get().syncConsumer(consumerId, cacheKeys) (or compute cacheKeys from descriptors
and pass them into syncConsumer) so newly-created entries receive the consumerId
and are not considered unreferenced by evictLruEntries.
In `@packages/deck/src/datasets/usePreparedDatasetStates.ts`:
- Around line 36-57: The hook currently depends directly on the datasets record
causing thrashing when callers pass inline objects; modify
usePreparedDatasetStates to derive a stableDatasets reference (e.g., keep a ref
of the previous datasets and perform a shallow equality check on top-level keys
and per-entry identity like sqlQuery/arrowTable) and use that stableDatasets in
the effect dependency list for
preparedDatasetStore.getState().syncDatasetsForConsumer and in the useMemo call
to resolvePreparedDeckDatasetStates; ensure removeConsumer and
syncDatasetsForConsumer still receive consumerId, executeSql and
sqlSourceIdentity (connector) but only re-run when the shallow-stable signature
actually changes.
In `@packages/deck/src/json/layerConfig.ts`:
- Around line 8-22: The two helpers disagree because hasExtensionKeys currently
uses the in-operator and returns true for {_sqlroomsBinding: undefined}; update
hasExtensionKeys to mirror getLayerConfig's criteria by checking that
props._sqlroomsBinding is a non-null object (e.g., typeof
layerProps._sqlroomsBinding === 'object' && layerProps._sqlroomsBinding != null)
so "has extension" only returns true when a usable binding object exists;
reference the hasExtensionKeys and getLayerConfig functions and ensure the same
truthy-object logic is used.
In `@packages/deck/src/prepare/detectGeometryColumn.ts`:
- Around line 53-63: The inferEncodingFromHint function currently drops the
"geoarrow" hint and returns undefined for it; update inferEncodingFromHint to
handle the 'geoarrow' case and return 'geoarrow' so an explicit
geometryEncodingHint is preserved, and make the same change in the other
identical branch referenced (around the second occurrence at lines 110-113) so
both locations return 'geoarrow' when the hint is 'geoarrow' (refer to the
function name inferEncodingFromHint to locate the code).
In `@packages/deck/src/prepare/wkbDecoder.ts`:
- Around line 131-158: supportsGeoArrowPromotion currently only checks the first
non-null geometry via getSampleGeometry which can disagree with
createPromotedPointTable and toGeoArrowLike on mixed-type columns; fix by
extracting a shared validator (e.g., validateAllPointGeometries) used by both
supportsGeoArrowPromotion and the promotion/conversion path
(createPromotedPointTable / toGeoArrowLike) that scans the entire column (or
caches a one-pass result keyed by table+column+encoding) and returns either "all
Point|null" or the offending type, then use that boolean/result in
supportsGeoArrowPromotion to only claim promotion when every non-null row is a
Point and reuse the same validation result during conversion to avoid
mid-pipeline failures.
In `@packages/deck/src/types.ts`:
- Around line 37-40: The union DeckDatasetInput is ambiguous because
DeckArrowTableDatasetInput currently has an optional arrowTable, so objects like
{geometryColumn: 'g'} match neither discriminant and normalizeDatasetEntry can
incorrectly treat them as Arrow entries; fix by making the discriminant
explicit: either change DeckArrowTableDatasetInput.arrowTable to a required
property (e.g., DeckTable | undefined) so the presence/absence is unambiguous,
or add a string discriminant (e.g., kind: 'sql' | 'arrow') to both
DeckSqlDatasetInput and DeckArrowTableDatasetInput and update the type, the
factory/normalizer (normalizeDatasetEntry) and predicates (isSqlDatasetInput,
isArrowTableDatasetInput) to use that discriminant. Ensure all uses of
DeckDatasetInput, isSqlDatasetInput, isArrowTableDatasetInput, and
normalizeDatasetEntry are updated to rely on the chosen discriminant.
---
Outside diff comments:
In `@packages/mosaic/src/MosaicSlice.ts`:
- Around line 189-317: The stored queryResultCallback semantics are
inconsistent: createClient stores a wrapper that converts raw data via
toArrowClientResult before calling the user callback, but ensureClient stores
options.onQueryResult raw, which can lead to callers receiving unconverted data;
update ensureClient so the stored
draft.mosaic.clients[options.id].queryResultCallback is the same wrapped form
used in createClient (i.e., if options.onQueryResult is provided, store a
function that takes the raw result, runs toArrowClientResult(result) and then
calls options.onQueryResult with that ArrowTable), and ensure the
TrackedClient.queryResultCallback type remains (result: ArrowTable) => void.
In `@packages/mosaic/src/useMosaicClient.ts`:
- Around line 55-75: The onQueryResult callback passed into ensureClient is
redundantly calling toArrowClientResult before forwarding the value to
queryResultRef.current even though ensureClient (see MosaicSlice.ensureClient)
already provides an ArrowTable to its callback; remove the extra conversion and
forward the result directly by changing the onQueryResult handler in
useMosaicClient to call queryResultRef.current?.(result) instead of
queryResultRef.current?.(toArrowClientResult(result)), keeping the rest of the
ensureClient call and related symbols (stableId, selectionName, directSelection,
queryRef) intact.
In `@packages/ui/src/theme/theme-provider.tsx`:
- Around line 139-150: The current useIsomorphicLayoutEffect only runs when the
React prop `theme` changes, so when `theme === 'system'` OS color-scheme changes
don't update `resolvedTheme` or the root class; modify the effect that uses
`useIsomorphicLayoutEffect` to, when window is present and `theme === 'system'`,
subscribe to window.matchMedia('(prefers-color-scheme: dark)') change events,
and in the listener re-compute `getResolvedTheme(theme)`, update the
documentElement classes and cause the context value's `resolvedTheme` to update
(e.g., via state or forcing an update), and clean up the listener on unmount;
ensure existing behavior remains for non-'system' themes.
---
Nitpick comments:
In `@examples/deckgl-mosaic/src/components/map/MapView.tsx`:
- Around line 207-222: The deckProps callbacks in MapView use unsafe "any"
coercions (onViewStateChange, onHover, getTooltip) which disables type checking;
replace these with the correct types from `@deck.gl/core`: import and use the
OnViewStateChange callback shape for onViewStateChange (accepting {viewState,
oldViewState?, interactionState?}) and use PickingInfo for the onHover and
getTooltip parameters (e.g., onHover: (info: PickingInfo) => void and
getTooltip: (info: PickingInfo) => string | null), update the signatures in the
deckProps object (references: deckProps, onViewStateChange, onHover, getTooltip)
and remove the "as any" casts so TypeScript enforces correct payload shapes.
- Around line 59-63: The useEffect that calls getSelection('brush') is redundant
because useMosaicClient already ensures the 'brush' selection via ensureClient
and respects connectionStatus; remove the entire useEffect block (the effect
referencing brush and getSelection) to avoid duplicate initialization and the
stale getSelection dependency, and rely on useMosaicClient to create/update the
selection (look for useEffect, getSelection('brush'), useMosaicClient,
ensureClient, and the selectionName 'brush' to locate the code to delete).
In `@packages/color-scales/__tests__/scale.test.ts`:
- Around line 30-52: The tests currently use a conditional guard (if
(legend?.type === ...)) that can hide a missing legend; before branching on
legend.type in both the categorical and quantile/stepped tests, add an explicit
assertion that legend is defined (e.g., expect(legend).toBeDefined()) and then
assert the discriminant (expect(legend.type).toBe('categorical') / 'stepped') so
the subsequent items length checks on buildColorScaleLegend's returned legend
always run; update the two test blocks around the legend variable to remove
optional chaining for the type and items checks after the presence assertion.
In `@packages/color-scales/__tests__/schemas.test.ts`:
- Around line 1-20: Add a negative test to assert ColorScaleConfig rejection for
invalid input: create a new it block that calls ColorScaleConfig.parse (or
safer: use try/catch or zod's safeParse) with an invalid payload (e.g., missing
type or invalid scheme like scheme: 'BadScheme') and assert it throws or returns
an error; reference ColorScaleConfig.parse in the test and mirror the existing
test style used for sequential scales so the suite verifies rejection behavior
as well.
In `@packages/color-scales/jest.config.js`:
- Around line 1-5: Add a JSDoc type annotation above the export to match the
sibling config: place /** `@type` {import('ts-jest').JestConfigWithTsJest} */
immediately before the export default block (which currently spreads
...nodeConfig imported from '@sqlrooms/preset-jest/node.js') so editors get the
same IntelliSense and parity with packages/deck/jest.config.js.
In `@packages/color-scales/src/ColorScaleLegend.tsx`:
- Around line 31-43: The tick alignment currently assumes only offsets 50 and
100 inside the legend.ticks map in ColorScaleLegend.tsx, causing misaligned
labels for other offsets; change the rendering of ticks in the map (where
tick.offset and tick.label are used and className is built) to position labels
absolutely relative to the gradient container using left: `${tick.offset}%` (or
equivalent CSS style) and adjust text alignment (e.g., translateX or text-align)
based on offset to avoid overlap; ensure the gradient container is position:
relative and each tick span is position: absolute so labels align with their
gradient stop instead of relying on justify-between.
In `@packages/color-scales/src/config.ts`:
- Around line 16-28: Refactor RGBAColor to avoid repeating the per-channel
constraint: create a reusable constant (e.g., channel =
z.number().int().min(0).max(255)) and then build RGBAColor using that symbol
(for example z.tuple([channel, channel, channel]).or(z.tuple([channel, channel,
channel, channel])) or a length-aware tuple/rest approach), replacing the
current explicit duplicated numeric constraints inside RGBAColor so future edits
only change the channel definition once.
In `@packages/deck/__tests__/DeckSpec.test.ts`:
- Around line 35-52: The test in DeckSpec.test.ts currently uses a broad
expect(...).toThrow() which can mask unrelated errors; update the assertion on
DeckJsonMapSpec.parse(...) to specifically assert that the rejection is due to
the invalid color scheme (e.g., use expect(() =>
DeckJsonMapSpec.parse(...)).toThrow(/scheme/i) or parse in a try/catch and
assert the ZodError contains an issue with the getFillColor.scheme path),
referencing the DeckJsonMapSpec.parse call and the getFillColor.scheme field so
the test only passes for the intended invalid-scheme error.
In `@packages/deck/src/datasets/helpers.ts`:
- Around line 8-21: Change the standalone declaration and export of
nextAccessTimestamp to an inline exported function (export function
nextAccessTimestamp()) to match the other helpers; leave the module-scoped
counters nextTableIdentity/nextSqlSourceIdentity and WeakMaps
tableIdentities/sqlSourceIdentities as-is for runtime behavior, but update any
tests (e.g., preparedDatasetStore.test.ts) that assert on literal identity
strings like "table:1" to avoid depending on module-level counter values across
Jest workers/HMR (use pattern matching or compare semantic properties instead).
In `@packages/deck/src/datasets/usePreparedDatasetStates.ts`:
- Around line 10-33: The module-global nextPreparedDatasetConsumerId used in
usePreparedDatasetStates can reset across HMR/module re-exec and may collide;
replace the manual counter with a stable id generator already used elsewhere
(e.g., call createId() as used by useMosaicClient or use React.useId()) so each
consumer gets a robust unique id; update the consumer id initialization in
usePreparedDatasetStates to use the chosen id generator instead of incrementing
nextPreparedDatasetConsumerId and remove or deprecate the module-global counter.
In `@packages/deck/src/DeckJsonMapSpec.ts`:
- Around line 7-13: The ColorScaleFunction currently intersects '@@function'
with the discriminated union ColorScaleConfig which prevents Zod's discriminator
fast-path and yields less precise types; to fix, export the individual variant
schemas (e.g., SequentialColorScaleConfig, DivergingColorScaleConfig, etc.) from
`@sqlrooms/color-scales`, then extend each variant with '@@function':
z.literal('colorScale') and reassemble them into a discriminated union before
exporting ColorScaleFunction so the discriminator stays intact and inference
improves (update references to ColorScaleFunction and ColorScaleConfig
accordingly).
In `@packages/deck/src/json/colorScaleFunction.ts`:
- Around line 22-30: isColorScaleMarker currently only checks for the
SQLROOMS_COLOR_SCALE_MARKER symbol, so objects with that symbol but missing or
invalid payload pass; update isColorScaleMarker to also verify the payload by
asserting the object has a colorScale property and that colorScale matches the
expected shape (e.g., non-null object or expected array/structure of
ColorScaleConfig) before returning true. Locate isColorScaleMarker and extend
its Boolean condition to check typeof (value as any).colorScale and/or
Array.isArray((value as any).colorScale) according to the intended
ColorScaleConfig shape; ensure this aligns with how createColorScaleMarker
constructs the marker and how getColorScale casts/uses value.colorScale.
In `@packages/deck/src/json/compileColorScale.ts`:
- Around line 80-127: compileColorScale currently materializes the full column
via getColumnValues(vector) every time and also implements an undocumented
GeoArrow fast-path that reads value.data?.data; to fix this, memoize the
materialized array per unique (table, field) pair (compute once where
getColumn(table, colorScale.field) is called and pass the resulting values array
into both compileColorScale and buildColorScaleLegend instead of calling
getColumnValues twice) and update getGeoArrowOrRowValue to avoid relying on
internal shapes (or at minimum annotate the function with a clear comment citing
the exact `@geoarrow/deck.gl-layers` version and the expected structure being
accessed), referencing the functions getColumn, compileColorScale,
buildColorScaleLegend, getColumnValues, and getGeoArrowOrRowValue so the change
is easy to locate.
In `@packages/deck/src/prepare/toGeoJsonBinary.ts`:
- Around line 15-36: The loop calls table.getChild(fieldName) for every
row×column which is expensive; hoist those lookups by resolving the child
vectors once into an array before iterating rows (e.g. build an array like
children = propertyColumns.map(name => table.getChild(name))) and then inside
the for loop use children[idx]?.get(rowIndex) to populate properties; update
references to
propertyColumns/table.getChild/getGeometryAt/features/table.numRows accordingly
and ensure you handle any undefined children when reading values.
In `@packages/mosaic/src/MosaicColorLegend.tsx`:
- Around line 46-61: The switch in getVgColorScaleType(colorScale:
ColorScaleConfig) is non-exhaustive and can return undefined if a new
ColorScaleKind is added; update the switch to include a default case that does
an exhaustive-never check (e.g., accept a never-typed value or throw with a
clear message) so the compiler will error for unknown ColorScale kinds and avoid
passing undefined into vg.colorScale; reference getVgColorScaleType and
ColorScaleConfig (and the ColorScaleKind union) when applying the fix.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 80b5d4d4-68ab-4f11-aaa9-c5ccf7ca64db
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (76)
AGENTS.mddocs/upgrade-guide.mdexamples/deckgl-mosaic/README.mdexamples/deckgl-mosaic/package.jsonexamples/deckgl-mosaic/src/components/map/MapView.tsxexamples/deckgl-mosaic/src/components/map/utils.tsexamples/deckgl/README.mdexamples/deckgl/package.jsonexamples/deckgl/src/components/MainView.tsxexamples/deckgl/src/components/MapView.tsxpackage.jsonpackages/color-scales/README.mdpackages/color-scales/__tests__/scale.test.tspackages/color-scales/__tests__/schemas.test.tspackages/color-scales/eslint.config.jspackages/color-scales/jest.config.jspackages/color-scales/package.jsonpackages/color-scales/src/ColorScaleLegend.tsxpackages/color-scales/src/colorSchemes.tspackages/color-scales/src/config.tspackages/color-scales/src/index.tspackages/color-scales/src/scale.tspackages/color-scales/tsconfig.jsonpackages/color-scales/typedoc.jspackages/crdt/package.jsonpackages/db/package.jsonpackages/deck/README.mdpackages/deck/__tests__/DeckSpec.test.tspackages/deck/__tests__/compileColorScale.test.tspackages/deck/__tests__/createDeckJsonConfiguration.test.tspackages/deck/__tests__/createDeckJsonSpecFromDatasets.test.tspackages/deck/__tests__/normalizeDatasets.test.tspackages/deck/__tests__/prepareDeckDataset.test.tspackages/deck/__tests__/preparedDatasetStore.test.tspackages/deck/eslint.config.jspackages/deck/jest.config.jspackages/deck/package.jsonpackages/deck/src/DeckJsonMap.tsxpackages/deck/src/DeckJsonMapSpec.tspackages/deck/src/createDeckJsonSpecFromDatasets.tspackages/deck/src/datasets/PreparedDatasetStore.tspackages/deck/src/datasets/helpers.tspackages/deck/src/datasets/normalizeDatasets.tspackages/deck/src/datasets/tableAdapter.tspackages/deck/src/datasets/types.tspackages/deck/src/datasets/usePreparedDatasetStates.tspackages/deck/src/index.tspackages/deck/src/json/colorScaleFunction.tspackages/deck/src/json/compileColorScale.tspackages/deck/src/json/compileGeoArrowAccessor.tspackages/deck/src/json/createDeckJsonConfiguration.tspackages/deck/src/json/defaultClasses.tspackages/deck/src/json/extractColorScaleLegends.tspackages/deck/src/json/layerCompatibility.tspackages/deck/src/json/layerConfig.tspackages/deck/src/json/rewriteGeoArrowAccessors.tspackages/deck/src/prepare/detectGeometryColumn.tspackages/deck/src/prepare/geoarrow.tspackages/deck/src/prepare/geometryDecoder.tspackages/deck/src/prepare/prepareDeckDataset.tspackages/deck/src/prepare/toGeoJsonBinary.tspackages/deck/src/prepare/types.tspackages/deck/src/prepare/wkbDecoder.tspackages/deck/src/types.tspackages/deck/tsconfig.jsonpackages/deck/typedoc.jspackages/mosaic/README.mdpackages/mosaic/__tests__/tableInterop.test.tspackages/mosaic/package.jsonpackages/mosaic/src/MosaicColorLegend.tsxpackages/mosaic/src/MosaicSlice.tspackages/mosaic/src/index.tspackages/mosaic/src/profiler/useMosaicProfiler.tspackages/mosaic/src/tableInterop.tspackages/mosaic/src/useMosaicClient.tspackages/ui/src/theme/theme-provider.tsx
💤 Files with no reviewable changes (1)
- examples/deckgl-mosaic/src/components/map/utils.ts
| function inferEncodingFromHint( | ||
| hint: GeometryEncodingHint | undefined, | ||
| ): ResolvedGeometryEncoding | undefined { | ||
| switch (hint) { | ||
| case 'wkb': | ||
| return 'wkb'; | ||
| case 'wkt': | ||
| return 'wkt'; | ||
| default: | ||
| return undefined; | ||
| } |
There was a problem hiding this comment.
Don’t silently ignore geometryEncodingHint: "geoarrow".
The public hint type allows "geoarrow", but this branch drops it. Without metadata, native GeoArrow vectors can resolve as "unknown" despite the explicit hint.
Proposed fix
const metadataEncoding =
normalizeEncoding(fieldMetadata[detectedGeometryColumn]?.encoding) ??
normalizeEncoding(geoMetadata?.columns?.[detectedGeometryColumn]?.encoding);
+ if (geometryEncodingHint === 'geoarrow' && !metadataEncoding) {
+ throw new Error(
+ 'geometryEncodingHint "geoarrow" requires GeoArrow schema metadata to identify the concrete geometry type.',
+ );
+ }
+
const encoding =
inferEncodingFromHint(geometryEncodingHint) ??
metadataEncoding ??
inferEncodingFromVector(vector);Also applies to: 110-113
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/deck/src/prepare/detectGeometryColumn.ts` around lines 53 - 63, The
inferEncodingFromHint function currently drops the "geoarrow" hint and returns
undefined for it; update inferEncodingFromHint to handle the 'geoarrow' case and
return 'geoarrow' so an explicit geometryEncodingHint is preserved, and make the
same change in the other identical branch referenced (around the second
occurrence at lines 110-113) so both locations return 'geoarrow' when the hint
is 'geoarrow' (refer to the function name inferEncodingFromHint to locate the
code).
Signed-off-by: Ilya Boyandin <[email protected]>
There was a problem hiding this comment.
Actionable comments posted: 7
🧹 Nitpick comments (2)
packages/mosaic/src/MosaicSlice.ts (1)
367-397: Update the adapter docs to match the new interop path.The implementation now uses
createMosaicTableFromArrowTable, but the JSDoc still describes the old direct IPC/decodeIPCconversion path.📝 Proposed doc update
- * For `'arrow'` queries the Apache Arrow table returned by the connector is - * serialized to IPC bytes and decoded via {`@link` decodeIPC} into a flechette - * `Table`, which is the shape Mosaic consumers expect (with `.toColumns()`). + * For `'arrow'` queries the Apache Arrow table returned by the connector is + * adapted via {`@link` createMosaicTableFromArrowTable} into a flechette + * `Table`, preserving interop metadata so Arrow results can be recovered.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@packages/mosaic/src/MosaicSlice.ts` around lines 367 - 397, The JSDoc above createDuckDbMosaicConnector is out of date: update the doc to describe that Arrow queries are converted by calling createMosaicTableFromArrowTable on the Arrow Table returned by DuckDbConnector (instead of describing the old IPC bytes/decodeIPC path), and remove references to decodeIPC/IPC serialization; keep notes about JSON path using Array.from and the intentional as any adapter casts, and reference DuckDbConnector, Connector, and the query methods in the updated wording.examples/deckgl-mosaic/src/components/map/MapView.tsx (1)
206-214: Optional: format numeric tooltip fields and harden against odd values.
object.Depthis rendered raw, which can yield long decimals (e.g.,10.123456789km). Also,Number(object.Magnitude).toFixed(1)will print"NaN"if Magnitude is missing. Low-impact polish for an example:✨ Proposed tweak
- getTooltip: ({object}: {object?: any}) => - !enableBrushing && - object && { - html: `<div style="font-family:system-ui; font-size:12px; padding:4px;"> - <strong>M ${Number(object.Magnitude).toFixed(1)}</strong><br/> - Depth: ${object.Depth}km<br/> - ${String(object.dateLabel ?? '')} - </div>`, - }, + getTooltip: ({object}: {object?: any}) => { + if (enableBrushing || !object) return null; + const mag = Number(object.Magnitude); + const depth = Number(object.Depth); + return { + html: `<div style="font-family:system-ui; font-size:12px; padding:4px;"> + <strong>M ${Number.isFinite(mag) ? mag.toFixed(1) : '—'}</strong><br/> + Depth: ${Number.isFinite(depth) ? depth.toFixed(1) : '—'}km<br/> + ${String(object.dateLabel ?? '')} + </div>`, + }; + },🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@examples/deckgl-mosaic/src/components/map/MapView.tsx` around lines 206 - 214, In getTooltip, sanitize and format numeric fields and guard against missing/odd values: ensure Number(object.Magnitude) is checked (or fallback to null/empty) before calling toFixed(1) and render a fallback like "M —" when not a valid number; format object.Depth to a fixed number of decimals (e.g., toFixed(1) or rounded) only if it is numeric and append "km", otherwise show a safe fallback like "Depth —"; also ensure dateLabel is coerced to string safely (e.g., default to empty string) so the tooltip never shows "NaN" or "undefined" for the fields referenced in getTooltip.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@packages/color-scales/src/scale.ts`:
- Around line 166-188: Replace the spread-based Math.min/Math.max calls in
getSequentialDomain and getDivergingDomain with an explicit loop-based extent
calculation to avoid argument-limit RangeErrors: iterate over the values array
to compute min and max (e.g., initialize min = Infinity, max = -Infinity and
update in a for loop), then reuse the existing branch logic in
getSequentialDomain and getDivergingDomain (including the equal-value and
diverging-zero handling) and return the same typed tuples.
In `@packages/deck/src/datasets/helpers.ts`:
- Around line 58-102: The cache key builder (resolvePreparedDatasetCacheKey and
buildGeometryKey) currently joins parts with '\u0001', which can collide when
parts contain that delimiter; change it to produce a structured, collision-safe
key by encoding parts as a JSON array or using length-prefixed concatenation
instead of a simple join. Update resolvePreparedDatasetCacheKey to serialize the
array ['sql'|'arrow', getSqlSourceIdentity(sqlSourceIdentity) or
getTableIdentity(table), input.sqlQuery (for SQL) or omitted, and
buildGeometryKey(input)] with JSON.stringify (or equivalent stable encoder) and
ensure buildGeometryKey returns a structured object or tuple (e.g., {column,
encodingHint}) that is included in the serialized key so distinct inputs cannot
collide. Ensure thrown error behavior in resolvePreparedDatasetCacheKey remains
unchanged and callers still expect a string | undefined.
- Around line 80-92: The cache key for SQL datasets omits data-revision info, so
include the optional version in the key and propagate it through the call chain:
update resolvePreparedDatasetCacheKey (the function containing the shown branch)
to accept a version (or include version from sqlSourceIdentity) and append it to
the returned join alongside 'sql', getSqlSourceIdentity(sqlSourceIdentity),
input.sqlQuery and buildGeometryKey(input); then update callers
(usePreparedDatasetStates and any ensureEntry/executeSql invocations) to pass
the same version through to resolvePreparedDatasetCacheKey and to executeSql so
prepared entries are invalidated when version changes.
In `@packages/deck/src/datasets/normalizeDatasets.ts`:
- Around line 45-52: normalizeDatasets currently throws synchronously on an
empty datasets record which can crash component renders; document the contract
and prevent unexpected throws by (1) adding a clear JSDoc to normalizeDatasets
describing that it throws when passed an empty record (mentioning the error
message) and (2) adding a guard at the caller (e.g., DeckJsonMap or its loader)
to check datasets is non-empty before calling normalizeDatasets (or handle the
empty case with a no-op render), referencing the normalizeDatasets function and
the DeckJsonMap consumer to locate where to apply the guard.
In `@packages/deck/src/prepare/wkbDecoder.ts`:
- Around line 37-48: The handler currently routes all non-WKT encodings to WKB
parsing, which breaks when encoding === 'unknown'; update parseGeometryValue to
treat encoding === 'unknown' specially by type-checking the incoming value: if
encoding === 'wkt' || encoding === 'geoarrow.wkt' || (encoding === 'unknown' &&
typeof value === 'string') call WKTLoader.parseTextSync with {wkt:{crs:false}},
otherwise (encoding === 'unknown' with non-string or other binary encodings)
call WKBLoader.parseSync with toArrayBuffer(value) and
{wkb:{shape:'geojson-geometry'}}; apply the same change to the similar branch in
tryPromotePointTable so string unknowns go to WKT parsing and binaries go to WKB
parsing.
- Around line 96-113: The loop that builds points for promotion must normalize
and validate coordinates: when reading geometry via parseGeometryValue (and
using geometry.coordinates), ensure you extract only the first two elements [x,
y] (drop z/m) and verify both are finite numbers (Number.isFinite) before
pushing into points; if the geometry is non-Point, has missing coords, or
contains non-finite values, push null (or return null only where appropriate) so
that vectorFromArray(promotedGeometry, geomType) receives only 2D finite
coordinate pairs or nulls; update the loop around parseGeometryValue, points,
and the FixedSizeList(2) promotion to perform this normalization and validation
prior to calling vectorFromArray.
In `@packages/mosaic/src/MosaicSlice.ts`:
- Around line 54-62: ensureClient currently treats an existing TrackedClient as
up-to-date by only comparing selection, which allows new query, onQueryResult,
or queryResult values to be ignored and produce stale bindings; update
ensureClient to compare and track the identities of query, onQueryResult, and
queryResultCallback (in addition to selection) on the stored TrackedClient and
recreate or rewrap the client when any of those identities differ, or
alternatively store both raw callback identities and the wrapped callback it
passes into makeClient so you can update the wrapped function when the external
callback changes; focus your changes in the ensureClient function and the
TrackedClient shape to add fields for query identity and external callback
identities and use those to decide when to re-create or update the
client/wrapped callbacks.
---
Nitpick comments:
In `@examples/deckgl-mosaic/src/components/map/MapView.tsx`:
- Around line 206-214: In getTooltip, sanitize and format numeric fields and
guard against missing/odd values: ensure Number(object.Magnitude) is checked (or
fallback to null/empty) before calling toFixed(1) and render a fallback like "M
—" when not a valid number; format object.Depth to a fixed number of decimals
(e.g., toFixed(1) or rounded) only if it is numeric and append "km", otherwise
show a safe fallback like "Depth —"; also ensure dateLabel is coerced to string
safely (e.g., default to empty string) so the tooltip never shows "NaN" or
"undefined" for the fields referenced in getTooltip.
In `@packages/mosaic/src/MosaicSlice.ts`:
- Around line 367-397: The JSDoc above createDuckDbMosaicConnector is out of
date: update the doc to describe that Arrow queries are converted by calling
createMosaicTableFromArrowTable on the Arrow Table returned by DuckDbConnector
(instead of describing the old IPC bytes/decodeIPC path), and remove references
to decodeIPC/IPC serialization; keep notes about JSON path using Array.from and
the intentional as any adapter casts, and reference DuckDbConnector, Connector,
and the query methods in the updated wording.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 629412c8-4a2c-4a45-a23c-8f15b59a285d
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (26)
examples/deckgl-mosaic/README.mdexamples/deckgl-mosaic/src/components/map/MapView.tsxpackages/color-scales/README.mdpackages/color-scales/__tests__/scale.test.tspackages/color-scales/__tests__/schemas.test.tspackages/color-scales/jest.config.jspackages/color-scales/src/ColorScaleLegend.tsxpackages/color-scales/src/config.tspackages/color-scales/src/scale.tspackages/db/package.jsonpackages/deck/__tests__/DeckSpec.test.tspackages/deck/__tests__/compileColorScale.test.tspackages/deck/package.jsonpackages/deck/src/datasets/PreparedDatasetStore.tspackages/deck/src/datasets/helpers.tspackages/deck/src/datasets/normalizeDatasets.tspackages/deck/src/datasets/usePreparedDatasetStates.tspackages/deck/src/json/colorScaleFunction.tspackages/deck/src/json/layerConfig.tspackages/deck/src/prepare/toGeoJsonBinary.tspackages/deck/src/prepare/wkbDecoder.tspackages/deck/src/types.tspackages/mosaic/src/MosaicColorLegend.tsxpackages/mosaic/src/MosaicSlice.tspackages/mosaic/src/useMosaicClient.tspackages/ui/src/theme/theme-provider.tsx
✅ Files skipped from review due to trivial changes (8)
- packages/color-scales/jest.config.js
- packages/color-scales/tests/scale.test.ts
- packages/color-scales/tests/schemas.test.ts
- packages/db/package.json
- packages/deck/package.json
- packages/color-scales/README.md
- packages/color-scales/src/config.ts
- packages/deck/src/types.ts
🚧 Files skipped from review as they are similar to previous changes (11)
- packages/color-scales/src/ColorScaleLegend.tsx
- packages/deck/tests/DeckSpec.test.ts
- packages/deck/src/json/colorScaleFunction.ts
- examples/deckgl-mosaic/README.md
- packages/deck/src/datasets/usePreparedDatasetStates.ts
- packages/mosaic/src/useMosaicClient.ts
- packages/deck/src/prepare/toGeoJsonBinary.ts
- packages/mosaic/src/MosaicColorLegend.tsx
- packages/deck/src/datasets/PreparedDatasetStore.ts
- packages/ui/src/theme/theme-provider.tsx
- packages/deck/src/json/layerConfig.ts
| export function buildGeometryKey(input: DeckDatasetInput): string { | ||
| return `${input.geometryColumn ?? ''}\u0001${input.geometryEncodingHint ?? ''}`; | ||
| } | ||
|
|
||
| /** | ||
| * Build the canonical cache key for one dataset input. | ||
| * | ||
| * The key intentionally ignores the user-facing dataset id and instead uses | ||
| * the underlying data identity: | ||
| * | ||
| * - SQL datasets: DuckDB connector identity + SQL text + geometry options | ||
| * - Arrow datasets: table object identity + geometry options | ||
| * | ||
| * Unresolved Arrow inputs (`arrowTable: undefined`) return `undefined`, which | ||
| * signals that the dataset should remain in `loading` until a table exists. | ||
| */ | ||
| export function resolvePreparedDatasetCacheKey(options: { | ||
| input: DeckDatasetInput; | ||
| sqlSourceIdentity?: object; | ||
| }): string | undefined { | ||
| const {input, sqlSourceIdentity} = options; | ||
|
|
||
| if (isSqlDatasetInput(input)) { | ||
| if (!sqlSourceIdentity) { | ||
| throw new Error( | ||
| 'SQL dataset cache keys require a sqlSourceIdentity object.', | ||
| ); | ||
| } | ||
|
|
||
| return [ | ||
| 'sql', | ||
| getSqlSourceIdentity(sqlSourceIdentity), | ||
| input.sqlQuery, | ||
| buildGeometryKey(input), | ||
| ].join('\u0001'); | ||
| } | ||
|
|
||
| const table = resolveArrowTable(input); | ||
| if (!table) { | ||
| return undefined; | ||
| } | ||
|
|
||
| return ['arrow', getTableIdentity(table), buildGeometryKey(input)].join( | ||
| '\u0001', | ||
| ); |
There was a problem hiding this comment.
Encode cache-key parts structurally to avoid collisions.
join('\u0001') can generate the same cache key for distinct inputs if SQL text, geometry column, or encoding values contain the delimiter. That can reuse the wrong prepared dataset. Prefer JSON/tuple encoding or length-prefixed parts.
🐛 Proposed fix
+function encodeCacheKeyParts(parts: readonly unknown[]): string {
+ return JSON.stringify(parts);
+}
+
export function buildGeometryKey(input: DeckDatasetInput): string {
- return `${input.geometryColumn ?? ''}\u0001${input.geometryEncodingHint ?? ''}`;
+ return encodeCacheKeyParts([
+ input.geometryColumn ?? null,
+ input.geometryEncodingHint ?? null,
+ ]);
}
@@
- return [
+ return encodeCacheKeyParts([
'sql',
getSqlSourceIdentity(sqlSourceIdentity),
input.sqlQuery,
buildGeometryKey(input),
- ].join('\u0001');
+ ]);
@@
- return ['arrow', getTableIdentity(table), buildGeometryKey(input)].join(
- '\u0001',
- );
+ return encodeCacheKeyParts([
+ 'arrow',
+ getTableIdentity(table),
+ buildGeometryKey(input),
+ ]);
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| export function buildGeometryKey(input: DeckDatasetInput): string { | |
| return `${input.geometryColumn ?? ''}\u0001${input.geometryEncodingHint ?? ''}`; | |
| } | |
| /** | |
| * Build the canonical cache key for one dataset input. | |
| * | |
| * The key intentionally ignores the user-facing dataset id and instead uses | |
| * the underlying data identity: | |
| * | |
| * - SQL datasets: DuckDB connector identity + SQL text + geometry options | |
| * - Arrow datasets: table object identity + geometry options | |
| * | |
| * Unresolved Arrow inputs (`arrowTable: undefined`) return `undefined`, which | |
| * signals that the dataset should remain in `loading` until a table exists. | |
| */ | |
| export function resolvePreparedDatasetCacheKey(options: { | |
| input: DeckDatasetInput; | |
| sqlSourceIdentity?: object; | |
| }): string | undefined { | |
| const {input, sqlSourceIdentity} = options; | |
| if (isSqlDatasetInput(input)) { | |
| if (!sqlSourceIdentity) { | |
| throw new Error( | |
| 'SQL dataset cache keys require a sqlSourceIdentity object.', | |
| ); | |
| } | |
| return [ | |
| 'sql', | |
| getSqlSourceIdentity(sqlSourceIdentity), | |
| input.sqlQuery, | |
| buildGeometryKey(input), | |
| ].join('\u0001'); | |
| } | |
| const table = resolveArrowTable(input); | |
| if (!table) { | |
| return undefined; | |
| } | |
| return ['arrow', getTableIdentity(table), buildGeometryKey(input)].join( | |
| '\u0001', | |
| ); | |
| function encodeCacheKeyParts(parts: readonly unknown[]): string { | |
| return JSON.stringify(parts); | |
| } | |
| export function buildGeometryKey(input: DeckDatasetInput): string { | |
| return encodeCacheKeyParts([ | |
| input.geometryColumn ?? null, | |
| input.geometryEncodingHint ?? null, | |
| ]); | |
| } | |
| /** | |
| * Build the canonical cache key for one dataset input. | |
| * | |
| * The key intentionally ignores the user-facing dataset id and instead uses | |
| * the underlying data identity: | |
| * | |
| * - SQL datasets: DuckDB connector identity + SQL text + geometry options | |
| * - Arrow datasets: table object identity + geometry options | |
| * | |
| * Unresolved Arrow inputs (`arrowTable: undefined`) return `undefined`, which | |
| * signals that the dataset should remain in `loading` until a table exists. | |
| */ | |
| export function resolvePreparedDatasetCacheKey(options: { | |
| input: DeckDatasetInput; | |
| sqlSourceIdentity?: object; | |
| }): string | undefined { | |
| const {input, sqlSourceIdentity} = options; | |
| if (isSqlDatasetInput(input)) { | |
| if (!sqlSourceIdentity) { | |
| throw new Error( | |
| 'SQL dataset cache keys require a sqlSourceIdentity object.', | |
| ); | |
| } | |
| return encodeCacheKeyParts([ | |
| 'sql', | |
| getSqlSourceIdentity(sqlSourceIdentity), | |
| input.sqlQuery, | |
| buildGeometryKey(input), | |
| ]); | |
| } | |
| const table = resolveArrowTable(input); | |
| if (!table) { | |
| return undefined; | |
| } | |
| return encodeCacheKeyParts([ | |
| 'arrow', | |
| getTableIdentity(table), | |
| buildGeometryKey(input), | |
| ]); | |
| } |
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/deck/src/datasets/helpers.ts` around lines 58 - 102, The cache key
builder (resolvePreparedDatasetCacheKey and buildGeometryKey) currently joins
parts with '\u0001', which can collide when parts contain that delimiter; change
it to produce a structured, collision-safe key by encoding parts as a JSON array
or using length-prefixed concatenation instead of a simple join. Update
resolvePreparedDatasetCacheKey to serialize the array ['sql'|'arrow',
getSqlSourceIdentity(sqlSourceIdentity) or getTableIdentity(table),
input.sqlQuery (for SQL) or omitted, and buildGeometryKey(input)] with
JSON.stringify (or equivalent stable encoder) and ensure buildGeometryKey
returns a structured object or tuple (e.g., {column, encodingHint}) that is
included in the serialized key so distinct inputs cannot collide. Ensure thrown
error behavior in resolvePreparedDatasetCacheKey remains unchanged and callers
still expect a string | undefined.
| if (isSqlDatasetInput(input)) { | ||
| if (!sqlSourceIdentity) { | ||
| throw new Error( | ||
| 'SQL dataset cache keys require a sqlSourceIdentity object.', | ||
| ); | ||
| } | ||
|
|
||
| return [ | ||
| 'sql', | ||
| getSqlSourceIdentity(sqlSourceIdentity), | ||
| input.sqlQuery, | ||
| buildGeometryKey(input), | ||
| ].join('\u0001'); |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Description: Inspect call sites to confirm sqlSourceIdentity changes when backing SQL data changes.
# Expected: sqlSourceIdentity should include or be replaced by a data/source revision, not only a long-lived client/connector object.
rg -n -C5 --type=ts --type=tsx '\bsqlSourceIdentity\s*[:=]'
rg -n -C8 --type=ts --type=tsx '\bsyncDatasetsForConsumer\s*\('Repository: sqlrooms/sqlrooms
Length of output: 116
🏁 Script executed:
#!/bin/bash
# Read the target file to understand the cache key construction
echo "=== File: packages/deck/src/datasets/helpers.ts (lines 70-100) ==="
sed -n '70,100p' packages/deck/src/datasets/helpers.ts
# Find sqlSourceIdentity assignments and usages with correct syntax
echo -e "\n=== sqlSourceIdentity assignments/usages ==="
rg -n -C4 'sqlSourceIdentity\s*[:=]' -t ts
echo -e "\n=== syncDatasetsForConsumer calls ==="
rg -n -C6 'syncDatasetsForConsumer\s*\(' -t tsRepository: sqlrooms/sqlrooms
Length of output: 12005
🏁 Script executed:
#!/bin/bash
# Check the ensureEntry method and cache lookup logic
echo "=== PreparedDatasetStore.ts: ensureEntry method ==="
sed -n '250,330p' packages/deck/src/datasets/PreparedDatasetStore.ts
echo -e "\n=== Look for version parameter usage in executeSql calls ==="
rg -n 'executeSql.*version' -t ts -A 2 -B 2
echo -e "\n=== Check how entries are created/cached ==="
sed -n '85,110p' packages/deck/src/datasets/PreparedDatasetStore.tsRepository: sqlrooms/sqlrooms
Length of output: 3703
🏁 Script executed:
#!/bin/bash
# Read the full usePreparedDatasetStates.ts to understand version tracking
echo "=== usePreparedDatasetStates.ts: full file ==="
cat packages/deck/src/datasets/usePreparedDatasetStates.ts
echo -e "\n=== Search for version parameter in sync/prepare calls ==="
rg -n 'version' packages/deck/src/datasets/ -t ts -B 2 -A 2Repository: sqlrooms/sqlrooms
Length of output: 3669
🏁 Script executed:
#!/bin/bash
# Find and read the ensureEntry method completely
echo "=== ensureEntry method in PreparedDatasetStore ==="
rg -n -A 50 'ensureEntry\(' packages/deck/src/datasets/PreparedDatasetStore.ts | head -80
echo -e "\n=== prepareDataset callback structure ==="
rg -n -B 5 -A 10 'prepareDataset:' packages/deck/src/datasets/PreparedDatasetStore.tsRepository: sqlrooms/sqlrooms
Length of output: 2857
🏁 Script executed:
#!/bin/bash
# Search for cache clearing, invalidation, or reset logic
echo "=== Cache invalidation/clearing logic ==="
rg -n 'clear|evict|invalidate|reset.*entries' packages/deck/src/datasets/PreparedDatasetStore.ts -t ts
echo -e "\n=== Check prepareDataset function signature ==="
rg -n -B 5 'export.*prepareDataset' packages/deck/src/datasets/PreparedDatasetStore.ts
echo -e "\n=== Check if version is tracked anywhere in entries ==="
rg -n 'type.*PreparedDatasetCacheEntry' -A 15 packages/deck/src/datasets/PreparedDatasetStore.tsRepository: sqlrooms/sqlrooms
Length of output: 536
🏁 Script executed:
#!/bin/bash
# Read type definitions at the start of PreparedDatasetStore.ts
echo "=== PreparedDatasetStore.ts: type definitions (lines 1-130) ==="
sed -n '1,130p' packages/deck/src/datasets/PreparedDatasetStore.ts
echo -e "\n=== Also check for any cache invalidation based on connector changes ==="
rg -n 'connector' packages/deck/src/datasets/usePreparedDatasetStates.ts -B 2 -A 2Repository: sqlrooms/sqlrooms
Length of output: 5412
Add data-revision tracking to cache keys to prevent stale prepared data.
ensureEntry skips re-execution when a cache key exists, and the cache key is built from source identity + SQL text + geometry only—it omits any data-revision tracking. If the same DuckDB connector object mutates (data inserts/updates/deletes), the cache serves stale prepared results indefinitely. The executeSql function accepts an optional version parameter (line 91) but it's never passed from usePreparedDatasetStates or included in the cache key. Either pass version through the sync call and incorporate it into resolvePreparedDatasetCacheKey, or ensure sqlSourceIdentity is replaced with an object that changes when backing data changes.
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/deck/src/datasets/helpers.ts` around lines 80 - 92, The cache key
for SQL datasets omits data-revision info, so include the optional version in
the key and propagate it through the call chain: update
resolvePreparedDatasetCacheKey (the function containing the shown branch) to
accept a version (or include version from sqlSourceIdentity) and append it to
the returned join alongside 'sql', getSqlSourceIdentity(sqlSourceIdentity),
input.sqlQuery and buildGeometryKey(input); then update callers
(usePreparedDatasetStates and any ensureEntry/executeSql invocations) to pass
the same version through to resolvePreparedDatasetCacheKey and to executeSql so
prepared entries are invalidated when version changes.
| try { | ||
| if (encoding === 'wkt' || encoding === 'geoarrow.wkt') { | ||
| return ( | ||
| WKTLoader.parseTextSync?.(String(value), {wkt: {crs: false}}) ?? null | ||
| ); | ||
| } | ||
|
|
||
| return ( | ||
| WKBLoader.parseSync?.(toArrayBuffer(value), { | ||
| wkb: {shape: 'geojson-geometry'}, | ||
| }) ?? null | ||
| ); |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
@loaders.gl/wkt 4.4.1 WKTLoader parseTextSync WKBLoader parseSync accepted input types
💡 Result:
In @loaders.gl/wkt version 4.4.1: WKTLoader.parseTextSync accepts a string containing Well-Known Text (WKT) geometry, such as 'LINESTRING (30 10, 10 30, 40 40)', parsing it into a GeoJSON-like geometry object {type: 'LineString', coordinates: [[30, 10], [10, 30], [40, 40]]}. As a text-format synchronous loader per the loaders.gl specification, it takes a single string input. WKBLoader.parseSync accepts binary data as an ArrayBuffer (e.g., new Uint8Array([...]).buffer), parsing Well-Known Binary (WKB) geometry into a binary geometry format like {positions: {value: Float64Array([1, 2]), size: 2}}. As a binary-format synchronous loader, it takes ArrayBuffer input. These match the standard loaders.gl parseSync/parseTextSync signatures and loader characteristics documented for these loaders.
Citations:
- 1: https://loaders.gl/docs/modules/wkt/api-reference/wkt-loader
- 2: https://loaders.gl/docs/specifications/loader-object-format
- 3: https://loaders.gl/docs/modules/core/api-reference/parse-sync
- 4: https://loaders.gl/docs/modules/wkt/api-reference/wkb-loader
🏁 Script executed:
cat -n packages/deck/src/prepare/wkbDecoder.tsRepository: sqlrooms/sqlrooms
Length of output: 6202
Handle unknown WKT strings before routing to WKB.
encoding === 'unknown' is supported in promotion (line 142) but parseGeometryValue does not explicitly handle it. String values with unknown encoding get routed to WBLoader.parseSync (which expects binary input), causing failures instead of WKT parsing.
Proposed fix
- if (encoding === 'wkt' || encoding === 'geoarrow.wkt') {
+ if (
+ encoding === 'wkt' ||
+ encoding === 'geoarrow.wkt' ||
+ (encoding === 'unknown' && typeof value === 'string')
+ ) {
return (
WKTLoader.parseTextSync?.(String(value), {wkt: {crs: false}}) ?? null
);
}When encoding is unknown, type-check the value to determine the appropriate parser: strings route to WKTLoader.parseTextSync, binary data to WBLoader.parseSync.
Also applies to: 98 (tryPromotePointTable)
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/deck/src/prepare/wkbDecoder.ts` around lines 37 - 48, The handler
currently routes all non-WKT encodings to WKB parsing, which breaks when
encoding === 'unknown'; update parseGeometryValue to treat encoding ===
'unknown' specially by type-checking the incoming value: if encoding === 'wkt'
|| encoding === 'geoarrow.wkt' || (encoding === 'unknown' && typeof value ===
'string') call WKTLoader.parseTextSync with {wkt:{crs:false}}, otherwise
(encoding === 'unknown' with non-string or other binary encodings) call
WKBLoader.parseSync with toArrayBuffer(value) and
{wkb:{shape:'geojson-geometry'}}; apply the same change to the similar branch in
tryPromotePointTable so string unknowns go to WKT parsing and binaries go to WKB
parsing.
| export type TrackedClient = { | ||
| id: string; | ||
| client: ReturnType<typeof makeClient>; | ||
| createdAt: number; | ||
| isLoading: boolean; | ||
| data: T | null; | ||
| data: unknown | null; | ||
| selection?: Selection; // Track for change detection | ||
| queryResultCallback?: (result: T) => void; // External callback | ||
| queryResultCallback?: (result: ArrowTable) => void; // External callback | ||
| }; |
There was a problem hiding this comment.
Track query/callback identity before treating ensureClient as a no-op.
Line 270 only compares selection, so a caller can pass a new query, onQueryResult, or queryResult for the same client id and still keep the old makeClient bindings. That makes ensureClient return stale results despite the comment saying query/callback changes recreate the client.
🐛 Proposed fix
export type TrackedClient = {
id: string;
client: ReturnType<typeof makeClient>;
createdAt: number;
isLoading: boolean;
data: unknown | null;
selection?: Selection; // Track for change detection
+ query: MosaicClientOptions['query'];
+ onQueryResultCallback?: (result: ArrowTable) => void;
queryResultCallback?: (result: ArrowTable) => void; // External callback
}; if (existing && existing.selection === selection) {
+ if (
+ existing &&
+ existing.selection === selection &&
+ existing.query === options.query &&
+ existing.onQueryResultCallback === options.onQueryResult &&
+ existing.queryResultCallback === options.queryResult
+ ) {
return; // No-op - client already exists with same selection
} draft.mosaic.clients[options.id] = {
id: options.id,
client,
createdAt: Date.now(),
isLoading: true,
data: null,
selection,
- queryResultCallback: options.onQueryResult
+ query: options.query,
+ onQueryResultCallback: options.onQueryResult,
+ queryResultCallback: options.queryResult,
+ queryResultCallback: options.onQueryResult
? (result: unknown) =>
options.onQueryResult!(toArrowClientResult(result))
: undefined,
};Adjust the final object shape to keep separate stored identities and wrapped callbacks if both callback types need to be retained.
Also applies to: 266-317
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.
In `@packages/mosaic/src/MosaicSlice.ts` around lines 54 - 62, ensureClient
currently treats an existing TrackedClient as up-to-date by only comparing
selection, which allows new query, onQueryResult, or queryResult values to be
ignored and produce stale bindings; update ensureClient to compare and track the
identities of query, onQueryResult, and queryResultCallback (in addition to
selection) on the stored TrackedClient and recreate or rewrap the client when
any of those identities differ, or alternatively store both raw callback
identities and the wrapped callback it passes into makeClient so you can update
the wrapped function when the external callback changes; focus your changes in
the ensureClient function and the TrackedClient shape to add fields for query
identity and external callback identities and use those to decide when to
re-create or update the client/wrapped callbacks.
Signed-off-by: Ilya Boyandin <[email protected]>
solves #548

Summary by CodeRabbit
New Features
Improvements
Documentation