Skip to content

chore: add Node and Wasm Quack feasibility experiment - #960

Draft
ilyabo wants to merge 4 commits into
feat/duckdb-2-compatibilityfrom
experiment/quack-node-wasm-v2
Draft

ilyabo wants to merge 4 commits into
feat/duckdb-2-compatibilityfrom
experiment/quack-node-wasm-v2

Conversation

@ilyabo

@ilyabo ilyabo commented Oct 22, 2026 •

Copy link
Copy Markdown
Collaborator

Roomie needs evidence that Node can host native DuckDB v2 over Quack and that DuckDB-Wasm can feed its existing chart renderer before choosing this transport for browserless cloud work. This draft preserves a reproducible experiment, a working source-build workaround, and the remaining implementation-plan checkpoints.

Stacked on #959 (feat/duckdb-2-compatibility, originally inspected at 8eb7a853ab98fa6c1dda6b7ac7832851fee0772f). Changes are confined to experiments/quack-node-wasm/.

Verified outcome

The remote browser histogram works using rebuilt httpfs and Quack extensions, the unmodified published @duckdb/[email protected] threaded runtime, and maximumThreads: 1.

  • Node embeds DuckDB v2.0.0 and serves Quack. The native smoke test queries a separate database instance and rejects an invalid token.
  • Browser queries reach the Node-owned table: remote version v2.0.0, 120 rows, total 12960. Arrow results feed the existing shared Roomie histogram renderer and produce nine visible bins.
  • A short liveness check succeeds after 12 seconds idle with a requested six-second heartbeat timeout.
  • This is an isolated chart fixture, not full Roomie application integration or a production-ready distribution. Unsigned-extension and metadata compatibility flags remain explicit.

Upstream blockers and workaround

The published Wasm binaries are identical to the previously tested CI artifact. Strict loading still rejects their d54cbbe3df metadata versus released v2.0.0 extensions, and the default extension endpoint returns 404. The single-thread bundle cannot start Quack's heartbeat thread.

Instrumenting the threaded loader exposes the hidden failure: the published wasm_threads extensions import non-shared memory while the runtime supplies shared memory. This matches extension-ci-tools #391 and duckdb-wasm #2041. A memory-only binary patch is insufficient here.

Rebuilding both extensions from pinned sources with Emscripten 6.0.9, shared-memory flags, and matching engine headers gets them working. More than one query thread can still hang during extension loading; one query thread avoids that in the verified configuration while retaining the threaded runtime for heartbeat support.

THREADED-INVESTIGATION.md records exact observations, the build recipe, existing upstream issues, and separate findings about missing error details and thread-count-dependent hangs. loader-probe.html reproduces the loader behavior without React, Mosaic, or Roomie. Upstream reports: v2 artifact reproduction on extension-ci-tools #391 and DuckDB-Wasm #2288: hidden loader errors and loading hang.

Validation

  • Source rebuild recipe completed; source and published download checksums passed.
  • Remote histogram rendered and was visually verified, including a fresh run using the saved build recipe's outputs.
  • Minimal probe: stock artifact fails; rebuilt pair loads with one query thread; two query threads time out.
  • Native smoke and browser-local chart control passed during the published-package retest.
  • JavaScript/shell syntax and diff checks passed. All 50 workspace packages built during the original experiment on main; that full build was not repeated in this stacked worktree.

Production adoption still needs a compatible signed distribution, full Roomie integration, cancellation/reconnect/concurrency and access-policy tests, plus startup/memory measurements. Browserless workspace semantics and agent lifecycle remain separate work.

@coderabbitai

coderabbitai Bot commented Oct 22, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant