Repository navigation
Conversation
|
@macunha1 is attempting to deploy a commit to the shashankss1205's projects Team on Vercel. A member of the Team first needs to authorize it. |
|
👋 Thanks for contributing to CodeGraphContext! Since this PR was opened, |
|
Triage update: thank you for the thorough work here, @macunha1 — upstream KuzuDB's status is a real concern and this deserves a proper decision rather than a drive-by review. Flagging for @Shashankss1205: this PR proposes deprecating the Kùzu backend and giving users a migration path. Main currently still ships Kùzu as a first-class backend (with Ladybug as the successor lineage), so the call on direction needs to come from the project. Leaving open until that decision is made. |
@Shashankss1205 done ✅ Rebased onto the latest
All good. Please let me know once the project direction is decided, or if you see the need for any more changes to the migration path or implementation in this PR. I can update it accordingly. |
|
Flagging this for a maintainer decision rather than reviewing it as a routine change. The PR is well-argued and the diff is coherent (24 files, mergeable, no conflicts). But it changes the product's supported-backend story — retiring KùzuDB as a runtime backend and providing a migration path — and that is a direction call for @Shashankss1205, not something to merge on technical correctness alone. The case you make is real and worth stating plainly for whoever decides:
Against it: KùzuDB is currently the zero-config default on Windows and the FalkorDB fallback, so this touches the default install path for a whole platform, and #1512 shows the simulator is already Kùzu/Ladybug-fragile in ways that would need to move with it. Two things that would help the decision regardless of outcome:
Holding rather than merging. |
|
Thanks for the very thorough write-up — the reasoning about carrying an archived upstream dependency is sound, and the migration guide plus Two things before this can land. 1. Three tests fail on a trial-merge onto (1140 pass.) The first two are expected fallout from the deprecation and just need updating to assert the new behaviour. The third points at something bigger — see below. 2. There's a second breaking change bundled in here that isn't in the PR description.
Today Beyond that, removing a documented backend is a product call rather than a code-review one, so I'm holding this for @Shashankss1205 to weigh in on the direction before merge — the implementation isn't the question here. Worth noting Please do fix the three tests in the meantime — that part is unambiguous. |
|
Thank you for the concrete follow-up @Shashankss1205 Quick status update after rebasing onto current main (v0.6.5):
Local validation covers:
|
Shashankss1205
left a comment
Shashankss1205
left a comment
There was a problem hiding this comment.
Thanks for the thorough rebase @macunha1 — the strict-selection commit, the XDG fixes, and the parity validation are all solid. But the trial-merge surfaced a structural problem that blocks this as-is, and it sits in the PR's headline feature.
The kuzu and ladybug pybind natives cannot coexist in one process
On the trial-merged tree, the unit suite segfaults at test_ladybug_unwind_batching.py (Linux, ladybug 0.19.1). The chain, fully reproduced:
get_database_manager()runs_maybe_migrate_legacy_kuzudbon every call. On any machine wherefind_legacy_kuzudb_sourcefinds old data (env/config/sibling — my dev HOME qualifies, as will most upgraders'), the migration path executesimport kuzu.- kuzu's pybind module registers its types. Any subsequent ladybug pybind import in the same process then fails with
ImportError: generic_type: type "Database" is already registered!(captured by instrumentingladybug._backend._import_pybind_module). ladybug._backend.get_pybind_modulecaches the failed attempt permanently and falls back to the C-API backend — which the PyPI wheel doesn't ship — so every later ladybug use raisesRuntimeError: Could not find lbug C API shared library, and some orderings segfault outright in_execute_with_pybind.
The critical part: the migration itself is this exact sequence — _export_legacy_kuzu_bundle(kuzu, …) reads via kuzu, then CGCBundle(target_manager) writes via ladybug, in one process. So the auto-migration a real upgrader triggers is precisely the crashing path. Your E2E parity validation wouldn't catch it because the e2e harness drives each backend in its own subprocess.
Suggested fixes
- Run the kuzu-side export in a subprocess (kuzu →
.cgcbundle on disk is already the design — apython -c/module subprocess that only imports kuzu keeps the natives apart), then import the bundle in-process with the target manager. This also improves the Python 3.14 story: the subprocess can even use a different interpreter if you ever want that escape hatch. - Gate the migration probe to once per process/path rather than every
get_database_managercall — besides the perf, the probe scanning env/config/sibling on every factory call is what let a mock-based unit test (test_factory_retries_falkordb_for_a_different_database_path) poison the process for the ladybug tests that follow, purely because my machine has legacy data. Tests shouldn't be able to reach the developer's real HOME here. - Pin a ladybug floor (
ladybug>=0.19): 0.17.1 segfaults inside_execute_with_pybindon the current schema DDL (reproduced before upgrading) — with no floor, users on older installs get a hard crash instead of a version message.
Happy to re-run the full gate as soon as these land — everything else in the trial-merge looked good (unit suite green once the collision is avoided by test selection, and the renames are clean).
|
heey @Shashankss1205!! Thanks for the detailed investigation. I implemented all three safeguards in the latest push because they address separate failure modes:
The subprocess also supports On top of that, the documented Docker migration remains available as the fallback when a compatible local Python interpreter is unavailable. Please, let me know in case you find more issues after running the full suite |
|
Coming back to this during a queue sweep. It conflicts with KùzuDB is no longer on an exit path for CGC — it's become the default embedded backend. The embedded layer ( So I can't take the deprecation framing. That said, two things in this PR are worth keeping, and I don't want them lost with the rest:
Concretely: I'd close this as-is and open a focused "migrate an indexed graph between backends" PR, which I'd review promptly. If you think I'm wrong about Kùzu's position, make that case here — I'll engage with it seriously, I just can't act on the deprecation without it. Thanks for the substantial work regardless; the fact that it no longer fits is about where the project went, not about the quality of what you wrote. |
|
Thanks for the direct answer @Shashankss1205. I agree that the current PR needs to be rebased and narrowed, and that the bundle migration should become backend-agnostic. KuzuDB is archived, and The compatibility layer is valuable, but it is CGC-owned mitigation around a frozen engine. It demonstrates that we can keep Kuzu working today; it does not give us an upstream path when a future Python version, compiler, platform, or planner behavior requires another engine change. We have already crossed that boundary once. #1444 stopped installing Kuzu on Python 3.14 for macOS and Windows because no compatible wheels exist and pip falls back to a native source build. I reproduced both sides:
That means the installation problem was avoided through platform markers, not fixed in Kuzu. macOS and Windows users have already moved away from Kuzu on Python 3.14, so keeping Linux on the archived backend gives us platform-dependent behavior and postpones the same decision until another wheel disappears. The data boundary makes this more urgent. I created a Kuzu LadybugDB is the maintained continuation of Kuzu and already delegates through the same I suggest a staged version of the same direction:
This keeps existing user data recoverable and preserves the substantial adapter work. It also moves ordinary users onto one maintained implementation before another Python or platform transition turns the archived dependency into a harder blocker. I am happy to rebuild this as a focused, rebased series rather than trying to land the current broad diff unchanged. My concern is with dropping the retirement direction entirely: every additional feature or workaround built around the archived runtime increases the eventual migration cost without creating an upstream capable of fixing the underlying engine. As a final step, I’ll push an updated implementation for one last review. The revised proposal will preserve compatibility with Kuzu as a legacy, upstream-archived backend while positioning LadybugDB as the maintained replacement for all embedded-database use cases currently covered by Kuzu. |
+ Remove Kuzu from normal dependency resolution and implicit selection, so clean Python 3.14 installs do not depend on an archived native package. + Migrate legacy stores through an isolated .cgc exporter before importing through the selected manager, which avoids native type collisions and direct storage-format reuse. + Gate probes by backend and path and refuse uncertain or populated targets, so factory retries cannot repeatedly inspect HOME or silently merge data. + Keep Kuzu explicit and pinned during the migration window, which preserves data recovery without presenting the archived engine as maintained.
+ Name Ladybug as the maintained embedded fallback across installation, configuration, architecture, and contributor guidance, so new users are not directed toward an archived runtime. + Add a one-way migration guide covering backups, an alternate interpreter, and Docker or nerdctl, which keeps existing Kuzu data recoverable when local wheels are unavailable. + Mark Kuzu as explicit legacy compatibility during the migration window and document that its storage directory cannot be reused by Ladybug.
Shashankss1205
left a comment
Shashankss1205
left a comment
There was a problem hiding this comment.
Conflicting broad Kuzu/Ladybug migration; CI is red. Latest contributor discussion agrees narrower backend-agnostic migration/rebase. Settle maintenance direction before retiring a backend.
This branch has not been deployed
2 participants
Why this is worth considering?
CGC currently treats KuzuDB as a normal runtime backend, but KuzuDB is deprecated: it was archived upstream and its Python package no longer installs cleanly on newer Python versions such as Python 3.14. As archived repos are read-only, we can't fix that. That turns Kuzu into an adoption blocker: a user can hit a packaging failure before they ever get to the value of CGC itself.
This exit path is, unfortunately, inevitable. The practical question is whether we make that path explicit and supported, or whether users discover it through broken installs. This PR proposes the cleaner route: keep KuzuDB data reachable through migration, but stop depending on KuzuDB as an active runtime backend.
The product story becomes cleaner for new users:
That direction lets us keep the "works locally, works with agents, works with teams" message without carrying an archived database dependency in the main install path.
What changed
This branch moves KuzuDB from "supported backend" to "deprecated legacy migration source":
falkordb,ladybugdb,falkordb-remote,neo4j, andnornic.Why this route
The alternative is to keep treating KuzuDB as an equal backend and try to work around its already limited state. That may buy short-term compatibility, but it leaves CGC tied to an archived upstream dependency and makes future Python support difficult. Since the deprecation pressure already exists, delaying KuzuDB migration does not avoid the exit.
This route keeps the useful part of KuzuDB support (protecting existing user data) while removing the part that creates ongoing install risk. It also makes the backend matrix easier to communicate:
That feels like the right product tradeoff: less dependency risk for new adoption, without leaving behind existing data and users.
User-facing impact
For new users, installation and setup should be more predictable because the default path no longer depends on an archived package. They will never have to worry about KuzuDB
For existing KuzuDB users, the recommended path is migration. CGC detects legacy KuzuDB stores, exports them through the bundle flow, and imports them into the configured supported backend when the target is empty. The docs also explain the Docker-based migration path for environments where KuzuDB only works under an older Python image. This makes the deprecation visible and gives users a controlled exit path before newer Python environments make that decision for them.
Validation
The branch adds and updates tests for:
Scope
Try and deprecate KuzuDB as a production runtime backend, with a narrow intent: acknowledge that KuzuDB was archived, won't be supported anymore, and consequently should also be deprecated in CGC's supported runtime surface. Reduce CGC installation and maintenance risk, and give existing KuzuDB data a clear exit path into supported backends.