Repository navigation
perf(neo4j): fix CALLS write performance — backend detection, transaction scope, and MATCH optimisation - #1192
Merged
Shashankss1205 merged 1 commit intoJun 11, 2026
Conversation
|
@ModQA is attempting to deploy a commit to the shashankss1205's projects Team on Vercel. A member of the Team first needs to authorize it. |
…tion scope, and MATCH optimisation Three compounding issues caused slow CALLS edge writes on Neo4j. 1. Broken backend detection in write_function_call_groups: the legacy getattr chain returned the method object (always truthy) instead of calling it, silently using MERGE instead of CREATE on neo4j/nornic backends — negating PR CodeGraphContext#1161 entirely. Replaced with the canonical get_backend_type() call already used elsewhere in the file. 2. Single managed transaction across all CALLS batches (introduced by PR CodeGraphContext#1171): execute_write wrapped the entire _work closure — all 100k+ edges in one transaction. Neo4j held write locks and accumulated WAL entries until the final commit, producing 46-minute silent gaps with ~167ms/edge observed in production logs. Replaced with a plain session so each 1000-edge batch gets its own execute_write — per-batch auto-retry and clean rollback without the lock contention. 3. Inefficient MATCH on called nodes: the called MATCH used only name+path (2 of 3 composite index properties), causing Neo4j to scan all matching nodes and filter line_number in a WHERE clause — 26,680 DbHits per lookup vs 2 DbHits with the full composite index. Split each batch into fast-path (called_line_number known: full 3-property composite index hit) and slow-path (ambiguous resolution: 2-property prefix scan, correct behaviour preserved). Same fix applied to post_resolution.py where the caller MATCH had no label (full node scan, no index possible) and the called MATCH had the same 2-property issue. 4. Fixed NameError in graph_builder.py where cgcignore_path was referenced in _build_graph_from_scip but never passed as a parameter, causing SCIP indexing to always fall back to Tree-sitter. Benchmarks (~97k fn→fn CALLS edges, Neo4j backend): Before (MERGE, single tx): ~138ms/edge, ~1697s total After, 1 repo (~13k functions): 3.3ms/edge, 320s (43x faster) After, 2 repos (~27k functions, CI): 7.1ms/edge, 689s (19x faster) Fast/slow path split: 28% fast (known called_line_number, 2 DbHits), 71% slow (ambiguous resolution, index prefix scan). The fast path delivers 13,340x fewer DbHits per row — proportionally larger benefit on graphs with hundreds of thousands of nodes. Co-Authored-By: Claude Sonnet 4.6 <[email protected]>
ModQA
force-pushed
the
perf/neo4j-calls-write-optimisation
branch
from
June 11, 2026 14:15
fce9187 to
d16577e
Compare
pull Bot
pushed a commit
to Mu-L/CodeGraphContext
that referenced
this pull request
Jun 12, 2026
PR CodeGraphContext#1183 accidentally removed incremental watcher relinking, and PR CodeGraphContext#1192's Neo4j fast/slow CALLS MATCH split caused a 1-edge Kuzu/Ladybug parity gap. Restore incremental _handle_modification, scope the perf split to Neo4j only, and align unit-test mocks with execute_write_operation. Co-authored-by: Cursor <[email protected]>
macunha1
pushed a commit
to macunha1/CodeGraphContext
that referenced
this pull request
Jun 21, 2026
…write-optimisation Fix Neo4j CALLS write performance via backend detection, per-batch transactions, and composite-index MATCH optimisation. Preserve Parameter and File special-case queries from main during conflict resolution. Co-authored-by: Cursor <[email protected]>
macunha1
pushed a commit
to macunha1/CodeGraphContext
that referenced
this pull request
Jun 21, 2026
PR CodeGraphContext#1183 accidentally removed incremental watcher relinking, and PR CodeGraphContext#1192's Neo4j fast/slow CALLS MATCH split caused a 1-edge Kuzu/Ladybug parity gap. Restore incremental _handle_modification, scope the perf split to Neo4j only, and align unit-test mocks with execute_write_operation. Co-authored-by: Cursor <[email protected]>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three compounding issues caused slow CALLS edge writes on Neo4j, producing ~167ms/edge and multi-hour indexing times on large graphs.
Root causes fixed
1. Broken backend detection (
writer.py)The legacy
getattrchain inwrite_function_call_groupsreturned the method object (always truthy) instead of calling it, silently usingMERGEinstead ofCREATEon neo4j/nornic backends — negating PR #1161 entirely. Replaced with the canonicalget_backend_type()call.2. Single managed transaction across all CALLS batches (
writer.py)PR #1171's
execute_writewrapped the entire_workclosure — all 100k+ edges in one transaction. Neo4j held write locks and accumulated WAL entries until the final commit, producing 46-minute silent gaps with zero progress logs. Replaced with a plain session so each 1000-edge batch gets its ownexecute_write— per-batch auto-retry and clean rollback without the lock contention.3. Inefficient MATCH on called nodes (
writer.py,post_resolution.py)The
calledMATCH used onlyname+path(2 of 3 composite index properties), causing Neo4j to scan all matching nodes and filterline_numberin a WHERE clause: 26,680 DbHits per lookup vs 2 DbHits with the full composite index. Split each batch into:called_line_number > 0): full 3-property composite index hit — 2 DbHitscalled_line_number <= 0): ambiguous resolution, 2-property prefix scan (correct behaviour preserved)Same fix applied to
post_resolution.pywhere the caller MATCH had no label (full node scan across all nodes) and the called MATCH had the same 2-property issue.4. SCIP
cgcignore_pathNameError (graph_builder.py)cgcignore_pathwas referenced in_build_graph_from_scipbut never passed as a parameter, causing SCIP indexing to always fall back to Tree-sitter with a NameError.Benchmarks
Measured on a large Java Spring Boot service (~97k fn→fn CALLS edges), Neo4j backend:
Production baseline (v0.4.18): ~167ms/edge, 46-minute CALLS write gaps with zero progress logging. Fast/slow path split: 28% fast (known
called_line_number, 2 DbHits), 71% slow (ambiguous resolution, index prefix scan). The fast path delivers 13,340× fewer DbHits per row — proportionally larger benefit on graphs with hundreds of thousands of nodes.Test plan
tests/unit/tools/test_graph_writer_transactions.py,tests/unit/core/)test_database_kuzu_kotlin_metadata.pyconfirmed pre-existing onmain, not introduced by this PR