Repository navigation
bug(indexer/falkordb): empty-index race during parallel writes silently skips all File-[:CONTAINS] linking; scoped cgc stats reports 0 functions/classes #1721
Description
Description
After a clean cgc index on FalkorDB Lite, cgc stats <repo_path> reports
Functions 0 / Classes 0 while cgc stats (global) and every name-based
query return the correct numbers. Root cause is two-fold:
- A property index created while the parallel write workers are still
running can come up operational but empty — it participates in query
routing yet returns zero rows for every equality/range predicate, while a
full-label scan finds all values. - The per-file linking statement in
writer.py
(MATCH (f:File {path}) MATCH (n:<Label> {name, path, line_number, occurrence_index}) MERGE (f)-[:CONTAINS]->(n)) includes
occurrence_index— the identity key introduced by bug(indexer): Node identity(name, path, line_number)is not unique; symbols merge and properties are clobbered #1393 — in its MATCH,
so every row is routed through the broken index, matches zero rows, and
theMERGEwrites nothing, with no error.
Net effect on our production index (473 files, 419 Python):
| Check | Before repair | After repair |
|---|---|---|
cgc stats <path> Functions / Classes |
0 / 0 | 3,589 / 250 |
MATCH (:File)-[:CONTAINS]->(:Function) |
0 edges | 3,589 |
MATCH (:File)-[:CONTAINS]->(:Class) |
0 edges | 250 |
MATCH (:File)-[:CONTAINS]->(:Variable) |
0 edges | 15,529 |
MATCH (n:Function {occurrence_index: 0}) (index route) |
0 | 3,589 (scan route) |
Same predicate forced off-index (toInteger(...)) |
3,589 | 3,589 |
Downstream, this silently degrades everything that traverses
Repository-[:CONTAINS*]->(...): scoped stats, dead-code scope analysis, and
class/file context resolution in callers analysis.
Reproduction
- CGC 0.6.13,
DEFAULT_DATABASE=falkordb(embedded Lite), Linux,
PARALLEL_WORKERS=4(default), a ~420-file Python monorepo. cgc index --force <repo>then:MATCH (f:File)-[:CONTAINS]->(n:Function) RETURN count(n) -- 0 MATCH (n:Function) WHERE n.occurrence_index = 0 RETURN count(n) -- 0 MATCH (n:Function) WHERE toInteger(n.occurrence_index) = 0 RETURN count(n) -- all
- A single-threaded synthetic repro does not trigger it (verified:
index-then-write, write-then-index, and UNWIND+MERGE sequences all behave);
the empty index needs the concurrent index-creation/write window that only
real parallel loads hit.index --forcereproduced it 2/2 times on our
machine; the repaired index (rebuilt while idle) served lookups correctly
until the next full re-index.
Why it is not the fixed #1529/#759 family
Those report over-counting from CONTAINS path traversal (fixed by #1411).
This is the opposite: the CONTAINS edges are missing entirely, so the
distinct-node count legitimately returns 0.
Suggested fix (PR coming)
A cheap post-index invariant check in GraphWriter, run at the end of the
pipeline's post-processing phase:
For each of
Function/Class/Variable: if the repo prefix has symbol
nodes but zeroFile-[:CONTAINS]edges, log a warning and back-fill
the edges from the already-storedpathproperty
(MATCH (f:File) ... MATCH (n) WHERE n.path = f.path MERGE (f)-[:CONTAINS]->(n)).
Partially-linked labels are deliberately left alone (that would be a
different defect). This is a self-heal for the symptom, not a fix for the
underlying FalkorDB Lite index-build race, which lives below CGC and deserves
a separate report upstream.
Related history: #1393 (introduced occurrence_index into the MERGE key),
#1710 / #1612 (same "batched writes silently drop rows" symptom class,
different backend), #682 (older zero-counts report, different cause).
Activity
github-actions commentedon Sep 20, 2026
Hi! 👋 Join our CodeGraphContext Discord channel to collaborate: https://discord.gg/dR4QY32uYQ
fix(indexer): self-heal missing File->CONTAINS edges caused by empty-…
Shashankss1205 commentedon Sep 24, 2026
Auto-closed by the #1722 merge — recording the resolution here for anyone who finds this later.
On FalkorDB Lite a property index built while the parallel write workers are still running can come up operational but empty: it participates in query routing and matches zero rows, while a label scan finds everything. Because the per-file linking statement keys part of its MATCH on
occurrence_index(since #1393), under that race it silently wrote zeroFile-[:CONTAINS]edges — nodes present, edges absent, no error raised, scopedcgc statsreporting 0 functions/classes.CGC now runs a post-index invariant check that detects the zero-edges-with-nonzero-nodes state per label and back-fills from the already-stored
pathproperty. Idempotent via MERGE, and deliberately conservative — a partially-linked label is left alone, so the repair can't mask ordinary per-file defects.The underlying race is in FalkorDB Lite's index building rather than in CGC, so this detects and repairs the consequence instead of patching third-party internals. @MorningStar0709 — please link the upstream report back here once you file it, so the trail survives.
Excellent debugging. Forcing the predicate off-index with
toInteger(n.occurrence_index) = 0and getting 3,589 rows back is what turned this from a guess into a diagnosis.MorningStar0709 commentedon Sep 24, 2026
Following up on the upstream-report request from #1722 and your closing note here.
We tried to isolate the empty-index state on the embedded backend alone, with no CodeGraphContext code in the loop: 4/8 threads streaming
UNWIND range(...) CREATEbatches into fresh labels,CREATE INDEX FOR (n:Label) ON (n.occurrence_index)issued while those writers were still live, then the dataset closed and reopened, then the same equality predicate read through the index and forced off-index withtoInteger(...)at the first/middle/last key of each label's range. Version pair:falkordblite0.9.0 withMODULE LIST → name=graph, ver=41602, Python 3.12, Linux x86_64.Result: no reproduction — 30 runs / 240 labels / 823 measurements that genuinely took the index route, zero cases where the indexed route came back empty while the scan route found the row. So we are not filing a confident upstream issue yet; an unreproducible "it broke once in someone's app" report would just burn the window there.
Two things did come out of it that are worth recording here, because they shape how anyone continues this:
GRAPH.EXPLAINkeeps returningNode By Label ScanafterCREATE INDEX; only after the dataset is closed and reopened does it switch toNode By Index Scanand reportOPERATIONAL. A coverage defect introduced during loading is therefore invisible until a reload, which lines up with how this bug surfaced for us — the failure showed on the next full re-index, not the one that was corrupted. (EXPLAINmust be sent as its ownGRAPH.EXPLAINcommand; routed throughGRAPH.QUERYit executes as a normal query.)[Indexing] k/N: UNDER CONSTRUCTION, reads are still correct — we sawnumRecords=0alongside a correct result, so this build falls back to a scan rather than serving an incomplete index. The window we are looking for is narrower than "build not finished yet".Method note for whoever picks this up: a harness that queries a constant key across labels whose writers each own a disjoint id range measures nothing — every label but the first matches zero rows on both routes and reads as a pass. And each reading should carry its own
GRAPH.EXPLAINresult, so a probe that never reached the index is recorded as "not measured" rather than as a clean result. That distinction is what turned our first 1096 probes from a null result into no result at all.Still open: what kind of build can finish
OPERATIONALwhile missing rows that a label scan finds.