Repository navigation
test(parity): print the differing CALLS edges in the CI output - #1441
Merged
Merged
Conversation
REL_CALLS is the only metric that drifts between backends, and a bare
count cannot tell "this backend dropped an edge" from "that backend wrote
a duplicate". The workflow log gave no way to act on the failure.
Each backend subprocess now also emits its CALLS edge set keyed on source
locations (caller path/name/line -> callee path/name/line, plus the call
site line), and the report prints, per backend: total vs distinct edges,
and the edges unique to or missing from each relative to the union.
REL_CALLS_DISTINCT is excluded from the pass/fail comparison — it is a
diagnostic, not a parity metric.
Run locally against an isolated Neo4j, this immediately narrows a spread
of 2 down to exactly two suspect edges, both of which look wrong
independently of any backend:
kuzudb total=648 distinct=637 duplicates=11
ladybugdb total=648 distinct=637 duplicates=11
falkordb total=647 distinct=636 duplicates=11
neo4j total=649 distinct=638 duplicates=11
union=638 common-to-all=636 differing=2
ToughCases.java:testAnonymous@42 -> Main.kt:process@4 (Java -> Kotlin)
tough_cases.ts:loadModuleDynamically@29
-> tough_circular_b.ts:tough_circular_b.ts@None (callee is a file
name, no line)
The first is a cross-language edge between unrelated fixtures; the second
points at a node whose name is a filename and whose line_number is null.
Both are unresolved/heuristic edges, which is why backends disagree about
them. No behaviour change here — this only makes the failure diagnosable.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
|
Hi! 👋 Join our CodeGraphContext Discord channel to collaborate: https://discord.gg/dR4QY32uYQ |
Contributor
🔍 PR Code Graph Analysistest(parity): print the differing CALLS edges in the CI output (#1441) 📊 Interactive VisualizationView the blast radius graph: PR Reviewer Dashboard 📦 ArtifactsThe graph JSON has been uploaded as a build artifact: Generated by CodeGraphContext using FalkorDB Lite |
Shashankss1205
added a commit
that referenced
this pull request
Jul 26, 2026
#1441 printed the whole CALLS edge set as a single JSON line per backend (~650 entries, ~100 KB). GitHub truncated the step log as a result, so the Actions output stopped after the first backend's STATS_JSON and neither the comparison table nor the new diagnostics were visible — the opposite of the intent. The subprocess now writes its edge list to <temp_test_dir>/<backend>_calls_edges.json and the parent reads that, leaving only the small human-readable diff in the log. Local run: log is 12 KB with a longest line of 831 chars (was ~100 KB on one line), and the diagnostics print in full. Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Makes the
Database Parity Checkfailure diagnosable from the Actions log, per #1440.REL_CALLSis the only metric that drifts, and a bare count can't distinguish "this backend dropped an edge" from "that backend wrote a duplicate". Each backend subprocess now also emits its CALLS edge set — keyed on source locations, not internal node ids, so the sets are comparable across backends — and the report prints per-backend totals, duplicate counts, and the edges unique to or missing from each.REL_CALLS_DISTINCTis excluded from the pass/fail comparison; it's a diagnostic, not a parity metric. No behaviour change.It already answers the question
Run locally against an isolated Neo4j, this narrows a spread of 2 to exactly two edges:
Both look wrong independently of any backend:
ToughCases.java:testAnonymouscallingMain.kt:process— two unrelated fixture projects.languages_are_compatibleis supposed to prevent cross-language CALLS; the audit spot-checked.py → .js/.java/.goand found none, so Java→Kotlin slipped through.tough_circular_b.ts:tough_circular_b.ts@None— the callee'snameis a filename and itsline_numberis null. That is the exact shape the existing tolerance comment blames for the drift ("cannot bind an exactcalled_line_number").Neither is a legitimate edge, so the fix is to stop emitting them rather than to widen the tolerance. That also explains why backends disagree: unresolved edges with a null line are exactly what the fast/slow MATCH split handles differently.
Also surfaced: 11 duplicate edges on every backend (648 total vs 637 distinct by caller→callee@call_line). Consistent across all four, so it isn't a parity problem, but it may be worth checking whether those are genuinely multiple calls on one line.
What this PR does not do
It does not make the workflow pass. The tolerance is untouched — see #1440 for why I don't think widening it is right. This is the diagnostic that makes the real fix possible, and it will print on the next run so the numbers can be read straight from the Actions output.