Skip to content

test(parity): print the differing CALLS edges in the CI output - #1441

Merged
Shashankss1205 merged 1 commit into
mainfrom
diag/parity-calls-edge-diff
Jul 26, 2026
Merged

Shashankss1205 merged 1 commit into
mainfrom
diag/parity-calls-edge-diff

Conversation

@Shashankss1205

Copy link
Copy Markdown
Collaborator

Makes the Database Parity Check failure diagnosable from the Actions log, per #1440.

REL_CALLS is the only metric that drifts, and a bare count can't distinguish "this backend dropped an edge" from "that backend wrote a duplicate". Each backend subprocess now also emits its CALLS edge set — keyed on source locations, not internal node ids, so the sets are comparable across backends — and the report prints per-backend totals, duplicate counts, and the edges unique to or missing from each.

REL_CALLS_DISTINCT is excluded from the pass/fail comparison; it's a diagnostic, not a parity metric. No behaviour change.

It already answers the question

Run locally against an isolated Neo4j, this narrows a spread of 2 to exactly two edges:

================= CALLS EDGE DIAGNOSTICS =================
kuzudb       total=648    distinct=637    duplicates=11
ladybugdb    total=648    distinct=637    duplicates=11
falkordb     total=647    distinct=636    duplicates=11
neo4j        total=649    distinct=638    duplicates=11

union=638  common-to-all=636  differing=2

--- MISSING from falkordb (2) ---
    sample_project_java/src/com/example/app/ToughCases.java:testAnonymous@42
        -> sample_project_kotlin/Main.kt:process@4 [call_line=54]
    sample_project_typescript/src/tough_cases.ts:loadModuleDynamically@29
        -> sample_project_typescript/src/tough_circular_b.ts:tough_circular_b.ts@None [call_line=31]

Both look wrong independently of any backend:

  1. A Java → Kotlin edge. ToughCases.java:testAnonymous calling Main.kt:process — two unrelated fixture projects. languages_are_compatible is supposed to prevent cross-language CALLS; the audit spot-checked .py → .js/.java/.go and found none, so Java→Kotlin slipped through.
  2. A callee that is a file, not a function. tough_circular_b.ts:tough_circular_b.ts@None — the callee's name is a filename and its line_number is null. That is the exact shape the existing tolerance comment blames for the drift ("cannot bind an exact called_line_number").

Neither is a legitimate edge, so the fix is to stop emitting them rather than to widen the tolerance. That also explains why backends disagree: unresolved edges with a null line are exactly what the fast/slow MATCH split handles differently.

Also surfaced: 11 duplicate edges on every backend (648 total vs 637 distinct by caller→callee@call_line). Consistent across all four, so it isn't a parity problem, but it may be worth checking whether those are genuinely multiple calls on one line.

What this PR does not do

It does not make the workflow pass. The tolerance is untouched — see #1440 for why I don't think widening it is right. This is the diagnostic that makes the real fix possible, and it will print on the next run so the numbers can be read straight from the Actions output.

REL_CALLS is the only metric that drifts between backends, and a bare
count cannot tell "this backend dropped an edge" from "that backend wrote
a duplicate". The workflow log gave no way to act on the failure.

Each backend subprocess now also emits its CALLS edge set keyed on source
locations (caller path/name/line -> callee path/name/line, plus the call
site line), and the report prints, per backend: total vs distinct edges,
and the edges unique to or missing from each relative to the union.
REL_CALLS_DISTINCT is excluded from the pass/fail comparison — it is a
diagnostic, not a parity metric.

Run locally against an isolated Neo4j, this immediately narrows a spread
of 2 down to exactly two suspect edges, both of which look wrong
independently of any backend:

  kuzudb       total=648 distinct=637 duplicates=11
  ladybugdb    total=648 distinct=637 duplicates=11
  falkordb     total=647 distinct=636 duplicates=11
  neo4j        total=649 distinct=638 duplicates=11
  union=638  common-to-all=636  differing=2

  ToughCases.java:testAnonymous@42 -> Main.kt:process@4   (Java -> Kotlin)
  tough_cases.ts:loadModuleDynamically@29
      -> tough_circular_b.ts:tough_circular_b.ts@None      (callee is a file
                                                            name, no line)

The first is a cross-language edge between unrelated fixtures; the second
points at a node whose name is a filename and whose line_number is null.
Both are unresolved/heuristic edges, which is why backends disagree about
them. No behaviour change here — this only makes the failure diagnosable.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
@vercel

vercel Bot commented Jul 26, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
code-graph-context-pack Ready Ready Preview, Comment Jul 26, 2026 3:49pm

@github-actions

Copy link
Copy Markdown
Contributor

Hi! 👋 Join our CodeGraphContext Discord channel to collaborate: https://discord.gg/dR4QY32uYQ

@Shashankss1205
Shashankss1205 merged commit 3cb2423 into main Jul 26, 2026
16 of 17 checks passed
@Shashankss1205
Shashankss1205 deleted the diag/parity-calls-edge-diff branch July 26, 2026 15:49
@github-project-automation github-project-automation Bot moved this from Backlog tasks to Done in CGC Progress Board Jul 26, 2026
@github-actions

Copy link
Copy Markdown
Contributor

🔍 PR Code Graph Analysis

test(parity): print the differing CALLS edges in the CI output (#1441)

📊 Interactive Visualization

View the blast radius graph: PR Reviewer Dashboard

📦 Artifacts

The graph JSON has been uploaded as a build artifact: pr-code-graph-1441


Generated by CodeGraphContext using FalkorDB Lite

Shashankss1205 added a commit that referenced this pull request Jul 26, 2026
#1441 printed the whole CALLS edge set as a single JSON line per backend
(~650 entries, ~100 KB). GitHub truncated the step log as a result, so the
Actions output stopped after the first backend's STATS_JSON and neither
the comparison table nor the new diagnostics were visible — the opposite
of the intent.

The subprocess now writes its edge list to
<temp_test_dir>/<backend>_calls_edges.json and the parent reads that,
leaving only the small human-readable diff in the log.

Local run: log is 12 KB with a longest line of 831 chars (was ~100 KB on
one line), and the diagnostics print in full.

Co-authored-by: Claude Opus 5 (1M context) <[email protected]>

This branch was successfully deployed

1 active deployment
Preview — bf3a6d03 Deployed Jul 26, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant