Skip to content

Improve progress bar handling during repository indexing - #1219

Merged
Shashankss1205 merged 141 commits into
CodeGraphContext:mainfrom
tanishqkr:improve-progress-handling
Jul 31, 2026
Merged

Shashankss1205 merged 141 commits into
CodeGraphContext:mainfrom
tanishqkr:improve-progress-handling

Conversation

@tanishqkr

Copy link
Copy Markdown
Contributor

Summary

This PR improves the existing indexing progress handling during cgc index runs by refining the current rich.progress integration rather than introducing a new progress system.

Changes

  • Added --no-progress CLI option to disable live progress rendering

  • Auto-disabled progress rendering in:

    • non-interactive terminals
    • CI environments
  • Prevented periodic indexing logs from interfering with the live progress display

Why

The repository already contained an internal progress system, but there were a few UX gaps:

  • no manual disable option
  • potential progress/log collision during long indexing runs
  • unnecessary rendering in CI or redirected output environments

This PR keeps the implementation minimal and aligned with the existing architecture.

Verification

Tested successfully in:

  • normal interactive terminal
  • redirected output (> output.txt)
  • CI mode (CI=true)
  • --no-progress mode

Closes #1050

@vercel

vercel Bot commented Jun 12, 2026

Copy link
Copy Markdown

@tanishqkr is attempting to deploy a commit to the shashankss1205's projects Team on Vercel.

A member of the Team first needs to authorize it.

Shashankss1205 and others added 19 commits June 12, 2026 18:58
…pass

Extract Kùzu/Ladybug into database_embedded_kuzu with GraphQueryInterface (M4),
add multi-backend embedding/vector resolve, harden deploy templates, lint CI,
and wire Tree-sitter CALLS resolution into the SCIP indexing pipeline.

Co-authored-by: Cursor <[email protected]>
Keep main's full incremental re-link path (callers, inheritors, embeddings)
while preserving audit fixes: cgcignore filtering on neighbor paths and
per-file edge deletion in synchronize_with_disk.

Co-authored-by: Cursor <[email protected]>
…lease-0.5.1-audit-remediation

Fix/release 0.5.1 audit remediation
…#1334)

Three crash sites fixed:
- tools/languages/python.py: _get_node_text() returns '' instead of None
- tools/indexing/resolution/inheritance.py: guard .split() calls in
  resolve_inheritance_link() when base_class_str is None
- tools/indexing/resolution/inheritance.py: guard .split() in
  build_embeds_links() when base list entry is None

Fixes CodeGraphContext#1334
…ce.py, graph_builder.py)

Additional None-guard fixes discovered during full-index testing:
- inheritance.py:95 — guard imp['name'].split('.') via (imp.get('name') or '')
  + filter out imports with no name
- inheritance.py:326 — guard imported.split('.') when imported is None
- calls.py:1754,2145 — same guard as inheritance.py:95
- graph_builder.py:531 — same guard

Fixes CodeGraphContext#1334
…graph-builder-none-split

Fix Python graph builder crash: NoneType split during relationship resolution
The parser golden regression tests (tests/integration/test_parser_goldens.py)
have failed on main for ~18 consecutive runs, blocking CI on every open PR.
The language parsers evolved but the recorded baselines were never refreshed.

Regenerated all 21 language baselines via the sanctioned --update-goldens flag
(tree-sitter 0.25.2 / language-pack 0.13.0, matching CI). Verified locally:
21/21 golden projects pass. Net effect: +71 nodes, +209 edges captured
(mostly parser improvements, incl. new Emacs Lisp support 0->26 nodes).

A few languages show a net node/edge loss reflecting genuine parser regressions
already shipped in v0.5.1 (Perl -17, C# -8, Swift -4, Java -1); these will be
filed as separate tracking bugs.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Beyond the golden re-baseline, main had 10 unit/integration failures + 1 e2e
failure, all from production changes that intentionally evolved behavior
without updating the tests. None are real regressions:

- python parser: exclude synthetic <module> frame (added by _attach_module_context)
- embeddings/vector resolver (5): fake driver now reports neo4j backend so the
  multi-backend detection path matches the mocked neo4j responses
- graph-builder CALLS queries (2): assert new inline line-match + backtick-quoted
  labels instead of the old WHERE-equality form
- CLI inventory (2): account for the new 'bundle merge' git-driver subcommand
- e2e test_clean_up: enable ALLOW_DB_DELETION for the delete journey

Verified locally: all 10 + e2e pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…-restore-green-parser-goldens

fix(ci): re-baseline stale parser goldens to restore green CI
…eadme-clarity

Improve README documentation with inline explanations
Shashankss1205 and others added 20 commits July 27, 2026 00:29
…e-target edges (CodeGraphContext#1445)

Database Parity Check has been red because REL_CALLS disagreed across
backends. Four defects, none of them in the database layer.

1. Nondeterministic resolution input (pipeline.py)
   all_file_data is appended in asyncio.as_completed order, so the list
   order varies with task scheduling. Everything built by iterating it —
   global_class_bases, interface_implementors, class_method_index —
   inherited that order, and last-writer-wins assignments then produced a
   different graph per run. The graph write loop already sorted for this
   reason; resolution did not. Sorted before post-processing.

2. Unguarded Java interface-implementor fan-out (calls.py)
   After resolve_function_call returns, extra edges are appended for
   every implementor of the resolved interface, gated only on
   caller_lang == "java". interface_implementors is keyed on the bare
   simple class name across the whole index, so Java's nested Processor
   and Kotlin's models.Processor collide and a Java call acquired a
   Kotlin callee in an unrelated fixture project. Gated on the
   implementor's language matching the caller's.

3. Fabricated dynamic-import edges (calls.py)
   The TS/JS dynamic-import branch never read the import's own
   specifier; it used static_imports[0] — the first *static* import in
   the file. `await import(`./tough_dynamic_${name}`)` therefore resolved
   to an unrelated module, while modules-namespaces.ts, which has four
   statically resolvable literal specifiers and no static imports, got
   nothing. Now resolved from the call's own argument, emitting nothing
   when the specifier is not a static string literal.

4. File-target CALLS edges silently dropped on Kuzu/Ladybug (writer.py)
   The CALLS write emitted
       WHERE (row.called_line_number <= 0 OR called.line_number = ...)
   for every target label. File nodes have no line_number
   (database_embedded_kuzu.py declares File as path, name, relative_path,
   package_name, is_dependency). Neo4j and FalkorDB are untyped and
   evaluate it to null, so the edge writes; Kuzu and Ladybug fail at BIND
   time, and the error is swallowed as a binder exception, discarding the
   whole Function->File batch. Verified directly against kuzu 0.11.3:

       WITH line predicate    -> Binder exception: Cannot find property
                                 line_number for called.
       WITHOUT line predicate -> OK, edge written

   This was real data loss on the default embedded backends, not just a
   parity artefact. The predicate is now built per label, the same way
   called_context_clause already was.

Verified locally against an isolated Neo4j:

    REL_CALLS   | KuzuDB | LadybugDB | FalkorDB | Neo4j | Match?
    before      |   647  |    648    |   649    |  648  |  NO   (and it varied run to run)
    after       |   651  |    651    |   651    |  651  |  YES
    union=640  common-to-all=640  differing=0
    tests/e2e/test_verify_databases_parity.py: 1 passed

tests/unit + tests/integration: 967 passed, 7 skipped, 0 failed.

test_typescript_dynamic_import_calls_static_import_file asserted the
fabricated edge (`assert edge["called_name"] == "tough_circular_b.ts"`),
so it is rewritten to assert that a non-literal specifier produces no
edge, with the history recorded in its docstring.

Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
CodeGraphContext#1452)

CodeGraphContext#613 swapped server.py's `from .prompts import LLM_SYSTEM_PROMPT` for
`build_system_prompt`, but left the `LLM_SYSTEM_PROMPT` reference in the
initialize response untouched. Every stdio `initialize` request therefore
raised NameError and came back as -32603, failing the handshake and making
the server unusable for any stdio MCP client (Claude Code, Cursor, ...).

Both names are needed: build_system_prompt() at the call site, and
LLM_SYSTEM_PROMPT for the `instructions` field.

Shipped in 0.5.3; 0.5.2 and earlier are unaffected.

Also adds a regression test that drives _run_loop with a real initialize
request. There was no test covering the handshake at all, which is how this
reached a release.

Co-authored-by: Claude Opus 5 (1M context) <[email protected]>
…GraphContext#1499)

`_import_edge_batch` decides an endpoint is missing from `_id_mapping` with a
truthiness test, so an endpoint that maps to a falsy-but-valid id is treated as
absent and the edge is skipped.

FalkorDB's `id()` is 0-based and the Repository node is written first, so it
maps to `0` on every import and every `Repository -> *` edge is dropped. On a
966-node bundle that is 18 of 774 CONTAINS edges; the Repository node survives
with its name and path but has no outgoing edges at all, so any traversal from
the repository root returns nothing.

Nothing surfaces it. `import_from_bundle` reports the edge count it intended
rather than the count it wrote, node counts match exactly, and a symbol lookup
never traverses from Repository — so the graph looks whole until someone
follows an edge out of the root.

Neo4j endpoints are `elementId()` strings and Kùzu/Ladybug endpoints are
(label, pk, value) tuples; neither is ever falsy, so this only affected
FalkorDB.

Reproduced on 0.4.7 and 0.5.3: export is correct — the bundle's edges.jsonl
carries all 774 CONTAINS including the 18 out of Repository — and re-importing
into an empty graph yields 1303 of 1321 edges, with 0 files reachable from the
repository root. With this change the round trip is exact.

Tests fail on the unpatched importer with "Skipping edge: node IDs not found in
mapping" and pass with it, and cover both endpoint directions, the guard's real
job (a genuinely absent id is still skipped), and the two backends whose
endpoint types were never falsy.

(cherry picked from commit 9d03bc4)

Co-authored-by: Owen Shoemaker <[email protected]>
…odeGraphContext#1453)

* fix(viz): handle real FalkorDB Node/Edge shape in offline renderer

The offline visualization fallback (added in CodeGraphContext#1419) assumed any
driver object carrying `.labels` was Neo4j-shaped: dict-convertible,
with `.start_node`/`.end_node`/`.type` on edges. FalkorDB's own driver
objects also carry `.labels` on nodes but are not iterable — properties
live in `.properties` — and its Edge exposes `.src_node`/`.dest_node`/
`.relation` instead. Since FalkorDB Lite is the default backend, every
`cgc visualize` on that backend crashed with
`TypeError: 'Node' object is not iterable`, and even once patched,
relationships were being silently dropped (`_is_relationship` had no
FalkorDB branch, so edges never rendered).

The existing regression test's "neo4j-style objects" fixture was a
dict subclass carrying `.labels`, which happens to also satisfy
FalkorDB's shape check but not its actual (non-dict) runtime shape —
so it never caught this. Added a fixture mirroring FalkorDB's real
Node/Edge classes and confirmed it fails on the old code.

* fix(viz): FalkorDB edge src_node/dest_node are bare ints, not Node objects

Follow-up to the previous commit in this PR. Confirmed against a live
query (falkordb.Edge.src_node / .dest_node) that FalkorDB returns the
raw integer node id here, not a Node object — unlike Neo4j, where
start_node/end_node are always full Node objects. The previous fix
still did `getattr(start, "id", None)` unconditionally, so on a real
edge `start` was an int like `0`, `getattr(0, "id", None)` was None,
and the edge got silently dropped again (0 edges rendered instead of
a crash).

Also corrects the test fixture, which had made the same mistake: it
passed Node objects as `_FalkorEdge`'s src/dest args instead of bare
ids, so it didn't catch this. Verified end-to-end against a real
FalkorDB-backed repo index (sktime, 1879 files): 263 nodes / 300 edges
rendered correctly after this commit, vs. 0 edges before it.
Co-authored-by: Loi Nguyen <loinguyen@local>
Co-authored-by: Loi Nguyen <loinguyen@local>
* feat: add Svelte and Vue (SFC) language support

Add .svelte and .vue parsers to the AST indexer. A shared sfc_common helper
splits the single-file component into <script>/<template>/<style>, detects
lang=ts, and pads the script block so the delegated JavaScript/TypeScript
parser line numbers map back to the SFC. svelte.py and vue.py mirror the
existing html.py module shape and register in the tree-sitter manager, parser
dispatch, graph builder, and pre-scan. Grammars load from
tree-sitter-language-pack. Adds SFC parser unit tests.

Closes CodeGraphContext#709

* fix(sfc): read SFC sources with errors="ignore" like every other parser

A non-UTF-8 .vue/.svelte file must degrade to lost bytes, not a lost file;
tests/unit/tools/test_parser_encoding_resilience.py enforces this for all
language parsers.

Co-Authored-By: Claude Fable 5 <[email protected]>

* fix(sfc): guard the temp-file write handle as well — the resilience check scans every open( call

Co-Authored-By: Claude Fable 5 <[email protected]>

---------

Co-authored-by: Shashank Shekhar Singh <[email protected]>
Co-authored-by: Claude Fable 5 <[email protected]>
…t.vercel.app website. (CodeGraphContext#1260)

* Fix CodeGraphContext#708: Add vercel analytics to our codegraphcontext.vercel.app webs

Signed-off-by: nandanadileep <[email protected]>

* fix(analytics): finish the inject() -> <Analytics /> migration

The @vercel/analytics/react import collided with the hand-rolled Analytics
component and its loose window.va declaration, and the package's stricter
Window.va type rejected the 3-arg window.va('event', name, props) calls.
Drop the manual component/declaration (the official one already tracks SPA
route changes) and move the custom events to the supported track() API.

Co-Authored-By: Claude Fable 5 <[email protected]>

---------

Signed-off-by: nandanadileep <[email protected]>
Co-authored-by: Shashank Shekhar Singh <[email protected]>
Co-authored-by: Claude Fable 5 <[email protected]>
Co-authored-by: Shashank Shekhar Singh <[email protected]>
Shashankss1205
Shashankss1205 previously approved these changes Jul 31, 2026

@Shashankss1205 Shashankss1205 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed and verified locally: brought the branch up to date with main (kept --summarize and the exit-code handling, threaded --no-progress through index and the i shortcut), full CLI unit suite green. Sensible CI/terminal detection too.

@tanishqkr
tanishqkr dismissed Shashankss1205’s stale review July 31, 2026 18:19

The merge-base changed after approval.

@Shashankss1205
Shashankss1205 merged commit 17434cc into CodeGraphContext:main Jul 31, 2026
1 check failed
@github-project-automation github-project-automation Bot moved this from Backlog tasks to Done in CGC Progress Board Jul 31, 2026
@Shashankss1205

Copy link
Copy Markdown
Collaborator

Thanks @tanishqkr for this contribution — merged! 🎉 --no-progress plus automatic CI detection makes indexing logs much cleaner. We'd love to have you contribute back again — feel free to pick up another open issue!

@Shashankss1205 Shashankss1205 added gssoc:approved GSSoC validation: counts toward scoring level:intermediate GSSoC difficulty: 35 pts contributor / 20 mentor mentor:Shashankss1205 GSSoC mentor attribution: credits reviewing mentor type:feature GSSoC type bonus: feature labels Aug 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gssoc:approved GSSoC validation: counts toward scoring level:intermediate GSSoC difficulty: 35 pts contributor / 20 mentor mentor:Shashankss1205 GSSoC mentor attribution: credits reviewing mentor type:feature GSSoC type bonus: feature

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

[FEATURE] Add progress bar display during indexing large repositories