Skip to content

fix(indexing): wire PARALLEL_WORKERS config to indexing concurrency - #1376

Merged
Shashankss1205 merged 1 commit into
CodeGraphContext:mainfrom
Sanjays2402:fix/parallel-workers-config-1340
Jul 31, 2026
Merged

Shashankss1205 merged 1 commit into
CodeGraphContext:mainfrom
Sanjays2402:fix/parallel-workers-config-1340

Conversation

@Sanjays2402

Copy link
Copy Markdown
Contributor

Fixes #1340.

PARALLEL_WORKERS is defined and validated in config_manager.py (default 4, range 1-32, "Number of parallel indexing workers"), but run_tree_sitter_index_async hardcoded concurrency_limit = 10, so setting it had no effect on indexing.

This reads the value through a small get_parallel_workers() helper that falls back to the previous limit of 10 when the config is unset or not a usable positive integer, so a missing or malformed value never breaks indexing.

Tests: tests/unit/tools/test_pipeline_parallel_workers.py covers valid values and each fallback case (unset, empty, non-numeric, zero, negative) — 8 passed; the module fails to import on main without the change.

@vercel

vercel Bot commented Jul 25, 2026

Copy link
Copy Markdown

@Sanjays2402 is attempting to deploy a commit to the shashankss1205's projects Team on Vercel.

A member of the Team first needs to authorize it.

@Shashankss1205

Copy link
Copy Markdown
Collaborator

👋 Thanks for contributing to CodeGraphContext! Since this PR was opened, main has moved forward significantly (we just shipped v0.5.2 and merged ~50 PRs), so this one now has merge conflicts. Could you please rebase onto the latest main and resolve the conflicts? Once it is conflict-free and CI is green, we will review and merge it. Really appreciate your work — thank you! 🙏

@Sanjays2402
Sanjays2402 force-pushed the fix/parallel-workers-config-1340 branch from 22211b0 to 06709c0 Compare July 30, 2026 17:35
@Sanjays2402

Copy link
Copy Markdown
Contributor Author

Rebased onto current main, now conflict-free (06709c0). The conflicts were all in files this PR never touched, so I reapplied the two real changes on top of main rather than replaying the old branch history: get_parallel_workers() in tools/indexing/pipeline.py and its unit test. concurrency_limit in run_tree_sitter_index_async now comes from the PARALLEL_WORKERS config with a fallback to 10 for unset/non-numeric/non-positive values.

pytest tests/unit/tools/test_pipeline_parallel_workers.py is 8 passed locally.

@Shashankss1205 Shashankss1205 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed and verified locally: merged onto current main, new config tests green. Thanks for wiring PARALLEL_WORKERS through (fixes #1340).

@Shashankss1205
Shashankss1205 merged commit cef622f into CodeGraphContext:main Jul 31, 2026
1 check failed
@github-project-automation github-project-automation Bot moved this from Backlog tasks to Done in CGC Progress Board Jul 31, 2026
@Shashankss1205

Copy link
Copy Markdown
Collaborator

Thanks @Sanjays2402 for this contribution — merged! 🎉 This closes a real gap (#1340). We'd love to have you contribute back again — feel free to pick up another open issue!

@Shashankss1205 Shashankss1205 added gssoc:approved GSSoC validation: counts toward scoring level:intermediate GSSoC difficulty: 35 pts contributor / 20 mentor mentor:Shashankss1205 GSSoC mentor attribution: credits reviewing mentor quality:clean GSSoC quality: x1.2 contributor / +5 mentor type:bug GSSoC type bonus: bug labels Aug 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gssoc:approved GSSoC validation: counts toward scoring level:intermediate GSSoC difficulty: 35 pts contributor / 20 mentor mentor:Shashankss1205 GSSoC mentor attribution: credits reviewing mentor quality:clean GSSoC quality: x1.2 contributor / +5 mentor type:bug GSSoC type bonus: bug

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Bug: PARALLEL_WORKERS config is not wired to indexing pipeline — concurrency hardcoded at 10

2 participants