Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/daemon-lifecycle.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,4 +58,4 @@ jobs:
- run: npm ci --ignore-scripts
- run: npm run build:kernel
- run: npm run build
- run: npx vitest run __tests__/runtime-control.test.ts __tests__/daemon-install-check.test.ts __tests__/daemon-older-version.test.ts __tests__/cli-version.test.ts __tests__/daemon-registry.test.ts __tests__/daemon-pid-reuse.test.ts __tests__/daemon-lock-refresh.test.ts __tests__/mcp-daemon.test.ts __tests__/daemon-client-liveness.test.ts __tests__/daemon-bind-failure.test.ts __tests__/daemon-socket-fallback.test.ts __tests__/daemon-rendezvous-case.test.ts __tests__/writer-lock.test.ts __tests__/writer-handover-ordering.test.ts __tests__/mcp-writer-lock.test.ts __tests__/mcp-projectpath-lifecycle.test.ts __tests__/query-pool.test.ts __tests__/status-index-health.test.ts __tests__/vbnet-project-ties.test.ts __tests__/vbnet-receiver-types.test.ts __tests__/upstream-native-scope-regressions.test.ts --maxWorkers=2 --minWorkers=1
- run: npx vitest run __tests__/runtime-control.test.ts __tests__/daemon-install-check.test.ts __tests__/daemon-older-version.test.ts __tests__/cli-version.test.ts __tests__/daemon-registry.test.ts __tests__/daemon-pid-reuse.test.ts __tests__/daemon-lock-refresh.test.ts __tests__/mcp-daemon.test.ts __tests__/daemon-client-liveness.test.ts __tests__/daemon-bind-failure.test.ts __tests__/daemon-socket-fallback.test.ts __tests__/daemon-rendezvous-case.test.ts __tests__/writer-lock.test.ts __tests__/writer-handover-ordering.test.ts __tests__/mcp-writer-lock.test.ts __tests__/mcp-projectpath-lifecycle.test.ts __tests__/query-pool.test.ts __tests__/status-index-health.test.ts __tests__/vbnet-project-ties.test.ts __tests__/vbnet-receiver-types.test.ts __tests__/upstream-native-scope-regressions.test.ts --maxWorkers=${{ runner.os == 'Windows' && '1' || '2' }} --minWorkers=1
2 changes: 2 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,8 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

### Fixes

- `codegraph_explore` reports requested-file limits and uncovered indexed continuation ranges so partial answers have a clear next query.

- Explore retains explicitly pinned code in mixed documentation/code requests, returns parameterized test callbacks, and preserves quoted source terms that match the filename stem.

- Python closures no longer link calls or method values to a captured receiver type when later replacement makes that type uncertain.
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ Compared with upstream `main` at `aeb8f955` (after v1.6.2). Each item here and i
| Session search (`codegraph sessions`, `codegraph_sessions`) | — | ✓ | Searches this project's earlier Claude Code, Codex, Cursor/T3, OpenCode, AGY, Devin and Grok transcripts and its git commit messages, so an agent can find what a past session decided. `codegraph_explore` also names the sessions that mentioned the symbols it returns. `"sessions": false` in `codegraph.json` turns it off. |
| Markdown indexing | — | ✓ | Headings, sections, tables and links become graph nodes; a documentation question gets the matching section. Explicit code-file pins remain eligible in mixed documentation/code requests. |
| Missing names reported | — | ✓ | After a complete scan of indexed source without skipped files, `codegraph_explore` lists requested names it cannot find, helping an agent catch a guessed name. |
| Requested-source limits reported | — | ✓ | Explore reports requested files that did not receive pin priority within the file limit, unexamined path-like references beyond its bounded scan, and uncovered indexed continuation ranges after a capped file gather. Existing output limits still apply. |
| Local callee source with file pins | — | ✓ | When a query names functions in pinned files, `codegraph_explore` prioritizes local callee bodies through two call or callback hops, within the file and output budgets. |
| Quoted prose source | — | ✓ | Quoted spans of three or more words find script strings and template text, ignoring case and punctuation, through a capped source scan. No re-index needed. |
| Requested test source | — | ✓ | Pinned JavaScript and TypeScript test files return matching `it`/`test` callbacks, including `.each` parameterized tests. Questions about tests include source from the nearest test callers of named symbols, within the file and output budgets. |
Expand Down
198 changes: 198 additions & 0 deletions __tests__/explore-cap-notices.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,198 @@
import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest';
import * as fs from 'fs';
import * as os from 'os';
import * as path from 'path';
import CodeGraph from '../src/index';
import { ToolHandler } from '../src/mcp/tools';
import { ExploreSessionState } from '../src/mcp/explore-session-state';
import { fileFingerprint } from '../src/mcp/explore-dedup';

let dir: string;
let cg: CodeGraph;
const bigSource = Array.from({ length: 360 }, (_, i) => [
`export function work${String(i).padStart(4, '0')}() {`,
` // independent source ${'x'.repeat(160)}`,
` return ${i};`,
'}',
].join('\n')).join('\n');

async function explore(query: string, extra: Record<string, unknown> = {}): Promise<string> {
const result = await new ToolHandler(cg).execute('codegraph_explore', { query, ...extra });
return result.content?.[0]?.text ?? '';
}

beforeAll(async () => {
dir = fs.mkdtempSync(path.join(os.tmpdir(), 'codegraph-cap-notices-'));
fs.writeFileSync(path.join(dir, 'big.ts'), bigSource);
fs.writeFileSync(path.join(dir, 'config.yaml'), 'flag: true\n');
fs.writeFileSync(path.join(dir, '1.ts'), 'export const numericFile = 1;\n');
fs.writeFileSync(path.join(dir, 'padding.ts'), 'export function padding() {\n' + Array.from({ length: 100 }, (_, i) => ` // padding row ${i} ${'p'.repeat(45)}`).join('\n') + '\n return 7;\n}');
fs.writeFileSync(path.join(dir, 'partial.md'), Array.from({ length: 300 }, (_, i) => `## Tiny${i}\n\nSmall.\n\n\n`).join('\n') + '## TailSection\n' + Array.from({ length: 1900 }, () => 'x').join('\n') + '\n');
fs.writeFileSync(path.join(dir, 'compact.ts'), Array.from({ length: 360 }, (_, i) => `export const val${i}=${i};`).join(' '));
fs.writeFileSync(path.join(dir, 'manual.md'), Array.from({ length: 301 }, (_, i) => `## Chapter${String(i).padStart(3, '0')}\n\nParagraph ${i}.\n`).join('\n'));
for (let j = 0; j < 3; j++) fs.writeFileSync(path.join(dir, `large-doc${j}.md`),
Array.from({ length: 301 }, (_, i) => `## Section${i}\n\n${i === 300 ? Array.from({ length: 110 }, (_, k) => `Last section row ${k} ${'q'.repeat(45)}`).join('\n') : 'Small paragraph.'}\n`).join('\n'));
if (process.platform !== 'win32') {
const nested = path.join(dir, ...Array.from({ length: 14 }, () => 'd'.repeat(200)));
fs.mkdirSync(nested, { recursive: true });
for (let i = 0; i < 7; i++) fs.writeFileSync(path.join(nested, `extra${i}.ts`), `export const extraValue${i} = ${i};`);
fs.writeFileSync(path.join(nested, 'enormous.ts'), bigSource);
}
for (let i = 0; i < 10; i++) fs.writeFileSync(path.join(dir, `unit${i}.ts`), `export function uniqueUnit${i}() { return ${i}; }`);
cg = CodeGraph.initSync(dir);
await cg.indexAll();
}, 180_000);

afterAll(() => {
cg?.destroy();
if (dir) fs.rmSync(dir, { recursive: true, force: true });
});

describe('bounded explore exclusion notices', () => {
it('offers a usable continuation for a pure-path gather tail', async () => {
const out = await explore('big.ts');
const pointer = /Additional indexed source not included: `big\.ts:(\d+)-(\d+)`/.exec(out);
expect(pointer).not.toBeNull();
expect(Number(pointer![1])).toBeGreaterThan(1000);
expect(out.length).toBeLessThanOrEqual(19500);
const continued = await explore(`big.ts:${pointer![1]}-${pointer![2]}`);
const firstLine = bigSource.split('\n')[Number(pointer![1]) - 1]!;
expect(continued).toContain(firstLine.trim());
});

it('suppresses gather warnings when the complete file covers more than300 nodes', async () => {
expect(cg.getNodesInFile('compact.ts').filter(n => !['file', 'import', 'export'].includes(n.kind)).length).toBeGreaterThan(300);
const out = await explore('compact.ts');
expect(out).toContain('export const val359=359;');
expect(out).not.toContain('Additional indexed source not included');
});

it('retains named source beyond the gather cap', async () => {
const out = await explore('big.ts work0359');
expect(out).toContain('return 359;');
});

it('names overflow references without adding source file slots', async () => {
const out = await explore('unit0.ts unit1.ts unit1.ts', { maxFiles: 1 });
expect(out).toContain("Requested files not pinned within this call's initial file limit: `unit1.ts`");
expect(out.match(/\*\*`unit[01]\.ts`/g)?.length).toBe(1);
expect(out).not.toContain('No indexed file uniquely matches');
});

it('reports an overflow file when the selected file has no code nodes', async () => {
const out = await explore('config.yaml 1.ts', { maxFiles: 1 });
expect(out).toContain('No relevant code found');
expect(out).toContain("Requested files not pinned within this call's initial file limit: `1.ts`");
});

it.runIf(process.platform !== 'win32' && process.getuid?.() !== 0)('does not promise a continuation for unreadable source', async () => {
const file = path.join(dir, 'big.ts');
const mode = fs.statSync(file).mode;
fs.chmodSync(file, 0);
try {
expect(() => fs.readFileSync(file, 'utf8')).toThrow(/EACCES/);
const out = await explore('big.ts');
expect(out).not.toContain('Additional indexed source not included');
expect(out).not.toContain('independent source');
} finally { fs.chmodSync(file, mode); }
});

it('offers the Markdown tail cut inside a surviving section', async () => {
const state = new ExploreSessionState();
const result = await new ToolHandler(cg).execute('codegraph_explore', {
query: 'padding.ts partial.md TailSection', maxFiles: 2,
}, state);
const out = result.content?.[0]?.text ?? '';
expect(out).toContain('output truncated to budget');
expect(out).toContain('**`partial.md`');
const pointer = /Additional indexed source not included: `partial\.md:(\d+)-(\d+)`/.exec(out);
expect(pointer).not.toBeNull();
const source = fs.readFileSync(path.join(dir, 'partial.md'), 'utf8').split('\n');
const firstMissing = Number(pointer![1]);
expect(out).not.toContain(`${firstMissing}\tx`);
const held = state.view().projects[0]!.calls[0]!.files.find(file => file.path === 'partial.md');
expect(held).toBeDefined();
expect(held!.ranges.every(range => range.end < firstMissing)).toBe(true);
expect(out.length).toBeLessThanOrEqual(19500);
const continued = await explore(`partial.md:${pointer![1]}-${pointer![2]}`);
expect(continued).toContain(`${firstMissing}\t${source[firstMissing - 1]}`);
});

it('keeps capacity notices off at the exact file limit', async () => {
const out = await explore('unit0.ts unit1.ts', { maxFiles: 2 });
expect(out).toContain('uniqueUnit0');
expect(out).toContain('uniqueUnit1');
expect(out).not.toContain('Requested files not pinned');
});

it('reports unexamined references beyond the scan budget', async () => {
const out = await explore(Array.from({ length: 10 }, (_, i) => `unit${i}.ts`).join(' '), { maxFiles: 1 });
expect(out).toContain('Further path-like references were not examined');
const notice = out.split("Requested files not pinned within this call's initial file limit: ")[1]!.split('. Explore')[0]!;
expect(notice).toContain('unit7.ts');
expect(notice).not.toContain('unit8.ts');
expect(out.length).toBeLessThanOrEqual(19500);
});

it('recognizes the selected Markdown section as delivered past the gather cap', async () => {
const out = await explore('manual.md Chapter300');
expect(out).toContain('Paragraph 300.');
expect(out).not.toContain('Additional indexed source not included');
});

it('reports an unexamined path even when no relevant source was found', async () => {
const out = await explore(Array.from({ length: 9 }, (_, i) => `zzmissing${i}.zzz`).join(' '));
expect(out).toContain('No relevant code found');
expect(out).toContain('Further path-like references were not examined');
});

it.runIf(process.platform !== 'win32')('bounds long path notices while retaining pinned source', async () => {
// POSIX allows this fixture below its4096-byte path limit. Windows long-path
// configuration is outside this retrieval test's contract.
const out = await explore('unit0.ts ' + Array.from({ length: 7 }, (_, i) => `extra${i}.ts`).join(' '), { maxFiles: 1 });
expect(out).toContain('uniqueUnit0');
expect(out).toContain("Requested files not pinned within this call's initial file limit:");
expect(out).toContain('(7 more)');
expect(out.length).toBeLessThanOrEqual(19500);
});

it.runIf(process.platform !== 'win32')('keeps gather guidance usable when its full path label cannot fit', async () => {
const out = await explore('enormous.ts');
expect(out).toContain('Additional indexed source not included: (1 more)');
expect(out).toContain('Explore fewer requested files or narrower line ranges');
expect(out).not.toContain('Explore these ranges to continue');
expect(out.length).toBeLessThanOrEqual(19500);
});

it('reports a capped section dropped at the final output boundary', async () => {
const out = await explore('large-doc0.md large-doc1.md large-doc2.md Section300', { maxFiles: 3 });
expect(out).toContain('output truncated to budget');
const missing = /Additional indexed source not included: `([^`]+):(\d+)-(\d+)`/.exec(out);
expect(missing).not.toBeNull();
expect(out).not.toContain('**`' + missing![1] + '`');
expect(out).toContain('Last section row');
expect(out.length).toBeLessThanOrEqual(19500);
});

it('counts verified source held earlier in the same conversation', async () => {
vi.stubEnv('CODEGRAPH_EXPLORE_DEDUP', '1');
const state = new ExploreSessionState();
state.record({ projectRoot: dir, query: 'prior', sourceBytes: bigSource.length, responseBytes: bigSource.length,
files: [{ path: 'big.ts', fingerprint: fileFingerprint(bigSource), bytes: bigSource.length,
ranges: [{ start: 1, end: bigSource.split('\n').length }] }] });
try {
const result = await new ToolHandler(cg).execute('codegraph_explore', { query: 'big.ts' }, state);
const out = result.content?.[0]?.text ?? '';
expect(out).not.toContain('Additional indexed source not included');
} finally { vi.unstubAllEnvs(); }
});

it('does not offer stale indexed continuation ranges', async () => {
fs.appendFileSync(path.join(dir, 'big.ts'), '\n// disk drift');
try {
const out = await explore('big.ts');
expect(out).toContain('changed on disk');
expect(out).not.toContain('Additional indexed source not included');
} finally { fs.writeFileSync(path.join(dir, 'big.ts'), bigSource); }
});
});
48 changes: 45 additions & 3 deletions __tests__/query-paths.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,7 @@ describe('extractQueryPaths — resolution and stripping', () => {

it('passes through untouched when nothing resolves', () => {
const q = 'plain prose question about scrolling';
const out = extractQueryPaths(q, INDEX);
const { capacityFiles, unexaminedPathSpans, ...out } = extractQueryPaths(q, INDEX);
expect(out).toEqual({ strippedQuery: q, pinnedFiles: [], unresolvedPathSpans: [], lineAnchors: [], setAsideMatches: [] });
});
});
Expand Down Expand Up @@ -199,13 +199,13 @@ describe('extractQueryPaths — extension-less kebab basenames', () => {

it('leaves kebab prose that names no indexed file untouched — and unreported', () => {
const q = 'how does cross-call dedup make explore non-blocking';
const out = extractQueryPaths(q, INDEX);
const { capacityFiles, unexaminedPathSpans, ...out } = extractQueryPaths(q, INDEX);
expect(out).toEqual({ strippedQuery: q, pinnedFiles: [], unresolvedPathSpans: [], lineAnchors: [], setAsideMatches: [] });
});

it('leaves a stem shared by too many files alone — one hot name must not pin half the repo', () => {
const q = 'refactor the user-profile rendering';
const out = extractQueryPaths(q, INDEX);
const { capacityFiles, unexaminedPathSpans, ...out } = extractQueryPaths(q, INDEX);
expect(out).toEqual({ strippedQuery: q, pinnedFiles: [], unresolvedPathSpans: [], lineAnchors: [], setAsideMatches: [] });
});

Expand Down Expand Up @@ -511,3 +511,45 @@ describe('extractQueryPaths — same-named files, narrowed by named symbols', ()
expect(l.asked).toEqual([]);
});
});


describe('explicit reference capacity reporting', () => {
it('reports overflow once while preserving the original matching query', () => {
const out = extractQueryPaths('src/lib/chat-manager.ts src/lib/task-runner-manager.ts src/lib/task-runner-manager.ts', INDEX, { maxPins: 1 });
expect(out.pinnedFiles).toEqual(['src/lib/chat-manager.ts']);
expect(out.capacityFiles).toEqual(['src/lib/task-runner-manager.ts']);
expect(out.strippedQuery).toBe('src/lib/task-runner-manager.ts src/lib/task-runner-manager.ts');
expect(out.unresolvedPathSpans).toEqual([]);
});

it('reports partial shared-basename capacity separately from ambiguity and missing files', () => {
const out = extractQueryPaths('generic-modal.tsx src/missing.ts and/or', INDEX, { maxPins: 1 });
expect(out.pinnedFiles).toEqual(['src/x/generic-modal.tsx']);
expect(out.capacityFiles).toEqual(['src/y/generic-modal.tsx']);
expect(out.setAsideMatches).toEqual([]);
expect(out.strippedQuery).toBe('src/missing.ts and/or');
});

it('does not count duplicate pins, ambiguous references or prose as capacity overflow', () => {
const out = extractQueryPaths('src/lib/chat-manager.ts src/lib/chat-manager.ts user-profile.tsx and/or', INDEX, { maxPins: 1 });
expect(out.capacityFiles).toEqual([]);
expect(out.unexaminedPathSpans).toBe(false);
});

it('marks the bounded scan incomplete without inventing overflow matches', () => {
const files = Array.from({ length: 10 }, (_, i) => `src/unit${i}.ts`);
const out = extractQueryPaths(files.join(' '), files, { maxPins: 1 });
expect(out.pinnedFiles).toEqual([files[0]]);
expect(out.capacityFiles).toEqual(files.slice(1, 8));
expect(out.unexaminedPathSpans).toBe(true);
expect(out.strippedQuery).toBe(files.slice(1).join(' '));
});
});


it('reports extensionless overflow after explicit paths fill pin capacity', () => {
const out = extractQueryPaths('background-image-table then src/lib/chat-manager.ts', INDEX, { maxPins: 1 });
expect(out.pinnedFiles).toEqual(['src/lib/chat-manager.ts']);
expect(out.capacityFiles).toEqual(['src/components/training-set-page/background-image-table.tsx']);
expect(out.strippedQuery).toBe('background-image-table then');
});
2 changes: 2 additions & 0 deletions site/src/content/docs/reference/mcp-server.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@ By default the server exposes `codegraph_explore` for code and `codegraph_sessio

Exposing one strong code tool is deliberate. Measured agent behavior showed that one well-aimed tool steers agents to a direct answer better than a menu of narrower ones — fewer mis-picks — and agents reach for it both when answering questions and while editing code.

Explore reports requested files that did not receive pin priority within the file limit. Those files may still appear through other matches. References beyond the bounded path scan are marked unexamined. When a capped file gather leaves indexed source uncovered, the summary offers continuation ranges for another explore call. Existing file and output limits still apply.

Explicit code-file pins remain eligible when the query also names documentation. Quoted terms used to select template or style source remain meaningful even when they match the filename stem. File and output budgets still apply.

When a query names functions in pinned files, `codegraph_explore` prioritizes their local callee bodies through two call or callback hops. It selects at most sixteen local helpers per file, each at most 200 lines. File and output budgets still apply.
Expand Down
Loading
Loading