Skip to content

feat: Phase 2 — Token Budget Manager and model limits - #4

Merged
getdownandcode merged 6 commits into
mainfrom
feat
Sep 9, 2026
Merged

getdownandcode merged 6 commits into
mainfrom
feat

Conversation

@getdownandcode

Copy link
Copy Markdown
Owner

Summary

Implements Phase 2: Token Budget Manager for gitmate according to plan.md, AGENTS.md, and agreed design decisions.

Key Deliverables

  1. TokenCounter Protocol & GeminiTokenCounter:

    • TokenCounter pure Python Protocol (count_tokens(text: str, model: str) -> int) decorated with @runtime_checkable.
    • GeminiTokenCounter wraps Google GenAI SDK (google-genai), using caller-passed credentials (SecretStore) rather than ambient os.environ.
    • Typed error hierarchy: TokenBudgetError, TokenCountError, and NoApiKeyError.
  2. Budget Decision Engine (TokenBudgetManager):

    • Computes effective input budget:
      99092\text{effective_budget} = \text{context_window} - \text{reserved_output_tokens} - \text{template_overhead}99092
    • Strategies (BudgetStrategy):
      • FITS: full diff fits within budget.
      • TRUNCATED: candidate files sorted by patch size descending are truncated with [patch omitted for size: +A/-D lines] until diff fits.
      • NEEDS_CHUNKING: signals Phase 3 to execute chunked summarization when diff metadata alone exceeds budget.
    • BudgetDecision retains the original, untruncated FileDiffs in omitted_diffs so Phase 3 can chunk without re-reading git.
    • Prevents token inflation by skipping truncation if len(patch_text) <= len(note).
  3. Model Limits & Config:

    • Context window mapping for known models (DEFAULT_MODEL_CONTEXT_WINDOWS).
    • Configurable max_context_tokens and reserved_output_tokens (with validation in load_config/save_config).
    • UnknownModelError raised if a model's context window cannot be resolved and no override is provided.
    • Conservative DEFAULT_TEMPLATE_OVERHEAD = 500 tokens configured for Phase 2, ready for exact template token counts in Phase 4.
  4. Testing & Verification:

    • 100% offline hermetic unit tests with FakeTokenCounter and mocked GenAI client (zero real network/API calls).
    • Full test suite: 86 passing tests across CLI, config, diff extractor, and token budget.
    • mypy src tests --strict and ruff check . pass cleanly.

- Reject non-positive effective budgets instead of emitting nonsense limits
- Truncate low-signal files first (snapshots, tests, generated), largest first within groups
- Working copies via dataclasses.replace; identity-keyed index handles duplicate paths
- Reuse last token count instead of a final paid recount
- Document BudgetDecision aliasing contract and note semantics
- Document why ignore_globs and _is_low_signal stay separate mechanisms
- Test whole low-signal group truncates before any normal file, in order
- Test many guard-skipped small files collectively fall through to chunking
@getdownandcode

Copy link
Copy Markdown
Owner Author

Follow-up fixes on top of the Phase 2 implementation, from the review round plus three clarification questions. Two commits:

75b3999 — review findings

  • Non-positive effective budgets now raise ValueError instead of emitting nonsense negative limits (guard sits in assess, so it covers constructor args and per-call overhead — an __init__-only check would have missed the latter).
  • Truncation drops low-signal files first (snapshots, tests, generated code, largest-first within groups), matching the plan's strategy; char length stays documented as the cost proxy to avoid a paid count call per file.
  • Working copies use dataclasses.replace() (no more 7-field manual copy to rot); identity-keyed index so duplicate paths can't collapse.
  • Final NEEDS_CHUNKING total reuses the last loop recount instead of a duplicate paid API call (pinned: 3 calls, not 4, for the 2-file chunking case).
  • BudgetDecision docstring now states the aliasing contract (omitted holds originals by design; included carries all files with truncated ones as note placeholders). Note wording unchanged — tests and plan language already bless it.
  • Config-side notes (model IDs, from_config, overhead plumbing) deliberately deferred to Phase 3.

5a9432f — ordering and coverage questions

  • _is_low_signal vs ignore_globs is a deliberate split, now stated in the docstring: "never send" (user-configured) vs "drop first under pressure" (built-in judgment, always sent whole when budget allows).
  • Sort key provably exhausts the whole low-signal group largest-first before any normal file, no matter the sizes — pinned by a test where a 150-char snap truncates before a 300-char normal file, then spills over in order.
  • The guard-skipped collective-overflow case (many tiny files, none individually truncatable, headers alone over budget) correctly falls through to NEEDS_CHUNKING — previously untested, now pinned.

Gate: 92 pytest green, ruff check, format --check, mypy --strict clean.

@getdownandcode

Copy link
Copy Markdown
Owner Author

Follow-up review fixes addressing the pre-merge review items and audit findings:

1. Truncation policy decision record & simplification (Option A)

  • Reverted _is_low_signal and aligned candidate sorting strictly by descending raw patch length (len(d.patch_text)).
  • Documented the deliberate policy decision on TokenBudgetManager:

    "Truncation orders candidate diffs strictly by descending patch length (largest raw patch first). Signal weighting was evaluated, but common generated artifacts and lockfiles are already filtered upstream by ignore_globs; its only non-redundant contribution (test-directory deprioritization) was actively harmful, stripping critical test diffs in test-focused commits. Reverted to size-only ordering as the deliberate policy."

2. Anti-inflation guard & NEEDS_CHUNKING interaction in mixed diffs

  • Fixed the summary_note formatting bug in Step 3 where mixed diffs (some files truncated, remainder guard-skipped) emitted misleading notes like "even with all 1 files truncated".
  • Formatted trunc_info with proper ratios ("even with 1/3 files truncated; requires chunking").
  • Added test_mixed_truncated_and_guarded_diffs_trigger_needs_chunking to pin this exact edge case.

3. template_overhead wired into GitmateConfig

  • Added template_overhead: int = DEFAULT_TEMPLATE_OVERHEAD (500) to GitmateConfig.
  • Added parsing, positive-integer validation, and persistence to load_config / save_config.
  • Enforced budget validation at config-load time (max_context_tokens > reserved_output_tokens + template_overhead) so invalid limits are rejected upfront instead of failing when assess() is called.
  • Added round-trip and validation tests in tests/test_config.py.

4. Domain typed exception

  • Added InvalidBudgetError(TokenBudgetError) to replace the raw generic ValueError when effective_budget <= 0.

Gate: 97 pytest passed, ruff check, ruff format --check, and mypy --strict clean.

- Reject negative reserved_output_tokens, default_template_overhead, and per-call template_overhead
- BudgetDecision docstring now matches copy-all implementation on truncate paths
@getdownandcode

Copy link
Copy Markdown
Owner Author

Follow-up commit addressing opencode's review findings:

  1. Named Risk for Phase 3 (Tracked in plan.md):

    • Documented under Phase 3 in plan.md that TokenBudgetManager defaults to a 1M context window (context_window: int = 1_048_576) independent of model. In Phase 3, when wiring up provider calls for models with smaller context windows (e.g., Claude's 200k limit), the caller must pass get_context_window(cfg) or use a TokenBudgetManager.from_config(cfg, counter) factory method rather than relying on constructor defaults.
  2. Validation Gap Closed for Budget Inputs (feat: Phase 1 diff/context extractor with parser hardening #2):

    • Extended InvalidBudgetError checks in TokenBudgetManager.__init__ and assess() to reject non-positive values (<= 0) for context_window, reserved_output_tokens, and template_overhead.
    • Updated and expanded tests in test_non_positive_budget_inputs_raise to verify that both 0 and negative inputs are rejected upfront with InvalidBudgetError.
  3. Docstring Clarification on Aliasing (fix: filename quoting, nested globs, and small display bugs #3):

    • Updated BudgetDecision docstring to accurately describe the safer behavior: on truncate and chunk paths, all entries in included_diffs are fresh copies created via dataclasses.replace() so caller objects are never mutated in-place.

Gate: 98 pytest passed, ruff check, ruff format --check, and mypy --strict clean.

@getdownandcode
getdownandcode merged commit dcd0129 into main Sep 9, 2026
2 checks passed
getdownandcode added a commit that referenced this pull request Sep 19, 2026
…d orchestrator consolidation

- #1 (Blocker): Fail loud with exit code 1 on git errors, budget cap breaches, and gh CLI errors across pr-summary and changelog CLI commands.
- #2: Prevent duplicate printing when no changes/commits exist by early returning None.
- #3: Route diagnostic warnings, budget alerts, and error messages to stderr console (err_console) to keep stdout clean for Unix piping.
- #4: Correctly parse and surface breaking changes (exclamation mark and BREAKING CHANGE: footers) in conventional commits and templates.
- #5: Bump changelog template to version 2, including from_ref..to_ref range and explicit release headers.
- #6: Support --title (-t) flag in pr-summary and propagate gh pr create failures.
- #7: Remove dead head argument from fallback PR summary.
- #8: Consolidate duplicate budget checking, safe telemetry recording, and heuristic token counters into orchestrator.py.
- #9: Enforce strict type hints with CommitLogEntry in fallback.py.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant