Skip to content

feat: Phase 3 provider abstraction and cache (Gemini-first) - #5

Merged
getdownandcode merged 5 commits into
mainfrom
feat
Sep 14, 2026
Merged

feat: Phase 3 provider abstraction and cache (Gemini-first)#5
getdownandcode merged 5 commits into
mainfrom
feat

Conversation

@getdownandcode

@getdownandcode getdownandcode commented Sep 10, 2026

Copy link
Copy Markdown
Owner

Overview

Phase 3: LLMProvider abstraction with a caching wrapper and a Gemini backend, resolving the plan.md vs AGENTS.md conflict in favor of Gemini-first (the locked-in default). Anthropic stays deferred to the stretch-ideas list.

What changed

providers/base.py (new)

  • LLMProvider Protocol (generate(prompt, model) -> LLMResponse)
  • LLMResponse dataclass: text, model, optional input_tokens/output_tokens
  • ProviderUnavailable (Phase 4 catches) + RetryableProviderError marker

providers/gemini.py (new)

  • GeminiProvider over google-genai, lazy genai.Client(api_key=...) like GeminiTokenCounter
  • tenacity retry on transient failures (5xx ServerError, 429, httpx.TransportError), 3 attempts, exponential backoff; final failure → ProviderUnavailable

providers/cache.py (new)

  • CachedProvider on diskcache, key sha256(template_version + model + prompt)
  • template_version hardcoded "1" until Phase 4 ships real templates (avoids a key migration)
  • default cache dir ~/.cache/gitmate/ (XDG), overridable

config.py

  • DEFAULT_MODEL → gemini-3.5-flash-lite (literal SDK ID, confirmed real), with a comment on Gemini's retirement cadence
  • new cache_dir override field + validation

token_budget.py

  • TokenBudgetManager.from_config(cfg, counter) resolves get_context_window(cfg) — closes the Phase 2 named risk (no 1M default against a 200k model)

plan.md — Phase 3 text corrected to Gemini-first (local-only file, gitignored)

Tests (113 green, no real API calls)

  • Cache consulted before generate() — two identical calls hit the underlying provider once
  • Cache key bump on template_version/model; cache_dir override honored
  • Gemini: success, retry-then-succeed (2 calls), retry-exhausted → ProviderUnavailable (3 calls), non-transient 400 raised once
  • from_config: gemini-3.5-flash-lite → 1,048,576; claude-3-5-sonnet → 200,000; override; unknown-model raises

Deferred (intentional)

  • AnthropicProvider (secondary) — stretch idea, after the abstraction is proven
  • Token-usage plumbing into metrics — Phase 6

@getdownandcode
feat: Phase 3 provider abstraction and cache layer (Gemini-first)
e31ce6f 
- LLMProvider protocol, LLMResponse, ProviderUnavailable in providers/base
- GeminiProvider over google-genai with tenacity retry on 5xx/429/transport
- CachedProvider on diskcache, keyed sha256(template_version+model+prompt), template_version=1
- cache_dir config override; DEFAULT_MODEL now gemini-3.5-flash-lite (literal SDK ID)
- TokenBudgetManager.from_config resolves real context windows (named risk fixed)
- plan.md Phase 3 corrected to Gemini-first (local-only, gitignored)
- Anthropic intentionally deferred to stretch ideas
- 113 tests green; fake-client only, no real API calls
@getdownandcode
fix: guard empty/filtered responses, dynamically wire config CLI fiel…
9e6721f 
…ds, and speed up tenacity tests
@getdownandcode

getdownandcode commented Sep 10, 2026

Copy link
Copy Markdown
Owner Author

Review Fixes Applied (commit 9e6721f)

All review feedback items have been addressed and verified:

  1. Empty / Filtered Response Guard & Cache Poison Prevention (docs: draw.io architecture, class, use-case, and sequence diagrams #1):

    • GeminiProvider._generate_with_retry: If text is empty, inspects resp.candidates[0].finish_reason if available and raises ProviderUnavailable(f"Gemini declined to respond (finish_reason: {reason_name}).") (or `ProviderUnavailable("Gemini returned an empty response.")"). This guarantees Phase 4 template fallback will fire when Gemini filters or returns empty content, instead of silently passing an empty message.
    • CachedProvider.generate: Checks if response.text.strip(): before self._cache.set(...), ensuring unhelpful/empty responses never permanently poison the disk cache.
    • Added unit tests: test_gemini_provider_empty_response_raises, test_gemini_provider_safety_declined_raises_with_finish_reason, and test_cache_does_not_store_empty_response.

  2. Root-Cause Config CLI Wiring (feat: Phase 1 diff/context extractor with parser hardening #2):

    • Rather than patching cache_dir as a one-off, refactored config show and config set in src/gitmate/cli.py to dynamically derive fields from dataclasses.fields(GitmateConfig).
    • Closes the schema drift accumulated since Phase 2: max_context_tokens, reserved_output_tokens, template_overhead, and cache_dir can now all be set and displayed via the CLI, with positive integer validation and budget boundary checks (max_context_tokens > reserved + overhead).
    • Added CLI integration tests in tests/test_cli.py.

  3. Tenacity Test Acceleration (fix: filename quoting, nested globs, and small display bugs #3):

    • Added an autouse fast_retries fixture in tests/test_providers.py monkeypatching tenacity's wait strategy with wait_none(). Eliminates ~3.5s of real time.sleep during retry tests, cutting total suite execution time in half (~9.6s → ~4.7s).

  4. Test Hygiene (feat: Phase 2 — Token Budget Manager and model limits #4):

    • Replaced the hardcoded /tmp/nope directory in test_is_like_provider_protocol with tmp_path and an explicit cached.close() call.

  5. Model Registry Completeness (feat: Phase 3 provider abstraction and cache (Gemini-first) #5):

    • Added "gemini-3.5-flash": 1_048_576 to DEFAULT_MODEL_CONTEXT_WINDOWS with a clarifying comment (# Supported for user override via 'config set model', not our default).

  6. YAGNI Adherence (Feat #6):

    • Left CachedProvider without speculative __enter__/__exit__ context manager methods until Phase 4/5 callers concretely need that pattern.

Verification

  • Test Suite: 118 passed in 4.99s (uv run pytest)
  • Linters & Formatters: Clean (uv run ruff check . and uv run ruff format --check .)
  • Type Checker: Clean (uv run mypy --strict src tests - 15 files checked, 0 errors)

@getdownandcode

getdownandcode commented Sep 10, 2026

Copy link
Copy Markdown
Owner Author

Note: Aligned README.md and config registry strictly on gemini-3.5-flash-lite (our locked-in default), removing the extra unneeded gemini-3.5-flash entry to avoid any ambiguity.

@getdownandcode

getdownandcode commented Sep 14, 2026

Copy link
Copy Markdown
Owner Author

Added httpx>=0.27 to pyproject.toml dependencies (commit 0c99c80), resolving the undeclared direct dependency imported in src/gitmate/providers/gemini.py.

@getdownandcode
getdownandcode merged commit 12891fc into main Sep 14, 2026
2 checks passed
@getdownandcode getdownandcode mentioned this pull request Sep 17, 2026
Merged
getdownandcode added a commit that referenced this pull request Sep 19, 2026
@getdownandcode
fix: address PR #12 review feedback on exit codes, stderr routing, an…
2cda9ef 
…d orchestrator consolidation

- #1 (Blocker): Fail loud with exit code 1 on git errors, budget cap breaches, and gh CLI errors across pr-summary and changelog CLI commands.
- #2: Prevent duplicate printing when no changes/commits exist by early returning None.
- #3: Route diagnostic warnings, budget alerts, and error messages to stderr console (err_console) to keep stdout clean for Unix piping.
- #4: Correctly parse and surface breaking changes (exclamation mark and BREAKING CHANGE: footers) in conventional commits and templates.
- #5: Bump changelog template to version 2, including from_ref..to_ref range and explicit release headers.
- #6: Support --title (-t) flag in pr-summary and propagate gh pr create failures.
- #7: Remove dead head argument from fallback PR summary.
- #8: Consolidate duplicate budget checking, safe telemetry recording, and heuristic token counters into orchestrator.py.
- #9: Enforce strict type hints with CommitLogEntry in fallback.py.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

See More

Successfully merging this pull request may close these issues.

1 participant

@getdownandcode