feat(k8s): Kubernetes restart-safety + Contabo + agent-budgets feature set - #248
Open
ciprianiacobescu wants to merge 34 commits into
Open
feat(k8s): Kubernetes restart-safety + Contabo + agent-budgets feature set#248ciprianiacobescu wants to merge 34 commits into
ciprianiacobescu wants to merge 34 commits into
Conversation
added 30 commits
June 23, 2026 12:25
P1 — Bootstrap sectioning + MCP Resource - Add XML-tag section parser to bootstrap.py (parsed once at import) - REST GET /v1/tools/bootstrap?section= returns compact index/section/full - REST default = 'full' (backward compat, D-2); MCP tool default = 'index' - Register mintkey://skill/agent-bootstrap as MCP Resource via resources/list + resources/read in the JSON-RPC dispatcher - mintkey_bootstrap inputSchema gains 'section' enum param P2 — Trim tools/list - Remove 'title' key from all 10 TOOLS entries - Shorten secret_put/get/list/delete descriptions to load-bearing text only P3 — Pointer-ize _BUILTIN_CAPABILITIES - Replace 474-char string with ~120-char pointer retaining all literals required by TestBuiltinCapabilitiesAdvertised P6 — Cap error hints - _upstream_to_tool_result: hint[:120] prevents unbounded context blowup Also: - admin_api_client.py: typed async httpx boundary for admin-api calls (INV-2) - 4 new test files (test_bootstrap_sections, test_mcp_resources, test_tools_list_trim, test_error_hint_cap); 339 tests pass, 5 skipped
…outcomes Remediations now have audit-trail state files committed: - 2026-05-29 ci-egress-and-playwright: CLOSED (Playwright OIDC + offline egress coverage) - 2026-05-31 apple-jwt: CLOSED (all 7 chunks merged, apple_jwt auth scheme on main) - 2026-05-31 ghcr-compose-install: CLOSED (PR #131 — install.sh + docker-compose.ghcr.yml) - 2026-05-31 vault-pg-migration: CLOSED (138 creds migrated, ADR-0021, d340c18) - 2026-06-01 ssh-credential-flow: CLOSED (PR #139 — services.base_url canonical SSH) - 2026-06-01 ssh-proxy-fix: CLOSED (C1–C9 all merged, SSH proxy fully functional) - 2026-06-03 email-imap-addr-and-grant-notify: CLOSED (PR #193 — IMAP addr, pg_notify, Gmail userinfo) - 2026-06-03 oauth2-authorized-ui-flag: CLOSED (PR #193 — oauth2_authorized derived from vault) - 2026-06-04 pr-triage-resolve: CLOSED (18/22 PRs merged, uuid vuln fixed, 4 conflicted open) - 2026-06-07 outlook-userinfo: CLOSED (PR #197 — Graph /me fetch, User.Read scope) Also: apps/proxy-plugin/.gitignore to exclude compiled binary from tracking.
Call-count ceilings per permission grant with automatic proxy circuit-breaking. When the ceiling is hit, the proxy returns 429 budget_exceeded without touching the upstream. Schema: - Liquibase changeset 028-budget-counters (composite PK, RLS, cascade FK) Contracts: - OpenAPI: Constraints.budget, BudgetStatus, GET/POST budget endpoints - OpenAPI: 4 new audit event types (threshold/exceeded/config/reset) - MCP tools: your_constraints.budget in describe_service output - JSON Schema: budget audit event payload schemas Admin API (Python): - BudgetCounter SQLAlchemy model + Pydantic schemas - Budget validation in grant CRUD (create/PATCH) - GET /budget + POST /budget/reset endpoints - Period boundary utility (hourly/daily/weekly/monthly, UTC-aligned) Proxy Plugin (Go): - Atomic budget.Check() with INSERT...ON CONFLICT upsert - 429 + Retry-After on ErrBudgetExceeded - Threshold audit emission with dedup tracker - Change-channel subscriber for budget invalidation - Prometheus metrics (used/ceiling/denied_total) MCP Server: - describe_service returns real-time budget status Observability: - Grafana dashboard (bar gauge + denied rate + 24h stat) - Prometheus alert BudgetNearExhaustion (>90% for 1m) Tests: - 24 unit tests for period bounds (Python) - 16 unit tests for budget check (Go) - Integration tests for config validation + endpoints - Acceptance tests (enforcement, rollover, reset, propagation) - Architecture tests (RLS coverage, cascade deletion) Documentation: - ADR-0029 promoted from draft - AGENTS.md + CLAUDE.md guardrails updated - CHANGELOG entry Refs: ADR-0029, P-011, FR-1..FR-10, NFR-1..NFR-6
… BFF routes, tests Implements ADR-0029 agent budget enforcement on the admin-ui layer. - BudgetStatusPanel: shows call ceiling, current count, period, threshold markers, and Edit/Reset/Remove actions on the permission-grant detail page - BudgetForm: create+edit form for ceiling/period/thresholds (AdminJS action) - BFF routes (src/routes/budget.ts): GET/POST proxy to admin-api budget endpoints - budget-validate.ts / budget-format.ts: pure utility functions (period labels, input validation with error messages) - admin-api permissions.py: budget removal via PATCH using model_fields_set to distinguish null (remove budget) from absent (keep budget) - 69 tests across 5 test files; 3 pre-existing failures unrelated to this work - Kiro spec budget-management-ui: all tasks complete
…roxy, long-lived-api-keys specs All tasks were implemented in earlier sessions; Kiro updated the task files to reflect [x] completion status.
…ix BFF paths
The BudgetStatusPanel was showing "Unable to load budget status" (502) because:
1. GET /v1/tenants/{tid}/permissions/{pid}/budget didn't exist — admin-api only
had the agent-scoped path and a flat list endpoint, not a single-record or
budget-status endpoint on the tenant router.
2. BFF budgetGetHandler was doing a two-hop via lookupPermission (which also
called a non-existent GET /permissions/{pid} endpoint) before calling the
budget endpoint.
3. budgetEditHandler/budgetRemoveHandler/budgetResetHandler were sending PATCH/POST
to /v1/tenants/{tid}/permissions/{pid} (flat router, no PATCH handler) instead
of the agent-scoped path.
Fixes:
- admin-api: add GET /{permission_id}/budget to tenant_permissions_router —
reads constraints.budget from permission_grants + used/period from
budget_counters; returns 404 when no budget configured.
- admin-api: add GET /{permission_id} to tenant_permissions_router — returns
id/agent_id/service_id/action/constraints (used by BFF lookupPermission for
write operations).
- admin-api: add POST /{permission_id}/budget/reset to agent-scoped router —
resets current-period counter to 0, emits budget.reset audit event.
- BFF budgetGetHandler: call /permissions/{pid}/budget directly (remove two-hop).
- BFF edit/remove/reset handlers: fix PATCH/POST paths to include agents/{aid}.
New AdminJS custom page showing all budget-configured permission grants
ranked by consumption percentage with:
- Server-side aggregation endpoint (admin-api): GET /v1/tenants/{tid}/budget-consumers
joins permission_grants, budget_counters, agents, services + audit_events
for requests_last_30_min count
- BFF proxy route: GET /admin/api/budget-consumers
- React page: data table with 30s auto-polling, client-side filtering
(threshold %, agent name, service name), red highlight on exhausted
rows, inline "Unlock" button to reset exhausted budgets
- AdminJS sidebar nav item "Budget Consumers" (icon: Activity)
- Pure utilities: filterConsumers (logical AND), isExhausted
- Full test suite: pytest (admin-api), vitest + supertest (BFF),
vitest + RTL (component), vitest (unit)
Spec: .kiro/specs/budget-consumers-dashboard
…clude all grants Two bugs: 1. audit_events timestamp column is 'at', not 'created_at' — the query was 500-ing on every call, causing the Budget Consumers page to never load. 2. WHERE filter excluded non-budgeted grants. Removed the two predicates so all permission grants appear (unlimited grants have null ceiling/used). Also: bc.used is no longer COALESCE'd to 0 in SELECT (becomes null for unlimited grants), consumption_percentage computation guards against None ceiling/used, and ORDER BY uses COALESCE to sort unlimited grants last.
…geted rows - BudgetConsumerRecord: widen ceiling/used/period/consumption_percentage to nullable (number | null) to reflect unlimited grants from the API - isExhausted: returns false when ceiling or used is null (unlimited = never exhausted) - filterConsumers: treats null consumption_percentage as 0 for threshold filter - BudgetConsumersPage: renders "Unlimited" for null ceiling/period, "—" for null used/consumption_percentage, relabels threshold filter to "Min consumption %", updates empty-state copy to "No permission grants found"
… exhausted state
Implements optional spec tasks 3.3, 4.3, 4.4 from budget-consumers-dashboard spec.
- budget-consumers-bff.property.test.ts (Task 3.3):
Property 3: BFF constructs /v1/tenants/{tid}/budget-consumers URL from session
Property 4: BFF proxies HTTP status + body byte-identically
- budget-consumers-filter.property.test.ts (Tasks 4.3 + 4.4):
Property A: filterConsumers applies all active filters as logical AND
Property B: isExhausted(record) iff used >= ceiling; false when either is null
Tasks 4.5 (consumption % computation) and 4.6 (descending sort) are N/A:
the computation is server-side Python and the sort is pre-applied by the API.
…or budget-counters
Adds a pre-configured Tailscale catalog entry to service_templates.yaml (category: networking, auth_type: bearer_token, test_path /api/v2/tailnet/-/devices). Data-only change — no new auth scheme, no proxy route, no DB migration. Includes OpenSpec proposal/design/tasks and registry test assertions.
The CREATE TABLE changeset lacked a preCondition guard, causing Liquibase to fail when the table already existed (renamed from 028-budget-counters). The RLS and index changesets had 028- IDs that didn't match the MARK_RAN entries already inserted. All three are now idempotent.
…teError
OTel's _get_route_details has a try/except for Match.FULL but not for
Match.PARTIAL, crashing with AttributeError when a _IncludedRouter
(produced by FastAPI's include_router) matches partially. This caused
every POST /v1/tenants/{tid}/agent-secrets request to 500 before the
handler ran. Monkey-patch the function after instrumentation.
…rash Move _safe_get_route_details to module level so it is importable for direct unit testing. Add four unit tests in test_otel_patch.py covering PARTIAL/FULL match on a router without .path, NONE match, and empty routes. Add an ASGI-level regression test in test_agent_secrets.py that applies configure_otel() to the test app and verifies POST returns non-500.
- New `contabo` HTTP service template using oauth2_password_grant auth. Credentials: client_id, client_secret, username, password, grant_type=password. Token endpoint: Contabo Keycloak ROPC at auth.contabo.com. test_path: GET /v1/users/client. - Extend oauth2_password_grant exchanger to support form-encoded token bodies. When token_request_headers["Content-Type"] is application/x-www-form-urlencoded, the credential_fields are posted as url.Values instead of JSON. Backward-compatible: existing JSON-body templates are unaffected. Two new tests cover both paths. - Auto-inject X-Request-Id UUID4 header on every proxied request when absent. Added to all three Director closures. Required by Contabo; harmless for all others. - Add token_request_headers to CredentialHint model and propagate it through the from-template service-creation endpoint (services.py) so the Content-Type override reaches the vault credential at creation time. - OpenSpec: openspec/changes/contabo-service-template/ (proposal + design + tasks).
Dependency ReviewThe following issues were found:
License Issuesapps/admin-api/uv.lock
OpenSSF Scorecard
Scanned Files
|
4 tasks
added 2 commits
July 5, 2026 21:29
…tate model; trivyignore CVE-2026-41992 + CVE-2026-54369
…RF suppressed (sameSite+signed-JWT); tighten URL assertion in email template test
…1.6.5→1.7.2; fix CodeQL suppression placement - oidc_login_state is intentionally platform-scoped (login precedes tenant context) — add to PLATFORM_SCOPED_TABLES to satisfy ADR-0008 architecture test - joserfc 1.6.5 has CVE-2026-49852 (HIGH, HMAC key empty/nil bypass); pin joserfc>=1.6.8, lock resolves to 1.7.2 - Move codeql[js/server-side-request-forgery] suppressions to the same line as fetch() in budget.ts (CodeQL requires inline suppression on the flagged line) - Move codeql[js/missing-token-validation] suppression to session({ line in index.ts (was one line above — not picked up by CodeQL) - Add codeql suppressions to test harness session() calls (CSRF + clear-text cookie false-positives in test-only code with secure:false)
Comment on lines
+32
to
+35
| const resp = await fetch( // codeql[js/request-forgery] ADMIN_API_URL is a fixed env var; permId is path-only, validated by admin-api | ||
| `${ADMIN_API_URL}/v1/tenants/${tenantId}/permissions/${permId}`, | ||
| { headers: { Cookie: cookie } } | ||
| ); |
Comment on lines
+64
to
+67
| const upstream = await fetch( // codeql[js/request-forgery] ADMIN_API_URL is a fixed env var; permId is path-only, validated by admin-api | ||
| `${ADMIN_API_URL}/v1/tenants/${tenantId}/permissions/${permId}/budget`, | ||
| { headers: { Cookie: cookie } } | ||
| ); |
Comment on lines
+66
to
+71
| session({ // codeql[js/missing-csrf-middleware] codeql[js/clear-text-cookie] test harness only | ||
| secret: "test-secret", | ||
| resave: false, | ||
| saveUninitialized: true, | ||
| cookie: { secure: false }, | ||
| }) |
Comment on lines
+66
to
+71
| session({ // codeql[js/missing-csrf-middleware] codeql[js/clear-text-cookie] test harness only | ||
| secret: "test-secret", | ||
| resave: false, | ||
| saveUninitialized: true, | ||
| cookie: { secure: false }, | ||
| }) |
Comment on lines
+67
to
+72
| session({ // codeql[js/missing-csrf-middleware] codeql[js/clear-text-cookie] test harness only | ||
| secret: "test-secret", | ||
| resave: false, | ||
| saveUninitialized: true, | ||
| cookie: { secure: false }, | ||
| }) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
budget_exceeded+Retry-After), budget-consumers admin UI page, live Grafana dashboard, OTel regression guardx-request-idauto-injection,credential_hint.token_request_headerspropagationbroker: signing key loaded fromMINTKEY_BROKER_SIGNING_KEY_FILE(seed file →ed25519.NewKeyFromSeed); dev falls back to ephemeral, production fails closedadmin-api: OIDC login state moved from in-processdictto Postgresoidc_login_statetable (Liquibase changeset 030, TTL-expiring, single-usepop())admin-ui/seed-job: Ed25519 keypair generated once by seed-job and mounted;signed_request.pyrejects unsigned writes in production when no public key is presentliquibase: DB role passwords parameterised via${db_app_password}/${db_subscriber_password}substitution vars instead of hard-coded literalsTest plan
go test ./apps/broker/internal/keyloader/...— 4 tests (file load, cross-validate, dev ephemeral, prod fail-closed)pytest apps/admin-api/tests/unit/admin_api/test_oidc_state_store.py— 4 testspytest apps/admin-api/tests/unit/admin_api/test_signed_request_prod.py— 2 testspytest apps/admin-api/tests/unit/admin_api/test_015_migration_no_literals.py— 3 testspytest apps/seed-job/tests/test_admin_ui_keypair.py— 4 testsdocker compose up -d— all 17 services healthy (no regression).envunset → compose starts with dev defaults (A4 backward compat)