Repository navigation
Testing: Tier budgets (T-13) #347
Description
Part of the test strategy (captain, 2026-10-05: scenarios that test the changes together; cleaner, faster suites; TESTING.md). Epic #305.
Scope. tests/budget.toml (unit = 60, component = 180, scenarios = 120, specs = 60, oracle_pr = 15 seconds); a step parses the finished in lines per binary (mapped to tiers by path), fails above budget + 25 percent, and uploads tests-timing.json.
Acceptance. The step passes on main; a PR that adds a 30 s sleep to a component test fails it.
Activity
jsam commentedon Oct 5, 2026
Add to this issue's scope: a perf-tier fixture measuring shop_pack(Size::Small) install time on the benchmark host with the 200 ms budget from #335 (the scenario harness reports the time but no longer asserts it, to avoid wall-clock flakes).
no-mistakes(review): Document shop_pack install budget as tracked in #…
no-mistakes(review): Document shop_pack install budget as tracked in #…
no-mistakes(review): Document shop_pack install budget as tracked in #…
no-mistakes(review): Document shop_pack install budget as tracked in #…
test(scenarios): scenario harness on the testkit, with the reactive e…
test: tier budgets, remaining scenarios, oracle shop corpus and maint…