ci: add tirith security scan and fail-fast preflight gate - #1383
Conversation
Add a security gate to the CI workflow so the expensive build/test work runs only after the cheap, high-signal security checks pass. Two new jobs run unconditionally and in parallel at the front of the pipeline (not gated on `changes`, so they cover docs-only PRs and fork PRs where local git hooks never run): - `tirith-scan`: scans the whole checkout with tirith (https://tirith.sh/docs/ci/), uploads SARIF to the Security tab, and gates downstream CI. Wired on `pull_request` (not `pull_request_target`), so fork PRs only get a read-only token and cannot leak secrets while their code is scanned; the SARIF upload is therefore skipped for forks (the scan and gate still run). tirith is installed from the flake's pinned nixpkgs to match the dev shell. - `preflight`: runs the existing `gitleaks` and `treefmt` flake checks (treefmt carries actionlint + zizmor with the repo's exact ignore config) as an early fail-fast gate before any build/test work. `test` and `build-native-packages` now `needs` both gates, so they are skipped unless the gates pass. tirith details: - `--fail-on high` blocks on high/critical findings. tirith's `--ci` exit code is 0 = clean, 1 = at/above threshold, 2 = findings below threshold; the gate treats 0/2 as pass and 1 (or any other code) as a block, and the scan exit code is passed via env to avoid template injection. - `*.snap` is excluded because terminal snapshot test fixtures intentionally contain ANSI escapes that trip the `ansi_escapes` rule; the rule stays active for real source files. Heavy gate jobs run on blacksmith runners, consistent with the rest of the nix-based CI jobs.
|
no API key found — this repo is configured to use To fix: add the key as a GitHub Actions secret (referenced from your workflow's Open repo secrets → · Configure model → · Setup docs → · Ask in Discord →
|
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
ccusage-guide | 8ff5cb8 | Commit Preview URL Branch Preview URL |
Jun 29 2026, 01:38 PM |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughThe CI workflow adds tirith-based security scanning, SARIF upload to the GitHub Security tab, and a preflight job for secret and formatting checks. Downstream test, build, and timeline jobs now wait for those new gates. ChangesCI Security Gate
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
You are seeing this message because GitHub Code Scanning has recently been set up for this repository, or this pull request contains the workflow file for the Code Scanning tool. What Enabling Code Scanning Means:
For more information about GitHub Code Scanning, check out the documentation. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/ci.yaml:
- Around line 41-47: The CI gates are currently bootstrapping from the PR
checkout, which lets PR-controlled code influence `tirith-scan` and `preflight`.
Update the workflow to source trusted helper/actions and tool pins from the base
ref instead of `github.event.pull_request.head.sha`, while still scanning the PR
worktree as input data. Adjust the `actions/checkout`, `setup-nix-cache`, and
`Install tirith` steps accordingly, and ensure the `tirith-scan` / `preflight`
jobs use trusted flake checks and local actions from the base branch rather than
the PR-defined versions; if SARIF upload is needed, split it into a separate
upload job with `security-events: write`.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: b25c433e-3f2a-4b86-8ca4-3d933a50d19a
📒 Files selected for processing (1)
.github/workflows/ci.yaml
ccusage
@ccusage/ccusage-darwin-arm64
@ccusage/ccusage-darwin-x64
@ccusage/ccusage-linux-arm64
@ccusage/ccusage-linux-x64
@ccusage/ccusage-win32-x64
commit: |
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
2 issues found across 1 file
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name=".github/workflows/ci.yaml">
<violation number="1" location=".github/workflows/ci.yaml:47">
P1: Security scanner is selected from the untrusted PR checkout. Pin tirith from a trusted/base ref so fork PRs cannot replace or neuter the gate that is supposed to scan them.</violation>
<violation number="2" location=".github/workflows/ci.yaml:120">
P1: Preflight gate trusts the PR to define the checks it is enforcing. Run the gate implementation from a trusted/base revision or otherwise prevent PR changes from disabling gitleaks/treefmt.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| persist-credentials: false | ||
| - uses: ./.github/actions/setup-nix-cache | ||
| - name: Secret scan and workflow lint | ||
| run: nix build --print-build-logs .#checks.x86_64-linux.gitleaks .#checks.x86_64-linux.treefmt |
There was a problem hiding this comment.
P1: Preflight gate trusts the PR to define the checks it is enforcing. Run the gate implementation from a trusted/base revision or otherwise prevent PR changes from disabling gitleaks/treefmt.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At .github/workflows/ci.yaml, line 120:
<comment>Preflight gate trusts the PR to define the checks it is enforcing. Run the gate implementation from a trusted/base revision or otherwise prevent PR changes from disabling gitleaks/treefmt.</comment>
<file context>
@@ -21,8 +21,106 @@ jobs:
+ persist-credentials: false
+ - uses: ./.github/actions/setup-nix-cache
+ - name: Secret scan and workflow lint
+ run: nix build --print-build-logs .#checks.x86_64-linux.gitleaks .#checks.x86_64-linux.treefmt
+
test:
</file context>
There was a problem hiding this comment.
Same pull_request (not pull_request_target) threat model: the PR already owns the workflow definition, so running gitleaks/treefmt from a base ref can't stop a fork from disabling them, and fork PRs are read-only with no secrets. The suggested change would also break the intentional flake-check reuse, since nix build .#checks… runs against the flake's own source rather than an external worktree.
There was a problem hiding this comment.
Same pull_request threat model as the tirith thread: the PR owns the workflow definition, so running gitleaks/treefmt from a base ref can't stop a fork from disabling them (it would just remove the step / needs: edge), and fork tokens are read-only with no secrets. It would also break the intentional flake-check reuse, since nix build .#checks.*.{gitleaks,treefmt} runs against the flake's own source rather than an external worktree. preflight holds contents: read only, so there is no privilege to protect here.
There was a problem hiding this comment.
The threat-model point is fair, but the parent comment is too broad for this PR: on pull_request, the PR already owns ci.yaml, fork tokens are read-only/no-secrets, and base-pinning would break the intended flake-check reuse. So the base-ref hardening doesn’t apply here.
| persist-credentials: false | ||
| - uses: ./.github/actions/setup-nix-cache | ||
| - name: Install tirith | ||
| run: nix profile install --inputs-from . nixpkgs#tirith |
There was a problem hiding this comment.
P1: Security scanner is selected from the untrusted PR checkout. Pin tirith from a trusted/base ref so fork PRs cannot replace or neuter the gate that is supposed to scan them.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At .github/workflows/ci.yaml, line 47:
<comment>Security scanner is selected from the untrusted PR checkout. Pin tirith from a trusted/base ref so fork PRs cannot replace or neuter the gate that is supposed to scan them.</comment>
<file context>
@@ -21,8 +21,106 @@ jobs:
+ persist-credentials: false
+ - uses: ./.github/actions/setup-nix-cache
+ - name: Install tirith
+ run: nix profile install --inputs-from . nixpkgs#tirith
+ # One pass produces the SARIF report and sets the gate exit code. tirith's
+ # `--ci` exit code is: 0 = no findings, 1 = findings at/above --fail-on,
</file context>
There was a problem hiding this comment.
Premise is right that tirith resolves through the PR's flake, but pinning it from base can't prevent neutering: on pull_request GitHub runs the PR's own ci.yaml, so a fork would just disable the step or needs: edge rather than swap the binary, and forks have a read-only token with no secrets to protect. Real tamper-resistance needs a base-defined workflow (pull_request_target/workflow_run), the secret-exposing model this PR intentionally avoids; it would also break the deliberate dev-shell-consistent install.
There was a problem hiding this comment.
Same reasoning as the CodeRabbit thread above. On pull_request the PR runs its own ci.yaml, so pinning tirith from a base ref can't prevent neutering — a fork would just drop the step or the needs: edge, not swap the binary. Fork tokens are read-only with no secrets, and base-pinning would break the deliberate dev-shell-consistent install (--inputs-from .). As defense-in-depth I isolated the only write scope (security-events: write, SARIF upload) into a job that executes no PR code (commit 8ff5cb8); tirith-scan is now contents: read only.
There was a problem hiding this comment.
The parent comment was wrong here: on pull_request, the PR owns the workflow, so base-pinning tirith wouldn’t stop a fork from dropping the step or the needs: edge. The meaningful hardening is the least-privilege split you added in tirith-sarif-upload, which keeps security-events: write out of the PR-code job.
ccusage performance comparisonPR SHA: This compares the Rust PR release binary against the configured base package on the same CI runner. Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
ccusage performance comparisonPR SHA: This compares the PR package against the configured base package on the same CI runner. Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
Address CodeRabbit's security review: keep the elevated `security-events: write` scope out of any job that executes PR-controlled code. - `tirith-scan` now holds only `contents: read`. It still runs the local action, the PR flake and the installed tirith (it must, to scan the PR), but with no write scope a neutered scan cannot abuse any privilege — the same trust model as the existing `test`/`build` jobs that run the PR's `nix flake check`. - The SARIF is handed off as an artifact and published by a new `tirith-sarif-upload` job that never executes PR code: it only checks out files for path mapping and runs trusted, pinned actions (download-artifact + codeql upload-sarif). This is the only job with `security-events: write`. - The upload runs `always()` so findings reach the Security tab even when the gate blocks the PR, and is skipped for fork PRs (read-only token cannot upload).
ccusage performance comparisonPR SHA: This compares the PR package against the configured base package on the same CI runner. Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
ccusage performance comparisonPR SHA: This compares the Rust PR release binary against the configured base package on the same CI runner. Package runtime diagnosticsCompares the PR package wrapper, the installed native optional dependency binary, and the workspace release binary on the same large fixture. This identifies whether slow package results come from JavaScript wrapper overhead, the published native binary build, or the Rust core itself. Fixtures: Claude
Committed fixture performanceCommitted small fixtures for stable PR-to-PR feedback and explicit Claude/Codex command coverage. Fixtures: Claude
Large real-world-shaped fixture performanceGenerated fixtures shaped from aggregate local log statistics: thousands of JSONL files, many small sessions, and a long tail of larger sessions. No real prompts, paths, or outputs are stored in the fixtures. Fixtures: Claude
Artifact size
Lower medians and smaller artifacts are better. CI runner noise still applies; use same-run ratios as directional PR feedback, not release guarantees. |
2 participants
Summary
Adds a security gate to the CI workflow (tirith) plus a fail-fast preflight, so the expensive build/test work runs only after the cheap, high-signal security checks pass.
What changed
Two new jobs run unconditionally and in parallel at the front of
ci.yaml(not gated onchanges, so they also cover docs-only PRs and fork PRs, where local git hooks never run):tirith-scan— scans the whole checkout with tirith, uploads SARIF to the Security tab, and gates downstream CI.pull_request(notpull_request_target): fork PRs only get a read-only token and cannot leak secrets while their code is scanned. The SARIF upload is therefore skipped for forks (the scan + gate still run); pushes tomainand same-repo PRs populate the Security tab.nixpkgsso CI matches the dev shell.--fail-on highblocks on high/critical findings. tirith's--ciexit code is0= clean,1= at/above threshold,2= findings below threshold; the gate treats0/2as pass and1(or any other code) as a block. The exit code is passed viaenvto avoid template injection.*.snapis excluded: terminal snapshot test fixtures intentionally contain ANSI escapes that trip theansi_escapesrule. The rule stays active for real source files.preflight— runs the existinggitleaksandtreefmtflake checks (treefmt carriesactionlint+zizmorwith the repo's exact ignore config) as an early fail-fast gate.testandbuild-native-packagesnowneedsboth gates, so the heavy pipeline is skipped unless the gates pass. Heavy gate jobs run on blacksmith runners, consistent with the rest of the nix-based CI.Why
code-changed-gated checks and local hooks.Testing
Validated locally against a CI-equivalent tree (
git archive HEAD):nix build .#checks.<sys>.gitleaks .#checks.<sys>.treefmt→ pass (no leaks found,0 changed).actionlintonci.yaml→ clean (only the pre-existingparallel:extension ignored).zizmor --offline --min-severity high --min-confidence high .github/workflows/→ no findings.--fail-on high --exclude '*.snap'→ exit 2 (pass, 0 high findings); a planted high/critical finding → blocks. The 8 remaining warnings (emoji variation selectors, instructional-commentconfig_injection) are below threshold and surface in the Security tab only.Need help on this PR? Tag
/codesmithwith what you need. Autofix is enabled.Summary by cubic
Adds a CI security gate with
tirithand a fast preflight so build/test only run after cheap, high-signal checks pass. Runs on all PRs (including docs-only and forks); SARIF publishing is isolated in a least-privilege job and uploads for same-repo PRs and pushes.tirith-scan: repo-widetirith --fail-on high; treats exit 0/2 as pass and 1 as block; excludes*.snap; installed from pinnednixpkgs; runs read-only and emits a SARIF artifact.tirith-sarif-upload: separate job withsecurity-events: writethat never executes PR code; runs always and publishes SARIF (skipped for forks).preflight: runsgitleaksandtreefmt(actionlint+zizmor) as an early gate.test,build-native-packages, andaction-timelineon the gates;action-timelinealso needstirith-sarif-upload.Written for commit 8ff5cb8. Summary will update on new commits.
Summary by CodeRabbit