| title | Pull request comments |
|---|---|
| description | The @e2e-dev/github reporter posts one comment per run from GitHub Actions and keeps it current. |
@e2e-dev/github posts test results as a pull request comment, including
failures, source links, and links to workflow artifacts. Reruns update the
same comment, and a --last-failed rerun updates it as one run. The reporter
also writes the results to the job summary.
pnpm add -D @e2e-dev/githubbun add -d @e2e-dev/githubimport type { E2EConfig } from 'e2e';
import { web } from '@e2e-dev/web';
import { github } from '@e2e-dev/github';
export default {
targets: [{ engine: web(), app: { url: 'http://localhost:3000' } }],
reporters: ['list', github()],
} satisfies E2EConfig;Give the workflow permission to write pull request comments and pass
GITHUB_TOKEN to the test step:
permissions:
contents: read
pull-requests: write
jobs:
e2e:
steps:
# ...checkout, install, browsers
- name: Run e2e
run: npx e2e run --reporter list,junit
env:
GITHUB_TOKEN: ${{ github.token }}
- name: Upload artifacts
if: ${{ !cancelled() }}
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: e2e-artifacts
path: .e2e/artifactsUpload artifacts even when a retry passes. Each failure's evidence links open
the run's Artifacts section; inside the download, every test has a directory
named after it and the agent that ran it
(web/tests_billing.e2e.ts__upgrades/default/attempt-0/), so keep
path: .e2e/artifacts and the download lines up with the report. A project
below the checkout root, such as packages/e2e-tests, gets source links under
that path. The checkout root is the nearest directory holding .git inside
GITHUB_WORKSPACE, so actions/checkout with a path input works too.
### 🔴 e2e: 1 failed, 1 flaky, 3 passed, 1 skipped
9 agent steps · 4 replayed from cache · 31 model calls · 128k tokens (62% cached) · $0.31
**🔴 billing › upgrades to Pro**
[tests/billing.e2e.ts:12](https://github.com/acme/shop/blob/1a2b3c/tests/billing.e2e.ts#L12)
**ASSERTION_FAILED** at step 4 of 6: `agent.act` "Pick the Pro plan and pay", after 38.0s and 12 model calls
> expected heading "Your cart" to be visible
> The Pay button opened a page that still shows the Free plan.
- Failed the same way on both attempts: **ASSERTION_FAILED** at step 4.
- Turn 9: `tap({"target":"n41"})` → Screen changed: /checkout/plan. New: #n50 heading "Free plan"
- Screen: `/checkout/plan`
Evidence: [screenshot](https://github.com/acme/shop/actions/runs/123#artifacts), [video](https://github.com/acme/shop/actions/runs/123#artifacts), [trace](https://github.com/acme/shop/actions/runs/123#artifacts)
<details>
<summary>⚠️ 1 flaky test passed on a retry</summary>
**⚠️ invite a member by email**
[tests/team.e2e.ts:21](https://github.com/acme/shop/blob/1a2b3c/tests/team.e2e.ts#L21)
**STEP_TIMEOUT** at step 3 of 3: `agent.waitFor` "the invitation email", after 30.0s
> the invitation email took too long
Evidence: [screenshot](https://github.com/acme/shop/actions/runs/123#artifacts), [trace](https://github.com/acme/shop/actions/runs/123#artifacts)
</details>
<details>
<summary>All 6 tests in 5 files</summary>
| | Test | Agent | Time |
| --- | --- | --- | --- |
| 🔴 | **tests/billing.e2e.ts** · 1 failed | 3 steps · 19 calls | 41.2s |
| 🔴 | billing › upgrades to Pro | 3 steps · 19 calls | 41.2s |
| ⚠️ | **tests/team.e2e.ts** · 1 flaky | 1 step | 6.4s |
| ⚠️ | invite a member by email (1 failed attempt first) | 1 step | 6.4s |
...
</details>
<sub>e2e 0.13.0 · 2m 15s · web · [run artifacts](https://github.com/acme/shop/actions/runs/123#artifacts)</sub>The comment shows test counts and agent usage, followed by each failure's
error, source line, and evidence links. Flaky tests and the full test list
are collapsed; the list is one table with a row per file above its tests,
and its agent column appears only when the run used the agent. Run-level
errors appear above the test failures. Large
reports are shortened to fit GitHub's comment limit. Text from the app under
test that GitHub would turn into a mention, an issue link, or a URL is shown
as code, so a screen reading @someone or https://... notifies no one and
links nowhere.
The built-in markdown reporter writes the same summary to
.e2e/summary.md.
A step that gives failed tests a second chance keeps one comment for both passes:
- name: Run e2e
run: npx e2e run || npx e2e run --last-failed
env:
GITHUB_TOKEN: ${{ github.token }}The comment covers the whole suite, not only the tests the rerun ran again. A test that failed and then passed shows as flaky, a test that failed twice stays failed, one that was interrupted and then passed shows as passed, and the rest keep their first-pass results. The comment matches the job's status: it stays red when another filter kept a failed test out of the rerun, and shows ⏹️ when it kept out a test the first pass never finished.
The reporter identifies a comment by project, workflow, and job. Matrix
jobs share those identifiers, so give each matrix entry a distinct key:
reporters: ['list', github({ key: process.env.MATRIX_BROWSER })],Set MATRIX_BROWSER: ${{ matrix.browser }} in the step's env. The key also
appears in the headline, such as e2e chromium: 77 passed. Keep it stable
across runs so reruns update the existing comment.
The reporter never changes the run's status or exit code. When it cannot post, it says why in one row under the summary:
Report .e2e/report.json
GitHub not posted: set GITHUB_TOKEN in the step's env (GITHUB_TOKEN: ${{ github.token }}); written to the job summary
| Situation | What happens |
|---|---|
| Not on GitHub Actions | Nothing is written; the row says so. |
A push, schedule, or workflow_dispatch run |
The job summary is written; there is no pull request to comment on. |
No GITHUB_TOKEN in the step's env |
The job summary is written; the row shows the line to add. GH_TOKEN is read too. |
A pull_request event from a fork |
Its token is read-only, so posting fails and the reporter explains why. |
An issue_comment event |
On a pull request's thread, the comment is posted without source links, since the checked-out commit is the default branch. |
Missing pull-requests: write |
The same failure, with the permission named. |
The token is read when the run finishes and is excluded from the report.
GitHub Enterprise Server uses GITHUB_SERVER_URL and GITHUB_API_URL.