Pull request comments

@e2e-dev/github posts test results as a pull request comment, including failures, source links, and links to workflow artifacts. Reruns update the same comment, and a --last-failed rerun updates it as one run. The reporter also writes the results to the job summary.

Setup

npm install --save-dev @e2e-dev/github

pnpm add -D @e2e-dev/github

bun add -d @e2e-dev/github

e2e.config.ts

import type { E2EConfig } from 'e2e';
import { web } from '@e2e-dev/web';
import { github } from '@e2e-dev/github';

export default {
  targets: [{ engine: web(), app: { url: 'http://localhost:3000' } }],
  reporters: ['list', github()],
} satisfies E2EConfig;

Give the workflow permission to write pull request comments and pass GITHUB_TOKEN to the test step:

.github/workflows/e2e.yml

permissions:
  contents: read
  pull-requests: write

jobs:
  e2e:
    steps:
      # ...checkout, install, browsers
      - name: Run e2e
        run: npx e2e run --reporter list,junit
        env:
          GITHUB_TOKEN: ${{ github.token }}
      - name: Upload artifacts
        if: ${{ !cancelled() }}
        uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
        with:
          name: e2e-artifacts
          path: .e2e/artifacts

Upload artifacts even when a retry passes. Each failure’s evidence links open the run’s Artifacts section; inside the download, every test has a directory named after it and the agent that ran it (web/tests_billing.e2e.ts__upgrades/default/attempt-0/), so keep path: .e2e/artifacts and the download lines up with the report. A --last-failed rerun keeps the first run’s directories and puts its own under rerun-1/, so a flaky test’s failed attempt still links to its own files. A project below the checkout root, such as packages/e2e-tests, gets source links under that path. The checkout root is the nearest directory holding .git inside GITHUB_WORKSPACE, so actions/checkout with a path input works too.

The comment

### 🔴 e2e: 1 failed, 1 flaky, 3 passed, 1 skipped
9 agent steps · 4 replayed from cache · 31 model calls · 128k tokens (62% cached) · $0.31

**🔴 billing › upgrades to Pro**  
[tests/billing.e2e.ts:12](https://github.com/acme/shop/blob/1a2b3c/tests/billing.e2e.ts#L12)

**ASSERTION_FAILED** at step 4 of 6: `agent.act` "Pick the Pro plan and pay", after 38.0s and 12 model calls

> expected heading "Your cart" to be visible
> The Pay button opened a page that still shows the Free plan.

- Failed the same way on both attempts: **ASSERTION_FAILED** at step 4.
- Turn 9: `tap({"target":"n41"})` → Screen changed: /checkout/plan. New: #n50 heading "Free plan"
- Screen: `/checkout/plan`

Evidence: [screenshot](https://github.com/acme/shop/actions/runs/123#artifacts), [video](https://github.com/acme/shop/actions/runs/123#artifacts), [trace](https://github.com/acme/shop/actions/runs/123#artifacts)

<details>
<summary>⚠️ 1 flaky test passed on a retry</summary>

**⚠️ invite a member by email**  
[tests/team.e2e.ts:21](https://github.com/acme/shop/blob/1a2b3c/tests/team.e2e.ts#L21)

**STEP_TIMEOUT** at step 3 of 3: `agent.waitFor` "the invitation email", after 30.0s

> the invitation email took too long

Evidence: [screenshot](https://github.com/acme/shop/actions/runs/123#artifacts), [trace](https://github.com/acme/shop/actions/runs/123#artifacts)
</details>

<details>
<summary>All 6 tests in 5 files</summary>

| | Test | Agent | Time |
| --- | --- | --- | --- |
| 🔴 | **tests/billing.e2e.ts** · 1 failed | 3 steps · 19 calls | 41.2s |
| 🔴 | billing › upgrades to Pro | 3 steps · 19 calls | 41.2s |
| ⚠️ | **tests/team.e2e.ts** · 1 flaky | 1 step | 6.4s |
| ⚠️ | invite a member by email (1 failed attempt first) | 1 step | 6.4s |
...
</details>

<sub>e2e 0.13.0 · 2m 15s · web · [run artifacts](https://github.com/acme/shop/actions/runs/123#artifacts)</sub>

The comment shows test counts and agent usage, followed by each failure’s error, source line, and evidence links. Flaky tests and the full test list are collapsed; the list is one table with a row per file above its tests, and its agent column appears only when the run used the agent. Run-level errors appear above the test failures. Large reports are shortened to fit GitHub’s comment limit. Text from the app under test that GitHub would turn into a mention, an issue link, or a URL is shown as code, so a screen reading @someone or https://... notifies no one and links nowhere. The built-in markdown reporter writes the same summary to .e2e/summary.md.

Rerunning the failed tests

A step that gives failed tests a second chance keeps one comment for both passes:

      - name: Run e2e
        run: npx e2e run || npx e2e run --last-failed
        env:
          GITHUB_TOKEN: ${{ github.token }}

The comment covers the whole suite, not only the tests the rerun ran again. A test that failed and then passed shows as flaky, a test that failed twice stays failed, one that was interrupted and then passed shows as passed, and the rest keep their first-pass results. The comment stays red while the rerun left out something it owed: a failed test, one --max-failures stopped before it ran, or a failed beforeAll or afterAll whose tests another filter kept out of the rerun. It shows ⏹️ when the rerun left out a test the first pass never finished. The rerun’s own exit code counts only what it ran. A failed attempt’s evidence links to the first run’s files, which the rerun keeps.

Matrix jobs

The reporter identifies a comment by project, workflow, and job. Matrix jobs share those identifiers, so give each matrix entry a distinct key:

reporters: ['list', github({ key: process.env.MATRIX_BROWSER })],

Set MATRIX_BROWSER: ${{ matrix.browser }} in the step’s env. The key also appears in the headline, such as e2e chromium: 77 passed. Keep it stable across runs so reruns update the existing comment.

When nothing is posted

The reporter never changes the run’s status or exit code. When it cannot post, it says why in one row under the summary:

       Report  .e2e/report.json

       GitHub  not posted: set GITHUB_TOKEN in the step's env (GITHUB_TOKEN: ${{ github.token }}); written to the job summary

Situation	What happens
Not on GitHub Actions	Nothing is written; the row says so.
A `push`, `schedule`, or `workflow_dispatch` run	The job summary is written; there is no pull request to comment on.
No `GITHUB_TOKEN` in the step’s `env`	The job summary is written; the row shows the line to add. `GH_TOKEN` is read too.
A `pull_request` event from a fork	Its token is read-only, so posting fails and the reporter explains why.
An `issue_comment` event	On a pull request’s thread, the comment is posted without source links, since the checked-out commit is the default branch.
Missing `pull-requests: write`	The same failure, with the permission named.

The token is read when the run finishes and is excluded from the report. GitHub Enterprise Server uses GITHUB_SERVER_URL and GITHUB_API_URL.

Get started

Web

Mobile

API

Models and agents

Sign-in and security

Debug and explore

Automate

Integrations

Migrate

Extend

Pull request comments

Setup

The comment

Rerunning the failed tests

Matrix jobs

When nothing is posted

​Setup

​The comment

​Rerunning the failed tests

​Matrix jobs

​When nothing is posted

Setup

The comment

Rerunning the failed tests

Matrix jobs

When nothing is posted