Skip to content

Fix quality gate silently passing when historyPath is set (fixes #895 and #827) (closes #861) - #924

Open
d-braun wants to merge 6 commits into
allure-framework:mainfrom
d-braun:fix/895-quality-gate-history-deadlock
Open

d-braun wants to merge 6 commits into
allure-framework:mainfrom
d-braun:fix/895-quality-gate-history-deadlock

Conversation

@d-braun

@d-braun d-braun commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

Context

Fixes #895 and #827 (closes #861).

When historyPath is configured, allure quality-gate exits with code 0 and prints nothing, no matter how many rules the results violate. The only trace it leaves is an allure-report/ directory containing a single test-results.json and no quality-gate.json, exactly as reported in the issue. allure generate and allure run hit the same defect; there it goes mostly unnoticed because the report has already been written by the time the process stalls.

Root cause. AllureLocalHistory opens streams on a FileHandle with autoClose: false and then awaits FileHandle.close() in a finally block without ever destroying them. Such streams are never destroyed on their own, and the handle stays referenced until they emit "close" — so FileHandle.close() waits for a reference that is never released and never settles. As a consequence AllureReport.done() never returns, the event loop drains, and Node exits with code 0 before the quality gate is ever validated.

This is why the issue looks like the config file is ignored: the reporter's config sets historyPath, while the --max-failures run they compared against was made without --config and therefore without history. Rule parsing was never the problem.

Note that running the CLI through yarn allure … hides the defect — Yarn PnP preloads .pnp.cjs, which shifts the timing enough for close() to settle. Under plain node / npx, which is what users run, it deadlocks. Vitest masks it as well. The numbers below come from a clean npm i [email protected] with the built history.js swapped in and run under plain node:

Scenario Before After
quality-gate with historyPath exit 0, no output, no quality-gate.json exit 1, both rule violations reported
4 consecutive runs, also with historyLimit deadlock exit 1, history appended and trimmed
generate with historyPath exit 0, but done() never returned exit 0, history grows as expected

Changes.

  • packages/core/src/history.ts — destroy every stream opened on the history file handle before closing it, in the order the streams were opened: destroying a reader that shares its handle with a still open writer never completes, whereas destroying the writer closes the reader along with it. Closing the handle tolerates EBADF, because a destroyed stream may already have closed the descriptor depending on the runtime.
  • packages/cli/src/commands/qualityGate.ts — safety net: a quality gate must never pass silently. If the process is about to exit before the validation has finished, the command now reports it and exits with code 1 instead of leaving the caller with a successful exit code. Checked against the unfixed deadlock, it turns the silent exit 0 into a loud failure.

Tests.

  • packages/core/test/history.streams.test.ts (new) — asserts that every stream opened on the history file handle is closed, for the new-file, existing-file and read paths. All three fail without the fix.
  • packages/cli/test/commands/qualityGate.test.ts — two tests for the new safety net: it fires on an interrupted run and stays silent on a completed one.

yarn format:check, oxlint and the @allurereport/core (528) and allure (217) suites pass.

Checklist

@d-braun
d-braun force-pushed the fix/895-quality-gate-history-deadlock branch 3 times, most recently from a372d01 to c5b08fb Compare September 7, 2026 22:23
@d-braun d-braun changed the title Fix quality gate silently passing when historyPath is set (fixes #895) Fix quality gate silently passing when historyPath is set (fixes #895 and #827) (closes #861) Sep 8, 2026
Comment thread packages/core/src/history.ts Outdated
Comment thread packages/cli/test/commands/qualityGate.test.ts Outdated
@formaceft-93

Copy link
Copy Markdown
Contributor

@d-braun Thank you for your contribution, I left two small comments, and could you please also make a rebase on current main because #958 also touched local history handling.

@d-braun
d-braun force-pushed the fix/895-quality-gate-history-deadlock branch from 76bdde4 to 370f193 Compare September 14, 2026 10:53
@d-braun

d-braun commented Sep 14, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @formaceft-93!
Both addressed and rebased onto current main, tests green.

@epszaw

epszaw commented Sep 14, 2026

Copy link
Copy Markdown
Member

Hello, @d-braun! Some tests windows failed. Please, resolve the problems before we continue

@d-braun
d-braun force-pushed the fix/895-quality-gate-history-deadlock branch from cfdf05c to 118787b Compare September 15, 2026 10:46
@d-braun

d-braun commented Sep 15, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @epszaw! Fixed. :)

Nothing in the suite could reproduce the silent exit that `historyPath`
caused (allure-framework#827, allure-framework#895): Yarn's PnP runtime replaces `fs/promises.open` with a
ZipFS-backed handle whose `close()` doesn't wait for stream references,
which hides the deadlock from every in-process test and from anything
started through `yarn node`.

- add a core test that copies the built packages into a real `node_modules`
  tree and runs the history code in a plain `node` process; without the fix
  it reports "Detected unsettled top-level await" and exits with code 13
- add an end-to-end `allure serve` test with `historyPath` configured,
  adapted from allure-framework#861; it covers the symptom of allure-framework#827 in environments with
  real `node_modules` and stays a smoke test under PnP
- hoist the CLI build in `run.integration.test.ts` to file level, so it
  can't wipe `dist` while another suite drives the built CLI
- create the write stream after the `limit === 0` early returns
…ssion test runs on Windows

`URL#pathname` keeps the leading slash and the percent-encoding on Windows
(`/C:/Users/RUNNER%7E1/...`), which `path.resolve` in `appendHistory` turned
into `C:\C:\...` and made the `mkdir` fail with ENOENT on the windows-latest job.
@formaceft-93
formaceft-93 force-pushed the fix/895-quality-gate-history-deadlock branch from 9fbdf3b to 904c695 Compare September 16, 2026 08:47
@formaceft-93

Copy link
Copy Markdown
Contributor

@d-braun Sorry to bother you again, but can you resolve conflicts one more time, please :)

# Conflicts:
#	packages/cli/src/commands/qualityGate.ts
#	packages/core/src/history.ts

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allure Quality Gate doesn't work with the config file anymore

3 participants