Improve realtime watch reliability and add scoped rerun - #816
Merged
Merged
Conversation
Allure perf metricsGenerated at:
Artifacts: allure-perf-metrics |
todti
force-pushed
the
realtime-watch-affected-tests
branch
from
July 23, 2026 16:22
4ffb746 to
1fcad62
Compare
todti
requested review from
epszaw and
formaceft-93
and removed request for
formaceft-93
July 29, 2026 09:29
todti
force-pushed
the
realtime-watch-affected-tests
branch
from
July 29, 2026 17:10
6401b6d to
01cc99e
Compare
formaceft-93
requested changes
Jul 30, 2026
epszaw
requested changes
Jul 30, 2026
formaceft-93
self-requested a review
July 30, 2026 12:02
formaceft-93
approved these changes
Jul 30, 2026
todti
force-pushed
the
realtime-watch-affected-tests
branch
2 times, most recently
from
August 4, 2026 11:52
0e00f74 to
411010a
Compare
allure run --watch: rerun only the changed spec file instead of the whole suite; kill the full test process tree on Ctrl+C with SIGTERM/SIGKILL escalation instead of hanging or leaving orphans; keep the watch session alive when a run crashes (e.g. spawn errors) instead of taking down the whole process; avoid EMFILE by skipping dotfiles/node_modules/build output when watching source files for changes. allure watch: discover allure-results directories dynamically instead of only at startup, so directories that don't exist yet get picked up once they appear; make shutdown interruptible mid-scan instead of waiting for a full directory walk to finish; add --new-only to skip the existing result backlog and react only to results written after the watch started. plugin-progress: fix runaway terminal output from overlapping renders and other plugins' log lines corrupting the in-place progress line. plugin-testops: stop re-uploading the same global attachments/errors on every realtime update cycle; announce newly found test results before uploading them.
Add tests for directory-watcher's default node_modules/dotfile ignoring and AbortSignal-based scan interruption, plugin-progress's render mutex/throttling/foreign-write isolation, and plugin-testops's global attachment/error upload deduplication. Shorten the multi-line comments added earlier in the session down to single lines per the repo's comment style.
yoctocolors wraps the bolded count in the middle of the "Found N new test results" message, splitting it with escape codes. That's invisible locally (no TTY, no colors), but CI runners enable color output, so the plain stringContaining assertion never matched there.
isIgnoredByDefault was splitting on node:path's sep, which is a backslash on Windows. chokidar normalizes paths to forward slashes regardless of platform, so the split never found a segment boundary there and node_modules/dot-directories went unfiltered. Split on either slash instead.
…watch
Classify TestOps upload failures into transient (5xx/429/timeout),
recoverable ("launch is closed"), or terminal (auth/validation/payload/
conflict) kinds, and retry only the retryable ones with exponential
backoff. Wired into all three upload call sites (test results, global
attachments, global errors) — previously a failed request just logged
and gave up with no retry at all.
Add a leaky-bucket pacer with independent per-window budgets for
requests, files, and bytes sent to TestOps, enabled by default with
sane defaults (20 req/s, 1000 files/s, 1 GiB/s) since the server-side
limits it approximates are real regardless of whether anyone configures
for them. Disable via a new uploadRateLimit: false plugin option, or
tune it via an explicit uploadRateLimit config.
Replace the ad-hoc SIGINT handler in run --watch, and add SIGTERM
handling (which it didn't have), with a small shared signals helper
(notifySignals/waitForAbort/boundedTerminationSignal) — conventional
exit codes (130/143), a graceful-shutdown deadline, and the same
kill-tree-with-escalation behavior as before. The `open` command's
temp-report cleanup now handles SIGTERM too.
allure run now watches source files and reruns just the changed spec by default when it's not running in CI (checked via the same CI-provider detection plugin-testops already uses, plus a CI=true fallback) — running it locally feels like starting a watch session, matching how the command is actually used day to day. In CI it still runs once and exits with a pass/fail code, unchanged. --watch / --no-watch still override the default explicitly either way.
Clearer than relying on clipanion's implicit --no-watch negation. --watch and --once together is a usage error.
…only dropping newly discovered directories terminationOf only listened for 'exit', so a failed spawn (e.g. bad command) emitted an unhandled 'error' event and crashed the whole watch session instead of being reported as a failed run. allure watch --new-only was passing ignoreInitial to every dynamically discovered allure-results directory, including ones that appear after watch has already started — dropping their backlog even though it's new content, not a pre-existing one to skip.
…tly instead of via a real spawn On Windows, runProcess spawns through a shell by default, so a bad command exits normally instead of emitting 'error' the way it does on POSIX with a direct spawn — making the spawn-based regression test platform-dependent. Emit the 'error' event on a fake process instead so the test is consistent across platforms.
Defaulting to watch mode outside CI would silently change behavior for every existing "allure run" workflow that isn't run in CI, since the command is what actually executes the test suite. Back to plain --watch, no CI auto-detection, no --once.
…ed directory's .gitignore Blanket-ignoring anything starting with "." could silently drop test data users keep in a dot-prefixed directory. Now only node_modules/dist/build/ out/coverage/.git are ignored unconditionally, plus whatever plain names the watched directory's own .gitignore lists. Also flips allure watch's --new-only to on by default (pass --no-new-only to ingest the existing backlog instead), since starting a watch session and getting flooded with every already-existing result is rarely what's wanted.
remove unnecessary logs from testops plugin (change log level) in realtime simplify testops uploader logic fix some progress plugin issues
epszaw
force-pushed
the
realtime-watch-affected-tests
branch
from
August 19, 2026 11:28
8d1b3c1 to
1cc93ac
Compare
epszaw
requested review from
epszaw and
formaceft-93
and removed request for
epszaw
August 19, 2026 11:29
Contributor
|
allure run --watch no longer works - it is okay? |
formaceft-93
requested changes
Aug 19, 2026
Member
It's ok because the previous implementation was unstable, and I decided to remove it due to the instability with different test runners and frameworks |
formaceft-93
approved these changes
Aug 19, 2026
…ast one allGlobalErrors() is append-only, but the code only ever sent allResults.at(-1): the same last error got re-uploaded every realtime cycle, and if multiple new errors landed between updates, all but the last were silently dropped. Restore the offset-based slice so every not-yet-uploaded error is sent.
TestError has no stable id, but the store only ever appends to allGlobalErrors() and never replaces existing entries, so the same logical error keeps the same object reference across calls. A Set<TestError> keyed by that reference dedupes the same way global attachments already do (Set<string> by attachment.id), without depending on array position staying meaningful.
Covers the exact scenario the at(-1)-based bug hit: several new errors landing between updates, another update with nothing new, then one more. Verified this fails against the old at(-1) logic (drops one of two errors added in the same cycle) and passes with the current Set-based dedup.
epszaw
approved these changes
Aug 20, 2026
2 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
allure run --watch(new): when a spec file changes, only that file is rerun instead of the entire suite — makes watch mode practical for large e2e suites where rerunning everything on every save was too slow.allure run --watch: Ctrl+C now reliably kills the full test process tree (SIGTERM, then SIGKILL if the process doesn't exit in time), instead of sometimes hanging or leaving orphaned processes behind.allure run --watch: if a single rerun fails unexpectedly (e.g. a spawn error), the watch session now logs it and keeps watching, instead of the whole process crashing.allure run --watch: source-file watching now skipsnode_modules, dotfiles/dot-directories, and build output by default, instead of exhausting the OS file-descriptor limit on large repos.allure watch:allure-resultsdirectories that don't exist yet at startup (e.g. a package that hasn't run its tests yet) are now picked up automatically once they appear, instead of only ever watching the directories found at launch.allure watch: Ctrl+C now interrupts an in-progress directory scan immediately instead of waiting for the whole scan to finish first.--new-onlyflag forallure watch: skips the existing backlog of results on disk at startup and only reacts to results written after the watch session started.failed/broken/passed/totalline no longer duplicates or corrupts itself when re-rendered rapidly or when other plugins log output at the same time.