Skip to content

chore: extend report generation performance metrics - #953

Open
formaceft-93 wants to merge 1 commit into
mainfrom
improve-metrics
Open

formaceft-93 wants to merge 1 commit into
mainfrom
improve-metrics

Conversation

@formaceft-93

@formaceft-93 formaceft-93 commented Sep 9, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This PR restores CI-visible perf data through the generated Allure Awesome report itself: CI now runs a lightweight perf probe, copies the produced *-perf.json into report inputs, and the final report shows the values in the Metrics section.

Changes

  • Extended ALLURE_PERF_METRICS with more detailed generation spans and aggregate timings.
  • Added core read metrics for xcresult check, directory read, path resolution, and total reader work.
  • Added store aggregate metrics for conversion, default labels, environment mapping, retry/history/resolution/indexing, and attachment metadata.
  • Added Awesome generation metrics for data loading, stats/charts, conversion, categories, timeline, testcase writes, tree/nav/search, environments, attachments, globals, static files, and plugin summary creation.
  • Added neutral workload counters for input files, generated files, attachment count/bytes, and converted test results.
  • Added .github/allurerc.perf.mjs for CI probe generation.
  • Documented the metric keys and PR baseline interpretation in docs/performance.md.

CI / History

On PRs, metrics are visible in the generated Awesome report Metrics page, not in a PR comment.

PR reports should show deltas when report history is available through the existing allure-framework/allure-action@v0 publishing/history flow. On pushes to main, the same report job runs, so main metrics should become part of the published report history that later PR reports can compare against.

This PR does not add a separate immutable perf baseline registry. The fixed benchmark baseline for optimization PRs should still be captured separately after PR1, using the agreed datasets and run matrix.

Metrics

Timing metrics are grouped into:

  • Allure lifecycle: total generation/restore/publish/summary timings.
  • Generate: result discovery, filesystem read phases, reader work, plugin completion.
  • Store: test-result ingestion, retry/history/resolution/indexing, attachment metadata.
  • Awesome: server-side report data/file generation.
  • Workload: neutral counters that explain input/output scale.

Checklist

@github-actions

github-actions Bot commented Sep 9, 2026 •

Copy link
Copy Markdown

Allure Report Summary

     Scope Duration Stats New Flaky Retry
     All tests 1h 11m 45s Passed tests⁠ 12498
Skipped tests⁠ 23
Unknown tests⁠ 12
339 0 6
     macos-latest 20m 13s Passed tests⁠ 4228
Skipped tests⁠ 6
Unknown tests⁠ 4
113 0 2
     ubuntu-latest 24m 33s Passed tests⁠ 4136
Skipped tests⁠ 6
Unknown tests⁠ 4
113 0 2
     windows-latest 26m 58s Passed tests⁠ 4134
Skipped tests⁠ 11
Unknown tests⁠ 4
113 0 2

Reports: Allure2, Awesome, Classic, Dashboard

Artifacts used (3)
Name Path
allure-results-macos-latest.zip allure-results-macos-latest.zip
allure-results-ubuntu-latest.zip allure-results-ubuntu-latest.zip
allure-results-windows-latest.zip allure-results-windows-latest.zip

@formaceft-93
formaceft-93 requested a review from epszaw September 10, 2026 09:18
@formaceft-93
formaceft-93 force-pushed the improve-metrics branch 2 times, most recently from e96bf9a to e26d713 Compare September 21, 2026 12:30

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant