Skip to content

chore: extend report generation performance metrics - #953

Open
formaceft-93 wants to merge 1 commit into
mainfrom
improve-metrics
Open

chore: extend report generation performance metrics#953
formaceft-93 wants to merge 1 commit into
mainfrom
improve-metrics

Conversation

@formaceft-93

@formaceft-93 formaceft-93 commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR restores CI-visible perf data through the generated Allure Awesome report itself: CI now runs a lightweight perf probe, copies the produced *-perf.json into report inputs, and the final report shows the values in the Metrics section.

Changes

  • Extended ALLURE_PERF_METRICS with more detailed generation spans and aggregate timings.
  • Added core read metrics for xcresult check, directory read, path resolution, and total reader work.
  • Added store aggregate metrics for conversion, default labels, environment mapping, retry/history/resolution/indexing, and attachment metadata.
  • Added Awesome generation metrics for data loading, stats/charts, conversion, categories, timeline, testcase writes, tree/nav/search, environments, attachments, globals, static files, and plugin summary creation.
  • Added neutral workload counters for input files, generated files, attachment count/bytes, and converted test results.
  • Added .github/allurerc.perf.mjs for CI probe generation.
  • Documented the metric keys and PR baseline interpretation in docs/performance.md.

CI / History

On PRs, metrics are visible in the generated Awesome report Metrics page, not in a PR comment.

PR reports should show deltas when report history is available through the existing allure-framework/allure-action@v0 publishing/history flow. On pushes to main, the same report job runs, so main metrics should become part of the published report history that later PR reports can compare against.

This PR does not add a separate immutable perf baseline registry. The fixed benchmark baseline for optimization PRs should still be captured separately after PR1, using the agreed datasets and run matrix.

Metrics

Timing metrics are grouped into:

  • Allure lifecycle: total generation/restore/publish/summary timings.
  • Generate: result discovery, filesystem read phases, reader work, plugin completion.
  • Store: test-result ingestion, retry/history/resolution/indexing, attachment metadata.
  • Awesome: server-side report data/file generation.
  • Workload: neutral counters that explain input/output scale.

Checklist

@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

Allure Report Summary

Name Duration Stats New Flaky Retry Report
Allure 3 Report (plugin-allure2) 1h 11m Passed tests 11391   Skipped tests 23   Unknown tests 12 159 0 34 View
Allure 3 Report 1h 11m Passed tests 11391   Skipped tests 23   Unknown tests 12 159 0 34 View
Allure 3 Report (plugin-classic) 1h 11m Passed tests 11391   Skipped tests 23   Unknown tests 12 159 0 34 View
My Dashboard 1h 11m Passed tests 11391   Skipped tests 23   Unknown tests 12 159 0 34 View
Allure Report 0s 0 0 0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant