Repository navigation
Keep ASV benchmark regressions visible in bot comments - #1872
Draft
anishpentyala wants to merge 3 commits into
Draft
anishpentyala wants to merge 3 commits into
anishpentyala wants to merge 3 commits into
Conversation
AI-generated and tested contribution. Exercises the actual inline workflow script with mocked artifact and GitHub calls, without executing fork code.
AI-generated and tested. Reorder existing split tables without hiding regressions, improvements or unfamiliar output. Only unchanged benchmarks are collapsed. No new privileged checkout or execution of fork code.
ASV BenchmarkingBenchmark Comparison ResultsBenchmarks that have stayed the same:
Benchmarks that have got worse:
|
Member
|
Hi @anishpentyala thanks very much for your contribution! FYI: I've added the "run-benchmark" label to this to trigger the ASV benchmark runs and see how their results show up. See it above in the "ASV Benchmarking" cell. Luckily, we have some of those benchmarks that got worse, but they are still shown in the bottom-most part of that cell. |
Author
|
@erogluorhan Thanks for checking the live output. That comment still uses the existing renderer: the comment workflow run ran from commit The workflow's The draft's offline tests exercise its actual inline JavaScript with mocked artifact reads and GitHub calls, including real ASV-formatted synthetic comparison output. Those checks support the proposed ordering, but they are not live validation of the patch against this benchmark run. The PR remains draft, and the personal-responsibility checkbox remains unchecked. AI disclosure: this source check and reply were prepared by the AI assistant operating for @anishpentyala. No personal human review or readiness attestation is claimed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1739
Overview
Keep ASV regressions visible at the top of the benchmark bot comment, followed by improvements. Only unchanged benchmarks go in a collapsed details section. Non-comparable and unfamiliar output stays visible; the table content is retained rather than interpreted or recalculated.
The change stays inside the existing trusted inline
github-script. It does not check out or execute PR/fork code in the write-token workflow, and it does not change benchmark thresholds, execution or artifact handling.Tests
Compare.print_tableoutput, including a 14x slowdown and introduced failure, was passed through the renderer. This is synthetic comparison data, not a UXarray performance claim.git diff --checkpass.Run the offline tests with Node.js available:
The full UXarray scientific test suite and live GitHub Actions execution were not run locally. No UXarray runtime code changes.
AI Disclosure
AI assistance generated, inspected and tested this patch and its tests. This is not a claim that Anish personally wrote or reviewed the changes. The personal-responsibility checkbox is left unchecked rather than making an attestation on his behalf.