fix(e2e): stabilize audit-log and clipboard scenarios - #117
Merged
Conversation
…ario The "Audit page loads for selected experiment" scenario intermittently failed with a 30s TimeoutException waiting for audit-entry rows. Root cause: the Then step branched on IsConfiguredAsync(), which relies on an instantaneous IsVisibleAsync() probe of the not-configured message. On the InteractiveServer circuit the empty-state can render slightly after the probe runs, so the step misclassifies the page as "configured" and then hard-waits 30s for audit entries that never appear. Fix (test-only): replace the racy branch with a single combined wait — AuditEntries.Or(NotConfiguredMessage).First — that settles on whichever valid end state appears first. No assertion is weakened; the page must still reach one of the two legitimate states. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Snapshot WarningsEnsure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice. Scanned FilesNone |
Contributor
Code Coverage |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #117 +/- ##
========================================
Coverage 80.81% 80.81%
========================================
Files 264 264
Lines 10508 10508
Branches 1836 1472 -364
========================================
Hits 8492 8492
Misses 2016 2016
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The
CIworkflowe2e-testsjob has been flaky onmain. The originally-reported flaky scenarios were Audit log displays entries and Copy YAML to clipboard; on the latest run (29892468989, head 503d238) those two passed but a related audit scenario failed instead:Scenario:
Audit page loads for selected experimentError:
System.TimeoutException : Timeout 30000ms exceeded— waiting forLocator(".audit-entry, tr.audit-row, [data-audit-entry]").Firstto be visible(
GovernanceAuditPage.AssertAuditEntriesVisibleAsync()line 162, viaThenIShouldSeeAuditTrailEntriesOrANotConfiguredMessage()line 70).Root cause (test race, not a product bug)
The
Then I should see audit trail entries or a not configured messagestep is meant to accept either audit entries or the not-configured empty-state. It branched onIsConfiguredAsync(), which uses an instantaneousIsVisibleAsync()probe of the not-configured message. On the Blazor InteractiveServer circuit the empty-state can render just after that probe, so the step misclassifies the page as "configured" and then hard-waits 30s for audit entries that never appear.Fix (test-only)
Replace the racy branch with a single combined wait —
AuditEntries.Or(NotConfiguredMessage).Firstwith a 30s budget — that settles on whichever valid end state appears first. No assertion is weakened: the page must still reach one of the two legitimate end states.Files:
tests/ExperimentFramework.E2E.Tests/PageObjects/GovernanceAuditPage.cs— newAssertAuditEntriesOrNotConfiguredAsync()helpertests/ExperimentFramework.E2E.Tests/StepDefinitions/Governance/GovernanceAuditStepDefinitions.cs— step now calls the combined waitBuilds clean (
dotnet build ... -c Release -f net10.0, 0 warnings / 0 errors).🤖 Generated with Claude Code