Repository navigation
Conversation
…rver awaits TestHubModule branched on the shared test instance's current state instead of the state its observer was dispatched for. Observers of an event run one after another. When the test body calls browser.reloadSession(), AutomateModule's TEST/POST observer awaits the session-name PUT for the new session before TestHub's observer runs. A screenshot or console log from the user's own afterTest, tracked meanwhile, moves the instance's current state to LOG, so TEST/POST took the LOG branch and its TestRunFinished was neither deferred nor sent. The test stayed in progress until the ~60-min reap and the build was reported as timed out. TestFramework.runHooks now puts the dispatched testFrameworkState/hookState on the observer args. TestHubModule (onAllTestEvents, sendTestFrameworkEvent, sendTestSessionEvent, sendLogCreatedEvent) acts on it, falling back to the instance's state for direct callers. Refs: SDK-8004 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the
|
|
✅ Good to go
Change map (generated deterministically from the diff)graph LR
subgraph nwdio_service["wdio-service"]
npackages_browserstack_service_tests_cli_modules_testHubModule_dispatchedState_test_ts["⚠ testHubModule.dispatchedState.test.ts<br/>~124 lines"]
npackages_browserstack_service_src_cli_modules_testHubModule_ts["testHubModule.ts<br/>~33 lines"]
n_changeset_pr_310_md["pr-310.md<br/>~5 lines"]
npackages_browserstack_service_src_cli_frameworks_testFramework_ts["testFramework.ts<br/>~5 lines"]
end
↻ This verdict comment is the review anchor — it's updated in place on each run (the gate posts its status separately). — SDK PR Review Agent |
|
🟢 SDK PR Review gate is green — the SDK PR Review Agent has run on the current head commit (verdict: This gate confirms a review ran on the latest commit. The verdict itself is advisory — read the findings and use your judgement; it does not block merge. A native GitHub reviewer approval is still separately required by branch protection before this PR can merge. |
1 similar comment
|
🟢 SDK PR Review gate is green — the SDK PR Review Agent has run on the current head commit (verdict: This gate confirms a review ran on the latest commit. The verdict itself is advisory — read the findings and use your judgement; it does not block merge. A native GitHub reviewer approval is still separately required by branch protection before this PR can merge. |
What is this about?
On the CLI flow, a Mocha test that calls
browser.reloadSession()inside its body can lose itsTestRunFinished. The test passes, but Test Reporting & Analytics keeps it in progress until the ~60-min reap, and the build is reported as timed out.The observers for an event run one after another, and
TestHubModuledecided what to send from the test instance's current state instead of the state it was dispatched for. That state is shared by everytrackEvent, so it can move on while an earlier observer is still awaiting:AutomateModule's session map. Its TEST/POST observer (onAfterTest) registers the session and awaits the session-name PUT (since 9.35.1). Without a reload this is a de-duplicated no-op.afterTesthook at the same time. If that hook takes a screenshot or prints to the console, the resulting LOGtrackEventsets the shared instance's current state to LOG while TEST/POST is still parked on the PUT.TestHubModule.onAllTestEventsthen reads LOG and takes the log branch. The TEST/POST is never deferred or sent. Nothing is logged as an error.This PR makes the observers act on the event they were called for:
TestFramework.runHooksputs the dispatchedtestFrameworkState/hookStateon the observer args.TestHubModulereads them inonAllTestEvents,sendTestFrameworkEvent,sendTestSessionEventandsendLogCreatedEvent. It falls back to the instance's state for direct callers.The same race could also drop a TEST/PRE or a hook finish if a log landed while an earlier observer was awaiting. The fix covers those too.
The other
getCurrentTestState()readers (custom tags, attachments, log loading, instance resolution) run from user commands or fromtrackEventitself. They want the live state, so they are unchanged.AutomateModule.onAfterTeststill awaits the session-name PUT. Making it fire-and-forget would race the de-dupe against the end-of-worker sweep, and once the observers act on their own state the await is harmless.The v8 line is not affected: its
onAfterTestdoes not await a session-name PUT.Verification
End to end, on real BrowserStack sessions (wdio 9.32 mocha, Chrome on Windows 11, Test Observability on). The sample spec has 4 tests that each call
browser.reloadSession()mid-test, plus 1 test without a reload. A config-levelafterTesthook either takes a screenshot or prints a console line 50 ms in, which always lands inside the session-name PUT.@wdio/browserstack-serviceafterTestTestRunFinishedWith the published package, a screenshot only hits when the PUT is slower than the screenshot command (here 1.3 s vs ~0.6 s). The test without a reload was never affected. With this PR the LOG still lands inside the PUT, but every TEST/POST is deferred and sent, and log delivery is unchanged (5 log events in each run).
Unit tests: new
tests/cli/modules/testHubModule.dispatchedState.test.ts(5). It uses the realTestFramework.runHooks, event dispatcher andTestFrameworkInstance:The first 4 fail without the fix. The related suites (TestHub, skip reporter, mocha/cucumber frameworks, Automate module, event dispatcher, custom tags, attachments) pass 194/194 on Node 22. The full suite has no new failures; its 78 pre-existing failures come from the local network proxy and are the same set with and without this change.
npm run lintandnpm run build(includingtsc) are clean.Related Jira task/s
Release (mandatory for every PR — required for the
ready-for-reviewlabel)Version bump: (required — tick exactly one)
Release notes type: (optional)
Release notes (customer-facing): (optional but encouraged)
browser.reloadSession()staying "in progress" on Test Reporting & Analytics, and their builds being reported as timed out, when anafterTesthook takes a screenshot or logs to the console.Release notes (internal): (required — engineer-facing; what actually changed / why)
TestFramework.runHooksnow passes the dispatchedtestFrameworkState/hookStateto observers, andTestHubModuleacts on it instead of the shared instance's current state.AutomateModule.onAfterTestawaited the session-name PUT for a session reloaded mid-test sent TEST/POST down the log branch, so itsTestRunFinishedwas never sent.Checklist
PR Validations
Run Tests: Comment RUN_TESTS to trigger sanity tests.
🤖 Generated with Claude Code