Skip to content

Cut the 4 slowest browser tests from 391 s to about 100 s - #21

Merged
EquityBot merged 2 commits into
mainfrom
ryan/faster-browser-tests
Oct 30, 2026
Merged

EquityBot merged 2 commits into
mainfrom
ryan/faster-browser-tests

Conversation

@ryanduguid

Copy link
Copy Markdown
Collaborator

Summary

These tests spent most of their time waiting out deadlines that fail by design. Test-only overrides shorten those waits, and only for the cases that wait them out. Production values don't change: the overrides exist only under cfg(test).

Wait Where Production and other cases Cases that wait it out
Attachment budget floor attachment_budget 45 s 5 s, the fixture's command budget
Watch after a refused close 3 close routines up to 5 s 1 s
  • The attachment override applies to the 4 upload faults that never settle. The successful upload keeps the 45 s floor, and wrong-receipt faults keep it too, so expiry can't stand in for the refusal they test.
  • The close override starts after the close command returns, so the command keeps its full budget. The watch only reads the target list, so it needs no room for a durable write.
  • The 5 s command budget is unchanged, including its warning about 1 s being too tight.

Evidence

Each test alone on Windows with --test-threads=1:

Test Before After
unconfirmed_upload_commands_or_cleanup_never_authorise_close_or_send 186 s 25 s
completed_close_retries_resume_only_the_exact_durable_marker 87 s 42 s
same_process_reconciliation_never_closes_running_or_uncertain_requests 69 s 26 s
owned_upload_must_reach_readiness_and_cleanup_before_send 49 s 10 s
Total 391 s 103 s

Another run of the same code with temporary logging took 80 s.

  • Coverage: for all 25 cases in these tests, final status, recoveryRequired, error text, cleanup warning, close and send counts and the CDP request sequence (repeats collapsed) match main. This came from temporary logging, not committed.
  • Before: library suite on main, 666 tests, 14 ignored.
    After: library suite, 666 passed, 0 failed, 14 ignored, on the commit before the final one.
  • On the final head: cargo fmt --check and cargo clippy --all-targets --locked -- -D warnings pass; browser_target 5 passed, browser_retirement 10 passed.

Merge danger

Door: two-way. Blast radius: tests

A panicking test could leave an override set for later tests on the same thread, as the existing TEST_CANCELLED_SEND_SETTLE can. That only matters in a run that has already failed.

Unverified

  • A clean before and after for the whole suite: both runs shared the machine with other work.
  • The attachment budget as the cause of the upload wait is inferred from the code and the drop after changing only that floor; it wasn't timed separately.
  • Not run locally on Linux; this PR's CI covers it.
  • The full library suite was not rerun after the final commit, which moves the close override and is test-only.
  • bridge_doctor_accepts_and_ignores_verbose_like_commander and local_browser_cli_and_mcp_admit_foreground_and_detached_rust_workers fail locally, as they do on main. No browser is installed here, which is the likely cause (not confirmed).

Most of the time in the 4 slowest browser tests went on deadlines that fail by
design: 45 s of attachment budget for each upload fault that never settles,
and 5 s of close watching for each refused close. Test-only overrides shorten
those waits for the cases that wait them out. Cases that refuse on a first
read keep the real budget, so expiry cannot stand in for the refusal they test.

The close watch only reads the target list, so its 1 s override has no durable
write to leave room for. The 5 s command budget is unchanged. Production values
are unchanged: the overrides exist only under cfg(test).
The override also capped the close command, so a slow but successful close
could have become a timeout that exercised a different path.
@sonarqubecloud

sonarqubecloud Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

@EquityBot
EquityBot merged commit 6c58009 into main Oct 30, 2026
12 of 13 checks passed
@EquityBot
EquityBot deleted the ryan/faster-browser-tests branch October 30, 2026 17:13
EquityBot pushed a commit that referenced this pull request Nov 1, 2026
* Cut browser test time by shortening deadlines those tests wait out

Most of the time in the 4 slowest browser tests went on deadlines that fail by
design: 45 s of attachment budget for each upload fault that never settles,
and 5 s of close watching for each refused close. Test-only overrides shorten
those waits for the cases that wait them out. Cases that refuse on a first
read keep the real budget, so expiry cannot stand in for the refusal they test.

The close watch only reads the target list, so its 1 s override has no durable
write to leave room for. The 5 s command budget is unchanged. Production values
are unchanged: the overrides exist only under cfg(test).

* Shorten only the watch after a refused close, not the close command

The override also capped the close command, so a slow but successful close
could have become a timeout that exercised a different path.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants