Repository navigation
chore(e2e): run the suite with Playwright workers instead of shards - #102
Merged
Merged
Conversation
Jenkins split the e2e into E2E_SHARDS shards: N Playwright containers, one worker each, with a hand-written balancer in playwright.config.ts that kept the platform-state specs in shard 1. That is Playwright's own worker pool, rebuilt by hand. Now one run uses E2E_WORKERS workers (1-16, default 4) and two projects: "platform-state" (the 13 specs that change platform-wide settings, held to one worker so they still run one after another) and "isolated" (the rest). The platform-state specs are 29 of the 112 tests and must run back to back, so past about 4 workers the suite gets no faster; the default is 4. - playwright.config.ts: E2E_SHARD/E2E_SHARD_TOTAL and the shard balancer are gone; files go to whichever worker is free; one HTML report; maxFailures now counts across the whole run. - Jenkinsfile: E2E_SHARDS -> E2E_WORKERS (quick runs use it too); one Playwright container, one exit file. The dev-server watchdog and post-mortem are unchanged. - e2e/helpers/stamp.ts: uniqueStamp() replaces Date.now().toString(36) in every spec (107 call sites) and adds a letter for the worker, so two workers can't create identically named fixtures in the same millisecond.
…kers-e2e-shards-b3276d # Conflicts: # packages/web/e2e/anonymity.spec.ts # packages/web/e2e/comment-replies.spec.ts # packages/web/e2e/core-journey.spec.ts # packages/web/e2e/engagement.spec.ts # packages/web/e2e/form-lock.spec.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The Jenkins e2e now runs as one Playwright run with
E2E_WORKERSworkers (default 4) insteadof
E2E_SHARDSseparate Playwright containers. Test and CI only, so there's no version bump and nochangelog entry.
Why
The shards did by hand what Playwright workers already do: N containers with one worker each, plus a
greedy balancer in
playwright.config.tsthat kept the specs changing platform-wide settingstogether in shard 1. Playwright 1.52+ can cap the workers of a single project, so that rule is now
config:
platform-state: the 13 specs that change platform-wide settings (AI provider, banner,settings, webhooks, featured, gamification, hype, public form, surveys),
workers: 1. They stillrun one after another. The project is listed first, so whenever its one worker frees up it takes
the next of those files at once; Playwright 1.64's dispatcher picks the first queued file whose
project is under its cap.
isolated: everything else, given to whichever worker is free.Expected speed-up
The 29 platform-state tests (of 112) must run back to back, and that sets the floor on the wall
time. Today's default of 2 shards runs about 55 tests end to end per shard. 4 workers bring that
down to the 29-test chain, roughly 2× faster. Beyond about 4 workers the run gets no shorter;
each test just slows as more browsers share the one dev server. Hence the default of 4. These are
estimates by test count, not measured durations.
Changes
e2e/playwright.config.ts:E2E_SHARD/E2E_SHARD_TOTALand the shard balancer are gone.E2E_WORKERStakes 1–16(default 4); any other value fails fast.
fullyParallelstays off), so two tests of one filenever overlap, exactly as with shards. Splitting files would gain nothing: the platform-state
chain is far longer than any one file (largest: 7 tests).
maxFailures: 10now counts across the whole run; before, the limit applied to each shardseparately.
Jenkinsfile:E2E_SHARDSparameter becomesE2E_WORKERS(default 4) and applies to the quick run too.[shard i]log prefixes.container instead of N.
e2e/helpers/stamp.ts:uniqueStamp()replacesDate.now().toString(36)at all 107 callsites in 46 specs. It adds a letter for the worker (from
TEST_PARALLEL_INDEX), so two workersstarting a test in the same millisecond can't create identically named fixtures. Shards already
carried that risk. The longest stamped namespace slug is 21 of the 40 characters allowed.
scripts/e2e-specs.mjsandpublic-submissions.spec.tsno longer mention shards.Checked
playwright test --list: 29 tests inplatform-state, 83 inisolated, every spec in exactlyone;
E2E_WORKERS=0is rejected.pnpm typecheckpasses; web unit tests: 570/570 pass.bash -n.pnpm typecheckdoesn't covere2e/, and that check also found one error that predates this PR (sign-out.spec.ts:42),left out of scope.
branch is the real test.
For the reviewer
e2e-host-stats.log: 4 browsers against one dev serveris more concurrent load than today's 2.
PLATFORM_STATE_SPECS, as before.