Skip to content

chore(e2e): run the suite with Playwright workers instead of shards - #102

Merged
krazymir merged 2 commits into
mainfrom
claude/playwright-workers-e2e-shards-b3276d
Oct 21, 2026
Merged

krazymir merged 2 commits into
mainfrom
claude/playwright-workers-e2e-shards-b3276d

Conversation

@krazymir

Copy link
Copy Markdown
Collaborator

What

The Jenkins e2e now runs as one Playwright run with E2E_WORKERS workers (default 4) instead
of E2E_SHARDS separate Playwright containers. Test and CI only, so there's no version bump and no
changelog entry.

Why

The shards did by hand what Playwright workers already do: N containers with one worker each, plus a
greedy balancer in playwright.config.ts that kept the specs changing platform-wide settings
together in shard 1. Playwright 1.52+ can cap the workers of a single project, so that rule is now
config:

  • platform-state: the 13 specs that change platform-wide settings (AI provider, banner,
    settings, webhooks, featured, gamification, hype, public form, surveys), workers: 1. They still
    run one after another. The project is listed first, so whenever its one worker frees up it takes
    the next of those files at once; Playwright 1.64's dispatcher picks the first queued file whose
    project is under its cap.
  • isolated: everything else, given to whichever worker is free.

Expected speed-up

The 29 platform-state tests (of 112) must run back to back, and that sets the floor on the wall
time. Today's default of 2 shards runs about 55 tests end to end per shard. 4 workers bring that
down to the 29-test chain, roughly 2× faster. Beyond about 4 workers the run gets no shorter;
each test just slows as more browsers share the one dev server. Hence the default of 4. These are
estimates by test count, not measured durations.

Changes

  • e2e/playwright.config.ts:
    • E2E_SHARD/E2E_SHARD_TOTAL and the shard balancer are gone. E2E_WORKERS takes 1–16
      (default 4); any other value fails fast.
    • Whole files are dealt out to workers (fullyParallel stays off), so two tests of one file
      never overlap, exactly as with shards. Splitting files would gain nothing: the platform-state
      chain is far longer than any one file (largest: 7 tests).
    • One HTML report instead of N.
    • maxFailures: 10 now counts across the whole run; before, the limit applied to each shard
      separately.
  • Jenkinsfile:
    • The E2E_SHARDS parameter becomes E2E_WORKERS (default 4) and applies to the quick run too.
    • One Playwright container, one exit file, no [shard i] log prefixes.
    • The dev-server watchdog, host stats and post-mortem are unchanged; the watchdog now stops one
      container instead of N.
    • The first build after this lands falls back to 4 until Jenkins picks up the new parameter.
  • e2e/helpers/stamp.ts: uniqueStamp() replaces Date.now().toString(36) at all 107 call
    sites in 46 specs. It adds a letter for the worker (from TEST_PARALLEL_INDEX), so two workers
    starting a test in the same millisecond can't create identically named fixtures. Shards already
    carried that risk. The longest stamped namespace slug is 21 of the 40 characters allowed.
  • Comments in scripts/e2e-specs.mjs and public-submissions.spec.ts no longer mention shards.

Checked

  • playwright test --list: 29 tests in platform-state, 83 in isolated, every spec in exactly
    one; E2E_WORKERS=0 is rejected.
  • pnpm typecheck passes; web unit tests: 570/570 pass.
  • The e2e stage's shell script passes bash -n.
  • The changed e2e files type-check under a temporary tsconfig. pnpm typecheck doesn't cover
    e2e/, and that check also found one error that predates this PR (sign-out.spec.ts:42),
    left out of scope.
  • Not run: the e2e suite itself and the Jenkins pipeline. The first Jenkins build of this
    branch is the real test.

For the reviewer

  • Watch the first full run's wall time and e2e-host-stats.log: 4 browsers against one dev server
    is more concurrent load than today's 2.
  • A new spec that changes platform-wide state must be added to PLATFORM_STATE_SPECS, as before.

Jenkins split the e2e into E2E_SHARDS shards: N Playwright containers, one worker each, with
a hand-written balancer in playwright.config.ts that kept the platform-state specs in shard 1.
That is Playwright's own worker pool, rebuilt by hand. Now one run uses E2E_WORKERS workers
(1-16, default 4) and two projects: "platform-state" (the 13 specs that change platform-wide
settings, held to one worker so they still run one after another) and "isolated" (the rest).

The platform-state specs are 29 of the 112 tests and must run back to back, so past about
4 workers the suite gets no faster; the default is 4.

- playwright.config.ts: E2E_SHARD/E2E_SHARD_TOTAL and the shard balancer are gone; files go to
  whichever worker is free; one HTML report; maxFailures now counts across the whole run.
- Jenkinsfile: E2E_SHARDS -> E2E_WORKERS (quick runs use it too); one Playwright container,
  one exit file. The dev-server watchdog and post-mortem are unchanged.
- e2e/helpers/stamp.ts: uniqueStamp() replaces Date.now().toString(36) in every spec (107
  call sites) and adds a letter for the worker, so two workers can't create identically named
  fixtures in the same millisecond.
…kers-e2e-shards-b3276d

# Conflicts:
#	packages/web/e2e/anonymity.spec.ts
#	packages/web/e2e/comment-replies.spec.ts
#	packages/web/e2e/core-journey.spec.ts
#	packages/web/e2e/engagement.spec.ts
#	packages/web/e2e/form-lock.spec.ts
@krazymir
krazymir merged commit 8be26ea into main Oct 21, 2026
13 checks passed
@krazymir
krazymir deleted the claude/playwright-workers-e2e-shards-b3276d branch October 21, 2026 06:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant