Skip to content

chore(deps): update dependency @playwright/test to v1.63.0 - #4658

Merged
gabemontero merged 1 commit into
mainfrom
renovate/rhdh-boost-dependencies-(minor)
Sep 10, 2026
Merged

chore(deps): update dependency @playwright/test to v1.63.0#4658
gabemontero merged 1 commit into
mainfrom
renovate/rhdh-boost-dependencies-(minor)

Conversation

@renovate

@renovate renovate Bot commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

This PR contains the following updates:

Package Change Age Confidence
@playwright/test (source) 1.61.11.63.0 age confidence

Release Notes

microsoft/playwright (@​playwright/test)

v1.63.0

Compare Source

v1.62.1

Compare Source

v1.62.0

Compare Source

🧱 New component testing model

Component testing moves to a stories and galleries model.
A story wraps your component in one specific scenario — hard-coded props, mock data, providers — and a

gallery page that you serve renders stories on demand. The new fixtures.mount() fixture navigates
to the gallery, mounts a story by id, and returns a Locator scoped to the story's root element:

test('click should expand', async ({ mount }) => {
  const component = await mount('components/Expandable/Stateful');
  await component.getByRole('button').click();
  await expect(component.getByTestId('expanded')).toHaveValue('true');
});

Pass a story type as a template argument to type-check its props, and use update(props) /
unmount() on the returned locator to re-render or tear down within a test.

🛑 Cancel operations with AbortSignal

Most operations and web-first assertions now accept a signal option that takes an
AbortSignal, letting you
cancel long-running actions, navigations, waits, and assertions:

const controller = new AbortController();
setTimeout(() => controller.abort(), 1000);

await page.getByRole('button', { name: 'Submit' }).click({ signal: controller.signal });
await expect(page.getByText('Done')).toBeVisible({ signal: controller.signal });

Providing a signal does not disable the default timeout; pass timeout: 0 to disable it.

🖼️ WebP screenshots

expect(page).toHaveScreenshot() and expect(locator).toHaveScreenshot()
can now store snapshots in the WebP format — just give the snapshot a .webp name:

// Visual comparisons store the golden snapshot as lossless WebP.
await expect(page).toHaveScreenshot('homepage.webp');

// Standalone screenshots can trade quality for size with lossy WebP.
await page.screenshot({ path: 'homepage.webp', quality: 50 });

page.screenshot() and locator.screenshot() also accept webp as a type,
where quality 100 (the default) is lossless and lower values use lossy compression.

🧩 Custom test filtering with Reporter.preprocess()

New reporter.preprocess() hook runs after the configuration is resolved and before
reporter.onBegin(), letting a reporter mark individual tests as skipped, excluded,
fixed, or failing through a TestRun object:

class MyReporter {
  async preprocess({ config, suite, testRun }) {
    for (const test of suite.allTests()) {
      if (shouldSkip(test))
        testRun.skip(test);
    }
  }
}
🔁 Isolated retries

New testConfig.retryStrategy controls when failed tests are retried. The default
'immediate' retries as soon as a worker is free; 'isolated' runs all retries at the end,
one by one in a single worker, to minimize interference with the rest of the suite:

// playwright.config.ts
export default defineConfig({
  retries: 2,
  retryStrategy: 'isolated',
});
New APIs
Browser and Context
  • New option credentials includes the context's virtual WebAuthn Credentials (passkeys) in the storage state, so they can be persisted and re-seeded into later contexts.
Actions
  • New scroll option ("auto" | "none") on actions to opt out of Playwright's automatic scroll-into-view.
Network
Evaluation
Command line & MCP
Reporters
  • The HTML report's Merge files grouping — previously only a UI toggle — can now be enabled from the config with the new mergeFiles reporter option:
// playwright.config.ts
export default defineConfig({
  reporter: [['html', { mergeFiles: true }]],
});
Announcements
  • ⚠️ Debian 11 is not supported anymore.
Browser Versions
  • Chromium 151.0.7922.34
  • Mozilla Firefox 153.0
  • WebKit 26.5

This version was also tested against the following stable channels:

  • Google Chrome 151
  • Microsoft Edge 151

Configuration

📅 Schedule: (UTC)

  • Branch creation
    • At any time (no schedule defined)
  • Automerge
    • At any time (no schedule defined)

🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.

Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

🔕 Ignore: Close this PR and you won't be reminded about this update again.


  • If you want to rebase/retry this PR, check this box

This PR was generated by Mend Renovate. View the repository job log.

@renovate renovate Bot added boost dependencies Pull requests that update a dependency file labels Sep 9, 2026
@fullsend-ai-review

fullsend-ai-review Bot commented Sep 9, 2026

Copy link
Copy Markdown

🤖 Review · ❌ Terminated · Started 5:30 PM UTC · Ended 5:32 PM UTC

Commit: 1438229 · View workflow run →

@codecov

codecov Bot commented Sep 9, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 62.60%. Comparing base (0754805) to head (0c21aed).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #4658   +/-   ##
=======================================
  Coverage   62.60%   62.60%           
=======================================
  Files        2635     2635           
  Lines      105191   105191           
  Branches    29539    29539           
=======================================
  Hits        65859    65859           
  Misses      37477    37477           
  Partials     1855     1855           
Flag Coverage Δ *Carryforward flag
adoption-insights 84.77% <ø> (ø) Carriedforward from 0754805
ai-integrations 78.80% <ø> (ø) Carriedforward from 0754805
app-defaults 53.07% <ø> (ø) Carriedforward from 0754805
augment 46.67% <ø> (ø) Carriedforward from 0754805
boost 82.94% <ø> (ø)
bulk-import 73.12% <ø> (ø) Carriedforward from 0754805
cost-management 13.35% <ø> (ø) Carriedforward from 0754805
dcm 73.47% <ø> (ø) Carriedforward from 0754805
e2e-adoption-insights 60.00% <ø> (ø) Carriedforward from 0754805
e2e-extensions 62.31% <ø> (ø) Carriedforward from 0754805
e2e-global-header 49.71% <ø> (ø) Carriedforward from 0754805
e2e-homepage 61.11% <ø> (ø) Carriedforward from 0754805
e2e-intelligent-assistant 46.74% <ø> (ø) Carriedforward from 0754805
e2e-orchestrator 49.52% <ø> (ø) Carriedforward from 0754805
e2e-orchestrator-plugin 49.51% <ø> (ø) Carriedforward from 0754805
e2e-quickstart 55.21% <ø> (ø) Carriedforward from 0754805
e2e-scorecard 50.05% <ø> (ø) Carriedforward from 0754805
e2e-theme 16.36% <ø> (ø) Carriedforward from 0754805
extensions 57.37% <ø> (ø) Carriedforward from 0754805
global-floating-action-button 71.18% <ø> (ø) Carriedforward from 0754805
global-header 67.88% <ø> (ø) Carriedforward from 0754805
homepage 48.48% <ø> (ø) Carriedforward from 0754805
install-dynamic-plugins 71.94% <ø> (ø) Carriedforward from 0754805
intelligent-assistant 76.51% <ø> (ø) Carriedforward from 0754805
konflux 91.98% <ø> (ø) Carriedforward from 0754805
lightspeed 69.02% <ø> (ø) Carriedforward from 0754805
mcp-integrations 84.46% <ø> (ø) Carriedforward from 0754805
orchestrator 72.02% <ø> (ø) Carriedforward from 0754805
quickstart 63.74% <ø> (ø) Carriedforward from 0754805
sandbox 79.56% <ø> (ø) Carriedforward from 0754805
scorecard 88.20% <ø> (ø) Carriedforward from 0754805
theme 87.91% <ø> (ø) Carriedforward from 0754805
translations 5.12% <ø> (ø) Carriedforward from 0754805
x2a 77.18% <ø> (ø) Carriedforward from 0754805

*This pull request uses carry forward flags. Click here to find out more.


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0754805...0c21aed. Read the comment docs.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@fullsend-ai-review

Copy link
Copy Markdown

🤖 Finished Review · ❌ Failure (validation failed after 2 iteration(s)) · Started 5:30 PM UTC · Completed 5:32 PM UTC

Commit: 1438229 · View workflow run →

Runtime: claude · Model: opus → claude-opus-5

@renovate
renovate Bot force-pushed the renovate/rhdh-boost-dependencies-(minor) branch from 1438229 to 65c9ea6 Compare September 9, 2026 18:11
@fullsend-ai-review

fullsend-ai-review Bot commented Sep 9, 2026

Copy link
Copy Markdown

🤖 Review · ❌ Terminated · Started 6:13 PM UTC · Ended 6:15 PM UTC

Commit: 65c9ea6 · View workflow run →

@fullsend-ai-review

Copy link
Copy Markdown

🤖 Finished Review · ❌ Failure (validation failed after 2 iteration(s)) · Started 6:13 PM UTC · Completed 6:15 PM UTC

Commit: 65c9ea6 · View workflow run →

Runtime: claude · Model: opus → claude-opus-5

@renovate
renovate Bot force-pushed the renovate/rhdh-boost-dependencies-(minor) branch from 65c9ea6 to a159edd Compare September 9, 2026 18:18
@fullsend-ai-review

fullsend-ai-review Bot commented Sep 9, 2026

Copy link
Copy Markdown

🤖 Finished Review · ❌ Failure (validation failed after 2 iteration(s)) · Started 6:19 PM UTC · Completed 6:21 PM UTC

Commit: a159edd · View workflow run →

Runtime: claude · Model: opus → claude-opus-5

@renovate
renovate Bot force-pushed the renovate/rhdh-boost-dependencies-(minor) branch from a159edd to f60cf47 Compare September 9, 2026 18:27
@fullsend-ai-review

fullsend-ai-review Bot commented Sep 9, 2026

Copy link
Copy Markdown

🤖 Finished Review · ❌ Failure (validation failed after 2 iteration(s)) · Started 6:28 PM UTC · Completed 6:30 PM UTC

Commit: f60cf47 · View workflow run →

Runtime: claude · Model: opus → claude-opus-5

@renovate
renovate Bot force-pushed the renovate/rhdh-boost-dependencies-(minor) branch from f60cf47 to 3ae9188 Compare September 9, 2026 18:44
@fullsend-ai-review

fullsend-ai-review Bot commented Sep 9, 2026

Copy link
Copy Markdown

🤖 Review · ❌ Terminated · Started 6:45 PM UTC · Ended 6:47 PM UTC

Commit: 3ae9188 · View workflow run →

@fullsend-ai-review

Copy link
Copy Markdown

🤖 Finished Review · ❌ Failure (validation failed after 2 iteration(s)) · Started 6:45 PM UTC · Completed 6:47 PM UTC

Commit: 3ae9188 · View workflow run →

Runtime: claude · Model: opus → claude-opus-5

@renovate
renovate Bot force-pushed the renovate/rhdh-boost-dependencies-(minor) branch from 3ae9188 to 952a7b7 Compare September 9, 2026 18:50
@fullsend-ai-review

fullsend-ai-review Bot commented Sep 9, 2026

Copy link
Copy Markdown

🤖 Finished Review · ❌ Failure (validation failed after 2 iteration(s)) · Started 6:52 PM UTC · Completed 6:54 PM UTC

Commit: 952a7b7 · View workflow run →

Runtime: claude · Model: opus → claude-opus-5

@renovate
renovate Bot force-pushed the renovate/rhdh-boost-dependencies-(minor) branch from 952a7b7 to 708cbfe Compare September 9, 2026 19:01
@fullsend-ai-review

fullsend-ai-review Bot commented Sep 9, 2026

Copy link
Copy Markdown

🤖 Finished Review · ❌ Failure (validation failed after 2 iteration(s)) · Started 7:03 PM UTC · Completed 7:05 PM UTC

Commit: 708cbfe · View workflow run →

Runtime: claude · Model: opus → claude-opus-5

@renovate
renovate Bot force-pushed the renovate/rhdh-boost-dependencies-(minor) branch from 708cbfe to 3702e10 Compare September 9, 2026 19:13
@fullsend-ai-review

fullsend-ai-review Bot commented Sep 9, 2026

Copy link
Copy Markdown

🤖 Finished Review · ❌ Failure (validation failed after 2 iteration(s)) · Started 7:14 PM UTC · Completed 7:16 PM UTC

Commit: 3702e10 · View workflow run →

Runtime: claude · Model: opus → claude-opus-5

@renovate
renovate Bot force-pushed the renovate/rhdh-boost-dependencies-(minor) branch from 3702e10 to 5c96c3d Compare September 9, 2026 19:21
@fullsend-ai-review

fullsend-ai-review Bot commented Sep 9, 2026

Copy link
Copy Markdown

🤖 Finished Review · ❌ Failure (validation failed after 2 iteration(s)) · Started 7:22 PM UTC · Completed 7:24 PM UTC

Commit: 5c96c3d · View workflow run →

Runtime: claude · Model: opus → claude-opus-5

@fullsend-ai-review fullsend-ai-review Bot added the ready-for-merge All reviewers approved — ready to merge label Sep 10, 2026
@renovate
renovate Bot force-pushed the renovate/rhdh-boost-dependencies-(minor) branch from 1990dc7 to ff1266f Compare September 10, 2026 14:01
@fullsend-ai-review

fullsend-ai-review Bot commented Sep 10, 2026

Copy link
Copy Markdown

🤖 Review · ❌ Terminated · Started 2:02 PM UTC · Ended 2:09 PM UTC

Commit: ff1266f · View workflow run →

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-review fullsend-ai-review Bot added ready-for-merge All reviewers approved — ready to merge and removed ready-for-merge All reviewers approved — ready to merge labels Sep 10, 2026
@fullsend-ai-review

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 2:02 PM UTC · Completed 2:09 PM UTC

Commit: ff1266f · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Cost: $1.06

@renovate
renovate Bot force-pushed the renovate/rhdh-boost-dependencies-(minor) branch from ff1266f to c691977 Compare September 10, 2026 14:41
@fullsend-ai-review

fullsend-ai-review Bot commented Sep 10, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 2:43 PM UTC · Completed 2:49 PM UTC

Commit: c691977 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Cost: $0.98

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-review fullsend-ai-review Bot added ready-for-merge All reviewers approved — ready to merge and removed ready-for-merge All reviewers approved — ready to merge labels Sep 10, 2026
@renovate
renovate Bot force-pushed the renovate/rhdh-boost-dependencies-(minor) branch from c691977 to c0d7848 Compare September 10, 2026 16:25
@fullsend-ai-review

fullsend-ai-review Bot commented Sep 10, 2026

Copy link
Copy Markdown

🤖 Review · ❌ Terminated · Started 4:27 PM UTC · Ended 4:33 PM UTC

Commit: c0d7848 · View workflow run →

fullsend-ai-review[bot]

This comment was marked as outdated.

@fullsend-ai-review fullsend-ai-review Bot added ready-for-merge All reviewers approved — ready to merge and removed ready-for-merge All reviewers approved — ready to merge labels Sep 10, 2026
@fullsend-ai-review

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 4:27 PM UTC · Completed 4:33 PM UTC

Commit: c0d7848 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Cost: $0.94

Signed-off-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
@renovate
renovate Bot force-pushed the renovate/rhdh-boost-dependencies-(minor) branch from c0d7848 to 0c21aed Compare September 10, 2026 18:53
@sonarqubecloud

Copy link
Copy Markdown

@fullsend-ai-review

fullsend-ai-review Bot commented Sep 10, 2026

Copy link
Copy Markdown

🤖 Finished Review · ✅ Success · Started 6:54 PM UTC · Completed 7:02 PM UTC

Commit: 0c21aed · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Cost: $1.34

@fullsend-ai-review fullsend-ai-review Bot added ready-for-merge All reviewers approved — ready to merge and removed ready-for-merge All reviewers approved — ready to merge labels Sep 10, 2026
@gabemontero
gabemontero merged commit 743c456 into main Sep 10, 2026
61 checks passed
@fullsend-ai-retro

fullsend-ai-retro Bot commented Sep 10, 2026

Copy link
Copy Markdown

🤖 Finished Retro · ✅ Success · Started 7:04 PM UTC · Completed 7:12 PM UTC

Commit: 0c21aed · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $2.08

@renovate
renovate Bot deleted the renovate/rhdh-boost-dependencies-(minor) branch September 10, 2026 19:08
@fullsend-ai-retro

Copy link
Copy Markdown

Retro: PR #4658 — Renovate dependency bump with 17 consecutive review failures

What happened

PR #4658 was a Renovate bot dependency bump (@playwright/test 1.61.1 → 1.63.0) — 16 additions, 39 deletions across 2 files. Despite being a trivial change, the review agent failed 17 consecutive times over ~18 hours (Sep 9 17:30 UTC – Sep 10 07:46 UTC), producing 41 comments before finally succeeding.

Root cause

The opus model alias resolved to claude-opus-5 in CLI v2.1.260, but the GCP Vertex AI service account lacked aiplatform.endpoints.predict permission for that model endpoint. Every API call returned a 403 PERMISSION_DENIED, causing the agent to produce zero output. The validation script then failed with "output/agent-result.json not found," which the status comment reported generically as "validation failed after 2 iteration(s)."

The fix came in CLI v2.1.263, which added automatic fallback: Warning: Opus: Opus 5 not available -- using Opus 4.6 for this session. After this update, all subsequent runs succeeded immediately.

Scope of impact

The failure was systemic, not PR-specific. At least 7 PRs were affected during the outage window, including Dependabot bumps (#4672, #4666, #4663, #4662, #4661) and a human-authored feature PR (#4667). PR #4658 accumulated the most failures (17) because Renovate kept rebasing the branch, each rebase producing a new commit SHA that triggered another review dispatch against the same broken model.

Agents repo

fullsend-ai/agents at commit 4bbe4f5 (discovered from workflow run logs).

Existing issue coverage

All improvement areas identified in this retro are already tracked as open issues in fullsend-ai/fullsend. No new proposals are warranted. Evidence from this retro supports the following existing issues:

  • Model fallback: #7026 (no automatic fallback when aliased model not served — the exact bug fixed by CLI v2.1.263), #7154, #6967, #6964 (validate model availability before dispatch).
  • Excessive Renovate triggers: #4596, #4652, #4401, #1355 (skip re-review when bot PR is rebased without diff changes). PR chore(deps): update dependency @playwright/test to v1.63.0 #4658's 17 dispatches across 17 different commit SHAs — all with identical diffs — is strong evidence for these.
  • Circuit breaker: #2992, #2587 (cap review dispatches per PR). Any cap ≤ 17 would have stopped the cascade.
  • Error classification: #7023 (status comments collapse infrastructure crashes and schema validation failures into generic "validation failed" message). The 403 permission denied was reported as "validation failed" — distinguishing infrastructure errors would have aided faster diagnosis.
  • Comment noise: #7050 (suppress status comments for superseded runs). The 41 comments created significant noise.
  • Similar prior pattern: #3985 documents the same compounding pattern: serial Renovate PRs accumulating 40+ review runs.

Autonomy observation

Once reviews succeeded, the agent correctly approved with "Looks good to me" on all 8 successful runs. The human reviewer (gabemontero) also approved without comments. For bot-authored dependency bumps with passing CI, agent and human review judgment aligned — consistent with existing proposals for lighter-weight review on trivial bot PRs (#2842, #6891).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

boost dependencies Pull requests that update a dependency file ready-for-merge All reviewers approved — ready to merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant