Skip to content

Restore Browser nightly coverage after UI and API changes - #279

Merged
Programcode merged 1 commit into
mainfrom
fix/browser-nightly-main
Oct 30, 2026
Merged

Programcode merged 1 commit into
mainfrom
fix/browser-nightly-main

Conversation

@Programcode

@Programcode Programcode commented Oct 30, 2026 •

Copy link
Copy Markdown
Owner

The Browser nightly failures on main came from tests that still assumed the UI, loading behavior, and usage API from before the October changes. This updates seven browser specs to exercise the current behavior. No application code, workflow, dependencies, Supabase stack, or verification tooling changes.

Each reported failure was checked against source and the failing browser output:

  • Refresh: select the shell's banner landmark; page-level headers are valid and must remain.
  • Factoids: open the researched-sources disclosure, then check both source URLs, Strong/Weak labels, fallback warning, and generated factoid. The quality checks are scoped to that disclosure.
  • Disabled Workspaces: keep checking that navigation omits Workspaces and its URL shows 404 plus the not-found heading.
  • Failed route chunk: fail both the initial fetch and the automatic retry, verify the error boundary, then click its offered Reload and verify recovery on the third fetch.
  • Subscription caching: count the current ai_usage_today RPC instead of the retired ai_requests query, retaining the exact one-read assertion across sibling routes. Return PostgREST's 406 for an empty required single-row read so settings hydration does not treat an invalid array response as an account and issue an unrelated billing read.
  • Localization: check the translated not-found message as the h1; 404 is now its eyebrow. Keep switching languages and checking document titles.
  • Hungarian landing: check the current Hungarian answer rather than a removed introduction with a hardcoded month. Keep the public-route and navigation checks.
  • Mobile Flowchart: use Playwright's locator tap, which scrolls and waits for stability, instead of tapping unvalidated viewport coordinates after selection reflows the page. Keep immediate deletion, no confirmation, and exact Undo restoration.
  • Onboarding: the app intentionally starts when its Home target exists, introduced in 2dc73ff3. Replace the obsolete 100ms post-navigation absence check with a controlled-clock check that the tour stays closed on the protected tool past its deadline, then opens at the visible Home target. No timeout increases or retries added.

Baseline at 151842f3: 14 deterministic failures reproduced locally in the reported cases; onboarding and Flowchart passed that baseline, consistent with timing-sensitive failures in Actions.

Local browser runs encountered Chromium ERR_NETWORK_CHANGED on localhost source-module fetches while other work changed Docker bridge/veth interfaces on the shared VPS. A read-only ip monitor link captured 54 lines of interface events in 40 seconds. Verification uses a temporary process network namespace, with the repository commands running as the workspace owner on Node 22, to keep those host interface changes out of Chromium. No host networking, application code, or other agents' containers were changed. The namespace has loopback and an isolated dummy interface so online-reload behavior is still exercised. A focused check of onboarding, chunk recovery, and subscription caching passed all six desktop/mobile cases there with retries disabled.

At head fc391496e096f68ada3a208aad167e7bc9748643, the complete seven named specs passed on desktop Chromium and mobile Chromium: 96 passed, with zero retries, in 6.1 minutes. Command run inside the isolated network namespace:

P=$(bash .superset/e2e-port.sh) && CI=1 PORT=$P PLAYWRIGHT_BASE_URL=http://127.0.0.1:$P npx playwright test e2e/loading-performance.spec.ts e2e/search-quality.spec.ts e2e/navigation-performance.spec.ts e2e/localization.spec.ts e2e/hungarian-parity.spec.ts e2e/quality-backlog.spec.ts e2e/home-navigation.spec.ts --project=desktop-chromium --project=mobile-chromium --retries=0 --reporter=list

The onboarding test then passed 20 consecutive runs, ten on desktop and ten on mobile, with one worker and zero retries, at the same head:

P=$(bash .superset/e2e-port.sh) && CI=1 PORT=$P PLAYWRIGHT_BASE_URL=http://127.0.0.1:$P npx playwright test e2e/home-navigation.spec.ts --project=desktop-chromium --project=mobile-chromium --grep "onboarding stays closed on a protected tool" --repeat-each=10 --workers=1 --retries=0 --reporter=list

npm run verify passed all ten steps at the same head in 147.7 seconds, started in a nonblocking background session. This includes docs, jargon, lint, typecheck, unit tests, collaboration, build, browser smoke, service-worker offline, and 505 Edge Function tests. No local Supabase stack was started, and the full local browser suite has not been run.

These are credential-free client checks with stub sessions and responses; they do not prove authenticated backend, staging, production, or live AI behavior. Desktop and mobile screenshots show the Hungarian not-found heading and language control exercised by the corrected spec.

Adam explicitly requested full send through merge. This PR will merge after final local verify, current-head Quality CI, and a clean substantive Codex review, with at most three review rounds. After merging, Browser nightly will be dispatched once on main and its result recorded here.

Hungarian not-found heading and language control on desktop

Hungarian not-found heading and language control on mobile

Copilot AI balanced review requested due to automatic review settings October 30, 2026 10:54

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota.

@coderabbitai

coderabbitai Bot commented Oct 30, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 54 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration
Commits

Reviewing files that changed from the base of the PR and between 151842f and fc39149.

Files selected for processing: 7
  • e2e/home-navigation.spec.ts
  • e2e/hungarian-parity.spec.ts
  • e2e/loading-performance.spec.ts
  • e2e/localization.spec.ts
  • e2e/navigation-performance.spec.ts
  • e2e/quality-backlog.spec.ts
  • e2e/search-quality.spec.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@Programcode
Programcode merged commit d690fa9 into main Oct 30, 2026
6 checks passed
@Programcode

Copy link
Copy Markdown
Owner Author

Codex review round 1 completed with no findings at head fc391496e096f68ada3a208aad167e7bc9748643, against merge base 151842f3af35533df81d844156e9460e98baa9ce. The read-only reviewer inspected all seven changed specs against application source, including the retained source-quality checks, single-row fixture semantics, usage RPC caching, chunk recovery, touch actionability, and Home-only onboarding. This was static review; the execution evidence came from the parent run.

Quality run 41044138632 passed lint, typecheck/build, unit tests, Edge Function tests, and browser smoke on this exact head. Local evidence is 96/96 cases in the seven named specs, 20/20 consecutive onboarding runs with one worker and no retries, and all ten npm run verify steps in 147.7 seconds.

Copilot returned a quota-only notice and is not counted as a substantive review. No open review findings remain. Adam explicitly authorized full send through merge; the local verification, current-head CI, and substantive Codex review gates are met. The post-merge Browser nightly run will be recorded in a follow-up.

@Programcode

Copy link
Copy Markdown
Owner Author

Post-merge verification is complete at d690fa9900613f12ce9d978e9702798979bb9e8a.

  • Browser nightly 41045151170 was dispatched on main after the merge and passed: 299 passed, 5 skipped, 0 failed, 0 flaky, with browser tests finishing in 10.0 minutes. The production-build service-worker offline step also passed, including offline routes, timer, language switching, recovery, logo, and zero API cache entries.
  • The merged tree is identical to locally verified head fc391496e096f68ada3a208aad167e7bc9748643: 96/96 cases in the seven requested specs, 20/20 sequential onboarding repetitions, and all ten npm run verify steps passed. No retries were used for the named specs or onboarding repetitions.
  • Quality passed all five jobs on the PR head. Codex review round 1 had no findings; no further review round was needed.
  • Vercel reported successful production and staging deployments for the merge commit, and staging-sync succeeded. The live production homepage returned HTTP 200. This is a test-only patch; no backend deploy, migration, secret change, local Supabase stack, or full local browser-suite run was needed.

The post-merge nightly dispatch was already complete when the resumed session checked it, so no duplicate workflow run was started. Browser verification on the VPS used a temporary isolated network namespace to avoid unrelated Docker bridge/veth churn; assertions and app configuration were unchanged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants