Skip to content

fix(e2e): start a Current Debate in the staging spec - #100

Merged
Programcode merged 3 commits into
mainfrom
fix/staging-e2e-current-debate
Oct 3, 2026
Merged

Programcode merged 3 commits into
mainfrom
fix/staging-e2e-current-debate

Conversation

@Programcode

@Programcode Programcode commented Oct 2, 2026 •

Copy link
Copy Markdown
Owner

Fixes the Protected staging E2E job. No staging run has passed since 2026-09-23, and every real run since Actions came back on 2026-10-01 has failed.

Root cause

The spec was stale. Staging was behind main too, but that didn't cause these failures.

  • Every failed run waited 180 seconds on getByRole('button', { name: 'Add Source' }), and the log shows that the locator resolved to a <button disabled>.
  • feat: add personal current debate context #72 (2026-09-27) made Library disable Add Source until a Current Debate exists. The last green staging run was on 2026-09-23.
  • On staging, the CI demo user (code label "Protected staging E2E") had 0 debates and no current_debates row, so the button could never become enabled.
  • The rewritten spec passed against staging before the backend sync below. The backend lag was real, but it did not cause this failure.

Changes

e2e/staging/authed-flows.spec.ts:

  • Starts a uniquely named debate from Library's "Start debate" button. A student has to do this before adding a source.
  • Asserts that Add Source is disabled before the debate exists and that the header shows the new Current Debate afterwards.
  • Clears both persisted stores (debate-trainer-storage and debatelab-current-debate) and reloads. It then waits for the sources, debates and current_debates reads and checks that the motion, the Current Debate indicator and the source all come back from staging. Codex caught that the first version cleared only the session store, so the motion could have come from the cache.
  • Deletes the source through the UI (same assertion as before, now waiting for the hydration request first). It then deletes the debate by id through the REST API, using the test user's own session, because the app can't delete the Current Debate. The debate's chat and pins cascade with it. Debate deletion runs even if source removal fails.
  • Sweeps debates whose motion starts with E2E staging debate at the start, so a run killed before cleanup can't leave a Current Debate behind for the next run.
  • Sets action and navigation timeouts to 30 seconds. A failed click now throws inside the 180-second test timeout, so finally cleanup still runs. Before this change a failure used up the whole test timeout.
  • Wraps each phase in a named test.step. On failure it logs Staging step "<name>" failed at path <pathname>. It logs only the pathname, never the query or hash, which can carry auth handoff tokens.

No assertion was removed or loosened. The source-removal check is stricter, because it now waits for the sources request before asserting the title is gone.

Docs: docs/operations.md records the real cause and adds an incident clue: run the staging suite locally before blaming staging. docs/testing.md describes the new suite steps, the required port, the failure output, and the error-context.md caveat below.

Staging sync (staging only, brifeyypjauyusmkptkx)

Production was not touched. I confirmed the ref before each command.

  • Applied migration 20261001100000_account_deletion_and_export.sql. The dry run listed exactly that file.
  • Deployed functions one at a time with --use-api: account (new, v1), voice (new, v1), workspace-invites (new, v1), admin v14→15, ai v19→20, ai-stream v20→21, create-checkout v11→12, create-portal v11→12, export-gdoc v11→12, import-source v12→13, redeem-promo v10→11, share v11→12.
  • npm run backend:status -- --project-ref brifeyypjauyusmkptkx --check now reports "Nothing pending" and exits 0.
  • Staging still lacks STRIPE_SECRET_KEY, STRIPE_WEBHOOK_SECRET and ELEVENLABS_API_KEY. I didn't touch secrets.

The CI demo code exists only in the GitHub staging environment. For the local runs I extended the expiry of an older staging test code ("Prompt evaluation PR42") by a few hours, and on 2026-10-03 by another 2 hours for the re-check (13 of 20 uses). I'll restore its original expires_at (2026-09-10 15:57 UTC) once the main run is green.

Verification

Local runs against staging used CI=1 PORT=18080 and --project=staging --retries=0:

  • unsynced staging: 2 passed (23.7 s);
  • forced failure (a temporary bad Ask locator, reverted): failed after 30 s and logged Staging step "complete one AI request in Ask" failed at path /app/ask. Cleanup still ran, leaving 0 debates, sources, chats and current pointers for the test user;
  • synced staging: 2 passed (33.5 s);
  • 36aba00: 2 passed (23.0 s);
  • 16b2f79 (hydration fix, Node 22): 2 passed (24.6 s). The run before it got through every step but failed the console-error assertion on net::ERR_NETWORK_CHANGED / ERR_SOCKET_NOT_CONNECTED, while another worktree's Docker stack was starting. I left the assertion as it is and reran. Both runs left 0 debates, sources, chats and current pointers for the test user.

GitHub Actions is blocked by billing. The database-tests rerun was not started ("recent account payments have failed or your spending limit needs to be increased"). Its first attempt at 36aba00 ran and failed one test that this PR doesn't touch: [mobile-chromium] full-stack/first-run.spec.ts:79 [hu], where the motion dialog never opened after the tour's start button. That spec passed on main at 34d1573. Because of the billing block, here are the local results at 16b2f79 (Node 22). They are a local pass, not a CI pass:

  • npm run verify: docs, lint, typecheck, unit with coverage, collaboration, edge (341 passed) and build all pass. test:sw timed out once while two browser suites shared the machine and passed on rerun. Browser E2E: 245 passed, 4 flaky, 1 failed (quality-backlog.spec.ts:20 [hu], a timeout). That spec then passed 12/12 alone with --retries=0. The desktop and mobile projects ignore e2e/staging/, so this PR's spec change can't reach them.
  • Database gate: see the comment below.

After merging I'll dispatch staging-e2e.yml on main once Quality is green for the merge commit. That run is the CI proof; this PR's checks don't run the staging job.

Proposal: redacted ARIA snapshot on failure (not implemented)

Playwright already writes an unredacted page ARIA snapshot to test-results/**/error-context.md on every failure, even with traces, screenshots and video off. It stays on the runner today because the workflow uploads nothing. It can include staging account text such as the test email, motions and chat content.

If the step and path line isn't enough, a safe next step could print a redacted snapshot into the log: run page.locator('main').ariaSnapshot(), keep only roles, test IDs, and the accessible names of buttons and links, and drop all text nodes, input values and anything that looks like an email. I'd hold off until a failure shows the step and path line isn't enough.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • Tests

    • Expanded staging end-to-end coverage for creating debates and sources, verifying data after reload, and completing an AI request.
    • Improved failure reports to identify the failed test step and page path. Added cleanup for test-created content and clearer action timeouts.
  • Documentation

    • Updated staging status guidance with recent run history and troubleshooting advice, including checking for outdated test steps when local and staging results differ.
    • Clarified staging test setup, execution, and failure-reporting instructions.

Since #72, Library disables Add Source until a debate is current. The
staging spec never started one, so every run since Actions came back on
2026-10-01 waited out its 180-second timeout on a disabled button.

The spec now starts a uniquely named debate from Library, sweeps debates
an interrupted run left behind, and deletes its debate with the test
user's own session after deleting the source through the UI. Actions
time out after 30 seconds so cleanup still runs, and a failure logs the
step name and URL path only.

Docs: record the real cause in operations.md and describe the new suite
steps, port and failure output in testing.md.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings October 2, 2026 20:51

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@vercel

vercel Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
debate-navigator Ready Ready Preview Oct 3, 2026 6:52am UTC
debatelab-staging Ready Ready Preview Oct 3, 2026 6:52am UTC

@coderabbitai

coderabbitai Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Free
  • Run ID: 1f4b10e5-4096-474f-bfe5-96120cae100f
📥 Commits

Reviewing files that changed from the base of the PR and between 36aba00 and 18741d5.

📒 Files selected for processing (3)
  • docs/operations.md
  • docs/testing.md
  • e2e/staging/authed-flows.spec.ts
 _______________________________________________________
< Your cache invalidation strategy is 'vibes and hope'. >
 -------------------------------------------------------
  \
   \   (\__/)
       (•ㅅ•)
       /   づ
ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Free
  • Run ID: 00e7c136-3bb1-45d6-8772-176b0e9114d0
📥 Commits

Reviewing files that changed from the base of the PR and between 34d1573 and 36aba00.

📒 Files selected for processing (3)
  • docs/operations.md
  • docs/testing.md
  • e2e/staging/authed-flows.spec.ts

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

The staging E2E test now creates and removes a uniquely named debate and source, verifies persisted state after reload, and completes a bounded Ask request. The operations and testing documentation describe the updated flow, failure diagnostics, and local-run requirements.

Changes

Staging E2E Flow

Layer / File(s) Summary
Test setup and debate creation
e2e/staging/authed-flows.spec.ts, docs/operations.md
The test adds action timeouts, step-level failure details, authenticated REST cleanup helpers, and uniquely named debate creation. Operations guidance records staging failures and recommends running the UI-driven suite locally before attributing failures to staging.
Source, Ask, and cleanup flow
e2e/staging/authed-flows.spec.ts, docs/testing.md
The test creates and verifies a source, checks debate and source hydration after reload, makes an Ask request with a 90-second response timeout, and cleans up the source and debate. Testing documentation describes the flow, local-run requirements, timeouts, and failure logging.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Test as Staging E2E test
  participant UI as Library UI
  participant REST as Supabase REST API
  participant AI as AI endpoint
  Test->>REST: Delete debates with the suite prefix
  Test->>UI: Create uniquely named debate
  UI->>Test: Return debate creation response and ID
  Test->>UI: Add source
  Test->>UI: Clear local state and reload
  Test->>UI: Verify debate and source
  Test->>AI: Send bounded Ask request
  AI->>Test: Return response
  Test->>UI: Delete source
  Test->>REST: Delete created debate by ID
Loading

Security Architecture Review

Security architecture risk: ⚪ Minimal · up to 36aba

The cleanup uses the staging user's existing permissions, and database ownership checks constrain deletion to that user's data. Production targeting is rejected, and protected CI runs are serialized. No material security risk introduced or worsened by this change was identified.

Retained concerns
No architecture-level concerns identified.

Security review details

Security Blast Radius

  • observed — The exact-ID helper can delete an addressed debate belonging to the authenticated staging user. The startup sweep can delete every debate for that user whose motion begins with the suite prefix, including cascaded dependents; its scope is broader than one run but remains owner-scoped.

Trust Boundaries and Controls

  • observed — The browser session becomes the REST request identity without a privilege transition. Database RLS evaluates auth.uid() for debate access, while composite owner/debate foreign keys prevent cross-owner dependent relationships.

Resilience and Maintainability Implications

  • inferred — Unique names alone would not isolate overlapping runs using the same account because recovery matches their common prefix. Existing CI serialization and the requirement for local runs to use their own bounded staging code counter that exposure in the documented execution paths.

Hardening Proposals

  • proposed — Preserve exclusive account use for destructive staging runs. If concurrent execution is introduced, use separate test identities or an account-scoped lease rather than relying on unique motion suffixes to protect active records from the recovery sweep.
  • Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Autopilot is currently an internal CodeRabbit preview.


Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Essentials by visiting https://app.coderabbit.ai/settings/billing.

Comment @coderabbitai help to get the list of available commands.

@Programcode

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@Programcode

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@Programcode

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Oct 2, 2026 •

Copy link
Copy Markdown
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

The Current Debate store persists under its own key, so clearing only the
session store let the restored motion come from localStorage. Clear both
stores, wait for the sources, debates and current_debates reads, and check
the current-debate indicator too.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@Programcode

Copy link
Copy Markdown
Owner Author

Local results at 16b2f79. GitHub Actions refused every job at this head ("recent account payments have failed or your spending limit needs to be increased"), so these are local passes, not CI passes.

  • ./.superset/db-gate.sh (Node 22): pgTAP and the database scripts pass (19 files, 451 tests, Result: PASS). The full-stack browser journey reports 71 passed and 1 skipped, including [mobile-chromium] first-run.spec.ts:79 [hu], the one test that failed on GitHub at 36aba00. The gate stopped its stack afterwards.
  • Staging suite against brifeyypjauyusmkptkx: 2 passed (24.6 s), and the test user was left with 0 debates, sources, chats and current pointers.
  • npm run verify: see the PR description. Everything passed except two load-related timeouts, test:sw and quality-backlog [hu], and both passed when rerun alone.

@Programcode

Copy link
Copy Markdown
Owner Author

Merged origin/main (ddb8187, which includes #95 and #101) into the branch at 18741d5. It merged cleanly. Re-checked at 18741d5: docs:check, ESLint on e2e/staging/ and typecheck pass, and the staging suite against brifeyypjauyusmkptkx reports 2 passed (29.9 s). Local only; Actions is still blocked by billing. Note: #95 changed _shared/cors.ts and demo-code-login, which this PR did not deploy, so staging functions are again behind main by that change.

@Programcode
Programcode merged commit 5ef8b24 into main Oct 3, 2026
1 of 7 checks passed
@Programcode
Programcode deleted the fix/staging-e2e-current-debate branch October 3, 2026 06:52

This branch was successfully deployed

2 active deployments
Preview – debate-navigator — 18741d5f Deployed Oct 3, 2026 by vercel[bot]
Preview – debatelab-staging — 18741d5f Deployed Oct 3, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants