Skip to content

test(e2e): full-stack feature journeys, plus the bugs they found - #67

Merged
Programcode merged 5 commits into
mainfrom
t3code/comprehensive-feature-e2e-tests
Sep 23, 2026
Merged

Programcode merged 5 commits into
mainfrom
t3code/comprehensive-feature-e2e-tests

Conversation

@Programcode

@Programcode Programcode commented Sep 23, 2026 •

Copy link
Copy Markdown
Owner

Why

Case generation broke in production and nobody knew until users reported it. The existing browser tests mock the Supabase session and the ai Edge Function, so they can't see failures between the browser, Auth, and the functions.

What this adds

New Playwright journeys in e2e/full-stack/, run by ./.superset/db-gate.sh (the CI database-tests job) on desktop and mobile Chromium.

  • Each test signs in through the real Login page with a fresh demo code, which calls the real demo-code-login function.
  • Local Supabase runs Auth, Postgres/RLS, and every Edge Function.
  • Only OpenRouter is replaced. The fake is a strict container (e2e/full-stack/fake-openrouter.mjs) that rejects prompts it doesn't recognize and records what the functions sent upstream. The functions reach it through a new OPENROUTER_API_URL override. It is unset in production, and the default stays https://openrouter.ai/api.
  • Each test asserts three things: what the page renders, what reached the provider, and which database rows were written. Tests also fail on any uncaught page error or horizontal page overflow.
  • In CI the specs throw instead of skipping when the stack env is missing, so a misconfigured gate can't go green.

Coverage:

  • Sign-in: demo code, reload, sign-out, a rejected code, and an expired token that refreshes before an AI request.
  • Generate: grounded in a large Library source, a provider failure with retry, a zero quota, and a saved case that loads from History in a fresh browser.
  • Other AI tools: Attack, Brainstorm, Factoids, and Ask. The streamed Ask test fails if the client silently falls back to the non-streaming path. Ask with citations is covered too.
  • Practice: a Lincoln-Douglas sparring round that is judged and saved, and a Crossfire debrief.
  • Account data: Library active/paused sources and deletion, output-language sync across devices, and flowchart saves.
  • Admin and sharing: an admin announcement that another user sees and dismisses, a non-admin denied the dashboard, and a signed-out share link.

Proof that it catches the case-generation bug

With the pre-#66 client (653aeac^ versions of client.ts, streaming.ts, and AskPage.tsx) the grounded case-generation test fails: no case appears. On current code it passes. I could not reproduce the #65 auth-relock hang locally. I tried an expired token, a tab visibility change, generating right after login, and a second tab, all against pre-#65 code. This suite does not claim to cover #65.

Bugs the new tests found (fixed here)

  1. Settings never reached the account. The 2026-08-19 hardening revoked INSERT on user_settings. The client saved with upsert, which needs INSERT, so every save failed with permission denied, and the failure was only a console warning. Affected: UI language, AI output language, theme, per-feature model choices, and the custom system prompt. The hardening grant also left out output_language. Fix: the client now updates the row the auth trigger provisions, and migration 20260923190000 grants UPDATE (output_language). Includes a pgTAP privileges test.

  2. Mobile pages widened past the screen.

    • Sonner's phone toaster stuck out by its left offset.
    • The Library grid let a source row stretch the card to about 660px.
    • The Flowchart toolbar didn't wrap, so the page was 965px wide on a 393px phone.

    Each problem pushed controls off-screen; the Flowchart one also made the save dialog's button untappable.

Verification (local, commit 8bace5b)

  • ./.superset/db-gate.sh: pgTAP (10 files, 299 tests) and 43 browser tests passed. 1 skip is pre-existing (mobile, workspace spec).
  • npm run docs:check, lint, typecheck: pass.
  • npm test -- --coverage: 1033 passed.
  • test:collaboration: 30 passed.
  • test:edge: 238 passed.
  • build and test:sw: pass.
  • Credential-free test:e2e: 226 passed, 6 skipped (unchanged).
  • WebKit: 112 passed, 4 skipped.

Not covered

Live provider output, web research (no search key locally), Jev decisions (the fake returns 503 and callers use their fallbacks), Stripe, Google export, and deploy drift between main and production.

Release notes

  • Migration 20260923190000_user_settings_output_language_grant.sql is forward-only. Rollback: REVOKE UPDATE (output_language) ON public.user_settings FROM authenticated. Apply it before the frontend deploy.
  • The Edge Function changes do nothing in production, so no function deploy is needed.

🤖 Generated with Claude Code

Summary by CodeRabbit

  • New Features

    • Added account-wide output-language preferences that can be updated and applied on a new device.
    • Added mobile-friendly toast notifications and more flexible layouts for flowchart actions.
  • Bug Fixes

    • Improved persistence of existing account settings.
  • Tests

    • Expanded full-stack browser coverage for authentication, AI tools, case generation, practice, sharing, account data, and admin announcements, using a simulated AI provider.

RetriggerConfidence Score: 5/5

Safe to merge.

Summary

This update makes the local database gate more resilient to transient container-registry failures by retrying Supabase startup with cleanup and bounded backoff. No outstanding issues were identified.

Reviews (2) · Last reviewed commit: "ci(db-gate): retry supabase start after ..."

Programcode and others added 4 commits September 23, 2026 19:47
Since the 2026-08-19 hardening revoked INSERT on user_settings, the client's
upsert was always denied, so language, output language, theme, model choices,
and the custom system prompt never left the browser. The hardening grant also
omitted output_language. Update the provisioned row instead and grant the
missing column.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sonner's mobile toaster overflowed by its left offset, the Library grid let a
source row's min-content width stretch the card, and the Flowchart toolbar did
not wrap. Each widened the phone layout viewport and pushed controls off-screen.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Add full-stack browser journeys that sign in through the real demo-code flow
and exercise Auth, RLS, quota, every Edge Function, streaming, and persistence
against local Supabase. Only OpenRouter is replaced, by a strict fake container
reached through the new OPENROUTER_API_URL override. The case-generation test
fails against the pre-#66 client, which silently dropped oversized requests.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Also warm the admin and announcements functions before the browser run.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 23, 2026 19:55
@vercel

vercel Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
debate-navigator Ready Ready Preview Sep 23, 2026 8:17pm UTC

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@coderabbitai

coderabbitai Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Warning

Review limit reached

Next included review available in 15 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Free

Run ID: 3c8e41f7-99d5-4ac6-a7ef-8ba3bb2f4def

📥 Commits

Reviewing files that changed from the base of the PR and between 8bace5b and 6e02fb5.

📒 Files selected for processing (1)
  • .superset/db-gate.sh
ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Free

Run ID: 441c3d3a-077c-411e-afd1-252df629c7ee

📥 Commits

Reviewing files that changed from the base of the PR and between 653aeac and 8bace5b.

📒 Files selected for processing (29)
  • .superset/db-gate.sh
  • .superset/workspace-test.env
  • docs/testing.md
  • e2e/full-stack/account-data.spec.ts
  • e2e/full-stack/admin.spec.ts
  • e2e/full-stack/ai-tools.spec.ts
  • e2e/full-stack/auth.spec.ts
  • e2e/full-stack/case-generation.spec.ts
  • e2e/full-stack/fake-openrouter.mjs
  • e2e/full-stack/practice.spec.ts
  • e2e/full-stack/public.spec.ts
  • e2e/full-stack/stack.ts
  • package.json
  • playwright.config.ts
  • playwright.workspace.config.ts
  • src/index.css
  • src/lib/stores/settings-store.ts
  • src/pages/FlowchartPage.tsx
  • src/pages/LibraryPage.tsx
  • src/test/security-hardening.test.ts
  • src/test/stores/settings-store.test.ts
  • supabase/README.md
  • supabase/functions/_shared/cors.ts
  • supabase/functions/_shared/jev.ts
  • supabase/functions/_shared/research.ts
  • supabase/functions/ai-stream/index.ts
  • supabase/functions/ai/index.ts
  • supabase/migrations/20260923190000_user_settings_output_language_grant.sql
  • supabase/tests/user_settings_privileges_test.sql

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

The pull request adds a local full-stack browser test gate with a fake OpenRouter provider and tests for account, AI, practice, admin, and public journeys. It also changes settings persistence and permissions, and adjusts responsive layouts.

Changes

Full-stack browser tests

Layer / File(s) Summary
Configure the local test stack
.superset/db-gate.sh, .superset/workspace-test.env, e2e/full-stack/fake-openrouter.mjs, e2e/full-stack/stack.ts, supabase/functions/..., package.json, playwright.config.ts, playwright.workspace.config.ts, docs/testing.md, supabase/README.md
The local gate starts a fake OpenRouter provider and serves all Supabase functions. The fake provider records requests and returns deterministic responses or configured failures. The AI endpoints and Jev decisions endpoint use the shared configurable OpenRouter base URL. The workspace Playwright configuration includes full-stack tests, and the documentation describes their coverage and limits.
Persist account settings with scoped updates
src/lib/stores/settings-store.ts, supabase/migrations/20260923190000_user_settings_output_language_grant.sql, supabase/tests/user_settings_privileges_test.sql, src/test/stores/settings-store.test.ts
Settings setters update the existing user_settings row by user_id. A migration grants authenticated users permission to update output_language; a privilege test checks permitted preference updates and restricted entitlement, insert, and delete operations.
Test authentication and account journeys
e2e/full-stack/auth.spec.ts, e2e/full-stack/account-data.spec.ts, e2e/full-stack/admin.spec.ts, e2e/full-stack/public.spec.ts
Tests cover demo-code sign-in and sign-out, account source and output-language behavior, flowchart persistence, admin access and announcement dismissal, and shared-content pages.
Test AI, case generation, and practice journeys
e2e/full-stack/ai-tools.spec.ts, e2e/full-stack/case-generation.spec.ts, e2e/full-stack/practice.spec.ts
Tests check AI tool results and provider requests, case generation with source and quota conditions, failure and retry, persistence, and Lincoln-Douglas and Crossfire practice flows.

Responsive layout adjustments

Layer / File(s) Summary
Adjust mobile and wrapping layouts
src/index.css, src/pages/FlowchartPage.tsx, src/pages/LibraryPage.tsx
Toasts use automatic container width and full-width toast elements on narrow screens. Flowchart header items and action buttons can wrap. The Library source list uses a single-column grid.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Playwright
  participant Vite
  participant SupabaseFunctions
  participant FakeOpenRouter
  Playwright->>Vite: Drive an AI feature journey
  Vite->>SupabaseFunctions: Send an AI request
  SupabaseFunctions->>FakeOpenRouter: Send chat-completions request
  FakeOpenRouter-->>SupabaseFunctions: Return deterministic response or configured failure
  SupabaseFunctions-->>Vite: Return response or stream
  Vite-->>Playwright: Render feature result
Loading

Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Essentials by visiting https://app.coderabbit.ai/settings/billing.

Comment @coderabbitai help to get the list of available commands.

Both database-tests attempts on this PR failed while pulling Supabase images
from ghcr.io with toomanyrequests, before any test ran. Retry the start up to
three times and stop a partially started stack between attempts.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@Programcode

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@Programcode

Copy link
Copy Markdown
Owner Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@Programcode
Programcode merged commit 7df7aa5 into main Sep 23, 2026
11 of 12 checks passed
@Programcode
Programcode deleted the t3code/comprehensive-feature-e2e-tests branch September 23, 2026 20:47

This branch was successfully deployed

1 active deployment
Preview — 6e02fb56 Deployed Sep 23, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants