From 885e6f0f3586c67682c13263fd0ec80b496c66ff Mon Sep 17 00:00:00 2001 From: Adam Date: Wed, 7 Oct 2026 14:57:18 +0000 Subject: [PATCH 1/3] fix: close the open findings from the 2026-09-24 site audit - AUD-001: the landing page groups its sections in a
landmark. - AUD-002: the build writes /terms and /privacy copies of the app shell whose canonical and og:url name their own route. Vercel serves them before the SPA fallback, so no canonical changes in JavaScript. - AUD-003: the login title is an h1 styled like the CardTitle it replaces. - AUD-004: a missing or expired share uses maybeSingle(), so PostgREST answers 200 with no row instead of 406. - AUD-005: the 404 page no longer logs console.error. - AUD-006: vercel.json sends no-store for / as it does for /index.html. Regression coverage: the public page sweep visits an unknown route, the agentic sweep visits a missing share and an unknown route and records PostgREST 406s, and the full-stack page fixture fails on console.error and failed local requests. Co-Authored-By: Claude Opus 5.5 (1M context) --- .claude/skills/verify-debatelab/SKILL.md | 2 +- AGENTS.md | 2 +- agentic/pages.e2e.ts | 18 ++++++++++-- agentic/support/capture-errors.js | 11 ++++--- agentic/support/fixtures.ts | 4 +-- docs/operations.md | 4 ++- docs/testing.md | 4 +-- e2e/full-stack/stack.ts | 20 +++++++++++-- e2e/public-pages.spec.ts | 2 ++ src/pages/LandingPage.tsx | 22 +++++++------- src/pages/LoginPage.tsx | 5 ++-- src/pages/NotFound.tsx | 7 ----- src/pages/SharedPage.tsx | 3 +- src/test/vercel-deployment-config.test.ts | 2 +- vercel.json | 9 ++++++ vite.config.ts | 36 ++++++++++++++++++++++- 16 files changed, 114 insertions(+), 37 deletions(-) diff --git a/.claude/skills/verify-debatelab/SKILL.md b/.claude/skills/verify-debatelab/SKILL.md index c23f5c4cc..417460058 100644 --- a/.claude/skills/verify-debatelab/SKILL.md +++ b/.claude/skills/verify-debatelab/SKILL.md @@ -118,7 +118,7 @@ $H e2e cache ls # recorded agent flo ``` - `e2e` runs `doctor` first and refuses a broken stack, like `drive`. It turns e2e telemetry off and prints `EVIDENCE `, which holds `summary.md`, `report.json`, and per-attempt screenshots, traces, and failure videos. -- The suite is tool journeys (tag `journey`) and a page sweep (tag `pages`) of every student and public page in English, Spanish and Hungarian (not `/app/admin` or `/share/:code`). Every test fails on an uncaught error, `console.error` (React warnings included), a Supabase 5xx, horizontal overflow, or a request to a hosted project: `agentic/support/capture-errors.js` records them in the page. A failure that names `errors the page logged` is a real app error to fix, not test noise. +- The suite is tool journeys (tag `journey`) and a page sweep (tag `pages`) of every student and public page in English, Spanish and Hungarian (not `/app/admin` or a valid `/share/:code`; a missing share and an unknown route are visited in English). Every test fails on an uncaught error, `console.error` (React warnings included), a Supabase 5xx or PostgREST 406, horizontal overflow, or a request to a hosted project: `agentic/support/capture-errors.js` records them in the page. A failure that names `errors the page logged` is a real app error to fix, not test noise. - The testing agent has its own OpenRouter models: `openai/gpt-6-luna` acts (test steps on a cache miss, and explorations) and `meta/muse-spark-1.3-contributor` judges. `E2E_AGENT_MODEL` and `E2E_JUDGE_MODEL` override them; the model is not part of the replay key. The key comes from `OPENROUTER_API_KEY`, `LIVE_OPENROUTER_API_KEY`, or the live keys file, and is needed in fake mode too. A recorded `agent.act` replays with no model call; every judgment and every exploration step is a call. Without a key, agent tests skip. An OpenRouter error such as `[Meta] The requested model was not found` is infrastructure: rerun with `--last-failed`. The Muse Spark judge cannot act. - `--session student` starts an exploration signed in as a fresh demo user, minted by `agentic/sessions.e2e.ts`. `--agent` picks a persona: `first-timer`, `coach`, `edge-input`, or `hungarian`. `--target mobile` explores the phone-sized viewport. - Explorations report `issue` (functional) and `warning` (cosmetic) findings, and exit 1 when there is an issue. Treat a finding as a lead: the stack serves Vite dev, so a route that is still compiling can look dead, and fake mode answers every motion with free-transport text. Prove a finding with a repro test tagged `repro` before you report it as a bug. diff --git a/AGENTS.md b/AGENTS.md index 544ee5c05..1d8050daf 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -98,7 +98,7 @@ $H down # always, including after failures - The stack holds the same lock as `./.superset/db-gate.sh`, so only one worktree can run it at a time. - Most tools work inside a Current Debate. Recipes start one with `startDebate` or answer the motion dialog with `chooseForMotion`; see the skill's Drive section. - Neither mode proves Stripe Checkout, real Stripe cancellation, ElevenLabs voice audio, Google sign-in or export, staging, or production. -- Agentic tests (`agentic/*.e2e.ts`) are an AI agent driving goals and judging screens, with exact assertions on the result. They cover a journey per tool and every student and public page in English, Spanish and Hungarian, and fail on page errors, `console.error`, and Supabase 5xx answers. They catch what selectors cannot: stray untranslated text, clipped layouts, off-topic model output in live mode. They also run explorations that hunt for bugs as a persona. The agent calls its own OpenRouter models (GPT-6 Luna acts, the Muse Spark contributor tier judges), even on a fake stack. A flow change that breaks an agentic test updates it in the same PR; commit new or re-recorded `agentic/replay/` entries with it. See [docs/testing.md](docs/testing.md#agentic-browser-tests). +- Agentic tests (`agentic/*.e2e.ts`) are an AI agent driving goals and judging screens, with exact assertions on the result. They cover a journey per tool and every student and public page in English, Spanish and Hungarian, and fail on page errors, `console.error`, Supabase 5xx answers, and PostgREST 406s. They catch what selectors cannot: stray untranslated text, clipped layouts, off-topic model output in live mode. They also run explorations that hunt for bugs as a persona. The agent calls its own OpenRouter models (GPT-6 Luna acts, the Muse Spark contributor tier judges), even on a fake stack. A flow change that breaks an agentic test updates it in the same PR; commit new or re-recorded `agentic/replay/` entries with it. See [docs/testing.md](docs/testing.md#agentic-browser-tests). ### Keep the skill working diff --git a/agentic/pages.e2e.ts b/agentic/pages.e2e.ts index 446531a29..d1684cc20 100644 --- a/agentic/pages.e2e.ts +++ b/agentic/pages.e2e.ts @@ -3,8 +3,9 @@ import type { Agent } from "e2e"; import { agentic, expect, LANGUAGES, localeTest, visitorTest, type Language } from "./support/fixtures"; // Every page a student or visitor reaches, in every interface language, on -// both viewports. Not covered: /app/admin (admins only) and /share/:code (no -// client code creates a share link yet). Selectors pin +// both viewports. Not covered: /app/admin (admins only) and a valid +// /share/:code (no client code creates a share link yet); a missing share and +// an unknown route are visited once below. Selectors pin // what they can (lang, a heading, no raw values); the judge reads the screen // for what they cannot: stray untranslated text, error notices, and clipped // or overlapping text. e2e/hungarian-parity.spec.ts and authed-pages.spec.ts @@ -66,3 +67,16 @@ for (const language of LANGUAGES) { }); } } + +// Expected misses show their notice without logging anything: the visitor +// fixture fails the test on a console error or on the 406 PostgREST answers +// when the page asks for exactly one share row and there is none. +const englishVisitor = visitorTest("en"); +englishVisitor("a missing share link says so without logging an error", { ...agentic, tags: ["pages", "en"] }, async ({ browser, visitor }) => { + await visitor.open("/share/agentic-missing-share"); + await expect(browser.locator("body")).toContainText("This shared content is no longer available"); +}); +englishVisitor("an unknown route shows the 404 page without logging an error", { ...agentic, tags: ["pages", "en"] }, async ({ browser, visitor }) => { + await visitor.open("/agentic-no-such-page"); + await expect(browser.locator("h1")).toHaveText("404"); +}); diff --git a/agentic/support/capture-errors.js b/agentic/support/capture-errors.js index 9fd06bb5d..56913fd04 100644 --- a/agentic/support/capture-errors.js +++ b/agentic/support/capture-errors.js @@ -1,7 +1,8 @@ // Runs before the app's own scripts in every document (web({ initScripts })). // It records what the page hides from the screen: uncaught errors, rejected -// promises, console.error, and 5xx answers from Supabase. The student fixture -// reads the list at teardown and fails the test on anything it does not expect. +// promises, console.error, 5xx answers from Supabase, and PostgREST 406s. The +// student fixture reads the list at teardown and fails the test on anything it +// does not expect. // sessionStorage keeps the list across reloads and navigations in the tab. (() => { const KEY = "__agentic_errors"; @@ -32,9 +33,11 @@ const response = await nativeFetch(input, init); // Recording must never change what the app's request returns. try { - if (response.status >= 500) { + // PostgREST answers 406 when a query asks for exactly one row + // (.single()) and gets none; a lookup that can miss uses .maybeSingle(). + if (response.status >= 500 || response.status === 406) { const url = new URL(input instanceof Request ? input.url : String(input), location.href); - if (/^\/(functions|rest|auth)\//.test(url.pathname)) { + if (response.status >= 500 ? /^\/(functions|rest|auth)\//.test(url.pathname) : url.pathname.startsWith("/rest/")) { const method = (init && init.method) || (input instanceof Request ? input.method : "GET"); record("http", `${response.status} ${method.toUpperCase()} ${url.pathname}`); } diff --git a/agentic/support/fixtures.ts b/agentic/support/fixtures.ts index 856891aba..c9015ee3d 100644 --- a/agentic/support/fixtures.ts +++ b/agentic/support/fixtures.ts @@ -66,8 +66,8 @@ async function signIn(screen: Screen, browser: Browser, code: string, language: /** * What `support/capture-errors.js` recorded in this tab: uncaught errors, - * rejected promises, console.error, and Supabase 5xx answers. Reading clears - * the list. + * rejected promises, console.error, Supabase 5xx answers, and PostgREST 406s. + * Reading clears the list. */ export async function pageErrors(browser: Browser): Promise { return browser.evaluate(() => { diff --git a/docs/operations.md b/docs/operations.md index 143711fbc..2db27e8d8 100644 --- a/docs/operations.md +++ b/docs/operations.md @@ -42,10 +42,12 @@ Vercel builds and serves the frontend from the Git repository. Merging to `main` - the Vite build command and `dist` output; - SPA fallback to `index.html`; -- no-cache headers for `index.html`, `theme-init.js`, and `sw.js`, and immutable caching for hashed `/assets/`; +- no-cache headers for `/`, `index.html`, `theme-init.js`, and `sw.js`, and immutable caching for hashed `/assets/`; - `noindex` headers for `/app`, `/login`, and `/share` routes; - Content Security Policy and the other response security headers, including the `Permissions-Policy` that allows the microphone for the app's own origin only. +The build also writes `dist/terms/index.html` and `dist/privacy/index.html`: copies of the app shell whose canonical URL names their own route instead of the landing page (`selfCanonicalShells` in [`vite.config.ts`](../vite.config.ts)). Vercel serves them before the SPA fallback. Add a route there when a new SPA page joins `public/sitemap.xml`. + [`src/test/vercel-deployment-config.test.ts`](../src/test/vercel-deployment-config.test.ts) guards these rules. Update it together with `vercel.json`, and do not paste the full configuration into another guide. A Vercel deploy ships only the frontend. Supabase migrations and Edge Functions are separate operations with their own authorization, so a frontend that depends on new schema must wait until that schema is applied to production. diff --git a/docs/testing.md b/docs/testing.md index 4ca4e2b0e..cd635deff 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -201,7 +201,7 @@ Each test signs in through the Login page with a fresh demo code and deletes tha - a promo code redeemed in Settings that restores AI access, and signed Stripe webhooks that grant and revoke Pro while a forged or replayed one changes nothing; - admin announcements seen by another user, denied admin access, and a public share link. -Tests assert the rendered result, what the Edge Function sent upstream (read back from the fake provider), and the resulting database rows. They also fail on any uncaught page error or horizontal page overflow. The fake provider rejects prompts it does not recognize, so a prompt change must update it. +Tests assert the rendered result, what the Edge Function sent upstream (read back from the fake provider), and the resulting database rows. They also fail on any uncaught page error, `console.error`, failed request (4xx or 5xx) to the local app or stack, or horizontal page overflow. A spec that causes a failure on purpose names it with `test.use({ tolerate: [...] })`. The fake provider rejects prompts it does not recognize, so a prompt change must update it. These journeys do not prove live provider behavior, web research (the local stack has no search key), real Jev judgement (the fake answers every Jev question with a fixed confident choice), Stripe Checkout, the billing portal, and subscription cancellation during account deletion (they call the Stripe API; the local key is a placeholder, and the Deno tests cover the cancellation logic), the `checkout.session.completed` and `invoice.payment_succeeded` webhooks (they fetch the subscription from Stripe), or Google export. @@ -254,7 +254,7 @@ $H down - Journeys (tag `journey`), one goal per tool, that survive copy and layout changes: the onboarding tour starting a first debate, Generate and reopening the case from History, Attack, Brainstorm filed in a new debate, Factoids, Ask, a full Sparring round to its verdict, Crossfire to its debrief, adding and pausing a Library source, saving a Flowchart, the speech Timer, and switching interface and AI output language in Settings. Each is pinned by a locator, the rows the app wrote (`ai_requests`, `brainstorms`, `cases`, `sparring_sessions`, `crossfires`, `sources`, `flowcharts`, `user_settings`), and in fake mode the provider's `FAKE-PROVIDER` marker on screen. - The page sweep (`pages.e2e.ts`, tag `pages`): the 14 student pages and 5 public pages, in English, Spanish and Hungarian, on both viewports. It skips `/app/admin`, which needs an admin, and `/share/:code`, because no client code creates a share link yet. Selectors check `lang`, a heading, and that no `undefined`, `NaN` or `[object Object]` leaked into the page. The judge checks for error notices, raw translation keys, untranslated interface text (which `e2e/hungarian-parity.spec.ts` deliberately cannot reject), and clipped or overlapping text. -- Hidden failures. [`agentic/support/capture-errors.js`](../agentic/support/capture-errors.js) runs before the app in every document and records uncaught errors, rejected promises, `console.error` (which includes React's warnings and the route error boundary), and 5xx answers from Supabase. Every test fails on any of them, on a request to a hosted Supabase project, on a page wider than its viewport, and on visible text that runs past the screen edge (which an `overflow-x: hidden` wrapper would otherwise hide). Text clipped away, cut short by an ellipsis, or inside a sideways scroller does not count; [`agentic/edge-guard.e2e.ts`](../agentic/edge-guard.e2e.ts) pins those rules on injected layouts. +- Hidden failures. [`agentic/support/capture-errors.js`](../agentic/support/capture-errors.js) runs before the app in every document and records uncaught errors, rejected promises, `console.error` (which includes React's warnings and the route error boundary), 5xx answers from Supabase, and PostgREST 406s (a `.single()` lookup that found no row). Every test fails on any of them, on a request to a hosted Supabase project, on a page wider than its viewport, and on visible text that runs past the screen edge (which an `overflow-x: hidden` wrapper would otherwise hide). Text clipped away, cut short by an ellipsis, or inside a sideways scroller does not count; [`agentic/edge-guard.e2e.ts`](../agentic/edge-guard.e2e.ts) pins those rules on injected layouts. - In live mode (`up --live`), whether model output is on topic: the generated case argues the motion, and factoids carry sources. Recipes never assert model wording. - Exploration. `e2e explore` sends a persona through the app with a goal and reports findings with expected and observed behavior, reproduction steps, and screenshots. The personas are the named agents in `e2e.config.ts`: `first-timer`, `coach`, `edge-input`, and `hungarian`. diff --git a/e2e/full-stack/stack.ts b/e2e/full-stack/stack.ts index 041eb8d78..24ac75b43 100644 --- a/e2e/full-stack/stack.ts +++ b/e2e/full-stack/stack.ts @@ -49,9 +49,12 @@ export async function chooseForMotion(page: Page, choice: "create" | "once") { await expect(dialog).toBeHidden(); } -type Fixtures = { user: DemoUser; signedIn: Page }; +type Fixtures = { user: DemoUser; signedIn: Page; tolerate: RegExp[] }; export const test = base.extend({ + // Problems a spec causes on purpose, such as a provider failure, matched + // against the lines the page fixture collects. Set with test.use(). + tolerate: [[], { option: true }], // Playwright requires a destructuring pattern even when a fixture has no dependencies. // eslint-disable-next-line no-empty-pattern user: async ({}, provide, testInfo) => { @@ -59,10 +62,22 @@ export const test = base.extend({ await provide(created); await deleteDemoUser(created); }, - page: async ({ page }, provide) => { + page: async ({ page, tolerate }, provide) => { // Any uncaught exception fails the test, even when the visible flow passes. const errors: string[] = []; page.on("pageerror", (error) => errors.push(error.message)); + // So does a console error or a failed request to the local app or stack: + // an expected miss, like an unknown share code, must not log one. + const problems: string[] = []; + page.on("console", (message) => { + if (message.type() === "error") problems.push(`console.error: ${message.text()}`); + }); + page.on("response", (response) => { + const url = new URL(response.url()); + if (response.status() >= 400 && /^(127\.0\.0\.1|localhost|\[::1\])$/.test(url.hostname)) { + problems.push(`${response.status()} ${response.request().method()} ${url.pathname}`); + } + }); // Init scripts rerun on every load, so seed only what the app has not written. await page.addInitScript(() => { const seed = (key: string, state: Record) => { @@ -74,6 +89,7 @@ export const test = base.extend({ await fake.reset(); await provide(page); expect(errors, "uncaught page errors").toEqual([]); + expect(problems.filter((line) => !tolerate.some((pattern) => pattern.test(line))), "console errors and failed local requests").toEqual([]); // Overflow widens the mobile layout viewport and pushes controls off-screen. const overflow = await page.evaluate(() => document.documentElement.scrollWidth - document.documentElement.clientWidth); expect(overflow, "horizontal page overflow in px").toBeLessThanOrEqual(1); diff --git a/e2e/public-pages.spec.ts b/e2e/public-pages.spec.ts index 63f602f68..3e97e1597 100644 --- a/e2e/public-pages.spec.ts +++ b/e2e/public-pages.spec.ts @@ -7,6 +7,8 @@ const PUBLIC_ROUTES = [ { path: '/terms', name: 'terms' }, { path: '/privacy', name: 'privacy' }, { path: '/login', name: 'login' }, + // An unknown route renders the 404 page, which must not log an error. + { path: '/no-such-page', name: 'not-found' }, ] as const; for (const locale of LOCALES) { diff --git a/src/pages/LandingPage.tsx b/src/pages/LandingPage.tsx index f3e826807..9c2afa324 100644 --- a/src/pages/LandingPage.tsx +++ b/src/pages/LandingPage.tsx @@ -15,16 +15,18 @@ const LandingPage = () => { return (
- - - - - - - - - - +
+ + + + + + + + + + +
); diff --git a/src/pages/LoginPage.tsx b/src/pages/LoginPage.tsx index 3f1c02a6e..827b969f1 100644 --- a/src/pages/LoginPage.tsx +++ b/src/pages/LoginPage.tsx @@ -2,7 +2,7 @@ import { useEffect, useRef, useState } from 'react'; import { useLocation, useNavigate } from 'react-router-dom'; import { useTranslation } from 'react-i18next'; import { Button } from '@/components/ui/button'; -import { Card, CardContent, CardDescription, CardHeader, CardTitle } from '@/components/ui/card'; +import { Card, CardContent, CardDescription, CardHeader } from '@/components/ui/card'; import { Input } from '@/components/ui/input'; import { Label } from '@/components/ui/label'; import { Tabs, TabsContent, TabsList, TabsTrigger } from '@/components/ui/tabs'; @@ -153,7 +153,8 @@ export default function LoginPage() {
- {t('login.title')} + {/* The page heading, styled like CardTitle (which renders an h3). */} +

{t('login.title')}

{t('login.subtitle')} diff --git a/src/pages/NotFound.tsx b/src/pages/NotFound.tsx index edba0062b..89d46bd1e 100644 --- a/src/pages/NotFound.tsx +++ b/src/pages/NotFound.tsx @@ -1,15 +1,8 @@ -import { useLocation } from "react-router-dom"; -import { useEffect } from "react"; import { useTranslation } from "react-i18next"; const NotFound = () => { - const location = useLocation(); const { t } = useTranslation(); - useEffect(() => { - console.error("404 Error: User attempted to access non-existent route:", location.pathname); - }, [location.pathname]); - return (
diff --git a/src/pages/SharedPage.tsx b/src/pages/SharedPage.tsx index 5efd8a393..ca5f83f8b 100644 --- a/src/pages/SharedPage.tsx +++ b/src/pages/SharedPage.tsx @@ -40,7 +40,8 @@ export default function SharedPage() { try { const { data, error: fetchError } = await supabase .rpc('get_shared_item', { p_share_code: code }) - .single(); + // A missing or expired code returns no row; .single() would turn that into an HTTP 406. + .maybeSingle(); if (fetchError || !data) { setError(t('sharedPage.contentNotFound')); diff --git a/src/test/vercel-deployment-config.test.ts b/src/test/vercel-deployment-config.test.ts index 2da68064f..1247cdddc 100644 --- a/src/test/vercel-deployment-config.test.ts +++ b/src/test/vercel-deployment-config.test.ts @@ -124,7 +124,7 @@ describe("Vercel frontend deployment configuration", () => { }, ); - it.each(["/index.html", "/theme-init.js", "/sw.js"])( + it.each(["/", "/index.html", "/theme-init.js", "/sw.js"])( "prevents stale caching for %s", (source) => { const config = loadConfig(); diff --git a/vercel.json b/vercel.json index 77295bebc..a9ca1e0b7 100644 --- a/vercel.json +++ b/vercel.json @@ -75,6 +75,15 @@ } ] }, + { + "source": "/", + "headers": [ + { + "key": "Cache-Control", + "value": "no-store, no-cache, must-revalidate, proxy-revalidate, max-age=0" + } + ] + }, { "source": "/index.html", "headers": [ diff --git a/vite.config.ts b/vite.config.ts index 3715316bb..add917c8d 100644 --- a/vite.config.ts +++ b/vite.config.ts @@ -2,6 +2,7 @@ import { defineConfig, loadEnv, type Plugin, type PluginOption } from "vite"; import react from "@vitejs/plugin-react-swc"; import { sentryVitePlugin } from "@sentry/vite-plugin"; import { execSync } from "node:child_process"; +import { mkdirSync, readFileSync, writeFileSync } from "node:fs"; import path from "path"; import { STAGING_PREVIEW_PROXY_PATH, SUPABASE_PROJECTS } from "./src/lib/supabase/projects"; @@ -54,6 +55,39 @@ function bundleGraph(): Plugin { }; } +/** + * The built index.html names the landing page as its canonical URL, and Vercel + * serves it for every route. Public routes listed in sitemap.xml get their own + * copy of the shell at `/index.html` naming themselves instead; Vercel + * serves a real file before the SPA rewrite. Google advises against changing a + * canonical from JavaScript, so this happens at build time. + */ +const SELF_CANONICAL_ROUTES = ["/terms", "/privacy"]; +const SITE_ORIGIN = "https://debate.pi-stack.org"; + +function selfCanonicalShells(): Plugin { + return { + name: "self-canonical-shells", + apply: (_config, env) => env.command === "build" && env.mode !== "capacitor", + writeBundle(options) { + const outDir = options.dir ?? "dist"; + const shell = readFileSync(path.join(outDir, "index.html"), "utf8"); + const canonical = ``; + const ogUrl = ``; + if (!shell.includes(canonical) || !shell.includes(ogUrl)) { + throw new Error("index.html no longer carries the landing canonical and og:url tags this plugin rewrites"); + } + for (const route of SELF_CANONICAL_ROUTES) { + const html = shell + .replace(canonical, ``) + .replace(ogUrl, ``); + mkdirSync(path.join(outDir, route), { recursive: true }); + writeFileSync(path.join(outDir, route, "index.html"), html); + } + }, + }; +} + /** * `npm run dev:preview` sets VITE_STAGING_PREVIEW=true, which makes the app call * Supabase at /__staging-supabase on its own origin (see src/lib/supabase/env.ts). Forward @@ -104,7 +138,7 @@ export default defineConfig(({ mode }) => { proxy, allowedHosts, }, - plugins: [react(), bundleGraph(), ...uploadSourceMaps], + plugins: [react(), bundleGraph(), selfCanonicalShells(), ...uploadSourceMaps], define: { "import.meta.env.VITE_SENTRY_RELEASE": JSON.stringify(release ?? ""), "import.meta.env.VITE_SENTRY_ENVIRONMENT": JSON.stringify(process.env.VERCEL_ENV ?? ""), From d761b3bdc50440dd8dcf340005ddc76e56e0a025 Mon Sep 17 00:00:00 2001 From: Adam Date: Wed, 7 Oct 2026 15:06:02 +0000 Subject: [PATCH 2/3] test(e2e): wait for each public page's h1 before reading its errors The pages are lazy chunks, so DOMContentLoaded can fire before the page mounts and logs anything. Co-Authored-By: Claude Opus 5.5 (1M context) --- e2e/public-pages.spec.ts | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/e2e/public-pages.spec.ts b/e2e/public-pages.spec.ts index 3e97e1597..b27d3d5c3 100644 --- a/e2e/public-pages.spec.ts +++ b/e2e/public-pages.spec.ts @@ -27,7 +27,9 @@ for (const locale of LOCALES) { const response = await page.goto(route.path, { waitUntil: 'domcontentloaded' }); expect(response?.ok()).toBeTruthy(); - await expect(page.locator('body')).toBeVisible(); + // Pages are lazy chunks: wait until the page itself has rendered, so + // errors it logs while mounting are in the list below. + await expect(page.locator('h1').first()).toBeVisible(); const fatalErrors = consoleErrors.filter((msg) => !msg.includes('Failed to load resource') && From b6d26e7e20a7e7227439137221e7111f671166ed Mon Sep 17 00:00:00 2001 From: Adam Date: Wed, 7 Oct 2026 15:44:00 +0000 Subject: [PATCH 3/3] chore: move stack-only regression coverage to its own PR The full-stack fixture and agentic sweep changes can only be proven on the shared local stack. They land separately from the audit fixes. Co-Authored-By: Claude Opus 5.5 (1M context) --- .claude/skills/verify-debatelab/SKILL.md | 2 +- AGENTS.md | 2 +- agentic/pages.e2e.ts | 18 ++---------------- agentic/support/capture-errors.js | 11 ++++------- agentic/support/fixtures.ts | 4 ++-- docs/testing.md | 4 ++-- e2e/full-stack/stack.ts | 20 ++------------------ 7 files changed, 14 insertions(+), 47 deletions(-) diff --git a/.claude/skills/verify-debatelab/SKILL.md b/.claude/skills/verify-debatelab/SKILL.md index 417460058..c23f5c4cc 100644 --- a/.claude/skills/verify-debatelab/SKILL.md +++ b/.claude/skills/verify-debatelab/SKILL.md @@ -118,7 +118,7 @@ $H e2e cache ls # recorded agent flo ``` - `e2e` runs `doctor` first and refuses a broken stack, like `drive`. It turns e2e telemetry off and prints `EVIDENCE `, which holds `summary.md`, `report.json`, and per-attempt screenshots, traces, and failure videos. -- The suite is tool journeys (tag `journey`) and a page sweep (tag `pages`) of every student and public page in English, Spanish and Hungarian (not `/app/admin` or a valid `/share/:code`; a missing share and an unknown route are visited in English). Every test fails on an uncaught error, `console.error` (React warnings included), a Supabase 5xx or PostgREST 406, horizontal overflow, or a request to a hosted project: `agentic/support/capture-errors.js` records them in the page. A failure that names `errors the page logged` is a real app error to fix, not test noise. +- The suite is tool journeys (tag `journey`) and a page sweep (tag `pages`) of every student and public page in English, Spanish and Hungarian (not `/app/admin` or `/share/:code`). Every test fails on an uncaught error, `console.error` (React warnings included), a Supabase 5xx, horizontal overflow, or a request to a hosted project: `agentic/support/capture-errors.js` records them in the page. A failure that names `errors the page logged` is a real app error to fix, not test noise. - The testing agent has its own OpenRouter models: `openai/gpt-6-luna` acts (test steps on a cache miss, and explorations) and `meta/muse-spark-1.3-contributor` judges. `E2E_AGENT_MODEL` and `E2E_JUDGE_MODEL` override them; the model is not part of the replay key. The key comes from `OPENROUTER_API_KEY`, `LIVE_OPENROUTER_API_KEY`, or the live keys file, and is needed in fake mode too. A recorded `agent.act` replays with no model call; every judgment and every exploration step is a call. Without a key, agent tests skip. An OpenRouter error such as `[Meta] The requested model was not found` is infrastructure: rerun with `--last-failed`. The Muse Spark judge cannot act. - `--session student` starts an exploration signed in as a fresh demo user, minted by `agentic/sessions.e2e.ts`. `--agent` picks a persona: `first-timer`, `coach`, `edge-input`, or `hungarian`. `--target mobile` explores the phone-sized viewport. - Explorations report `issue` (functional) and `warning` (cosmetic) findings, and exit 1 when there is an issue. Treat a finding as a lead: the stack serves Vite dev, so a route that is still compiling can look dead, and fake mode answers every motion with free-transport text. Prove a finding with a repro test tagged `repro` before you report it as a bug. diff --git a/AGENTS.md b/AGENTS.md index 1d8050daf..544ee5c05 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -98,7 +98,7 @@ $H down # always, including after failures - The stack holds the same lock as `./.superset/db-gate.sh`, so only one worktree can run it at a time. - Most tools work inside a Current Debate. Recipes start one with `startDebate` or answer the motion dialog with `chooseForMotion`; see the skill's Drive section. - Neither mode proves Stripe Checkout, real Stripe cancellation, ElevenLabs voice audio, Google sign-in or export, staging, or production. -- Agentic tests (`agentic/*.e2e.ts`) are an AI agent driving goals and judging screens, with exact assertions on the result. They cover a journey per tool and every student and public page in English, Spanish and Hungarian, and fail on page errors, `console.error`, Supabase 5xx answers, and PostgREST 406s. They catch what selectors cannot: stray untranslated text, clipped layouts, off-topic model output in live mode. They also run explorations that hunt for bugs as a persona. The agent calls its own OpenRouter models (GPT-6 Luna acts, the Muse Spark contributor tier judges), even on a fake stack. A flow change that breaks an agentic test updates it in the same PR; commit new or re-recorded `agentic/replay/` entries with it. See [docs/testing.md](docs/testing.md#agentic-browser-tests). +- Agentic tests (`agentic/*.e2e.ts`) are an AI agent driving goals and judging screens, with exact assertions on the result. They cover a journey per tool and every student and public page in English, Spanish and Hungarian, and fail on page errors, `console.error`, and Supabase 5xx answers. They catch what selectors cannot: stray untranslated text, clipped layouts, off-topic model output in live mode. They also run explorations that hunt for bugs as a persona. The agent calls its own OpenRouter models (GPT-6 Luna acts, the Muse Spark contributor tier judges), even on a fake stack. A flow change that breaks an agentic test updates it in the same PR; commit new or re-recorded `agentic/replay/` entries with it. See [docs/testing.md](docs/testing.md#agentic-browser-tests). ### Keep the skill working diff --git a/agentic/pages.e2e.ts b/agentic/pages.e2e.ts index d1684cc20..446531a29 100644 --- a/agentic/pages.e2e.ts +++ b/agentic/pages.e2e.ts @@ -3,9 +3,8 @@ import type { Agent } from "e2e"; import { agentic, expect, LANGUAGES, localeTest, visitorTest, type Language } from "./support/fixtures"; // Every page a student or visitor reaches, in every interface language, on -// both viewports. Not covered: /app/admin (admins only) and a valid -// /share/:code (no client code creates a share link yet); a missing share and -// an unknown route are visited once below. Selectors pin +// both viewports. Not covered: /app/admin (admins only) and /share/:code (no +// client code creates a share link yet). Selectors pin // what they can (lang, a heading, no raw values); the judge reads the screen // for what they cannot: stray untranslated text, error notices, and clipped // or overlapping text. e2e/hungarian-parity.spec.ts and authed-pages.spec.ts @@ -67,16 +66,3 @@ for (const language of LANGUAGES) { }); } } - -// Expected misses show their notice without logging anything: the visitor -// fixture fails the test on a console error or on the 406 PostgREST answers -// when the page asks for exactly one share row and there is none. -const englishVisitor = visitorTest("en"); -englishVisitor("a missing share link says so without logging an error", { ...agentic, tags: ["pages", "en"] }, async ({ browser, visitor }) => { - await visitor.open("/share/agentic-missing-share"); - await expect(browser.locator("body")).toContainText("This shared content is no longer available"); -}); -englishVisitor("an unknown route shows the 404 page without logging an error", { ...agentic, tags: ["pages", "en"] }, async ({ browser, visitor }) => { - await visitor.open("/agentic-no-such-page"); - await expect(browser.locator("h1")).toHaveText("404"); -}); diff --git a/agentic/support/capture-errors.js b/agentic/support/capture-errors.js index 56913fd04..9fd06bb5d 100644 --- a/agentic/support/capture-errors.js +++ b/agentic/support/capture-errors.js @@ -1,8 +1,7 @@ // Runs before the app's own scripts in every document (web({ initScripts })). // It records what the page hides from the screen: uncaught errors, rejected -// promises, console.error, 5xx answers from Supabase, and PostgREST 406s. The -// student fixture reads the list at teardown and fails the test on anything it -// does not expect. +// promises, console.error, and 5xx answers from Supabase. The student fixture +// reads the list at teardown and fails the test on anything it does not expect. // sessionStorage keeps the list across reloads and navigations in the tab. (() => { const KEY = "__agentic_errors"; @@ -33,11 +32,9 @@ const response = await nativeFetch(input, init); // Recording must never change what the app's request returns. try { - // PostgREST answers 406 when a query asks for exactly one row - // (.single()) and gets none; a lookup that can miss uses .maybeSingle(). - if (response.status >= 500 || response.status === 406) { + if (response.status >= 500) { const url = new URL(input instanceof Request ? input.url : String(input), location.href); - if (response.status >= 500 ? /^\/(functions|rest|auth)\//.test(url.pathname) : url.pathname.startsWith("/rest/")) { + if (/^\/(functions|rest|auth)\//.test(url.pathname)) { const method = (init && init.method) || (input instanceof Request ? input.method : "GET"); record("http", `${response.status} ${method.toUpperCase()} ${url.pathname}`); } diff --git a/agentic/support/fixtures.ts b/agentic/support/fixtures.ts index c9015ee3d..856891aba 100644 --- a/agentic/support/fixtures.ts +++ b/agentic/support/fixtures.ts @@ -66,8 +66,8 @@ async function signIn(screen: Screen, browser: Browser, code: string, language: /** * What `support/capture-errors.js` recorded in this tab: uncaught errors, - * rejected promises, console.error, Supabase 5xx answers, and PostgREST 406s. - * Reading clears the list. + * rejected promises, console.error, and Supabase 5xx answers. Reading clears + * the list. */ export async function pageErrors(browser: Browser): Promise { return browser.evaluate(() => { diff --git a/docs/testing.md b/docs/testing.md index cd635deff..4ca4e2b0e 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -201,7 +201,7 @@ Each test signs in through the Login page with a fresh demo code and deletes tha - a promo code redeemed in Settings that restores AI access, and signed Stripe webhooks that grant and revoke Pro while a forged or replayed one changes nothing; - admin announcements seen by another user, denied admin access, and a public share link. -Tests assert the rendered result, what the Edge Function sent upstream (read back from the fake provider), and the resulting database rows. They also fail on any uncaught page error, `console.error`, failed request (4xx or 5xx) to the local app or stack, or horizontal page overflow. A spec that causes a failure on purpose names it with `test.use({ tolerate: [...] })`. The fake provider rejects prompts it does not recognize, so a prompt change must update it. +Tests assert the rendered result, what the Edge Function sent upstream (read back from the fake provider), and the resulting database rows. They also fail on any uncaught page error or horizontal page overflow. The fake provider rejects prompts it does not recognize, so a prompt change must update it. These journeys do not prove live provider behavior, web research (the local stack has no search key), real Jev judgement (the fake answers every Jev question with a fixed confident choice), Stripe Checkout, the billing portal, and subscription cancellation during account deletion (they call the Stripe API; the local key is a placeholder, and the Deno tests cover the cancellation logic), the `checkout.session.completed` and `invoice.payment_succeeded` webhooks (they fetch the subscription from Stripe), or Google export. @@ -254,7 +254,7 @@ $H down - Journeys (tag `journey`), one goal per tool, that survive copy and layout changes: the onboarding tour starting a first debate, Generate and reopening the case from History, Attack, Brainstorm filed in a new debate, Factoids, Ask, a full Sparring round to its verdict, Crossfire to its debrief, adding and pausing a Library source, saving a Flowchart, the speech Timer, and switching interface and AI output language in Settings. Each is pinned by a locator, the rows the app wrote (`ai_requests`, `brainstorms`, `cases`, `sparring_sessions`, `crossfires`, `sources`, `flowcharts`, `user_settings`), and in fake mode the provider's `FAKE-PROVIDER` marker on screen. - The page sweep (`pages.e2e.ts`, tag `pages`): the 14 student pages and 5 public pages, in English, Spanish and Hungarian, on both viewports. It skips `/app/admin`, which needs an admin, and `/share/:code`, because no client code creates a share link yet. Selectors check `lang`, a heading, and that no `undefined`, `NaN` or `[object Object]` leaked into the page. The judge checks for error notices, raw translation keys, untranslated interface text (which `e2e/hungarian-parity.spec.ts` deliberately cannot reject), and clipped or overlapping text. -- Hidden failures. [`agentic/support/capture-errors.js`](../agentic/support/capture-errors.js) runs before the app in every document and records uncaught errors, rejected promises, `console.error` (which includes React's warnings and the route error boundary), 5xx answers from Supabase, and PostgREST 406s (a `.single()` lookup that found no row). Every test fails on any of them, on a request to a hosted Supabase project, on a page wider than its viewport, and on visible text that runs past the screen edge (which an `overflow-x: hidden` wrapper would otherwise hide). Text clipped away, cut short by an ellipsis, or inside a sideways scroller does not count; [`agentic/edge-guard.e2e.ts`](../agentic/edge-guard.e2e.ts) pins those rules on injected layouts. +- Hidden failures. [`agentic/support/capture-errors.js`](../agentic/support/capture-errors.js) runs before the app in every document and records uncaught errors, rejected promises, `console.error` (which includes React's warnings and the route error boundary), and 5xx answers from Supabase. Every test fails on any of them, on a request to a hosted Supabase project, on a page wider than its viewport, and on visible text that runs past the screen edge (which an `overflow-x: hidden` wrapper would otherwise hide). Text clipped away, cut short by an ellipsis, or inside a sideways scroller does not count; [`agentic/edge-guard.e2e.ts`](../agentic/edge-guard.e2e.ts) pins those rules on injected layouts. - In live mode (`up --live`), whether model output is on topic: the generated case argues the motion, and factoids carry sources. Recipes never assert model wording. - Exploration. `e2e explore` sends a persona through the app with a goal and reports findings with expected and observed behavior, reproduction steps, and screenshots. The personas are the named agents in `e2e.config.ts`: `first-timer`, `coach`, `edge-input`, and `hungarian`. diff --git a/e2e/full-stack/stack.ts b/e2e/full-stack/stack.ts index 24ac75b43..041eb8d78 100644 --- a/e2e/full-stack/stack.ts +++ b/e2e/full-stack/stack.ts @@ -49,12 +49,9 @@ export async function chooseForMotion(page: Page, choice: "create" | "once") { await expect(dialog).toBeHidden(); } -type Fixtures = { user: DemoUser; signedIn: Page; tolerate: RegExp[] }; +type Fixtures = { user: DemoUser; signedIn: Page }; export const test = base.extend({ - // Problems a spec causes on purpose, such as a provider failure, matched - // against the lines the page fixture collects. Set with test.use(). - tolerate: [[], { option: true }], // Playwright requires a destructuring pattern even when a fixture has no dependencies. // eslint-disable-next-line no-empty-pattern user: async ({}, provide, testInfo) => { @@ -62,22 +59,10 @@ export const test = base.extend({ await provide(created); await deleteDemoUser(created); }, - page: async ({ page, tolerate }, provide) => { + page: async ({ page }, provide) => { // Any uncaught exception fails the test, even when the visible flow passes. const errors: string[] = []; page.on("pageerror", (error) => errors.push(error.message)); - // So does a console error or a failed request to the local app or stack: - // an expected miss, like an unknown share code, must not log one. - const problems: string[] = []; - page.on("console", (message) => { - if (message.type() === "error") problems.push(`console.error: ${message.text()}`); - }); - page.on("response", (response) => { - const url = new URL(response.url()); - if (response.status() >= 400 && /^(127\.0\.0\.1|localhost|\[::1\])$/.test(url.hostname)) { - problems.push(`${response.status()} ${response.request().method()} ${url.pathname}`); - } - }); // Init scripts rerun on every load, so seed only what the app has not written. await page.addInitScript(() => { const seed = (key: string, state: Record) => { @@ -89,7 +74,6 @@ export const test = base.extend({ await fake.reset(); await provide(page); expect(errors, "uncaught page errors").toEqual([]); - expect(problems.filter((line) => !tolerate.some((pattern) => pattern.test(line))), "console errors and failed local requests").toEqual([]); // Overflow widens the mobile layout viewport and pushes controls off-screen. const overflow = await page.evaluate(() => document.documentElement.scrollWidth - document.documentElement.clientWidth); expect(overflow, "horizontal page overflow in px").toBeLessThanOrEqual(1);