Cut CI to a two-job gate and weekly E2E and AI evals

.claude/skills/verify-debatelab/features/ai-failures.md

@@ -24,4 +24,4 @@ When an AI request fails, the tool says why in one plain sentence and offers the
 ## Gotchas
 
 - A real 401 from a function makes the client refresh the session once (`verifySessionAfterRejection`). On this stack the refresh succeeds, so the tab stays signed in until `Sign in again` is clicked.
-- A timeout needs the 200-second client budget to pass, so no recipe forces one. `src/lib/ai/failure-reason.test.ts` covers its mapping.
+- A timeout needs the 200-second client budget to pass, so no recipe forces one. Its message mapping lives in `src/lib/ai/failure-reason.ts`.

.claude/skills/verify-debatelab/features/auth-demo-login.md

@@ -41,4 +41,4 @@ Preconditions:
 - A first sign-in on a new account starts the onboarding tour on `/app`. Its first step (`Welcome to DebateOwl`, `Start your first debate here.`) offers `Start debate`. When the one-time DebateOwl welcome is due, closing it (`Got it` or Esc) continues straight into that step; the verify stack keeps the welcome closed unless the URL is `?intro=show`. The fixtures pre-seed `debate-trainer-onboarding` to skip the tour. The tour is tracked per account (store version 1, `completedUserIds`). A version 0 seed with `hasCompletedOnboarding: true` covers only the next account that signs in, so a second account in the same browser sees the tour. Interactive runs click `Skip tour` (`[data-test-id="button-skip"]`). The tour itself is covered in [current-debate](./current-debate.md).
 - Sessions are stored under the `debate-auth-local` localStorage key in local-preview mode. Don't reuse stub-session helpers (`e2e/helpers/auth.ts`). They fake auth and prove rendering only.
 - Google sign-in is not wired to the local stack. Demo codes, email codes and the email link work here.
-- The revoked-session check forces one token refresh before it signs out. An Auth outage (network error, 5xx, 429) keeps the session, and that path is covered by `src/test/auth/session-expiry.test.ts`, not by a recipe.
+- The revoked-session check forces one token refresh before it signs out. An Auth outage (network error, 5xx, 429) keeps the session. No recipe covers that path.

.claude/skills/verify-debatelab/features/voice-rounds.md

@@ -6,7 +6,7 @@ Sparring and Crossfire can run as spoken rounds. The student speaks through the
 
 - `voice-spar` runs a spoken Lincoln-Douglas round. The opponent is told the round is spoken (`SPOKEN ROUND:` in its system prompt). Each reply comes back as clean text plus signed voice segments, which may keep allowed audio tags such as `[sighs]` and drop invented ones. The browser voice reads the clean text, and no tag reaches the page or the saved round. The scorecard shows `Spoken delivery, measured` (test id `spoken-delivery`), and the saved user turns carry `delivery` measurements.
 - `voice-cross` runs a spoken Crossfire exchange. The question arrives by microphone, the request carries `SPOKEN EXCHANGE:`, and the clean answer is read aloud. The Jev signal still attaches, and the debrief shows `spoken-delivery`.
-- `voice-unavailable`: when the browser has no speech recognition or the microphone is blocked, the switch is replaced by the test id `voice-unavailable`. This is covered by `src/test/pages/SparringVoice.test.tsx` and `CrossfireVoice.test.tsx`, not by a recipe.
+- `voice-unavailable`: when the browser has no speech recognition or the microphone is blocked, the switch is replaced by the test id `voice-unavailable`. No recipe covers it; check it by hand in a browser with the microphone blocked.
 
 ## How to get to it (user POV)
 

.github/workflows/agentic-e2e.yml

@@ -3,16 +3,17 @@ name: Agentic E2E
 # The e2e suite in agentic/ against the local stack in fake mode: an AI agent
 # drives goal-level flows and judges screens. Recorded flows replay from
 # agentic/replay without model calls; judgments always call the model.
-# It runs nightly, not on pull requests: it takes far longer than the
-# pull-request check and each judgment is a paid model call. Run it on a
-# branch with `gh workflow run agentic-e2e.yml --ref <branch>`. A weekly bug
-# bash sends each persona exploring, and on demand one exploration runs
-# instead of the suite.
+# It runs weekly, not on pull requests: it takes far longer than the
+# pull-request check (about 40 billed minutes) and each judgment is a paid
+# model call. Run it on a branch with
+# `gh workflow run agentic-e2e.yml --ref <branch>`. A monthly bug bash sends
+# each persona exploring, and on demand one exploration runs instead of the
+# suite. The two job conditions below match the bug-bash cron string.
 # Not a required check. See docs/testing.md#agentic-browser-tests.
 on:
   schedule:
-    - cron: "47 3 * * *" # the suite
-    - cron: "13 5 * * 0" # the weekly bug bash
+    - cron: "47 3 * * 3" # the suite, Wednesdays
+    - cron: "13 5 1 * *" # the bug bash, on the 1st of each month
   workflow_dispatch:
     inputs:
       explore:
@@ -42,7 +43,7 @@ env:
 
 jobs:
   suite:
-    if: github.repository == 'Programcode/DebateOwl' && github.event.schedule != '13 5 * * 0' && (github.event_name != 'workflow_dispatch' || inputs.explore == '')
+    if: github.repository == 'Programcode/DebateOwl' && github.event.schedule != '13 5 1 * *' && (github.event_name != 'workflow_dispatch' || inputs.explore == '')
     runs-on: ubuntu-latest
     timeout-minutes: 45
     # Four legs keep each well under the timeout: the page sweep makes about
@@ -123,9 +124,9 @@ jobs:
           if-no-files-found: ignore
           retention-days: 7
 
-  # The weekly bug bash explores with every persona; a manual run explores once.
+  # The monthly bug bash explores with every persona; a manual run explores once.
   plan-explore:
-    if: github.repository == 'Programcode/DebateOwl' && (github.event.schedule == '13 5 * * 0' || (github.event_name == 'workflow_dispatch' && inputs.explore != ''))
+    if: github.repository == 'Programcode/DebateOwl' && (github.event.schedule == '13 5 1 * *' || (github.event_name == 'workflow_dispatch' && inputs.explore != ''))
     runs-on: ubuntu-latest
     outputs:
       legs: ${{ steps.plan.outputs.legs }}

.github/workflows/browser-nightly.yml → .github/workflows/browser-weekly.yml

@@ -1,20 +1,20 @@
-name: Browser nightly
+name: Browser weekly
 
 # The full credential-free browser suite on desktop and mobile Chromium, and
 # the service worker's offline test against a production build. Pull requests
 # run only the @smoke subset (deploy.yml), so this catches what the subset
-# skips, at most a day late. Run it on a branch with
-# `gh workflow run browser-nightly.yml --ref <branch>`.
+# skips, at most a week late. A change that spans many pages runs it on its
+# branch with `gh workflow run browser-weekly.yml --ref <branch>`.
 on:
   schedule:
-    - cron: "47 2 * * *"
+    - cron: "47 2 * * 2"
   workflow_dispatch:
 
 permissions:
   contents: read
 
 concurrency:
-  group: browser-nightly
+  group: browser-weekly-${{ github.ref }}
   cancel-in-progress: true
 
 jobs:
@@ -53,7 +53,7 @@ jobs:
         if: failure()
         uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
         with:
-          name: chromium-nightly-failure-${{ github.run_id }}
+          name: chromium-weekly-failure-${{ github.run_id }}
           path: |
             test-results/
             playwright-report/

.github/workflows/database.yml

@@ -3,7 +3,8 @@ name: Database
 # Pull requests that change the schema, its tests, or the gate run pgTAP and
 # the database scripts against Postgres alone, about 2 minutes. The full gate
 # (local Supabase, pgTAP, and the full-stack browser journeys on desktop and
-# mobile, about 20 minutes) runs nightly and on demand.
+# mobile, about 20 minutes) runs weekly and on demand. Weekly also keeps the
+# Supabase image pulls under public.ecr.aws's anonymous data limit.
 on:
   pull_request:
     paths:
@@ -16,7 +17,7 @@ on:
       - ".superset/lib/**"
       - ".github/workflows/database.yml"
   schedule:
-    - cron: "23 5 * * *"
+    - cron: "23 5 * * 4"
   workflow_dispatch:
 
 permissions:

.github/workflows/deploy.yml

@@ -1,8 +1,10 @@
 name: Quality
 
-# The pull-request check. Five jobs run side by side so the check takes as long
-# as the slowest one, about 2.5 minutes. Slow, paid, and full-stack suites run
-# on a schedule instead; docs/operations.md lists them.
+# The pull-request check: two jobs side by side, about 2.5 minutes. `check`
+# proves the app builds and the repo rules hold; `browser-smoke` proves the
+# built pages load in a browser. There is no unit-test job: behavior is proven
+# end to end, and the slow, paid and full-stack suites run weekly or on demand
+# (docs/operations.md lists them).
 on:
   push:
     branches: [ "main" ]
@@ -17,7 +19,10 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  lint:
+  # One job, so a run pays for one runner start and one cache restore. Every
+  # step is a few seconds to half a minute; the guards are the Vitest files
+  # listed in vitest.config.ts, about 2 seconds.
+  check:
     runs-on: ubuntu-latest
     timeout-minutes: 10
     steps:
@@ -28,69 +33,35 @@ jobs:
 
       - uses: ./.github/actions/node-deps
 
+      - name: Set up Deno
+        uses: denoland/setup-deno@22d081ff2d3a40755e97629de92e3bcbfa7cf2ed # v2.0.5
+        with:
+          deno-version: 2.9.6
+          cache: true
+
       - name: Documentation
         run: npm run docs:check
 
       - name: Lint
         run: npm run lint
 
-  typecheck-build:
-    runs-on: ubuntu-latest
-    timeout-minutes: 10
-    steps:
-      - name: Checkout code
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
-        with:
-          persist-credentials: false
-
-      - uses: ./.github/actions/node-deps
-
       - name: Typecheck
         run: npm run typecheck
 
-      - name: Build
-        run: npm run build
-
-  unit:
-    runs-on: ubuntu-latest
-    timeout-minutes: 10
-    steps:
-      - name: Checkout code
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
-        with:
-          persist-credentials: false
-
-      - uses: ./.github/actions/node-deps
-
-      - name: Test
+      - name: Guards
         run: npm test
 
-      - name: Collaboration service tests
-        run: npm run test:collaboration
-
-  # Edge Function tests import only jsr:, npm: and https: modules, so this job
-  # needs Deno and no npm install.
-  edge:
-    runs-on: ubuntu-latest
-    timeout-minutes: 10
-    steps:
-      - name: Checkout code
-        uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
-        with:
-          persist-credentials: false
-
-      - name: Set up Deno
-        uses: denoland/setup-deno@22d081ff2d3a40755e97629de92e3bcbfa7cf2ed # v2.0.5
-        with:
-          deno-version: 2.9.6
-          cache: true
-
+      # Edge Functions hold auth, quota and billing, which no pull-request
+      # browser test reaches.
       - name: Edge function tests
         run: npm run test:edge
 
+      - name: Build
+        run: npm run build
+
   # The @smoke set: the app boots, sign-in works, every public and signed-in
   # page renders without errors, and the first-run tour starts a debate. The
-  # full desktop and mobile suite runs in browser-nightly.yml.
+  # full desktop and mobile suite runs weekly in browser-weekly.yml.
   browser-smoke:
     if: github.event_name == 'pull_request'
     runs-on: ubuntu-latest

.github/workflows/live-e2e.yml

@@ -1,26 +1,43 @@
-name: Live provider E2E
+name: AI evals
 
-# Real OpenRouter, Jev, and Exa behind the local stack. Every run spends
-# provider credit and takes about 17 minutes, so it runs nightly and on demand,
-# never on pull requests. To check a branch that changes AI or research code:
-# `gh workflow run live-e2e.yml --ref <branch>`. Nightly runs also catch
-# provider drift without code changes.
+# The AI evals: the local stack against real OpenRouter, Jev, and Exa. Every
+# run spends provider credit, so it runs weekly and on demand, never on pull
+# requests. The production canary's daily run makes two real AI requests, so
+# provider drift still shows up within a day.
+# - live: the live provider journeys, about 8 minutes.
+# - model-matrix: one case from every catalog model, unpinned, so a model that
+#   times out or returns invalid JSON fails here before users hit it. About $1.
+# A branch that changes AI, research, prompts or model routing runs them with
+# `gh workflow run live-e2e.yml --ref <branch>` (add `-f suite=model-matrix`
+# or `-f suite=both`).
 on:
   schedule:
-    - cron: "17 3 * * *"
+    - cron: "17 3 * * 1"
   workflow_dispatch:
+    inputs:
+      suite:
+        description: "Which eval to run"
+        type: choice
+        default: live
+        options: [live, model-matrix, both]
 
 permissions:
   contents: read
 
 concurrency:
-  group: live-e2e-${{ github.ref }}
+  group: live-e2e-${{ github.ref }}-${{ inputs.suite || 'schedule' }}
   cancel-in-progress: true
 
 jobs:
-  live-e2e:
+  eval:
     # Forks have none of the secrets this job needs.
     if: github.repository == 'Programcode/DebateOwl'
+    strategy:
+      fail-fast: false
+      matrix:
+        # The schedule runs both; a dispatch runs the suite it names.
+        suite: ${{ fromJSON((github.event_name == 'schedule' || inputs.suite == 'both') && '["live","model-matrix"]' || format('["{0}"]', inputs.suite)) }}
+    name: ${{ matrix.suite }}
     runs-on: ubuntu-latest
     timeout-minutes: 45
     steps:
@@ -52,8 +69,9 @@ jobs:
       - name: Live provider journeys
         env:
           LIVE_PROVIDERS: 1
-          # Repository variable override; empty falls back to db-gate.sh's default.
-          LIVE_MODEL: ${{ vars.LIVE_MODEL }}
+          # live: a repository variable override, empty falls back to
+          # db-gate.sh's default. model-matrix: every catalog model.
+          LIVE_MODEL: ${{ matrix.suite == 'model-matrix' && 'as-requested' || vars.LIVE_MODEL }}
           LIVE_OPENROUTER_API_KEY: ${{ secrets.LIVE_OPENROUTER_API_KEY }}
           LIVE_EXA_API_KEY: ${{ secrets.LIVE_EXA_API_KEY }}
           SUPABASE_INTERNAL_IMAGE_REGISTRY: public.ecr.aws
@@ -64,7 +82,7 @@ jobs:
         if: failure()
         uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
         with:
-          name: live-e2e-failure-${{ github.run_id }}
+          name: ${{ matrix.suite }}-failure-${{ github.run_id }}
           path: |
             ${{ runner.temp }}/workspace-gate/
             test-results/