Skip to content
1 change: 1 addition & 0 deletions .claude/skills/verify-debatelab/features/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ Each feature file starts with an H1 title and one paragraph describing the user-
- [Library sources](./library-sources.md): add, pause, delete, grounding, and URL import (`recipes/library.spec.ts`).
- [AI tools](./ai-tools.md): Attack, Brainstorm, Factoids, and Ask streamed, cited, and with web search (`recipes/ai-tools.spec.ts`).
- [Current Debate](./current-debate.md): start, the first-debate offer and the different-motion choice, sources and results scoped to a debate, switching, and the first onboarding step (`recipes/current-debate.spec.ts`); debate context review and pins (`e2e/full-stack/debate-context.spec.ts`).
- [Model defaults](./model-defaults.md): Pro accounts start on Claude Opus 5.5, Basic accounts on Claude Haiku 5.5 with Pro models locked, and retired model IDs move to Opus 5.5 (`recipes/model-defaults.spec.ts`).
- [Practice](./practice.md): a full Lincoln-Douglas sparring round with verdict, and Crossfire in both directions with debrief, each filed in its debate (`recipes/practice.spec.ts`).
- [Voice rounds](./voice-rounds.md): spoken Sparring and Crossfire on the browser voice, with no ElevenLabs key (`recipes/voice-rounds.spec.ts`).
- [Workspace autosave](./workspace-autosave.md): a slow Crossfire or Sparring save from a Workspace never overwrites newer text (`recipes/workspace-autosave.spec.ts`). Needs `up --collab`; skips without it.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,5 +41,5 @@ Preconditions:
- `Save` appears both on the result toolbar and in the dialog. Scope the second click to `getByRole("dialog")`. The toolbar `Save` exists only for one-off results.
- A case filed in a debate is titled with the motion, which the header control also shows (truncated and hidden). Look for it inside the History panel, not with a page-wide `getByText`.
- The client files a result after it renders. Wait for the result on screen (and the row, if the next step reloads) before a `page.goto`, or the reload can drop it.
- The model label in the result header (for example `anthropic/claude-sonnet-5`) is the model the function requested. The answer itself came from the fake provider.
- The model label in the result header (for example `anthropic/claude-opus-5.5`) is the model the function requested. The answer itself came from the fake provider.
- `Google Docs` export needs Google OAuth and cannot be proven locally.
24 changes: 24 additions & 0 deletions .claude/skills/verify-debatelab/features/model-defaults.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Model defaults and the model selector

Every AI tool has a model selector. Basic accounts can pick Claude Haiku 5.5, GPT-6 Luna, or Muse Spark 1.3 (`meta/muse-spark-1.3-contributor`); everything else is Pro. A new Pro account starts every tool on Claude Opus 5.5, and a Basic (trial) account on Claude Haiku 5.5. Pro-only models show a `Pro` badge and are disabled for Basic accounts. A setting saved with a retired catalog ID is rewritten on the next load: GPT-5.6 Luna moves to Claude Haiku 5.5, Claude Sonnet 5 and Opus 5 to Claude Opus 5.5, GPT-5.6 Sol and Terra to GPT-6.1 Sol, DeepSeek V4 Flash to Claude Haiku 5.5, and Muse Spark 1.3 to its contributor tier. The Edge Functions resolve a retired ID from an older client the same way. A retired Basic ID never resolves to a Pro model. The catalog and defaults live in `supabase/functions/_shared/model-catalog.ts`.

## Sub-features

- `model-pro-default`: a new Pro account's `user_settings.feature_configs` hold `anthropic/claude-opus-5.5` for every tool, the selector shows `Claude Opus 5.5`, and a Generate run records `model: anthropic/claude-opus-5.5` in `ai_requests`.
- `model-basic-default`: a trial account with no saved settings gets `anthropic/claude-haiku-5.5` everywhere. `Claude Opus 5.5` and `Claude Sonnet 5.5` are disabled options, and a Generate run records Haiku 5.5.
- `model-retired`: saved `anthropic/claude-sonnet-5` configs become `anthropic/claude-opus-5.5` on screen and in `user_settings`.

## How to get to it (user POV)

- Any tool page, for example `/app/generate`: the combobox named `Select AI model for generate` sits beside the tool's inputs.

## Driving it with verify.sh + Playwright

- `bin/verify.sh drive recipes/model-defaults` covers all three sub-features.
- Demo sign-in grants Pro. For the Basic path the recipe sets `subscription_tier: "trial"`, a future `trial_ends_at`, clears `grace_period_ends_at` and `daily_limit_override`, and empties `feature_configs`, then reloads so the client re-hydrates as a new trial account.
- End states are read from `user_settings.feature_configs` and `ai_requests.model` through `admin`, and from the fake provider's recorded `model`.

## Gotchas

- Fake mode records the requested model but answers with fake text, and live mode rewrites every chat call to the Muse Spark contributor pin. Neither proves that OpenRouter accepts the Claude 5.5 IDs; that needs a direct OpenRouter call.
- Existing saved choices of a model still in the catalog are kept. Only retired IDs and unsaved settings move to the new defaults.
99 changes: 99 additions & 0 deletions .claude/skills/verify-debatelab/recipes/model-defaults.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
import type { Page } from "@playwright/test";
import { admin, AI_TIMEOUT, aiResponse, chooseForMotion, expect, providerCalls, test } from "./fixtures";

// Feature map: features/model-defaults.md.
const motion = "This House would make public transport free at the point of use";
const OPUS = "anthropic/claude-opus-5.5";
const HAIKU = "anthropic/claude-haiku-5.5";

async function settings(userId: string) {
const { data, error } = await admin.from("user_settings").select("feature_configs").eq("user_id", userId).single();
if (error) throw error;
return data.feature_configs as Record<string, { model: string }>;
}

async function updateSettings(userId: string, values: Record<string, unknown>) {
const { error } = await admin.from("user_settings").update(values).eq("user_id", userId);
if (error) throw error;
}

async function generateOnce(page: Page) {
await page.getByRole("textbox", { name: "Motion / Topic *" }).fill(motion);
const response = aiResponse(page, "generate");
await page.getByRole("button", { name: "Generate Debate Case" }).click();
await chooseForMotion(page, "once");
return response;
}

test("model-pro-default: a new Pro account starts on Claude Opus 5.5 and generates with it", async ({ signedIn: page, user, evidence }) => {
const userId = await user.userId();
await expect.poll(async () => (await settings(userId)).generate?.model, { timeout: AI_TIMEOUT }).toBe(OPUS);

await page.goto("/app/generate");
const selector = page.getByRole("combobox", { name: "Select AI model for generate" });
await expect(selector).toContainText("Claude Opus 5.5");
await selector.click();
await expect(page.getByRole("option", { name: /Claude Haiku 5\.5/ })).toBeEnabled();
await expect(page.getByRole("option", { name: /Claude Sonnet 5\.5/ })).toBeEnabled();
await evidence.shot("01-pro-model-list");
await page.keyboard.press("Escape");
await evidence.step("New Pro account: selector shows Claude Opus 5.5, all Claude 5.5 models enabled");

const body = await generateOnce(page);
expect((body.meta as { resolvedModel?: string } | undefined)?.resolvedModel ?? OPUS).toBe(OPUS);
const calls = await providerCalls();
// Live mode rewrites the upstream model to the pin, so read what the app asked for.
expect(calls.filter((call) => call.feature === "generate").map((call) => call.requestedModel ?? call.model)).toContain(OPUS);
const { data: usage } = await admin.from("ai_requests").select("feature,status,model").eq("user_id", userId);
expect(usage).toContainEqual(expect.objectContaining({ feature: "generate", status: "success", model: OPUS }));
await evidence.json("02-ai-requests", usage);
await evidence.shot("02-generated-with-opus");
});

test("model-basic-default: a Basic account starts on Claude Haiku 5.5 with only Haiku, Luna and Muse Spark unlocked", async ({ signedIn: page, user, evidence }) => {
const userId = await user.userId();
// Demo sign-in grants Pro; turn the account into a fresh trial with nothing saved.
await updateSettings(userId, {
subscription_tier: "trial",
trial_ends_at: new Date(Date.now() + 2 * 86_400_000).toISOString(),
grace_period_ends_at: null,
daily_limit_override: null,
feature_configs: {},
});
await page.goto("/app/generate");
await expect.poll(async () => (await settings(userId)).generate?.model, { timeout: AI_TIMEOUT }).toBe(HAIKU);

const selector = page.getByRole("combobox", { name: "Select AI model for generate" });
await expect(selector).toContainText("Claude Haiku 5.5");
await selector.click();
await expect(page.getByRole("option", { name: /Claude Opus 5\.5/ })).toBeDisabled();
await expect(page.getByRole("option", { name: /Claude Sonnet 5\.5/ })).toBeDisabled();
await expect(page.getByRole("option", { name: /Gemini 3\.8 Flash/ })).toBeDisabled();
await expect(page.getByRole("option", { name: /GPT-6 Luna/ })).toBeEnabled();
await expect(page.getByRole("option", { name: /Muse Spark 1\.3/ })).toBeEnabled();
await evidence.shot("01-basic-model-list");
await page.keyboard.press("Escape");
await evidence.step("Basic account: selector shows Claude Haiku 5.5; Luna and Muse Spark enabled; Opus, Sonnet and Gemini locked behind Pro");

await generateOnce(page);
const { data: usage } = await admin.from("ai_requests").select("feature,status,model").eq("user_id", userId);
expect(usage).toContainEqual(expect.objectContaining({ feature: "generate", status: "success", model: HAIKU }));
await evidence.json("02-ai-requests", usage);
});

test("model-retired: a saved Claude Sonnet 5 setting moves to Claude Opus 5.5", async ({ signedIn: page, user, evidence }) => {
const userId = await user.userId();
const retired = Object.fromEntries(["generate", "attack", "brainstorm", "factoids", "ask", "sparring"].map((feature) => [
feature,
{ provider: "openrouter", model: "anthropic/claude-sonnet-5", temperature: 0.7, reasoningEffort: "low" },
]));
await updateSettings(userId, { feature_configs: retired });
await page.evaluate(() => localStorage.removeItem("debate-trainer-settings"));
await page.goto("/app/generate");

await expect(page.getByRole("combobox", { name: "Select AI model for generate" })).toContainText("Claude Opus 5.5");
await expect.poll(async () => Object.values(await settings(userId)).map((config) => config.model), { timeout: AI_TIMEOUT })
.toEqual(Array(6).fill(OPUS));
await evidence.shot("01-retired-migrated");
await evidence.step("Saved Sonnet 5 configs were rewritten to Opus 5.5 in user_settings");
});
4 changes: 2 additions & 2 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -128,9 +128,9 @@ Supabase Auth owns user sessions. Protected routes provide a client navigation b

Checkout and portal functions create Stripe sessions after authenticated ownership and redirect validation. The webhook updates subscription state and records event processing for idempotency. AI handlers record the provider-reported usage and charge for each request.

All model requests use the single OpenRouter backend route and `OPENROUTER_API_KEY`. The shared ten-model catalog, Luna Basic default, Pro-only entries, and request controls live in [`supabase/functions/_shared/model-catalog.ts`](../supabase/functions/_shared/model-catalog.ts). OpenRouter usage remains raw in the audit row, and the reported charge is unknown when the provider does not return a complete cost. The opponent's voice is the one exception to the OpenRouter route: the `voice` function calls ElevenLabs with `ELEVENLABS_API_KEY`. ElevenLabs reports billed characters, not dollars, so [`supabase/functions/_shared/cost.ts`](../supabase/functions/_shared/cost.ts) prices those rows from a dated list-price snapshot; promotions and plan credits are not modelled, so a voice row can overstate the charge.
All model requests use the single OpenRouter backend route and `OPENROUTER_API_KEY`. The shared ten-model catalog, its defaults (Claude Haiku 5.5 for Basic, Claude Opus 5.5 for Pro), Pro-only entries, retired-ID successors, and request controls live in [`supabase/functions/_shared/model-catalog.ts`](../supabase/functions/_shared/model-catalog.ts). OpenRouter usage remains raw in the audit row, and the reported charge is unknown when the provider does not return a complete cost. The opponent's voice is the one exception to the OpenRouter route: the `voice` function calls ElevenLabs with `ELEVENLABS_API_KEY`. ElevenLabs reports billed characters, not dollars, so [`supabase/functions/_shared/cost.ts`](../supabase/functions/_shared/cost.ts) prices those rows from a dated list-price snapshot; promotions and plan credits are not modelled, so a voice row can overstate the charge.

`Auto Fast` and `Auto Pro` are virtual selections, not provider model IDs. The Edge Functions resolve them through TypeSafe Jev on OpenRouter before calling the concrete writing model. Auto Fast is available to Basic accounts and normally resolves to GLM 5.3 Flash; manual GLM selection keeps its Pro restriction. Auto Pro remains Pro-only. Provider calls, usage rows, and response metadata record the concrete resolved model, while response metadata also retains the requested automatic selection. A Jev router call uses the same OpenRouter key and does not consume a second daily quota unit.
`Auto Fast` and `Auto Pro` are virtual selections, not provider model IDs. The Edge Functions resolve them through TypeSafe Jev on OpenRouter before calling the concrete writing model. Auto Fast is available to Basic accounts, routes only among the Basic models (Claude Haiku 5.5, GPT-6 Luna, Muse Spark 1.3 Contributor), and normally resolves to Claude Haiku 5.5. Auto Pro remains Pro-only and normally resolves to Claude Opus 5.5. Provider calls, usage rows, and response metadata record the concrete resolved model, while response metadata also retains the requested automatic selection. A Jev router call uses the same OpenRouter key and does not consume a second daily quota unit.

## Observability

Expand Down
8 changes: 4 additions & 4 deletions docs/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ OPENROUTER_API_KEY=... deno run --allow-read --allow-write --allow-env=OPENROUTE
--allow-net=openrouter.ai scripts/prompt-examples.ts supabase/functions /tmp/after.json
```

This sends the fourteen prompt examples to one real model (GPT-5.6 Luna by default) and writes the raw outputs for manual review. Point the first argument at a directory holding an older `_shared/prompts.ts` and `ai/prompt-builder.ts` to compare revisions. It runs no web research and is a bounded manual check, not a model matrix or an automated quality gate.
This sends the fourteen prompt examples to one real model (GPT-6 Luna by default) and writes the raw outputs for manual review. Point the first argument at a directory holding an older `_shared/prompts.ts` and `ai/prompt-builder.ts` to compare revisions. It runs no web research and is a bounded manual check, not a model matrix or an automated quality gate.

### Live Jev contract smoke

Expand Down Expand Up @@ -215,7 +215,7 @@ To check one feature by hand, or keep evidence for a change, use the same stack
LIVE_PROVIDERS=1 LIVE_OPENROUTER_API_KEY=... LIVE_EXA_API_KEY=... ./.superset/db-gate.sh
```

Live mode uses the same locked stack but skips pgTAP and the fake-provider specs. It runs `e2e/live/` on desktop Chromium against real OpenRouter, Jev, and Exa. Imported web pages are fetched live too. The fake provider becomes a recording proxy. It forwards every chat call to OpenRouter with the model pinned to `LIVE_MODEL` (default `meta/muse-spark-1.3-contributor`; CI reads the `LIVE_MODEL` repository variable, which is unset), passes Jev decisions through unchanged, and refuses calls past `MAX_UPSTREAM_CALLS` (200). The default is the cheapest model that supports every request parameter the app sends, including `response_format` and reasoning effort. It is test-only and not in the app catalog, so users cannot pick it. Meta is its only provider, so an outage there fails the gate instead of failing over.
Live mode uses the same locked stack but skips pgTAP and the fake-provider specs. It runs `e2e/live/` on desktop Chromium against real OpenRouter, Jev, and Exa. Imported web pages are fetched live too. The fake provider becomes a recording proxy. It forwards every chat call to OpenRouter with the model pinned to `LIVE_MODEL` (default `meta/muse-spark-1.3-contributor`; CI reads the `LIVE_MODEL` repository variable, which is unset), passes Jev decisions through unchanged, and refuses calls past `MAX_UPSTREAM_CALLS` (200). The default is the cheapest model that supports every request parameter the app sends, including `response_format` and reasoning effort. It is also the Basic catalog model "Muse Spark 1.3", so a live run that selects it cannot be told apart from the pin by the upstream model alone; recipes check the requested model instead. Meta is its only provider, so an outage there fails the gate instead of failing over.

The specs cover web-researched case generation, verified Factoids citations, Ask with web search, URL import, streamed Ask, Attack, Brainstorm, a judged sparring round, and Crossfire with live Jev signals. Outputs vary, so tests assert structure rather than text:

Expand Down Expand Up @@ -288,8 +288,8 @@ npx playwright test --config=playwright.production.config.ts
- every browser-facing Edge Function answers a preflight that allows the site origin (`workspace-invites` is excluded until production enables Workspaces);
- `ai` returns 401, not a 5xx, to an unauthenticated call;
- the canary account signs in, opens Generate, and Settings shows its Pro plan (a row-level-security read), without spending an AI request;
- the dedicated canary account gets a streamed GPT-5.6 Luna answer from `ai-stream`;
- the canary account generates a three-contention case with GPT-5.6 Luna through `ai`.
- the dedicated canary account gets a streamed Claude Haiku 5.5 answer from `ai-stream`;
- the canary account generates a three-contention case with Claude Haiku 5.5 through `ai`.

The last two checks are the only production writes, and they touch only the canary account: its `ai_requests` usage rows and whatever the Generate page saves for it. The dedicated canary account has Pro promo access capped at 10 requests a day. Captures are disabled because they could retain its session.

Expand Down
10 changes: 5 additions & 5 deletions e2e/production/canary.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,10 @@ async function signInAsCanary(page: Page, { aiRequest }: { aiRequest: boolean })
}, data.session);
}

// GPT-5.6 Luna is the Basic default, so this is the path most users take.
async function pickLuna(page: Page, feature: string) {
// Claude Haiku 5.5 is the Basic default, so this is the path most users take.
async function pickBasicDefault(page: Page, feature: string) {
await page.getByRole("combobox", { name: `Select AI model for ${feature}` }).click();
await page.getByRole("option", { name: /GPT-5\.6 Luna/ }).first().click();
await page.getByRole("option", { name: /Claude Haiku 5\.5/ }).first().click();
}

test("the production site loads and guards the app", async ({ page, baseURL }) => {
Expand Down Expand Up @@ -89,7 +89,7 @@ test("the canary account signs in and loads its account", async ({ page }) => {
test("the canary account gets a streamed answer from a real model", async ({ page }) => {
await signInAsCanary(page, { aiRequest: true });
await page.goto("/app/ask");
await pickLuna(page, "ask");
await pickBasicDefault(page, "ask");
await page.getByPlaceholder("Ask your debate coach...").fill("Reply with one short sentence: what is a rebuttal?");
const stream = page.waitForResponse((response) => response.url().endsWith("/functions/v1/ai-stream"), { timeout: 90_000 });
await page.getByRole("button", { name: "Send message" }).click();
Expand All @@ -111,7 +111,7 @@ test("the canary account generates a case", async ({ page }) => {
test.setTimeout(180_000);
await signInAsCanary(page, { aiRequest: true });
await page.goto("/app/generate");
await pickLuna(page, "generate");
await pickBasicDefault(page, "generate");
await page.getByRole("textbox", { name: "Motion / Topic *" }).fill("This House would make public transport free");
const generated = page.waitForResponse((response) => response.url().endsWith("/functions/v1/ai") && response.request().method() === "POST", { timeout: 150_000 });
await page.getByRole("button", { name: "Generate Debate Case" }).click();
Expand Down
Loading
Loading