Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .claude/skills/verify-debatelab/features/case-generation.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Case generation

A signed-in user enters a motion on Generate and receives a structured debate case: an opening (hook and "we will show N arguments" line), arguments (title, point, explanation, evidence, why it matters), an "If they say…" list, a closing line and a sources list. The role shapes it: the Introductor may add one definition line; R2 brings no new arguments, only `Backing up our case` (the team's earlier arguments) and "If they say…" with one full answer for each thing the other side is likely to say; the Conclusor has 2-3 "Why we win" points instead of new arguments. Saved cases from before still show their old sections under plain names (`How to judge this debate`, `What we have to show`, `They may also say`) when they have content.
A signed-in user enters a motion on Generate and receives a structured debate case: an opening (hook and "we will show N arguments" line), arguments (title, point, explanation, evidence, why it matters), an "If they say…" list, a closing line and a sources list. The role shapes it: the Introductor may add one definition line; R2 brings no new arguments, only `Backing up our case` (the team's earlier arguments) and "If they say…" with one full answer for each thing the other side is likely to say; the Conclusor has 2-3 "Why we win" points instead of new arguments. An R2 case makes five provider calls, all `generate` in `providerCalls()`: a plan, then four parts written at the same time; the last one in the list still carries `Role: r2`. Saved cases from before still show their old sections under plain names (`How to judge this debate`, `What we have to show`, `They may also say`) when they have content.

## Sub-features

Expand All @@ -10,7 +10,7 @@ A signed-in user enters a motion on Generate and receives a structured debate ca
- `gen-quota` shows the dialog `Daily AI uses used up` when the account has no quota, without calling the provider.
- `gen-save` saves a one-off case (the motion dialog answered with `Not now, use it once`) under a title into `cases` with `debate_id: null`. It appears in History under `One-off and unsorted work`, including from a fresh browser context.
- `gen-regenerate` regenerates one section. Only that section shows the overlay `Regenerating <section>…`; the rest of the case and the result toolbar stay on screen, and every regenerate button and the form's submit button are disabled until it finishes.
- `gen-roles` shows the role's shape: on `/app/generate` pick the `Speaker Role` option `Introductor (4 min)` and the Opening card holds a `What we mean:` line; pick `R2 - Rebuttal/Build (6 min)` and there is no `Arguments` section, `Backing up our case` holds the team's earlier arguments, and `If they say…` holds six full answers covering their main arguments, supporting points and answers to our case (standard length); pick `Conclusor (3 min)` and `Why we win` replaces `Arguments`.
- `gen-roles` shows the role's shape: on `/app/generate` pick the `Speaker Role` option `Introductor (4 min)` and the Opening card holds a `What we mean:` line; pick `R2 - Rebuttal/Build (6 min)` and there is no `Arguments` section, `Backing up our case` holds the team's earlier arguments, and `If they say…` holds twelve full answers covering their main arguments, supporting points and answers to our case (standard length); pick `Conclusor (3 min)` and `Why we win` replaces `Arguments`.
- `gen-export` downloads a `.pdf` (starting with `%PDF`) and a `.docx` (starting with `PK`).
- `gen-flowchart` builds a flowchart from the Current Debate's case. `/app/flowchart` opens the template; the button `Build from current case` replaces it with the thesis, one node per argument (title, point and evidence, or a Conclusor's reasons why we win), what the other side will say, and our answers, fitted to the canvas. Node titles the app wrote (`Thesis`, `Argument 1`) follow the UI language.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -194,12 +194,12 @@ test("gen-roles: the speaker role shapes the case: Introductor, R2 and Conclusor
await expect(page.getByRole("button", { name: "Regenerate If they say…" })).toBeVisible();
await expect(page.getByText("They say", { exact: true })).toHaveCount(r2.counters.length);
// One full answer for each thing they are likely to say: standard length
// asks for six, and the Edge Function keeps a live reply with at least four.
// asks for twelve, and the Edge Function keeps a live reply with at least nine.
if (LIVE) {
expect(r2.counters.length).toBeGreaterThanOrEqual(4);
expect(r2.counters.length).toBeLessThanOrEqual(6);
expect(r2.counters.length).toBeGreaterThanOrEqual(9);
expect(r2.counters.length).toBeLessThanOrEqual(12);
} else {
expect(r2.counters).toHaveLength(6);
expect(r2.counters).toHaveLength(12);
}
await evidence.shot("02-r2");
await evidence.aria("02-r2");
Expand Down
2 changes: 1 addition & 1 deletion docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ Every structured answer is shown while the model writes it. A page passes a `liv

A streamed run keeps its worker alive with `EdgeRuntime.waitUntil` ([`_shared/background.ts`](../supabase/functions/_shared/background.ts)) from the first event until the HTTP layer has read the last one. Supabase otherwise counts the response as sent once its headers are out and may retire the worker mid-run (`EarlyDrop`). Streamed Ask (`ai-stream`) is held the same way until the browser has read its final event. A run still unanswered about 142 s after its request arrived is aborted and gets a retryable timeout `result` instead of a connection cut at the 150 s wall clock; when the aborted run answers within 2 s, that answer is sent instead, so it says `charged: false` when the units went back. Storing the result and waiting for the browser to read it are cut short to finish before the wall clock. The request sends its id as `resultKey`. The function then writes a row to `ai_run_results` when the run starts, and stores the `result` there before sending it ([`ai/run-results.ts`](../supabase/functions/ai/run-results.ts); one row per user and feature, readable only by its owner). When the stream drops, stalls (no keepalive for 45 s), or delivers a result it cannot read, [`src/lib/ai/run-recovery.ts`](../src/lib/ai/run-recovery.ts) polls that row until the run's result appears, then resolves or fails exactly as if the event had arrived. A row still without a status long after it started is a run its worker never finished. A browser Stop, reload, or closed tab still cancels the run. The server cannot tell those from a lost connection, so it stops spending as soon as the connection goes.

A reply that hits its token cap is written again once with twice the cap, after a `reset`. The Generate cap (12,000 tokens) leaves room for a full comprehensive case on Claude Haiku, so a long case finishes on its first attempt. Claude rejects an assistant prefill, so a cut-off reply cannot be continued where it stopped. Sparring calls send a strict `json_schema` (`sparringOutputSchema` in `ai/sparring-persistence.ts`). Anthropic's structured outputs take no numeric or array-size limits, so the 0-25 range, the four categories, and the 12-item lists are enforced by the parser. When a reply parses but has the wrong shape, the log records only the failing field path, never the model's text. The call is then retried once with a repair prompt that names the field. A second failure answers `invalid_output` (502, logged as `invalid_output`), which the browser shows as a malformed answer. The judge also retries once when Claude is briefly unavailable or its stream breaks. The round has already paid, so retries cost the student nothing.
A reply that hits its token cap is written again once with twice the cap, after a `reset`. The Generate cap (12,000 tokens) leaves room for a full comprehensive case on Claude Haiku, so a long case finishes on its first attempt. Claude rejects an assistant prefill, so a cut-off reply cannot be continued where it stopped. An R2 case is the longest reply (the speech and 10-12 full answers), and one call wrote it in up to 124 s of the 130 s budget, which cannot grow because both Supabase projects are on the Free plan and stop a worker at 150 s. So [`ai/r2-case.ts`](../supabase/functions/ai/r2-case.ts) asks for a short plan first (our two arguments and the claims to answer, at low effort), then writes four parts at the same time at the student's model and effort: part 1 the speech and its share of the answers, streamed to the browser, the others only answers. Each part has its own smaller cap (`caseWriterPart`), which also keeps its reasoning budget small enough to finish well inside the budget. One failed part fails the case, stops the others, and bills every call. Sparring calls send a strict `json_schema` (`sparringOutputSchema` in `ai/sparring-persistence.ts`). Anthropic's structured outputs take no numeric or array-size limits, so the 0-25 range, the four categories, and the 12-item lists are enforced by the parser. When a reply parses but has the wrong shape, the log records only the failing field path, never the model's text. The call is then retried once with a repair prompt that names the field. A second failure answers `invalid_output` (502, logged as `invalid_output`), which the browser shows as a malformed answer. The judge also retries once when Claude is briefly unavailable or its stream breaks. The round has already paid, so retries cost the student nothing.

For a validated Crossfire next turn, the server returns the reply first with a short-lived signed sidecar token. The client then requests a Jev classification of only the newest completed question-answer exchange. The token is atomically claimed through its own one-use counter, separate from the product's daily AI quota. The optional directness, concession, and follow-up signal is attached by turn ID to the answer; a missing, slow, replayed, forged, or failed sidecar never delays or invalidates the main reply. Draft version 1 remains compatible because turn signals are optional.

Expand Down
28 changes: 26 additions & 2 deletions e2e/full-stack/fake-openrouter.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,10 @@ const counter = (index) => ({
"Cities cannot afford the extra buses.",
"Fares already have discounts for people who need them.",
"Free transport has been tried and dropped elsewhere.",
"Better buses would help more than free buses.",
"Fare-free cities still have empty buses at night.",
"Taxpayers who never ride would pay for everyone else.",
"Free fares crowd out walking and cycling.",
][index],
counterpoint: `${MARK} answer ${index + 1}: fare collection itself costs a large share of what it brings in, so the real gap is far smaller.`,
...(index % 2 === 0 ? { source: "UITP, 2024" } : {}),
Expand All @@ -87,13 +91,33 @@ const r2Answer = (index) => ({
// theirs, and the Conclusor brings reasons why we win. Neither R2 nor the
// Conclusor brings new arguments.
const generate = (user) => {
// R2 is planned, then written in parts (supabase/functions/ai/r2-case.ts).
const planCount = /PLANNING STEP[\s\S]*?theirClaims: exactly (\d+)/.exec(user)?.[1];
if (planCount) {
return {
ourArguments: [
{ title: "Fares shut people out", claim: "Fares keep the riders who need transit most off the bus." },
{ title: "More people ride", claim: "Free fares bring more passengers onto public transport." },
],
theirClaims: Array.from({ length: Number(planCount) }, (_, index) => `Free transit claim ${index + 1}: it costs the city too much.`),
};
}
const part = /=== WRITING PART (\d+) OF \d+: claims (\d+)-(\d+) ===/.exec(user);
if (part && part[1] !== "1") {
const [first, last] = [Number(part[2]), Number(part[3])];
return {
counters: Array.from({ length: last - first + 1 }, (_, index) => r2Answer(first - 1 + index)),
sources: [{ org: "UITP", year: "2024", claim: "Ridership rose after fares were removed", url: "https://www.uitp.org/free-fares-2024" }],
};
}
const length = /^Length: (\w+)/m.exec(user)?.[1] ?? "standard";
const role = /^Role: (\w+)/m.exec(user)?.[1] ?? "r1";
const count = { brief: 2, standard: 3, comprehensive: 4 }[length] ?? 3;
const conc = role === "conc";
const r2 = role === "r2";
// R2: one full answer for each thing they are likely to say.
const answers = length === "brief" ? 5 : 6;
// R2: one full answer for each thing they are likely to say; part 1 of a
// split R2 case writes only its own share of them.
const answers = part ? Number(part[3]) - Number(part[2]) + 1 : length === "brief" ? 10 : 12;
return {
opening: {
hook: `${MARK} opening hook: picture a nurse who skips the night bus because the fare is a tenth of her shift's pay.`,
Expand Down
36 changes: 32 additions & 4 deletions supabase/functions/_shared/control-enforcement.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -239,10 +239,11 @@ Deno.test("a Conclusor case without at least two reasons we win is rejected, not
});

Deno.test("an R2 case must back up the team's arguments and answer theirs, or it is rejected", () => {
const answers = (count: number) => Array.from({ length: count }, () => ({}));
const cases: Array<[string, Record<string, unknown>]> = [
["no reinforce", { reinforce: [], counters: [{}, {}, {}, {}, {}], contentions: [], metadata: {} }],
["only blank reinforce", { reinforce: [{ argument: "Our first", text: " " }], counters: [{}, {}, {}, {}, {}], contentions: [], metadata: {} }],
["too few answers", { reinforce: [{ argument: "Our first", text: "Backed up." }], counters: [{}, {}, {}], contentions: [], metadata: {} }],
["no reinforce", { reinforce: [], counters: answers(12), contentions: [], metadata: {} }],
["only blank reinforce", { reinforce: [{ argument: "Our first", text: " " }], counters: answers(12), contentions: [], metadata: {} }],
["too few answers", { reinforce: [{ argument: "Our first", text: "Backed up." }], counters: answers(8), contentions: [], metadata: {} }],
];
for (const [label, output] of cases) {
let rejected = false;
Expand All @@ -257,14 +258,41 @@ Deno.test("an R2 case must back up the team's arguments and answer theirs, or it
// New arguments from the model are dropped: R2 brings none.
const output: Record<string, unknown> = {
reinforce: [{ argument: "Our first", text: "Backed up." }],
counters: [{}, {}, {}, {}, {}],
counters: answers(12),
contentions: [{ claim: "A new point", whyItMatters: "x" }],
metadata: {},
};
enforceGenerateControls(output, "con", "custom", "standard", "r2");
if ((output.contentions as unknown[]).length !== 0) throw new Error("R2 must not keep new arguments");
});

Deno.test("an R2 case a few answers short of its target is kept, one well short is rejected", () => {
const run = (length: string, count: number) => {
const output: Record<string, unknown> = {
reinforce: [{ argument: "Our first", text: "Backed up." }],
counters: Array.from({ length: count }, (_, index) => ({ targetArgument: `claim ${index}` })),
contentions: [],
metadata: {},
};
enforceGenerateControls(output, "con", "custom", length, "r2");
return (output.counters as unknown[]).length;
};
const rejects = (length: string, count: number) => {
try {
run(length, count);
return false;
} catch (error) {
return error instanceof Error && error.message.includes(`R2 needs ${length === "brief" ? 10 : 12}`);
}
};
// Standard asks for 12 and keeps a reply of 9; brief asks for 10 and keeps 7.
if (run("standard", 9) !== 9 || run("brief", 7) !== 7) throw new Error("a slightly short R2 reply must be kept as it is");
if (run("standard", 15) !== 12 || run("brief", 15) !== 10) throw new Error("extra R2 answers must be trimmed to the target");
if (!rejects("standard", 8) || !rejects("brief", 6) || !rejects("comprehensive", 8)) {
throw new Error("an R2 reply well short of its target must be rejected");
}
});

Deno.test("why it matters is filed under the key saved cases already use", () => {
const output: Record<string, unknown> = {
contentions: [{ claim: "a", whyItMatters: "People reach work." }, { claim: "b", impact: "Kept.", whyItMatters: "Ignored." }],
Expand Down
7 changes: 4 additions & 3 deletions supabase/functions/_shared/control-enforcement.ts
Original file line number Diff line number Diff line change
Expand Up @@ -192,12 +192,13 @@ export function enforceGenerateControls(output: unknown, side: string, format: s
: [];
const answers = Array.isArray(generateOutput.counters) ? generateOutput.counters : [];
// One full answer for each thing they are likely to say: their main
// arguments, supporting points and answers to our case.
const target = length === "brief" ? 5 : 6;
// arguments, supporting points and answers to our case. A reply a few
// short of the target is still a usable speech, so only a thin one fails.
const target = length === "brief" ? 10 : 12;
if (reinforce.length < 1) {
throw new Error("Generate response returned no reinforce items, but R2 must back up the team's arguments.");
}
if (answers.length < 4) {
if (answers.length < target - 3) {
throw new Error(`Generate response returned ${answers.length} counters, but R2 needs ${target}.`);
}
generateOutput.reinforce = reinforce.slice(0, 3);
Expand Down
13 changes: 13 additions & 0 deletions supabase/functions/_shared/output-caps.ts
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,18 @@ export const REPLY_TOKEN_CAPS = {
* limit. The cap only bounds the reply, so room costs nothing unless used.
*/
generate: 12_000,
/**
* The R2 plan: our two arguments and the claims the other side will make,
* one line each. Written at low effort before the parts below.
*/
casePlan: 1_500,
/**
* One of the parts an R2 case is written in at the same time: the speech
* and three answers, or three answers. A part is a quarter of the case, so
* its smaller cap also keeps its reasoning budget (a share of max_tokens)
* small enough that the slowest part finishes well inside the 130 s budget.
*/
caseWriterPart: 6_000,
attack: 4_000,
brainstorm: 4_000,
factoids: 4_000,
Expand Down Expand Up @@ -66,6 +78,7 @@ const REASONING_HEADROOM: Record<ReasoningEffort, number> = { low: 2_048, medium
*/
export const REASONING_FLOOR_TOKENS: Partial<Record<ReplyCap, number>> = {
generate: 10_000,
caseWriterPart: 10_000,
};

/**
Expand Down
Loading
Loading