Repository navigation
Fix case generator issues from case-quality batch 1 - #263
Conversation
- Spoken text never mentions the research or missing data; gaps go to sourceQuery with needsSource. - Generate research prefers Hungarian or Spanish sources for cases in those languages, with international sources as the fallback. - R2 asks for 10-12 full "If they say" blocks; reinforce paragraphs are 100-120 words in both the role text and the schema; the R2 counter target is 10/12 and a reply up to three short is kept. - Each role states its word budget at 130 spoken words a minute. - The no-conceding, no-study-method and no-jargon rules apply to every speaker role. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
| Layer / File(s) | Summary |
|---|---|
R2 prompts and speech budgets supabase/functions/_shared/prompts/generate.ts, supabase/functions/_shared/prompts/prompt-examples.test.ts, supabase/functions/ai/prompt-builder.test.ts |
Generation prompts add role-specific speech budgets and revise spoken-content and evidence instructions. R2 prompts specify answer counts, lengths, and plan and part-writing instructions. |
R2 planning and parallel writing supabase/functions/ai/r2-case.ts, supabase/functions/ai/r2-case_test.ts, supabase/functions/ai/features/generate.ts, supabase/functions/_shared/output-caps.ts, e2e/full-stack/fake-openrouter.mjs, docs/architecture.md |
The R2 writer validates a plan, writes four claim parts concurrently, streams part 1, merges outputs, and tracks usage. The generation entry point routes R2 requests to the writer. Output caps, fake responses, tests, and architecture documentation cover this flow. |
R2 answer counts and validation supabase/functions/_shared/control-enforcement.ts, supabase/functions/_shared/control-enforcement.test.ts, supabase/functions/ai/control-matrix.test.ts, .claude/skills/verify-debatelab/features/case-generation.md, .claude/skills/verify-debatelab/recipes/case-generation.spec.ts |
R2 targets increase to 10 answers for brief cases and 12 otherwise. Enforcement tests and case-generation checks cover the updated counts. |
Locale-aware research
| Layer / File(s) | Summary |
|---|---|
Locale-specific source selection and ranking supabase/functions/_shared/source-quality.ts, supabase/functions/ai/source-quality.test.ts |
Source-quality helpers resolve Hungarian and Spanish locale preferences. Search requests can add a national-source query, and result ranking adds a bonus for matching country domains. Tests cover supported and unsupported locales. |
Locale propagation through research supabase/functions/ai/features/generate.ts, supabase/functions/_shared/research.ts, supabase/functions/_shared/research.test.ts |
Generation passes language and derived country preferences to deep research. Deep research uses the locale for search requests, result ranking, and follow-up-query prompts. Tests check Hungarian source selection and English behavior. |
Estimated code review effort: 3 (Moderate) | ~25 minutes
Sequence Diagram(s)
sequenceDiagram
participant handleGenerate
participant deepResearchInner
participant buildSearchRequests
participant Exa
participant rankSearchResults
handleGenerate->>deepResearchInner: Pass language and country preferences
deepResearchInner->>buildSearchRequests: Build seed requests with locale
buildSearchRequests-->>deepResearchInner: Return locale-aware requests
deepResearchInner->>Exa: Search with seed and follow-up requests
Exa-->>deepResearchInner: Return search results
deepResearchInner->>rankSearchResults: Rank results with locale
rankSearchResults-->>deepResearchInner: Return ranked results
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Note
🎁 Summarized by CodeRabbit Free
-
Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Essentials by visiting https://app.coderabbit.ai/settings/billing.
Comment @coderabbitai help to get the list of available commands.
…ce-limited rule in the no-research-talk override, give general-role cases R1 counters
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… target R2 cases came within 6 s of the 130 s request budget (124 s for a Hungarian case at medium effort). Both Supabase projects are on the Free plan, which stops a worker at 150 s, so the budget cannot grow. An R2 case is now planned at low effort (our two arguments and the claims to answer), then written in four parts at the same time at the student's model and effort. Part 1 writes the speech and streams to the browser. Each part has its own smaller cap, which keeps its reasoning budget at 8,000 tokens at medium. Measured: 68 s, 58 s and 53 s, down from 124, 109 and 105. A failed part stops the others and every call is billed. R1 speeches ran to 752 and 823 words against a 600-word budget. The Speech Budget line now gives each argument field a word target derived from the speaker's time and the number of arguments. Measured: 533 and 531. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fixes the case generator (Generate tool) against the findings from case-quality batch 1 (12 cases,
~/.t3/scratch/case-quality/batch1/). The reference is Adam's own debate prints. In those, R2 brings no new arguments, spends about 2 minutes backing up our arguments, then about 4 minutes on many full "If they say X, we answer Y" blocks.Fixes, mapped to batch 1
1. Spoken text no longer talks about the research.
Batch 1 had spoken lines like "We do not have the survey numbers in our research" (12, en intro), "a kutatásban nincs olyan adat…" (10, hu R1), "A kapott kutatásból nem tudunk nemzetközi összehasonlító adatot idézni" (09) and "the research shows even governments are unsure" (08). The generate prompt now says the research is for the writer. No spoken field may mention it, what it doesn't cover, missing data or an undated source. With no number, the speaker makes the point without one. A note on missing evidence goes only into
sourceQuerywithneedsSource: true. Theevidencerule used to say "describe the evidence still needed", which pushed the gap into a spoken field. It now says to give the best example or reasoning instead. The prompt also overrides the shared research block's "state that evidence is limited" line, which other features still use.2. Hungarian and Spanish cases look for national sources first.
Batch 1's Hungarian R2 on safe schools (06) cited Brazil, the CDC, YouthTruth and Victoria, and no Hungarian source.
deepResearchnow accepts optionallanguageandcountryfields on its existing options object. The signature is unchanged, and calls without them, or withlanguage: "en", send exactly the same requests as before (tested).huores, the first seed search goes to that country's sources: KSH, kormany.hu, Oktatási Hivatal, Hungarian universities and press; or INE, gob.es, BOE, CIS, Spanish universities and press. It comes first so the four-query limit can't drop it. When it finds nothing, it retries without the domain filter. The usual international searches follow as the fallback..hu/.essite) +12. Without that bonus, the five pages read in full would still be mostly foreign.3. R2: 10-12 full "If they say" blocks, one reinforce length.
Batch 1's R2 cases had 6 blocks of 170-190 words. The prompt now asks for 10-12 blocks covering their main arguments, then the supporting points behind each, then their answers to each of ours. Each block must be complete enough to say word for word. Brief asks for 10 answers of 100-130 words, standard for 12 of 120-150, and comprehensive for 12 of 150-180. That puts comprehensive at about 2,200 words of answers, inside the 12,000-token reply cap even in Hungarian. The contradiction is fixed: the role text and the JSON schema both say reinforce paragraphs are about 100-120 words (two paragraphs is roughly 2 minutes with the opening).
control-enforcement.tsnow targets 10 (brief) or 12 counters and keeps a reply up to three short (7 or 9 minimum), so a slightly short reply doesn't throw. Extra answers are trimmed. R1 still has no counter minimum in code: batch 1's R1s already had 5-8, the prompt asks for 5-8, and a minimum would break the fake provider's three-counter R1 used by the browser recipes.4. Word budgets per speech role.
Batch 1's spoken text often ran over the time (the analyzer counted 960 spoken words for case 02, a 6-minute R1). Each role's request now carries a
Speech Budget:line at 130 spoken words a minute, and the role text repeats it:prompt-builder.test.tschecks that the prompt for every role and length carries its budget, and that a case with no speaker role carries none.5. Rules that now apply to every speaker, not only R2.
Batch 1's R1 cases had "that cost is real" (01), "we agree" (04), "cross-sectional" (01, 07) and "meta-analysis" (03), and "contentions" turned up in every case. A new "SPOKEN TEXT RULES (every speaker)" section bans conceding, study-method words (say "a study of 10,000 pupils" instead), and debate jargon in sentences ("contentions", "weighing", "warrant", "framework" and similar). It notes these may be JSON keys but never words the speaker says. Sources from the last five years are preferred (it was two).
The case JSON shape is unchanged: no field was added or renamed.
Kept in step
e2e/full-stack/fake-openrouter.mjsreturns 10 or 12 R2 answers.case-generationrecipe expects 12 R2 answers on the fake stack and 9-12 live. Its feature doc says twelve.MAX_CASE_COUNTERS), within the 12,000-character case digest. I left that limit alone.Verification
All local, at head
98e7a984. This is a local pass, not CI.NODE_OPTIONS=--max-old-space-size=3072 npm run verify: exit 0. docs:check, seo:jargon, lint, typecheck clean; vitest 714/714; collaboration 30/30; edge (Deno) 502/502; build ok; test:sw ok; Playwright smoke 47/47.$H up,drive recipes/case-generation,$H down): 10/10 ondesktop-chromiumand 10/10 onmobile-chromium, including the R2 check for 12 answers.ai/source-quality.test.ts); national ranking bonus;deepResearchin Hungarian searches, asks for and reads Hungarian sources first, and an English call is identical to the old call shape (_shared/research.test.ts); R2 targets 10/12 with the three-short floor (control-enforcement.test.ts,control-matrix.test.ts); speech budget per role and new R2 counts (prompt-builder.test.ts); the new prompt rules (prompt-examples.test.ts).No live model, staging function or Exa call was made. Everything above is unit and prompt-text tests plus the fake-provider stack, so none of it proves the model now writes better cases.
Before this ships
aiEdge Function needs a deploy, staging first and then production. A merge tomainonly ships the frontend.🤖 Generated with Claude Code
Summary by CodeRabbit