Repository navigation
Add /debate-case-example: three real, unedited DebateOwl cases - #304
Merged
Merged
Conversation
The page shows a British Parliamentary and a Policy case in English and a Classroom case in Hungarian, each an R1 and an R2 generated through the Generate tool on the live verify stack with Claude Sonnet 5.5. The raw output, with the request, serving model and database rows, is in scripts/case-examples/, and the page renders from it unchanged. The page names each sample by the format setting it was generated under and the 4-6-6-3 structure the product writes. It links from the case generator guide, the case-writing guide and the home page guide hub, and is listed in llms.txt and llms-full.txt. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Caution Review failedThe pull request is closed.
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. CodeRabbit Verify
|
Two sources in the generated cases do not load in a browser: the tandfonline page for Levin (2025) stays behind a Cloudflare check, and the Senate Commerce Committee's February 2026 report PDF answers Access Denied. The page now links the author's APSA preprint of the same paper and the Internet Archive's copy of the same report. The counter's source line for Levin named the wrong journal, so it now names APSA Preprints. The raw output stays as generated. scripts/case-examples/corrections.json lists each replaced field with its raw value, the page applies it and says which links were replaced and why, and the generator fails if a raw file no longer matches a correction. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first "If they say" answer in the British Parliamentary sample says 16-year-olds have voted "since 1984 in Nicaragua and Brazil". Brazil's year is 1988, per the cited Levin paper and Brazil's 1988 constitution. The page keeps the generated wording and adds "[correction: Brazil since 1988]" right after it, linked to the renamed "Corrections after generation" list, which now explains this correction beside the two replaced links. corrections.json gains marker entries: the generator fails unless the words a marker follows appear exactly once in the raw text and the marker reads "[correction: ...]". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Programcode
marked this pull request as ready for review
November 3, 2026 17:59
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
/debate-case-example: three real DebateOwl cases, generated through the product's own Generate tool and shown without edits apart from formatting. Draft, for Adam to review. Do not merge.What's on the page
AI Debate Case Example: Real DebateOwl Output | DebateOwl, H1What does an AI-written debate case look like?. The first<p>in<main>is the 59-word answer box.<details>block: 2 backing-up paragraphs and 12 answers. All of it is in the static HTML.anthropic/claude-sonnet-5.5, the Pro default) and the generation date and time. It also covers how the cases were made, the one failed attempt, and the single difference from production (below)./. The page makes no claims about other tools.about→/#software,hasPart→ the samples), one CreativeWork per sample (creator→/#software), FAQPage with "Can AI write a debate case?" and "What should an AI debate case include?", and BreadcrumbList./ai-debate-case-generatorand/how-to-write-a-debate-case("See a real example") and from the homepage guide hub (Skills guides). Listed inllms.txtunder Main pages and inllms-full.txt. The sitemap picks it up with lastmod 2026-11-03.I used the guide hub rather than the homepage FAQ: the FAQ question there is "What does a DebateOwl case include?", and its answer is shared text in three languages.
How the cases were generated
.claude/skills/verify-debatelab/recipes/case-examples.spec.ts, ran on the localverify-debatelabstack in live mode (VERIFY_LIVE_MODEL=anthropic/claude-sonnet-5.5). For each sample it signs in a demo user, starts a debate with the motion and the Format setting, generates R1, then generates R2 in the same debate. It skips unlessCASE_EXAMPLESis set, sodrive allnever spends money on it.~/.config/debatelab/live-verify.env. No new secret, no production project, no production rows. 44 upstream calls, against a cap of 120.scripts/case-examples/*.json: the request body, the fullairesponse, the model that served every call, and theai_requestsandcasesrows.scripts/case-examples/README.mdexplains how to reproduce them.scripts/seo_case_example.pyrenders the page from those files, and the generator exits if a raw file's model, role, language or format doesn't match the page's label.generatecall was served byanthropic/claude-sonnet-5.5; the debate'sFormat: bp|policy|classroomreached the prompt (the Generate request itself still sendsformat: custom; format comes from the debate setting); and every R2 part had the R1 case in its debate context.Finding: one product error during the run
The first Hungarian R2 failed in the Edge Function with
Generate response is missing metadata and cannot satisfy the requested output shape.(supabase/functions/_shared/control-enforcement.ts:150). I reran the Hungarian sample from the start and it passed. Nothing else was rerun. A student would see "Generation failed" with Retry. This PR doesn't fix it; it probably deserves its own issue.Evidence links (checked 3 November 2026)
22 of 31 return 200. 9 return 403 behind a Cloudflare or Akamai bot check for every automated client I tried (Python, headless Chromium, Camofox). Those aren't counted as passes. The Edge Function drops any link its own research didn't return, so each of the 9 was an Exa result at generation time. Please open them in a normal browser.
tandfonline.com/…/17457289.2025.2560983→ https://preprints.apsanet.org/engage/apsa/article-details/6886c339728bf9025e8fa41acommerce.senate.gov/…/2026.02.05 CHSRA Follow-Up Report….pdf→ https://web.archive.org/web/20260811041844/https://www.commerce.senate.gov/wp-content/uploads/media/doc/2026.02.05%20CHSRA%20Follow-Up%20Report%20(Part%202%20of%20Acela%20Report).pdfReplaced citations (follow-up commit)
Adam checked the 9 bot-walled links in a real browser. Seven load and two don't, and the page now replaces those two and says so under "Corrections after generation":
curl200), whose page links the JEPOP version. The counter's source line said "Political Studies Review", which is the wrong journal (Crossref: Journal of Elections, Public Opinion and Parties, 6 Oct 2025), so it now reads "Levin, APSA Preprints, 2025". No other wording changed.curl200). I checked that the archived PDF says every claim the case attributes to the committee: no train after a quarter of a century and $2.6 billion, the first segment won't be profitable, and the $287 million NextGen Acela delay with slower trains. The case text is unchanged.The raw JSON stays as generated.
scripts/case-examples/corrections.jsonlists each replaced field with its raw value. The generator applies it and fails if a raw file no longer matches.Brazil year (follow-up commit
7d1eb333): the same BP counter says 16-year-olds "have voted since 1984 in Nicaragua and Brazil". Per the cited paper and Brazil's 1988 constitution, it's Nicaragua since 1984 and Brazil since 1988. The generated wording stays, followed by an inline[correction: Brazil since 1988]marker that links to the page section, now called "Corrections after generation", where the correction is explained.corrections.jsonlogs it as a marker entry, and the generator fails unless the marked words appear exactly once in the raw text.I didn't check that each quoted figure appears on its source page. The page tells readers to check every source before a round. Per the no-edit rule, no evidence item was dropped or changed.
Checks
npm run verifyat 7d1eb33 (and earlier at 1b94710 and 421f509): all 9 steps passed (docs, jargon, lint, build including check-site and check-links, sw-offline, typecheck, guards, edge, smoke).🤖 Generated with Claude Code