Stream every AI answer live, token by token by Programcode · Pull Request #149 · Programcode/DebateOwl

Programcode · 2026-10-17T15:39:43Z

Not merging yet: you asked for full send without merge. Video and screenshots are attached below (live mode: real OpenRouter pinned to Muse Spark contributor, real Exa).

live-generation-demo.mp4

Every AI answer in DebateLab now appears while the model writes it, not after the whole reply lands.

What changes for students

Generate, Attack, Brainstorm, Factoids show a live draft under a "Writing live" bar with Cancel. The result's own sections (Case Structure, Arguments, Weaknesses, …) fill in token by token, with a caret where text is arriving. Before the model starts, the status line says what the request is doing ("Researching sources…", "Writing…", "Checking which sources the answer used…"). When the answer is complete, the normal result with its toolbar replaces the draft.
Ask with citations, web search, pros/cons, or argument map now fills its bubble live too (plain Ask already streamed through ai-stream, unchanged). Stop now cancels these requests as well.
Crossfire: the opponent's question or answer is written in place as the next transcript card; the debrief fills in section by section.
Sparring: the opponent's speech is written in place; the judge's scorecard fills in while it is judged.
In voice rounds the live text has audio tags removed, as the finished reply does.

How it works

ai Edge Function: a request with stream: true gets Server-Sent Events (supabase/functions/ai/stream.ts): phase, delta (raw model text), reset (a JSON repair retry starts again), and one final result with the status and body a plain request returns. Every check before the feature handler (auth, input, debate context, quota, model routing, source ranking) still answers with plain JSON and its real status.
provider.ts: only the main call whose output the user sees streams (stream: true + stream_options.include_usage). Research sub-calls and Jev never stream. The joined text goes through the same parseProviderContent, so outputs are identical; streamed usage is billed, and marked incomplete if the stream breaks.
Client: src/lib/ai/live.ts reads the stream and turns a failed result into the same FunctionsHttpError a plain request gets, so limit, paywall, and provider-error handling is unchanged. A JSON body is accepted too, so this client works against the currently deployed ai function until it is redeployed.
packages/core/src/ai/partial-json.ts: a tolerant prefix parser (cut strings, escapes, surrogate pairs, partial literals and numbers, fences). Drafts never show a value that may still change.
Requests without a live callback (section regeneration, Crossfire signals, Workspace practice) are unchanged.

Tests

New: partial-json.test.ts (parser, every prefix of a real answer), src/test/ai/live.test.ts (stream reader: JSON fallback, split events, reset, error mapping, truncated stream), supabase/functions/ai/stream.test.ts (streamed body equals the plain body byte for byte, repair retry resets and bills both attempts, provider failure, pre-checks stay JSON).
verify-debatelab: new recipes/live-generation.spec.ts (draft on screen mid-stream, Cancel, every tool's event stream) and expectStreamed in the practice recipes. features/live-generation.md, SKILL.md, and the feature index are updated. The fake provider streams structured answers in small chunks.
Test helpers read ai responses through e2e/helpers/ai-response.ts (event stream or JSON), including the live, canary, and staging specs.
Full-stack specs now wait for the finished result (expectResult), since a draft shows result text before the result is saved. Three of them also had stale Library selectors from the design shell; fixed.

Screenshots (live mode, mid-stream):

Verification (local; GitHub Actions is blocked by billing)

Every Actions job is refused before it starts ("recent account payments have failed or your spending limit needs to be increased"), so these are local runs, not CI passes.

At the current head:

npm run verify (lint, typecheck, unit, edge, build, service worker, desktop smoke 30/30): pass.
Database gate (.superset/db-gate.sh): pgTAP 419/419, browser suite 71 passed, 1 skipped.
npm test 301 passed, npm run test:edge 339 passed.

Earlier in the PR (each run again after the later changes it touched):

verify-debatelab fake mode: drive all 72 passed, 4 skipped (Workspace autosave needs --collab), desktop and mobile; e2e/full-stack/ 68 passed; agentic journeys 24 passed; ai-tools, auth, live-generation, full-stack case-generation rerun at the end.
verify-debatelab live mode: recipes/live-generation 6/6 (desktop + mobile, Cancel included), recipes/practice 6/6, e2e/live/ 8/8.

Codex review: five rounds. Rounds 1–3 found five issues (frame-size cap, rejoined Sparring drafts, phase order, and two cases of live text crossing an account change); all are fixed with tests, and the replies are in the comments below. Rounds 4 and 5 (current head): no findings.

Found and fixed along the way: the Jam recorder wraps window.fetch and keeps an aborted response streaming, so Cancel did not stop a run (the request finished in the background and still autosaved its sources). The live reader now cancels itself on abort. Plain requests without live generation still have that limitation; it predates this PR.

Not covered: staging and production (the ai Edge Function is not deployed by a merge), WebKit, voice audio from ElevenLabs.

Deploy notes

Merging ships the client only. The client falls back to JSON, so it works against today's ai function, just without live drafts. Live generation needs the ai Edge Function deployed (staging, then production).

🤖 Generated with Claude Code

Summary by CodeRabbit

New Features
- AI responses now appear as they’re generated across Generate, Attack, Brainstorm, Factoids, Ask with citations or web search, Crossfire, and Sparring.
- Follow progress through research, writing, and source-checking status updates. Cancel generation where available.
- Sparring now shows the opponent’s speech and judging details as they arrive.
Documentation
- Added guidance on live generation and its progress indicators.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Live drafts show result text while the request still runs, so specs now wait for the draft to close before reading rows. Also fix the Library form selectors these specs still used from before the design shell. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The Jam recorder wraps window.fetch and keeps an aborted response streaming, so Cancel left the page waiting for the whole answer. The live reader now cancels itself on abort. The live proxy records the answering model from the first chunk and stops the upstream call when its caller hangs up. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…ded sources Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

vercel · 2026-10-17T15:39:51Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
debate-navigator	Ready	Preview	Oct 17, 2026 4:32pm UTC
debatelab-staging	Ready	Preview	Oct 17, 2026 4:32pm UTC

coderabbitai · 2026-10-17T15:40:13Z

Warning

Review limit reached

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

Next included review available in 8 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)
Review profile: CHILL
Plan: Free
Run ID: 938e3905-6e0e-4b4f-96d8-d13c0516efbc

📥 Commits

Reviewing files that changed from the base of the PR and between d74f845 and baf3f8d.

📒 Files selected for processing: 15

.claude/skills/verify-debatelab/SKILL.md
.claude/skills/verify-debatelab/recipes/live-generation.spec.ts
e2e/full-stack/fake-openrouter.mjs
e2e/full-stack/stack-core.ts
src/components/features/tools/tool-run.tsx
src/lib/ai/features/sparring.ts
src/lib/ai/live.ts
src/lib/supabase/AuthProvider.tsx
src/pages/AskPage.tsx
src/test/ai/live.test.ts
src/test/ai/sparring-live.test.ts
src/test/pages/AskAccountSwitch.test.tsx
src/test/tools/tool-run.test.tsx
supabase/functions/ai/provider.ts
supabase/functions/ai/stream.test.ts

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)
Review profile: CHILL
Plan: Free
Run ID: b8bc9853-f1cb-4efc-a78c-17c75076722c

📥 Commits

Reviewing files that changed from the base of the PR and between df25e98 and d74f845.

📒 Files selected for processing: 69

.claude/skills/verify-debatelab/SKILL.md
.claude/skills/verify-debatelab/features/README.md
.claude/skills/verify-debatelab/features/ai-tools.md
.claude/skills/verify-debatelab/features/live-generation.md
.claude/skills/verify-debatelab/recipes/fixtures.ts
.claude/skills/verify-debatelab/recipes/live-generation.spec.ts
.claude/skills/verify-debatelab/recipes/practice.spec.ts
docs/architecture.md
e2e/full-stack/account-data.spec.ts
e2e/full-stack/ai-tools.spec.ts
e2e/full-stack/auth.spec.ts
e2e/full-stack/billing.spec.ts
e2e/full-stack/case-generation.spec.ts
e2e/full-stack/current-debate.spec.ts
e2e/full-stack/debate-context.spec.ts
e2e/full-stack/fake-openrouter.mjs
e2e/full-stack/first-run.spec.ts
e2e/full-stack/stack.ts
e2e/full-stack/voice.spec.ts
e2e/helpers/ai-response.ts
e2e/live/live.ts
e2e/live/research.spec.ts
e2e/production/canary.spec.ts
e2e/staging/authed-flows.spec.ts
packages/core/src/ai/partial-json.ts
src/components/features/sparring/SparringRound.tsx
src/components/features/tools/live-draft.tsx
src/components/features/tools/live-sections.ts
src/components/features/tools/tool-run.tsx
src/lib/ai/client.ts
src/lib/ai/features/ask.ts
src/lib/ai/features/attack.ts
src/lib/ai/features/brainstorm.ts
src/lib/ai/features/crossfire.ts
src/lib/ai/features/factoids.ts
src/lib/ai/features/generate.ts
src/lib/ai/features/sparring.ts
src/lib/ai/live.ts
src/lib/ai/partial-json.test.ts
src/lib/ai/request.ts
src/lib/ai/sse.ts
src/lib/ai/streaming.ts
src/lib/supabase/client.ts
src/lib/voice/live-text.ts
src/locales/en.json
src/locales/es.json
src/locales/hu.json
src/pages/AskPage.tsx
src/pages/AttackPage.tsx
src/pages/BrainstormPage.tsx
src/pages/CrossfirePage.tsx
src/pages/FactoidsPage.tsx
src/pages/GeneratePage.tsx
src/pages/SparringPage.tsx
src/test/ai/live.test.ts
supabase/README.md
supabase/functions/_shared/research.ts
supabase/functions/ai/features/ask.ts
supabase/functions/ai/features/attack.ts
supabase/functions/ai/features/brainstorm.ts
supabase/functions/ai/features/crossfire.ts
supabase/functions/ai/features/factoids.ts
supabase/functions/ai/features/generate.ts
supabase/functions/ai/features/sparring.ts
supabase/functions/ai/features/types.ts
supabase/functions/ai/index.ts
supabase/functions/ai/provider.ts
supabase/functions/ai/stream.test.ts
supabase/functions/ai/stream.ts

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

📝 Walkthrough

The change adds streamed structured AI responses across the server, client, and interface. The server sends progress and text events followed by a final result. The client parses partial JSON and pages display drafts. Tests and documentation cover streaming, cancellation, and response handling.

Changes: Live AI generation

Layer / File(s)	Summary
Provider and server streaming `supabase/functions/ai/provider.ts`, `supabase/functions/ai/stream.ts`, `supabase/functions/ai/index.ts`, `supabase/functions/ai/features/*`, `supabase/functions/ai/stream.test.ts`, `supabase/functions/_shared/research.ts`, `docs/architecture.md`, `supabase/README.md`	The AI endpoint accepts `stream: true` and sends SSE phase, delta, reset, and result events. Feature handlers pass stream callbacks to the provider. Provider streaming handles usage, repair retries, errors, and incomplete streams.
Client stream parsing and request wiring `packages/core/src/ai/partial-json.ts`, `src/lib/ai/*`, `src/lib/supabase/client.ts`, `e2e/helpers/ai-response.ts`, `e2e/live/live.ts`, `e2e/full-stack/voice.spec.ts`, `e2e/production/canary.spec.ts`, `e2e/staging/authed-flows.spec.ts`, `.claude/skills/verify-debatelab/recipes/fixtures.ts`	The client parses partial JSON and SSE events, processes streamed responses, and forwards live callbacks through AI feature requests. Response helpers support both SSE results and plain JSON.
Draft state and live interface `src/components/features/tools/`, `src/components/features/sparring/SparringRound.tsx`, `src/pages/AskPage.tsx`, `src/pages/AttackPage.tsx`, `src/pages/BrainstormPage.tsx`, `src/pages/CrossfirePage.tsx`, `src/pages/FactoidsPage.tsx`, `src/pages/GeneratePage.tsx`, `src/pages/SparringPage.tsx`, `src/lib/voice/live-text.ts`, `src/locales/`	Tool pages display formatted drafts and live status while requests run. Crossfire and Sparring display provisional speech, and Sparring can display a live scorecard.
Streaming verification and documentation `.claude/skills/verify-debatelab/features/`, `.claude/skills/verify-debatelab/recipes/`, `e2e/full-stack/*`, `e2e/live/research.spec.ts`, `src/lib/ai/partial-json.test.ts`, `src/test/ai/live.test.ts`	Tests cover partial parsing, streamed results, cancellation, and provider calls. The fake provider emits structured responses in chunks, and full-stack assertions wait for live drafts to finish.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant User
  participant GeneratePage
  participant AIEndpoint
  participant OpenRouter
  User->>GeneratePage: Start case generation
  GeneratePage->>AIEndpoint: Send request with stream true
  AIEndpoint->>OpenRouter: Request streamed model output
  OpenRouter-->>AIEndpoint: Send text deltas and progress
  AIEndpoint-->>GeneratePage: Send SSE events and final result
  GeneratePage-->>User: Display draft, then completed result

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Essentials by visiting https://app.coderabbit.ai/settings/billing.

_{Comment @coderabbitai help to get the list of available commands.}

…order - Cap a streamed answer's text at the plain reply's 1 MiB; the SSE frames around it get room for their per-token overhead. - A Sparring screen that rejoins a running request gets the draft so far and every later one. - A throttled draft is delivered before the next phase, so Checking never flips back to Writing live. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Programcode · 2026-10-17T15:53:01Z

Codex review, round 1 (head 2e24502): three findings, all fixed in 0416044.

Medium: the 1 MiB cap counted SSE framing. Fixed. The answer text keeps the 1 MiB cap and the raw frames get 32 MiB of room for their per-token overhead. Test: frame overhead does not count against the answer's size cap (1.5 MB of frames around 10 KB of text, which the old cap rejected).
Medium: a Sparring screen that rejoined a running request got no drafts. Fixed. sharedSparringRequest sends each draft to every screen following the request and replays the latest one to a screen that joins late. Test: src/test/ai/sparring-live.test.ts.
Low: a throttled draft could land after checking and switch the label back to Writing live. Fixed. The reader delivers any pending draft before it reports a new phase. Test: delivers the last draft before a later phase.

After the fixes: npm test 299 passed, npm run test:edge 339 passed, lint clean. Live mode: recipes/practice 6/6 (desktop + mobile), recipes/live-generation 3/3 (desktop). Round 2 is running on 0416044.

Live drafts and run controllers are module-level, keyed by tool. Sign-out and account switches reset the session store but left them, so the next account could see the previous one's draft. The session reset now aborts every tool run and clears every draft first. (Codex review r2) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Programcode · 2026-10-17T15:59:42Z

Codex review, round 2 (head 0416044): one finding, fixed in c2c512d.

Medium: live drafts survived an account change. The run controllers and drafts in tool-run.tsx are kept per tool, so if account A signed out mid-answer and account B opened the same tool in that tab, B could see A's draft. Fixed: AuthProvider's session reset, which runs on sign-out, account switch and local discard, now calls abortToolRuns() first. It aborts every running tool request and clears every draft, and A's late updates are ignored. Test: stops every run and drops its draft when the account changes.

After the fix: npm test 300 passed, lint clean. npm run verify at 0416044 passed (smoke 30/30). Round 3 is running on c2c512d.

AskPage already dropped a late answer when the debate scope changed. The account is now part of that scope, so a reply, or its live text, asked by one account never lands in the next one's chat. (Codex review r3) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Programcode · 2026-10-17T16:07:10Z

Codex review, round 3 (head c2c512d): one finding, fixed in a6ee0c1.

Medium: a structured Ask could keep writing into another account's chat. In one-off mode both accounts have the same (empty) chat scope, so an Ask asked by account A stayed current after B signed in, and its next live text recreated A's bubble. The plain final reply had the same gap before this PR. Fixed: the signed-in account is now part of AskPage's request scope. On an account change the page aborts the request and clears the live bubble and loading state. Test: src/test/pages/AskAccountSwitch.test.tsx (fails without the fix: A's text appears for B).

npm test 301 passed, lint and typecheck clean. Round 4 is running on a6ee0c1.

Programcode · 2026-10-17T16:12:51Z

Codex review, round 4 (head a6ee0c1): no findings. It checked the Ask account-switch fix and its test, the earlier fixes, the SSE transport, repair retries, cancellation, usage accounting, draft rendering, and the repo standards. Every finding from rounds 1–3 is fixed.

On mobile, a full-page screenshot could outlast the one-second fake stream before the draft's ARIA snapshot. fake.streamDelay(ms) slows streamed chunks for one test; the next reset restores the default. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Programcode · 2026-10-17T16:33:51Z

Codex review, round 5 (head baf3f8d, the fake-provider stream-delay control): no findings. That's two clean rounds in a row on the current code.

Programcode and others added 5 commits October 17, 2026 13:12

Stream every AI answer live, token by token

a1b5e9e

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Bring the live draft into view on phones when writing starts

2e24502

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Live gate: a close after the finish reason is not a hang-up; open fol…

d74f845

…ded sources Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI balanced review requested due to automatic review settings October 17, 2026 15:39

Copilot AI reviewed Oct 17, 2026

vercel Bot deployed to Preview – debatelab-staging October 17, 2026 15:40

View deployment

vercel Bot deployed to Preview – debate-navigator October 17, 2026 15:40

View deployment

vercel Bot deployed to Preview – debatelab-staging October 17, 2026 15:53

View deployment

vercel Bot deployed to Preview – debate-navigator October 17, 2026 15:53

View deployment

vercel Bot deployed to Preview – debatelab-staging October 17, 2026 16:00

View deployment

vercel Bot deployed to Preview – debate-navigator October 17, 2026 16:00

View deployment

vercel Bot deployed to Preview – debatelab-staging October 17, 2026 16:07

View deployment

vercel Bot deployed to Preview – debate-navigator October 17, 2026 16:09

View deployment

vercel Bot deployed to Preview – debatelab-staging October 17, 2026 16:31

View deployment

vercel Bot deployed to Preview – debate-navigator October 17, 2026 16:32

View deployment

Programcode merged commit 0b3e9f0 into main Oct 17, 2026
8 of 19 checks passed

Programcode deleted the t3/token-streaming-live-generation branch October 17, 2026 17:00

Programcode mentioned this pull request Oct 17, 2026

Pre-ship security pass: 25-item audit and fixes #148

Merged

Programcode mentioned this pull request Oct 28, 2026

iOS app: catch up with main and restyle as DebateOwl #253

Merged

Navigation Menu

Repository navigation

Stream every AI answer live, token by token - #149

Stream every AI answer live, token by token#149
Programcode merged 9 commits into
mainfrom
t3/token-streaming-live-generation

Programcode commented Oct 17, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

vercel Bot commented Oct 17, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Oct 17, 2026 •

edited

Loading

Review limit reached

Sequence Diagram(s)

Uh oh!

Programcode commented Oct 17, 2026

Uh oh!

Programcode commented Oct 17, 2026

Uh oh!

Programcode commented Oct 17, 2026

Uh oh!

Programcode commented Oct 17, 2026

Uh oh!

Programcode commented Oct 17, 2026

Uh oh!

Uh oh!

This branch was successfully deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Programcode commented Oct 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes for students

How it works

Tests

Verification (local; GitHub Actions is blocked by billing)

Deploy notes

Summary by CodeRabbit

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

vercel Bot commented Oct 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

coderabbitai Bot commented Oct 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review limit reached

Sequence Diagram(s)

Uh oh!

Programcode commented Oct 17, 2026

Uh oh!

Programcode commented Oct 17, 2026

Uh oh!

Programcode commented Oct 17, 2026

Uh oh!

Programcode commented Oct 17, 2026

Uh oh!

Programcode commented Oct 17, 2026

Uh oh!

Uh oh!

This branch was successfully deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Programcode commented Oct 17, 2026 •

edited

Loading

vercel Bot commented Oct 17, 2026 •

edited

Loading

coderabbitai Bot commented Oct 17, 2026 •

edited

Loading