[Programcode]

Batch T18: AI stream reliability, Auto Pro charging, refunds, and the Sparring judge error state (QA round 3, bugs 1, 2, 3, 4 and 6), plus Sentry JAVASCRIPT-REACT-A.

What changed, by bug

Bug 1 (Auto Fast Generate dies at ~90 s).

• Streamed runs in ai and ai-stream now register their whole lifecycle with EdgeRuntime.waitUntil (_shared/background.ts). That covers the run, the stored result, the final event, and the close. Without it, Supabase can retire the worker (EarlyDrop) once the stream's headers are out. In tests and plain Deno the helper does nothing.
• The SSE body in ai/stream.ts is now pull-driven, so the worker is held until the HTTP layer has read the last chunk (with a 10 s cap and a 0.5 s settle).
• A run still unanswered about 142 s after the request arrived ends with a retryable timeout result, instead of a connection cut at the 150 s wall clock.
• Cap change instead of a continuation. The Generate reply cap goes from 6,000 to 12,000 tokens, the size the old retry used. A long Haiku case now finishes on its first attempt, with no reset and no text streamed twice. The cap only bounds the reply, so it costs nothing unless the model uses it.
  • I first built a continuation from the cut-off text (assistant prefill) and then removed it. Claude Haiku 5.5 and Sonnet 5.5 reject assistant prefill with a 400, so it would never have run.
  • A reply that still hits the cap restarts once, as before.

Bug 2 (Auto Pro succeeds on the server, the client shows failure).

• When the client sends resultKey (its request id), the function writes an ai_run_results row as the run starts. It stores the final result there before sending it (ai/run-results.ts).
• The client's stream reader (src/lib/ai/live.ts) turns each failure into a typed LiveStreamError:
  • dropped: a read error, such as ERR_CONNECTION_CLOSED;
  • stalled: no bytes for 45 s, which is three missed keepalives;
  • ended: the body ended without a result;
  • unreadable: a result event whose data does not parse.
• For any of these, src/lib/ai/run-recovery.ts polls the stored row and then resolves or fails exactly as if the event had arrived. It polls up to 30 s for a dropped stream, and up to the 155 s window for a stalled one.
• I could not confirm from here why the event never reached the client in QA's run. The fix covers the delivery path both ways: the worker is held until the last byte is read, and the browser recovers the stored result if it still loses it.

Bug 3 (Auto Pro charges 5).

• usage-limits.ts now derives the Auto unit costs from FAST_CANDIDATES and PRO_CANDIDATES. Auto Pro reserves 2 (Sonnet), and the cost can't drift from the pools again.
• The UI's costs come from the server's quotaCheck, so the displayed cost and uses-left follow automatically. No client file hard-codes Auto units.

Bug 4 (dropped runs keep their charge, no ai_requests row).

• Every refund goes through refundOnce, so a run is refunded at most once.
• Each run logs exactly one ai_requests row. Model-routing and source-ranking failures now log routing_failed and source_ranking_failed. A failure after a handler has logged its success row doesn't log a second row.
• New refund rule:
  • A run that fails on the server is refunded however much of it streamed: a provider failure, a reply cut off at its cap, an answer that failed validation, or the time budget running out.
  • A run the browser left (Stop, reload, a lost connection; the server can't tell these apart) keeps the existing floor. It stays charged once 200 characters have streamed or the model has finished.
  • This is the policy call review round 1 raised, so it's worth Cody's look. Without the floor, a student could read most of a case and press Stop to get the unit back every time.
• Safety net: a row in ai_run_results that still has no status long after started_at is a run whose worker never finished.

Bug 6 (the Sparring judge error leaves a stale status).

• On a judge failure, the round header switches to "Scoring stopped" and "The judge could not finish scoring." in EN, ES and HU. Retrying brings back "The judge is reading the transcript."
• The judge retries once inside the request on a transient provider failure (Claude 5xx/429, or a dropped provider stream). Key and credit errors (401/402/403) are not retried. The round has already paid, so the retry is free.
• Error bodies for transient failures carry retryable: true. The client no longer reports those, or Claude provider_unavailable, as Sentry exceptions; ai_request_failed still records them. Other provider failures are still reported.

Sentry JAVASCRIPT-REACT-A

The root cause, from Cody: Sonnet returns valid JSON for the score action, but in a shape the parser rejected, and that surfaced as a provider 500.

• Strict schema: speech and scorecard calls now send a strict json_schema (sparringOutputSchema), modelled on the Crossfire schemas. It has four named categories, a winner enum, and additionalProperties: false everywhere. Anthropic's structured outputs reject minimum/maximum and maxItems, so the 0–25 range, the exact count of four categories, and the 12-item lists are stated in field descriptions and enforced by the parser.
• Failing field path: checkSparringProviderOutput / checkSparringScorecard return the first failing field (for example scorecard.categories[1].userScore). Only that path is logged, never model text.
• Repair retry: a reply in the wrong shape gets one retry through withJsonRepairPrompt, with the field named in the system prompt. If it fails again, the handler throws AIResponseValidationError. ai/index.ts then answers 502 with code invalid_output and logs error_code = invalid_output. JSON that is still invalid after callAI's own repair maps the same way.
• Client: shows "The scorecard came back malformed. Try scoring again." in EN, ES and HU, and the header leaves the "judge is reading" state.
• Normalizer: moves a stray top-level overallFeedback into scorecard, and flattens {text|moment|description} objects in keyMoments and improvementAreas into strings.
• Refund: a failed call is refunded once (refund_ai_session_quota for the call that paid). A judge call inside a round that already paid has nothing to refund. That charge covers the whole round, the failed attempt doesn't charge again, and "Try scoring again" is free.

Deploy checklist (I did none of this)

1. Migration, additive: supabase/migrations/20261023140000_ai_run_results.sql creates public.ai_run_results with owner-only SELECT and service-role writes. It also re-creates clear_current_user_data() with the same body plus one DELETE for the new table. Apply it before the functions.
2. Edge Functions: ai and ai-stream. Both use the new _shared/background.ts; ai also changes _shared/usage-limits.ts, _shared/output-caps.ts and _shared/entitlements.ts (comment only).
3. Frontend: ships when this merges.

Compatibility, in any deploy order:

• An old cached client sends no resultKey, so nothing is stored and streaming works as before. The extra retryable field is ignored.
• A new client against old functions, or a database without the table, finds no row and falls back to the usual connection error.
• invalid_output on an old client shows the generic server error.

Verification

• npm run verify (full ladder, local, at the final head 7391338; also passed at 5346a03): passed, exit 0. docs:check, lint, typecheck; vitest 645/645 (89 files); collaboration 30/30; edge 468/468; build; test:sw; e2e smoke 47/47 (desktop Chromium). This is a local pass: GitHub Actions minutes are used up, so CI did not run.
• ./.superset/db-gate.sh --db-only: all 29 pgTAP files pass (522 tests), including the new ai_run_results_test.sql, plus the default-model migration check, exit 0. Account deletion and export now include the new table; account_deletion_test.sql caught that it was missing.
• verify-debatelab, fake mode, desktop and mobile Chromium:
  • case-generation, live-generation, practice and ai-tools: all passed (18, 6, 10 and 14 tests) on the first commit.
  • Re-run on the final code: practice, live-generation and ai-tools passed. case-generation passed 18/18 on a clean re-run, after two failures in an earlier run that overlapped my edits under the hot-reloading stack.
  • The temporary proof passed 4/4 (desktop and mobile).
• Temporary proof recipe (not committed):
  • A Generate run streaming for 103 s completed. The case was saved, with one ai_requests row (success) and one finished ai_run_results row (status 200).
  • A stream whose result event was cut off by a Playwright route still showed the stored case and saved it.
  • Limit: the local edge runtime logs wall-clock warnings but never EarlyDrop, so the prod drop can't be reproduced locally. The waitUntil registration is covered by a unit test with a fake EdgeRuntime.
• Tests added:
  • waitUntil held until the body is drained;
  • a result stored before it is sent (the test fails if the order flips);
  • the deadline answering a retryable result;
  • a result row written only with resultKey;
  • parser edge cases (a large result split mid-character, an unterminated final event, damaged, dropped, stalled);
  • recovery (stored success, stored failure with charged: false, a still-running row, a missing row, another run's key, the dropped-run limit);
  • Auto units equal to the most expensive candidate;
  • a server failure refunded after 500 streamed characters;
  • invalid_output with one repair retry, one refund and one usage row;
  • the judge retrying on an outage but not on an exhausted key;
  • normalizer and field-path cases;
  • the error header in SparringRound.
• Not verified: live models, because there is no live key in this VM. Whether Sonnet now passes the strict scorecard schema consistently needs a live or staging check.

Review rounds

• Round 1 (fresh-context Opus 5.5, claude-opus-5-5, on 37c1baa): no Critical or High findings. Two were Medium:
  • The refund rule allowed Stop abuse. Fixed: the floor is back for client-left runs.
  • The prefill continuation was unproven. Resolved: it was removed, because the models reject prefill.
  • Of the Lows: fixed a slow start write that could land after the finish, the dropped-run wait limit, recovery running on any fetch error, the judge's retry stacking on top of JSON repair, a test that could not fail, and stale docs.
  • Left open (Low): after the 142 s deadline answer, a step that ignores the abort could still finish and keep the charge.
• Round 2 (fresh-context Opus 5.5, claude-opus-5-5, on 5346a03): one Medium finding, no Critical or High.
  • Medium (fixed in 7391338): the 45 s stall watchdog cancelled the reader. That closed the fetch, so the function cancelled the very run the client was about to recover, which hits students behind SSE-buffering proxies. A stalled stream now stays open while the stored result is polled. It is cancelled when the caller aborts, or after 160 s.
  • Lows, not fixed (deadline):
    1. The 142 s deadline result has no charged: false. The refund happens later, in the aborted run's own error path.
    2. With a slow database, the recorder's waits (up to 10 s for the start write plus 4 s for the finish) can push the result past the 150 s wall clock.
    3. If successBody throws after a handler logged its success row, the run is refunded but keeps that success label. There is still exactly one row.
    4. ai-stream still enqueues from start and gives the last bytes a 1 s settle, not the pull-driven drain that ai uses.
  • Per the wrap-up instruction, no further review rounds.

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features
  • AI responses can now recover after a live stream disconnects, helping completed results reach you without restarting the run.
  • Longer generated replies are supported, and Sparring can retry malformed scorecards or certain temporary failures.
• Bug Fixes
  • Improved messages for stopped scoring and invalid AI output, including when a failed run was not charged.
  • Updated quota reservations and refunds to better reflect model costs and whether a response was delivered.
• Documentation
  • Clarified AI stream recovery, timeout, retry, and refund behavior.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[coderabbitai]

ℹ️ Recent review info

⚙️ Run configuration

• Source mode: Auto
• Repository: Programcode/DebateLab
• Branch: fix/t18-ai-stream-reliability
• File: .coderabbit.yaml
• Commit: cced73a
• Layers: Repository YAML (base), Organization UI (inherited)
• Review profile: CHILL
• Plan: Free
• Run ID: 6acb7bfe-0707-4a58-b334-f3c371902819

📥 Commits

Reviewing files that changed from the base of the PR and between 5346a03 and cced73a.

📒 Files selected for processing: 15

• docs/architecture.md
• src/lib/ai/live.ts
• src/lib/ai/run-recovery.ts
• src/locales/en.json
• src/locales/es.json
• src/locales/hu.json
• src/pages/SparringPage.tsx
• src/test/ai/live.test.ts
• supabase/functions/_shared/background.ts
• supabase/functions/ai-stream/handler_test.ts
• supabase/functions/ai-stream/index.ts
• supabase/functions/ai/index.ts
• supabase/functions/ai/run-results.ts
• supabase/functions/ai/stream.test.ts
• supabase/functions/ai/stream.ts

🚧 Files skipped from review as they are similar to previous changes: 4

• src/locales/hu.json
• src/locales/es.json
• src/locales/en.json
• docs/architecture.md

Included review availability: This review used your included allowance. Your plan provides up to 1 included review per hour; 0 remain after this review.

📝 Walkthrough

The changes add streamed AI result persistence and recovery, worker-lifetime handling, and timeout responses. They also add Sparring output validation and retries, update scoring messages, and revise Auto quota weights, refund handling, and the generate token cap.

Changes: Streamed run persistence and recovery

Layer / File(s)	Summary
Persist and deliver streamed results supabase/functions/ai/run-results.ts, supabase/migrations/20261023140000_ai_run_results.sql, supabase/functions/ai/stream.ts, supabase/functions/ai/index.ts, supabase/functions/_shared/background.ts, supabase/functions/ai-stream/index.ts, supabase/functions/ai/provider.ts, supabase/README.md, supabase/functions/ai/stream.test.ts, supabase/tests/*, docs/architecture.md	Streamed requests can pass a resultKey; the AI function can record the result before sending it. Stream handling adds worker retention, result delivery, and deadline behavior. Provider stream errors distinguish incomplete streams from completed output with a missing usage trailer. The migration adds ai_run_results with access controls and account-deletion cleanup.
Classify stream failures and recover results src/lib/ai/live.ts, src/lib/ai/run-recovery.ts, src/lib/ai/request.ts, src/lib/observability.ts, src/test/ai/live.test.ts, src/lib/ai/run-recovery.test.ts	The client classifies malformed, dropped, stalled, and ended streams. It polls for a matching stored result after eligible stream failures and records successful recovery with ai_stream_recovered.

Sparring output validation and retries

Layer / File(s)

Summary

Validate and retry Sparring output supabase/functions/ai/sparring-persistence.ts, supabase/functions/ai/features/sparring.ts, supabase/functions/ai/provider.ts, src/lib/ai/errors.ts, src/lib/ai/client.ts, src/pages/SparringPage.tsx, src/components/features/sparring/SparringRound.tsx, src/locales/*.json, supabase/functions/ai/sparring-*-test.ts, supabase/functions/ai/quota-sessions_test.ts, src/lib/ai/errors.test.ts, src/test/sparring/sparring-round.test.tsx

Sparring output validation reports the first invalid field path and uses action-specific schemas. Eligible malformed output and scoring failures receive one retry. Invalid output maps to a localized error, and the scoring UI displays a failure message when judging stops.

Quota weights and refund rules

Layer / File(s)	Summary
Update quota weights, refunds, and output cap supabase/functions/_shared/usage-limits.ts, supabase/functions/_shared/entitlements.ts, supabase/functions/_shared/output-caps.ts, supabase/functions/_shared/auto-model-router.test.ts, supabase/functions/ai/quota-sessions_test.ts, docs/architecture.md	Auto quota units now use the highest candidate cost in each pool. The generate reply cap increases from 6,000 to 12,000 tokens. Refund documentation and tests cover the revised rules.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant callSupabaseAI
  participant liveStreamReader
  participant recoverRunResult
  participant ai_run_results
  callSupabaseAI->>liveStreamReader: Read live response
  liveStreamReader-->>callSupabaseAI: Report stream failure
  callSupabaseAI->>recoverRunResult: Poll with feature and result key
  recoverRunResult->>ai_run_results: Query stored result
  ai_run_results-->>recoverRunResult: Return matching result
  recoverRunResult-->>callSupabaseAI: Return status and response body

Loading

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Note

🎁 Summarized by CodeRabbit Free

Your organization is on the Free plan. CodeRabbit will generate a high-level summary and a walkthrough for each pull request. For a comprehensive line-by-line review, please upgrade your subscription to CodeRabbit Essentials by visiting https://app.coderabbit.ai/settings/billing.

_{Comment @coderabbitai help to get the list of available commands.}