Repository navigation
Repair model JSON locally and enforce schemas before paying for a retry - #176
Conversation
An invalid reply used to cost a full second generation. Now: - the parser fixes fences, prose around the object, trailing commas and raw line breaks in strings before giving up; - a reply cut off at the output token cap is not retried, since the same cap would cut the retry too; - Crossfire turns and debriefs, the Ask search decision and research follow-up queries send a strict json_schema, so the provider holds the reply to the shape the server already validates. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
|
Note Currently processing new changes in this PR. This may take a few minutes, please wait... ⚙️ Run configuration
📒 Files selected for processing: 8
Note 🎁 Summarized by CodeRabbit Free
Comment |
|
All 11 failing Actions jobs on 451eae7 failed in about 2 seconds without starting. The job annotation says: "The job was not started because recent account payments have failed or your spending limit needs to be increased." That's the repo-wide Actions billing block, not this change. The same checks ran locally at this commit with |
Review of 451eae7 found three ways the parser took the wrong text: - the fence regex backtracked for seconds on long whitespace, and ran before the plain parse; - a fence inside a string of an unfinished object was taken as the answer; - the first parsable span won, so "{} means omitted" beat the real object. Plain JSON now parses first, a fence counts only when it wraps the whole reply (found with indexOf, no regex), and a span is accepted only when it is the one span in the reply that parses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Codex review round 1 on 451eae7 had three findings, all valid, all fixed in b0cfe00:
All three cases are in |
Review of b0cfe00 found two ways the repair still accepted an example: - a reply that opened with a fence was taken even with text after the closing fence; - the scan stopped after eight spans, so a second valid object later in the reply went unseen. A fence now has to close the reply. The span scan reads every span and rejects the reply on a second parsable span, on an unclosed brace, or past 16 spans (which bounds parse work on a reply full of braces: a 1 MB reply of tiny spans went from 4.7 s to 118 ms). Rejection means the existing paid retry, as before this PR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Codex review round 2 on b0cfe00 had two findings, both valid, both fixed in 3608445:
I ran typecheck, |
A slip in the real answer (an unescaped quote) made its span fail, and any other span that parsed, such as an example object or a nested one, was returned as the answer.
|
Deployment failed for project debate-navigator with the following error: Learn More: https://vercel.com/adam-gaborjanis-projects?upgradeToPro=build-rate-limit |
|
Deployment failed for project debatelab-staging with the following error: Learn More: https://vercel.com/adam-gaborjanis-projects?upgradeToPro=build-rate-limit |
Keep #182 retry of a reply cut off at its per-feature cap (now with the strict schema) and drop this branch no-retry-on-truncation path, which the per-feature caps superseded.
Cost fix 7 of 7. A reply that isn't valid JSON used to cost a full second generation. This PR makes that second call rarer and skips it when it can't help.
What changes
Local repair before any paid retry (
provider.ts). Before the server gives up on a reply, it now fixes the usual slips itself:```json,```JSON, plain```);{...}match failed in that case. If two objects in the reply both parse, the answer is ambiguous, so the parser rejects the reply and the normal retry runs;}or];The parser still rejects a reply that was cut off mid-object. Closing it would ship half a case as if it were complete.
A reply cut off at its cap is retried once at double the cap. If the provider reports
finish_reason: "length", the server doesn't send the reply to the JSON repair retry, because a second call under the same cap would be cut off again. Since #182, which gave each feature its own reply cap, the server instead retries once at double that cap. The strict schema stays on the retry, and both attempts are billed. This works for streamed and non-streamed replies.As first opened, this PR failed straight away on a cut-off reply and billed one attempt. #182 landed before the merge and its retry was kept, because with a smaller per-feature cap a doubled cap can actually fit the reply.
Strict schemas where the server already checks the shape. These calls now send
response_formatwithtype: "json_schema"andstrict: true, so the provider has to return the right shape:typeis fixed to the expected reply kind;turnIdscan only be IDs from this transcript. Before, an invented ID failed the validator and the student had to run the debrief again, which cost another paid call;Every other feature (Generate, Attack, Brainstorm, Factoids, Ask answers, Sparring) stays on plain JSON mode. Their contract lives in prompt prose, and the client reshapes them before validating. A strict schema makes every field required, so a schema that drifted from the prompt would turn rare failures into constant ones. They can move over one at a time once each has a code-side validator to copy.
Routing note
We send
require_parameters: true. With a schema attached, OpenRouter then routes only to endpoints that support structured outputs. For the four schema calls, OpenRouter drops these endpoints from the pool. Amazon Bedrock (and, for Haiku and Sonnet, Claude Platform on AWS) for the Claude models, DeepSeek's own API for V4 Pro, Z.AI's own API for GLM, and several smaller hosts for the DeepSeek and GLM models. Every catalog model still has several endpoints that support it. Calls without a schema route exactly as before.Verification
npm run verifyat 451eae7: passed (docs check, lint, typecheck, 305 unit tests, 30 collaboration tests, 358 Edge Function tests, build, service worker check, 30 desktop Chromium smoke tests)npm run test:edge(358 passed). That commit only changes the parser and its test.provider-errors_test.tscover each repair case, the refusals (a nested object is never taken for the answer, a cut-off object is not completed even around a fenced example, two parsable objects are ambiguous), and a 20 KB fence-like string that must parse in under a second. After the merge with Cap each AI feature's reply length instead of 32,768 tokens #182, the cap test checks for one retry at a larger cap with the strict schema kept.anyOfand a turn-IDenum) to all ten catalog models withrequire_parameters: true. All ten routed and returned valid JSON in the right shape. DeepSeek V4.1 Flash failed once, but only because my probe capped output at 4,000 tokens and reasoning used them up. It passed four out of four reruns. The production cap was then 32,768; since Cap each AI feature's reply length instead of 32,768 tokens #182 each feature has its own cap.stream: trueto all ten models, using the then-production 32,768 output cap. All 20 streamed replies parsed, each ended withfinish_reason: stopand[DONE], and every next turn came back with the expectedtype.verify-debatelabstack because a parallel cost-fix thread held the shared Supabase lock the whole time. The direct probes above are the live proof for the schema path. The fake provider ignoresresponse_format, so a fake stack run wouldn't prove it anyway.GitHub Actions is broken across the repo, so these are local results, not CI.
🤖 Generated with Claude Code