Skip to content

Commit 451eae7

Browse files
Programcodeclaude
andcommitted
Repair model JSON locally and enforce schemas before paying for a retry
An invalid reply used to cost a full second generation. Now: - the parser fixes fences, prose around the object, trailing commas and raw line breaks in strings before giving up; - a reply cut off at the output token cap is not retried, since the same cap would cut the retry too; - Crossfire turns and debriefs, the Ask search decision and research follow-up queries send a strict json_schema, so the provider holds the reply to the shape the server already validates. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent 4f9e0a0 commit 451eae7

8 files changed

Lines changed: 262 additions & 28 deletions

File tree

‎docs/architecture.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -134,6 +134,8 @@ Checkout and portal functions create Stripe sessions after authenticated ownersh
134134

135135
All model requests use the single OpenRouter backend route and `OPENROUTER_API_KEY`. The shared ten-model catalog, its defaults (Claude Haiku 5.5 for Basic, Claude Opus 5.5 for Pro), Pro-only entries, retired-ID successors, and request controls live in [`supabase/functions/_shared/model-catalog.ts`](../supabase/functions/_shared/model-catalog.ts). OpenRouter usage remains raw in the audit row, and the reported charge is unknown when the provider does not return a complete cost. The opponent's voice is the one exception to the OpenRouter route: the `voice` function calls ElevenLabs with `ELEVENLABS_API_KEY`. ElevenLabs reports billed characters, not dollars, so [`supabase/functions/_shared/cost.ts`](../supabase/functions/_shared/cost.ts) prices those rows from a dated list-price snapshot; promotions and plan credits are not modelled, so a voice row can overstate the charge.
136136

137+
Every model call asks for JSON. Calls whose shape the server checks in code (the Crossfire next turn and debrief, the Ask search decision, and research follow-up queries) send a strict JSON Schema, so OpenRouter routes them only to endpoints that enforce it; the other features use plain JSON mode. [`supabase/functions/ai/provider.ts`](../supabase/functions/ai/provider.ts) fixes common slips in a reply locally (markdown fences, prose around the object, trailing commas, raw line breaks in strings) before it pays for one repair retry, and it does not retry a reply the provider cut off at the output token cap.
138+
137139
`Auto Fast` and `Auto Pro` are virtual selections, not provider model IDs. The Edge Functions resolve them through TypeSafe Jev on OpenRouter before calling the concrete writing model. Auto Fast is available to Basic accounts, routes only among the Basic models (Claude Haiku 5.5, GPT-6 Luna, Muse Spark 1.3 Contributor), and normally resolves to Claude Haiku 5.5. Auto Pro remains Pro-only and normally resolves to Claude Opus 5.5. Provider calls, usage rows, and response metadata record the concrete resolved model, while response metadata also retains the requested automatic selection. A Jev router call uses the same OpenRouter key and does not consume a second daily quota unit.
138140

139141
## Observability

‎supabase/functions/_shared/crossfire-response.ts‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
import type { JsonOutputSchema } from "./research.ts";
2+
13
export const MAX_CROSSFIRE_REPLY_CHARS = 2_500;
24
export const MAX_CROSSFIRE_DEBRIEF_ITEMS = 8;
35
export const MAX_CROSSFIRE_REFERENCE_IDS = 6;
@@ -81,6 +83,56 @@ function crossfireDebriefItem(
8183
return { value: { turnIds: turnIds.value, claim: value.claim.trim() } };
8284
}
8385

86+
/**
87+
* The provider-side shape of a debrief, mirroring validateCrossfireDebrief.
88+
* Turn IDs are pinned to the transcript's, so a reply cannot cite a turn that
89+
* does not exist. Counts, lengths, and duplicates stay with the validator.
90+
*/
91+
export function crossfireDebriefSchema(transcript: Array<{ id: string }>): JsonOutputSchema {
92+
const ids = [...new Set(transcript.map((turn) => turn.id))];
93+
const turnIds = { type: "array", items: ids.length ? { type: "string", enum: ids } : { type: "string" } };
94+
const item = {
95+
type: "object",
96+
properties: { turnIds, claim: { type: "string" } },
97+
required: ["turnIds", "claim"],
98+
additionalProperties: false,
99+
};
100+
const items = { type: "array", items: item };
101+
return {
102+
name: "crossfire_debrief",
103+
schema: {
104+
type: "object",
105+
properties: {
106+
concessionsSecured: items,
107+
controlledQuestions: items,
108+
weakOrRepetitiveQuestions: items,
109+
evasiveOrIncompleteAnswers: items,
110+
missedFollowUps: items,
111+
betterFollowUpQuestions: {
112+
type: "array",
113+
items: {
114+
type: "object",
115+
properties: { turnIds, question: { type: "string" }, reason: { type: "string" } },
116+
required: ["turnIds", "question", "reason"],
117+
additionalProperties: false,
118+
},
119+
},
120+
strongestExchange: { anyOf: [item, { type: "null" }] },
121+
},
122+
required: [
123+
"concessionsSecured",
124+
"controlledQuestions",
125+
"weakOrRepetitiveQuestions",
126+
"evasiveOrIncompleteAnswers",
127+
"missedFollowUps",
128+
"betterFollowUpQuestions",
129+
"strongestExchange",
130+
],
131+
additionalProperties: false,
132+
},
133+
};
134+
}
135+
84136
export function validateCrossfireDebrief(
85137
value: unknown,
86138
transcript: Array<{ id: string }>,

‎supabase/functions/_shared/research.ts‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -80,9 +80,42 @@ export interface ResearchDeps {
8080
messages: Array<{ role: string; content: string }>;
8181
reasoningEffort: ReasoningEffort;
8282
signal?: AbortSignal;
83+
outputSchema?: JsonOutputSchema;
8384
}) => Promise<{ output: unknown }>;
8485
}
8586

87+
/**
88+
* A JSON Schema the provider must hold the reply to. Only for outputs the
89+
* server already validates in code: with `strict` every property is required,
90+
* so the schema has to match that validator exactly. Keep to keywords every
91+
* routed provider accepts: type, properties, required, items, enum, anyOf,
92+
* and additionalProperties false.
93+
*/
94+
export interface JsonOutputSchema {
95+
name: string;
96+
schema: Record<string, unknown>;
97+
}
98+
99+
const QUERIES_SCHEMA: JsonOutputSchema = {
100+
name: "follow_up_queries",
101+
schema: {
102+
type: "object",
103+
properties: { queries: { type: "array", items: { type: "string" } } },
104+
required: ["queries"],
105+
additionalProperties: false,
106+
},
107+
};
108+
109+
const SEARCH_DECISION_SCHEMA: JsonOutputSchema = {
110+
name: "search_decision",
111+
schema: {
112+
type: "object",
113+
properties: { shouldSearch: { type: "boolean" }, query: { type: "string" } },
114+
required: ["shouldSearch", "query"],
115+
additionalProperties: false,
116+
},
117+
};
118+
86119
export function emptyResearchContext(): ResearchContext {
87120
return { searches: [], rankedResults: [], readContent: [], verifiedUrls: new Set() };
88121
}
@@ -377,6 +410,7 @@ export async function decideAskSearch(
377410
messages: [{ role: "user", content: prompt }],
378411
reasoningEffort: deps.reasoningEffort,
379412
signal: deps.signal,
413+
outputSchema: SEARCH_DECISION_SCHEMA,
380414
});
381415
const decision = decisionResult.output;
382416
if (decision && typeof decision === "object") {
@@ -415,6 +449,7 @@ async function generateFollowUpQueries(
415449
messages: [{ role: "user", content: prompt }],
416450
reasoningEffort: deps.reasoningEffort,
417451
signal: deps.signal,
452+
outputSchema: QUERIES_SCHEMA,
418453
});
419454
const parsed = typeof result.output === "object" && result.output !== null ? result.output : {};
420455
if (Array.isArray((parsed as { queries?: string[] }).queries)) {

‎supabase/functions/ai/features/crossfire-input.ts‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,6 +15,7 @@ import {
1515
type PromptSource,
1616
} from "../../_shared/prompts.ts";
1717
import { MAX_CROSSFIRE_REPLY_CHARS } from "../../_shared/crossfire-response.ts";
18+
import type { JsonOutputSchema } from "../../_shared/research.ts";
1819
import { hasOnlyKeys, isRecord, sanitizePromptInput } from "../request.ts";
1920
import { sparringIntegritySecret, supabase } from "../runtime.ts";
2021

@@ -290,6 +291,19 @@ export function boundCrossfireSources(value: unknown, enabled: boolean): { value
290291
return { value: bounded, truncated };
291292
}
292293

294+
/** The provider-side shape of a next turn, mirroring crossfireReplyOutput. */
295+
export function crossfireReplySchema(expectedReply: "question" | "answer"): JsonOutputSchema {
296+
return {
297+
name: "crossfire_turn",
298+
schema: {
299+
type: "object",
300+
properties: { type: { type: "string", enum: [expectedReply] }, text: { type: "string" } },
301+
required: ["type", "text"],
302+
additionalProperties: false,
303+
},
304+
};
305+
}
306+
293307
export function crossfireReplyOutput(value: unknown, expectedReply: "question" | "answer"): { value: { type: "question" | "answer"; text: string } } | { error: string } {
294308
if (!isRecord(value) || !hasOnlyKeys(value, ["type", "text"])) return { error: "Crossfire next-turn response must contain only type and text" };
295309
if (value.type !== expectedReply) return { error: `Crossfire response type must be ${expectedReply}` };

‎supabase/functions/ai/features/crossfire.ts‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,11 +3,12 @@ import { callAI } from "../provider.ts";
33
import { cleanGeneratedText, getCurrentDateContext } from "../text.ts";
44
import { buildCrossfireNextTurnPrompt, buildCrossfireDebriefPrompt } from "../prompt-builder.ts";
55
import { buildSystemPrompt, type DebatePromptContext, type PromptSource } from "../../_shared/prompts.ts";
6-
import { validateCrossfireDebrief } from "../../_shared/crossfire-response.ts";
6+
import { crossfireDebriefSchema, validateCrossfireDebrief } from "../../_shared/crossfire-response.ts";
77
import {
88
createCrossfireSignalToken,
99
type CrossfireInput,
1010
crossfireReplyOutput,
11+
crossfireReplySchema,
1112
MAX_CROSSFIRE_PROMPT_CHARS,
1213
} from "./crossfire-input.ts";
1314
import { prepareVoiceReply, type VoicePayload } from "../../_shared/voice-token.ts";
@@ -65,6 +66,9 @@ export async function handleCrossfire(ctx: FeatureContext, options: CrossfireHan
6566
reasoningEffort,
6667
workSignal,
6768
ctx.stream,
69+
crossfireInput.operation === "next-turn"
70+
? crossfireReplySchema(crossfireInput.expectedReply)
71+
: crossfireDebriefSchema(crossfireInput.transcript),
6872
));
6973
const cleanedOutput = cleanGeneratedText(aiResult.output);
7074
const output = crossfireInput.operation === "next-turn"

‎supabase/functions/ai/index.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -638,7 +638,7 @@ export async function handleRequest(req: Request): Promise<Response> {
638638
},
639639
callModel: async (args) => {
640640
try {
641-
const result = await callAI(args.provider, args.model, args.systemPrompt, args.messages, args.reasoningEffort, args.signal);
641+
const result = await callAI(args.provider, args.model, args.systemPrompt, args.messages, args.reasoningEffort, args.signal, undefined, args.outputSchema);
642642
// A provider that ignores cancellation must not append usage after the
643643
// research timeout has already been finalized by the parent request.
644644
if (result.usage && !args.signal?.aborted) researchUsage.push(result.usage);

‎supabase/functions/ai/provider-errors_test.ts‎

Lines changed: 38 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
import { assertEquals } from "jsr:@std/assert@1.0.19";
1+
import { assertEquals, assertRejects, assertThrows } from "jsr:@std/assert@1.0.19";
22
for (const [key, value] of Object.entries({
33
SUPABASE_URL: "https://supabase.test", SUPABASE_ANON_KEY: "test-anon-key",
44
SUPABASE_SERVICE_ROLE_KEY: "test-service-key", OPENROUTER_API_KEY: "test-provider-key",
@@ -40,3 +40,40 @@ Deno.test("AI JSON and SSE failures never expose provider response bodies", asyn
4040
}
4141
} finally { globalThis.fetch = originalFetch; }
4242
});
43+
44+
Deno.test("common JSON slips are repaired locally instead of paying for a retry", async () => {
45+
const { parseJsonFromText } = await import("./provider.ts");
46+
const parse = (text: string) => parseJsonFromText(text, "OpenRouter");
47+
assertEquals(parse('```JSON\n{"a":1}\n```'), { a: 1 });
48+
assertEquals(parse('```\n{"a":1}\n```'), { a: 1 });
49+
assertEquals(parse('Here you go: {"a":"x"} Note: {not json}'), { a: "x" });
50+
assertEquals(parse('Use {braces} like so. {"a":[1,2,],}'), { a: [1, 2] });
51+
assertEquals(parse('{"a":"line one\nline two\t!"}'), { a: "line one\nline two\t!" });
52+
assertEquals(parse('{"a":"keep , ] inside strings",}'), { a: "keep , ] inside strings" });
53+
// A broken outer object never yields one of its nested objects.
54+
assertThrows(() => parse('{"case": {"title": "x"}, oops}'));
55+
// A reply cut off mid-object is not completed.
56+
assertThrows(() => parse('{"contentions": [{"title": "x"}, {"title": "y"'));
57+
});
58+
59+
Deno.test("a reply cut at the output cap is not retried, and a schema is sent as strict json_schema", async () => {
60+
const { callAI } = await import("./provider.ts");
61+
const originalFetch = globalThis.fetch;
62+
const bodies: Array<Record<string, unknown>> = [];
63+
globalThis.fetch = (async (_input: Request | URL | string, init?: RequestInit) => {
64+
bodies.push(JSON.parse(String(init?.body)));
65+
return Response.json({
66+
choices: [{ message: { content: '{"queries": ["a", "b' }, finish_reason: "length" }],
67+
usage: { prompt_tokens: 10, completion_tokens: 5, cost: 0.01 },
68+
});
69+
}) as typeof fetch;
70+
try {
71+
const schema = { name: "follow_up_queries", schema: { type: "object" } };
72+
const error = await assertRejects(() =>
73+
callAI("openrouter", "openai/gpt-6-luna", "system", [{ role: "user", content: "hi" }], "low", undefined, undefined, schema)
74+
) as { usage?: { inputTokens?: number } };
75+
assertEquals(bodies.length, 1);
76+
assertEquals(bodies[0].response_format, { type: "json_schema", json_schema: { name: "follow_up_queries", strict: true, schema: { type: "object" } } });
77+
assertEquals(error.usage?.inputTokens, 10);
78+
} finally { globalThis.fetch = originalFetch; }
79+
});

0 commit comments

Comments
 (0)