Retry with max_completion_tokens when the endpoint rejects max_tokens

[Schro-code]

Why

Newer OpenAI reasoning models (o1, o3, GPT-5 family) renamed max_tokens to max_completion_tokens. Sending the old name returns a 400 with error.code: "unsupported_parameter", and every extraction chunk fails.

What changes

An AtomicBool on LlmClient (use_max_completion_tokens) learns the rename from a single 400 and remembers it for the client and its clones, the same way completion_ceiling already works.

In chat_at_streaming's retry loop:

• The token-limit key is chosen based on the flag, so a learned rename does not get undone by the next iteration's body["max_tokens"] = json!(ceiling).
• A 400 with error.param: "max_tokens" / error.code: "unsupported_parameter" sets the flag and retries. The first chunk pays the 400; the remaining 69 use the correct name from the start.

chat_at (non-streaming) is not touched — it never sends max_tokens.

Temperature rejection (unsupported_value) is left out deliberately: silently dropping temperature=0 changes extraction from deterministic to sampling, which is a semantic decision the operator should make, not a silent fallback. Happy to add it if the project disagrees.

A wiremock test verifies the rename, learning across calls, and learning across clones.

How it was checked

Tested against an Azure OpenAI endpoint running gpt-5.6-sol, which rejects max_tokens. Before: 70/70 chunks fail with 400. After: first chunk retries once, remaining 69 pass directly. cargo fmt, cargo clippy -D warnings, and cargo test -p utopia-llm (54 tests) all pass.

Before review

• Every commit is signed off (git commit -s)
• cargo fmt --all --check, cargo clippy --workspace --all-targets -- -D warnings and cargo test --workspace pass
• For changes under web/: pnpm build and pnpm test pass — N/A, no web changes
• SQL under crates/utopia-store/ was tested with UTOPIA_DATABASE_URL set — N/A, no SQL changes
• A new migration takes the next free number on dev — N/A, no migrations
• UI strings are in both web/src/i18n/en.ts and zh.ts — N/A, no UI strings
• A change to the data model, the ontology contract or a public API has its ADR in docs/decisions/ — N/A

[WaylandYang]

Thank you, the problem is real and reading error.param / error.code is the right way to detect it.

Two things to change:

• In chat_at_streaming each attempt rewrites body["max_tokens"], so after the rename the next request carries it again and the retry can loop without end.
• Every call pays the 400 again (once per chunk). Learn the parameter name and a rejected temperature once on the client, as completion_ceiling already does.

Please add a wiremock test like the existing "max_tokens is too large" one, and run cargo fmt (the lint failure). Thanks again.

[Schro-code]

Thanks for the review. Addressed all three points:

• The streaming loop now picks max_tokens or max_completion_tokens based on a learned AtomicBool on the client, so the next iteration no longer re-adds the old key.
• The flag is Arc-shared, so one 400 teaches the client and all its clones — same pattern as completion_ceiling.
• Added a wiremock test (an_unsupported_max_tokens_is_retried_as_max_completion_tokens) that verifies the rename, learning across calls, and learning across clones.
• cargo fmt passes.

Left out temperature deliberately: silently dropping temperature=0 switches extraction from deterministic to sampling, which felt like a semantic decision for the operator rather than a silent fallback. Happy to add it if you disagree.