fix: align stale provider defaults and branch Claude request shaping on model generation - #71
Merged
Conversation
Three registered defaults named models absent from models.yaml, so selecting those providers with no explicit model applied a default that failed the extension's own registry validation and printed a stderr warning -- the extension warning about its own default. - anthropic: claude-3-5-haiku-latest -> claude-haiku-4-5-20251001 (the old default resolved to claude-3-5-haiku-20241022, retired 19 Feb 2026; the replacement is the current registry entry that still supports extended thinking) - gemini: gemini-1.5-flash -> gemini-2.5-flash - ollama: llama3.2 -> llama3.2:3b (bare llama3.2 is not a pullable tag; the registry carries the :3b and :1b tags) Also corrects the Ollama help text, which told users to pull the same non-pullable bare tag. Adds ProviderDefaultsSpec as a deterministic drift guard asserting every registered descriptor's defaultModel resolves in the bundled registry, so this cannot silently regress. Verified the guard fails when a stale default is reintroduced. Fixes #62
The Claude path sent one request shape to every model, which current
Anthropic models reject with HTTP 400:
- temperature was set unconditionally to 1.0 when thinking was on, and
passed through otherwise. Non-default sampling parameters return 400
on Opus 4.7/4.8, Opus 5, Sonnet 5 and Fable/Mythos, on every request
-- not only thinking ones -- so the non-thinking path is gated too.
- thinking used {type: "enabled", budget_tokens: N} for all models.
That shape is rejected on 4.7 and later, which take
{type: "adaptive"} with depth steered by output_config.effort.
Adds ClaudeModelCapabilities holding the per-generation table:
thinkingMode (Extended for 4.5 and earlier, Adaptive otherwise,
defaulting forward for unknown identifiers), supportsSamplingParams,
and the reasoning_effort -> output_config.effort mapping. The
extension's "none" has no Anthropic equivalent and is dropped so the
API default applies.
Placing this beside ClaudeProvider rather than in
ReasoningModelDetector keeps the shared cross-provider matcher from
growing another hardcoded family check (#31). A models.yaml capability
flag was considered and rejected: the schema is a flat provider ->
[model] list, so per-model attributes would require changing the
parser, ProviderModels, merge semantics and every user's
models-override.yaml.
Also pins anthropic-version to 2023-06-01. The header was bumped to
"2025-04-15" whenever thinking was on; Anthropic publishes no such
version.
Adds ClaudeRequestSpec covering extended-thinking, adaptive-thinking
and non-thinking requests, asserting no temperature key is sent to
4.7+ models. Tests inspect the built request body directly, so they
need no network or API key. Verified they fail when the temperature
gate is removed.
Fixes #63
Three follow-ups from review of this branch. 1. models.yaml carried no model that takes the adaptive-thinking path, so the new request shaping was unreachable out of the box — every bundled Anthropic entry was 4.5-or-earlier. Adds claude-fable-5, claude-opus-5, claude-sonnet-5, claude-opus-4-8/4-7 and the 4.6 pair, with IDs taken from Anthropic's live model overview (dateless from 4.6 on, still pinned snapshots). Drops claude-3-7-sonnet, retired 19 Feb 2026 — listing a retired model is the defect #62 fixed. 2. supportsSamplingParams was a denylist of models that reject temperature, so claude-sonnet-4-7 or claude-opus-4-9 matched nothing, fell through as permitted, and would be sent a temperature they 400 on. It is now an allowlist of generations known to ACCEPT sampling params, so unknown names default forward — the same direction thinkingMode already defaulted. The two checks are now consistent. 3. FALLBACK_CONFIG still held claude-3-5-haiku-latest, gemini-1.5-flash and bare llama3.2 — the exact retired models #62 removed elsewhere. The existing drift guard reads the loaded registry, so it could not see them. Updated, and a second guard now covers the fallback map. The new guard was mutation-tested: reintroducing a stale Ollama default fails it with the drifted name, and passes once restored. 73 tests pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #62 and #63. Together these restore the Anthropic path end to end.
#62 — three provider defaults did not exist in the bundled registry
ProviderRegistrations.scalashippedclaude-3-5-haiku-latest,gemini-1.5-flash, andllama3.2. None appear inmodels.yaml(the registry hasllama3.2:3b/:1b, not barellama3.2).Consequence:
llm:set-provider "gemini"with no model applied a default that failed registry validation and printed a warning — the extension warning about its own default. For Ollama it was worse:llama3.2is not a pullable tag. This is the first thing a new user hits.claude-3-5-haiku-latest→claude-haiku-4-5-20251001(the old default resolved toclaude-3-5-haiku-20241022, retired 19 Feb 2026)gemini-1.5-flash→gemini-2.5-flashllama3.2→llama3.2:3bollama pull llama3.2— the same non-pullable tagdefaultModelexists in the bundled registry, so this cannot silently recur#63 — Claude request shape broke every current model
ClaudeProviderunconditionally settemperature = 1.0andthinking: {type: "enabled"}whenever thinking was on. Both are now wrong.Verified against Anthropic's live documentation:
thinking.type: "enabled"returns 400 on Opus 4.7, Opus 4.8, Opus 5, Sonnet 5, Fable 5, Mythos 5. Those usethinking: {type: "adaptive"}with depth steered byoutput_config: {effort: ...}— nested, not a top-leveleffortfield.temperatureis rejected on those models on every request, regardless of whether thinking is used — so it is suppressed on the non-thinking path too."adaptive".low|medium|high|xhigh|max— there is nonone. The extension's existingnonemaps to omittingoutput_configso the API default applies.Also fixed: the code sent
anthropic-version: 2025-04-15. That version does not exist — the documented value is2023-06-01. Pre-existing bug, in scope because #63 is about restoring this path.Design decision
The generation split lives in a new
ClaudeModelCapabilities, not inReasoningModelDetectorand not inmodels.yaml.ReasoningModelDetectoranswers the cross-provider "should thinking be on" question. Putting Anthropic request-shape details there grows exactly the shared hardcoded-family-check surface that cleanup: remove or wire currently unused provider and registry helpers #31 is about.models.yamlcapability flag sounds cleaner, but the schema is a flatprovider -> [model strings]list. Per-model attributes would mean changing the parser,ProviderModels, merge semantics, and every user's existingmodels-override.yaml— a breaking format change.Unknown model identifiers default to adaptive, since an unrecognised name is more likely new than old.
Tests
71 passing, 0 failed (baseline 54, +17). Two new deterministic specs — no network, no API keys.
The new tests were mutation-tested: reintroducing both bugs produced exactly three failures (drift guard caught the stale Gemini default; both temperature guards caught the regression), then green again after restoring.
Known follow-up, not in this PR
models.yamlhas noclaude-opus-5,sonnet-5,opus-4-7/4-8, orfable-5— the newest Anthropic entry isclaude-opus-4-5-20251101. The adaptive path therefore only activates for models a user adds via override, and the default here is Haiku 4.5 rather than a 5-series model. Refreshing the registry deserves its own issue and strengthens the case for #64.