Repository navigation
fix(chatgpt): infer image input from OpenAI catalogue - #4528
Merged
Merged
Conversation
There was a problem hiding this comment.
Copilot review overview
🟢 Approval recommended
The fallback is narrowly scoped, preserves capability precedence, and has focused regression coverage with no identified blocking issues.
Review effort: Balanced
Findings: None
What changed in this PR
Restores image input for known ChatGPT models by using matching OpenAI catalogue entries when direct entries are absent.
Changes:
- Adds an image-only fallback while preserving direct entries and explicit overrides.
- Tests capability precedence, attachment filtering, and backend serialization.
- Documents the behavior and adds a runnable example.
| File | Description |
|---|---|
| pkg/runtime/transforms.go | Applies the fallback before filtering messages. |
| pkg/runtime/transforms_test.go | Tests image retention and capability precedence. |
| pkg/modelinfo/resolve_caps_test.go | Tests fallback scope, precedence, and misses. |
| pkg/modelinfo/modelinfo.go | Integrates fallback into capability loading. |
| pkg/modelinfo/catalog_caps.go | Implements image-only OpenAI catalogue lookup. |
| pkg/model/provider/openai/chatgpt_test.go | Tests attachment and tool-image serialization. |
| pkg/model/provider/capability_override_test.go | Verifies ChatGPT overrides survive construction. |
| examples/README.md | Lists the image-input example. |
| examples/chatgpt-images.yaml | Adds an example without capability overrides. |
| docs/providers/chatgpt/index.md | Explains image support and fallback limits. |
| docs/configuration/models/index.md | Clarifies fallback and override behavior. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
gtardif
approved these changes
Oct 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ChatGPT models use the OpenAI-compatible Codex backend, but the models.dev catalogue has no
chatgptprovider entries. As a result, the capability lookup treated known image-capable ChatGPT models as text-only and stripped image attachments—including images returned by tools—before sending the request.Fix
When a ChatGPT model has no direct catalogue entry, use the matching OpenAI entry to infer image input only. Direct ChatGPT entries and explicit capability overrides retain precedence; unknown models remain text-only. This does not inherit PDF, audio, video, pricing, context limits, tool support, output capabilities, or
max_tokens.Regression coverage checks lookup precedence and misses, filtering, and backend serialization for both user attachments and tool-result images. A live authenticated run of
examples/chatgpt-images.yamlsent the attached PNG and returned: “A red square is centered on a bright blue square background.” The sample needs noimage: trueoverride.The issue also reported that an explicit
image: trueoverride had no effect; that symptom was not reproduced in testing, so it remains unresolved here. Related catalogue alias gaps for fireworks, together, moonshot, and opencode-zen, and an OVH model-case mismatch, were found in a read-only audit and are intentionally out of scope for this PR; no live tests were run for those providers. They should be addressed separately.Validation
Related to #4521.