[BUG] Error loading an image with glm-5.3-flash

[felocru]

🐛 What happened?

Whenever I instruct the agent to open the internal browser and correct a specific webpage using screenshots, I consistently receive this error.

🔄 Steps to reproduce

1. Request the agent to: I would like you to try viewing the website using the built-in browser, in order to correct any details.

✅ What did you expect?

🖥️ Environment

• Extension version: 0.7.5
• VS Code version: 1.137.0
• Provider: OpenCode Go
• Model: glm 5.3 flash

📸 Logs or screenshots (optional)

Sorry, your request failed. Please try again.

Client Request Id: fe5bdf71-5bd6-4a1e-a24a-f9af95f8210b

Reason: OpenCode Go API request failed (422) model=glm-5.3-flash payloadBytes=949611: Error from provider (Console Go): Upstream request failed: [invalid_request_error] Input should be a valid string: OpenCodeRequestError: OpenCode Go API request failed (422) model=glm-5.3-flash payloadBytes=949611: Error from provider (Console Go): Upstream request failed: [invalid_request_error] Input should be a valid string at buildOpenCodeRequestError (/home/felocru/.vscode-server/extensions/ltmoerdani.opencode-copilot-chat-0.7.5/out/errors.js:39:12) at streamOpenCodeResponse (/home/felocru/.vscode-server/extensions/ltmoerdani.opencode-copilot-chat-0.7.5/out/transports/engine.js:320:73) at process.processTicksAndRejections (node:internal/process/task_queues:104:5) at async streamChatCompletions (/home/felocru/.vscode-server/extensions/ltmoerdani.opencode-copilot-chat-0.7.5/out/transports/chatCompletions.js:26:9) at async OpenCodeProvider.provideLanguageModelChatResponse (/home/felocru/.vscode-server/extensions/ltmoerdani.opencode-copilot-chat-0.7.5/out/provider/OpenCodeProvider.js:388:13)

### 23. glm-5.3-flash

- time: 2026-09-13T13:22:57.279Z
- endpoint: chat-completions
- initiator: github.copilot-chat
- metadataSource: models.dev
- status: 422
- durationMs: 648
- ttfbMs: n/a
- totalBytes: 208
- totalEvents: 0
- tokens: prompt=n/a, completion=n/a, total=n/a, cached=n/a
- cacheHitRatio: n/a
- finishReason: n/a
- requestId: req-3f07876c
- sessionId: vscode-3938f5d5
- url: https://opencode.ai/zen/go/v1/chat/completions
- error: OpenCode Go API request failed (422) model=glm-5.3-flash payloadBytes=949611: Error from provider (Console Go): Upstream request failed: [invalid_request_error] Input should be a valid string

[wallase16]

Root-cause trace for this one (reproduced on 0.7.5, same stack, different conversation/OS). Reposting here since it belongs in this issue:

What the upstream is actually rejecting

The 422 body is a pydantic-style validation failure from the Console Go upstream:

[invalid_request_error] Input should be a valid string

That message shape ("Input should be a valid string") is what a schema with a str-typed field emits when it receives something else — in this case content as an OpenAI-style array of content parts ([{type:"text"...},{type:"image_url"...}]) rather than a plain string. So the upstream chat-completions schema for glm-5.3-flash appears to accept string content only, and rejects the whole request the moment any image part is present.

Why the extension forwards images for this model

• Model capability comes from provider metadata (metadataSource: models.dev in the request log). detectModalityFlags() in out/models/metadata.js maps an advertised image/image_url/input_image input modality to supportsVision.
• In out/provider/chatPrep.js, image parts are only replaced (vision proxy) or stripped when !metadata.supportsVision. Since the gateway advertises glm-5.3-flash as image-capable, image_url parts are forwarded verbatim — and the upstream rejects the whole request.
• Net effect: a capability-metadata mismatch between what the gateway advertises for glm-5.3-flash and what its console upstream schema accepts.

Second, related defect: the payload backstop doesn't apply

MAX_REQUEST_PAYLOAD_BYTES = 512 * 1024 (out/config.js) bounds the trimmed history, but both failing requests exceeded it (949 KB here, 2.4 MB in mine). trimOldImagesFromHistoryInPlace keeps the most recent MAX_HISTORY_IMAGES_KEPT = 2 images, so a conversation where the last turn carries screenshot(s) always ships those bytes — history trimming alone cannot bound it.

Suggested directions

1. Treat "upstream rejects array content" as a capability mismatch: when a vision-advertised model 422s with "Input should be a valid string", retry once with images stripped (surfacing that images were dropped), and/or add a per-model vision override so users can force-strip.
2. Extend the payload-bytes backstop to cap the current turn's image attachments too, not just history.
3. Possibly also file against the gateway: models.dev metadata for glm-5.3-flash advertises image input that the upstream rejects.

Workaround until fixed

Keep image-bearing turns out of the conversation with this model (no screenshots/paste-images), or start a new conversation after any turn that had images.

Environment: extension 0.7.5, VS Code 1.136.1, OpenCode Go, model glm-5.3-flash. Repro: any request whose history contains an image attachment fails with this 422 in <1 s, zero stream events.

[wallase16]

Follow-up to my earlier comment — with a minimal reproducer, the hypothesis there was partly wrong, and the actual trigger is narrower (and easier to fix):

What the upstream accepts (verified directly against zen/go/v1/chat/completions, model glm-5.3-flash)

#	Payload shape	Result
1	user message, string content	200
2	user message, array with image_url part	200 — the model reads the image (streaming and non-streaming)
3	user message, array of two text parts	200
4	assistant tool_calls with arguments as string + tool message, string content	200
5	tool message, array of one text part	200
6	tool message whose content array contains an image_url part	422 — the exact error in this issue
7	assistant tool_calls with arguments as an object instead of a string	422 — same error

The gateway returns the offending field, which is very actionable:

{"error":{"param":"messages.2.tool.content.str","type":"invalid_request_error",
 "message":"... [invalid_request_error] Input should be a valid string"}}

{"error":{"param":"messages.1.assistant.tool_calls.0.function.arguments", ...}}

What this means

The upstream does accept multimodal content in user messages — the catalog entry isn't lying, and no image-stripping is needed for normal attachments. What its schema rejects is non-string content in tool-role messages (and object-typed tool_calls[].function.arguments). In the failing conversations, the screenshots enter the history as tool results (e.g. a browser/screenshot tool's output), which the extension forwards as a tool message whose content is an array containing an image part → 422 on every subsequent turn, because the tool result stays in history.

Suggested fix (extension-side, narrow)

When serialising for a chat-completions gateway, normalise tool-role messages only:

• tool message content: if it's an array, either flatten to string, move image parts out of the tool message into a follow-up user-role message, or run them through the existing vision proxy (the current code only proxies/strips images based on metadata.supportsVision and only for user messages — tool-result images are only size-capped, never converted);
• tool_calls[].function.arguments: always JSON.stringify if it isn't already a string.

That also explains why the payload backstop can't help here: the offending parts aren't a size problem, they're a schema problem.

Repro payloads available on request; reproduces on 0.7.5 with any conversation containing a tool result that carries an image.

[ltmoerdani]

@felocru This one is fixed in #242, and credit for pinning it down goes to @wallase16, the repro matrix in the comments above saved a lot of guessing.

To recap what it showed: the upstream accepts images fine in user messages, it only rejects list-type content (and image_url parts) inside role: "tool" messages. Our converter was forwarding screenshot tool results as a multimodal array, and since the tool result stays in history, every following turn failed with the same 422.

The fix: for glm-5.3* we now flatten the tool message to a plain string and move the images into a follow-up user message instead, so the model still sees them (that path returns 200 per the same matrix). MiMo keeps its existing placeholder behavior, and models that accept multimodal tool content (kimi, minimax, qwen, glm-5.1/5.2) are untouched. Also added the serialization cases to our e2e suite so this can't quietly come back.

Will be in the next release. Thanks @felocru for the report and @wallase16 for the follow-up investigation.