feat(tui): render inline images for ReadMediaFile tool results#492
feat(tui): render inline images for ReadMediaFile tool results#492linxinhong wants to merge 1 commit into
Conversation
🦋 Changeset detectedLatest commit: 38142f3 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 38142f3346
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| }; | ||
| const dims = parseOriginalSize(summary.originalSize); | ||
| const image = new Image( | ||
| summary.base64, |
There was a problem hiding this comment.
Avoid sending huge images to the terminal
When a supported Kitty/iTerm2 terminal expands a large ReadMediaFile image, this path passes the full data-URL payload directly into the inline image component even though ReadMediaFile permits files up to 100 MB (packages/agent-core/src/tools/builtin/file/read-media.ts). The maxHeightCells/maxWidthCells options cap the displayed cell size, but they do not reduce the base64 payload being emitted, so expanding a large screenshot/photo can push tens or hundreds of MB of escape-sequence data through the TUI and make it appear hung; gate inline rendering by summary.bytes or generate a smaller thumbnail before constructing Image.
Useful? React with 👍 / 👎.
| const theme: ImageTheme = { | ||
| fallbackColor: (s: string) => chalk.hex(ctx.colors.textDim)(s), | ||
| }; | ||
| const dims = parseOriginalSize(summary.originalSize); |
There was a problem hiding this comment.
Parse the dimensions format emitted by ReadMediaFile
For real ReadMediaFile results with known image dimensions, core emits the leading system text as Original dimensions: WxH pixels. (packages/agent-core/src/tools/builtin/file/read-media.ts), but this new inline-image path only looks for the older WxHpx summary format before passing dimensions to Image. That means the renderer almost always constructs the image without the original pixel size, so terminals cannot reliably reserve/scale the image with the intended aspect ratio; accept the current Original dimensions wording or source the dimensions from the actual output format before rendering.
Useful? React with 👍 / 👎.
Problem
(
image · 45.2 KB) in the TUI transcript. Users cannot visually inspect images without switching to the visweb interface or running an external viewer.
support the Kitty or iTerm2 inline graphics protocol, render the actual image using pi-tui's
Imagecomponent.