fix(tools): persist large file outputs before response limits (#7781)

* fix(tools): persist large file outputs before response limits

* fix(tools): cover late attachments and simplify file outputs

Author: icecrasher321

.agents/skills/add-block/SKILL.md

@@ -927,6 +927,11 @@ bun run apps/sim/scripts/check-canvas-sentences.ts --block={service}
 
 ## Generated artifacts
 
+When adding or changing `sunset.replacedBy`, run `bun run generate:block-successors` and commit
+`apps/sim/lib/permission-groups/block-successors.generated.ts`. Authorization uses this generated
+map to resolve legacy and current block IDs consistently without importing the executable registry.
+Verify it with `bun run check:block-successors`.
+
 Adding a block on its own needs no **tool metadata** regeneration — a block references existing
 tool IDs through `tools.access` and does not change any tool's shape.
 
@@ -969,6 +974,7 @@ changes.
 - [ ] Tools.config.tool returns correct tool ID (snake_case)
 - [ ] Outputs match tool outputs
 - [ ] Block + meta registered in registry-maps.ts (`BLOCK_REGISTRY` / `BLOCK_META_REGISTRY`)
+- [ ] If `sunset.replacedBy` changed: regenerated and committed the block successor map; `bun run check:block-successors` passes
 - [ ] If any tool was added, changed or removed alongside the block: ran `bun run tool-metadata:generate` and committed the artifacts
 - [ ] Ran `bun run scripts/generate-docs.ts`, reviewed the generated diff, and committed the integration catalog changes
 - [ ] `bun run integration-catalog:check` passes

.agents/skills/add-tools/SKILL.md

@@ -264,6 +264,38 @@ stale/missing sidecars, and scope isolation.
 
 ## Critical Rules for Outputs
 
+### File Downloads and Generated Files
+
+Internal operations return `createInternalToolFileResult` / `createInternalToolFilesResult` from
+`lib/internal/tool-operations/file-result.ts` with bounded Buffers and a callback that places the
+stored descriptors in the response. Their handlers preserve this result through dispatch, using
+`InternalToolOperationHandler<InternalToolOperationResult>`. Do not serialize file bytes as base64
+JSON: the executor's 10 MiB response cap runs before ordinary file postprocessing or large-value
+externalization. The shared executor stores files using trusted run or Copilot ownership.
+
+External endpoints that return raw binary files explicitly declare `request.responseType: 'binary'`
+and return `output.file` with `{ name, mimeType, data: buffer, size }` from `transformResponse`.
+The executor applies the bounded file-transfer budget and persists the descriptor. This opt-in is
+for raw binary responses, not provider JSON containing base64 or tools that fetch attachments later.
+Keep provider-specific limits and bounded reads; a file declaration is not permission to enlarge
+arbitrary JSON responses.
+
+Attachment readers that download files inside `transformResponse` need their own bounded reads:
+the first response cap does not cover subsequent fetches. Accept `ToolResponseContext` as the third
+transform argument, forward its `signal`, and share one `AttachmentDownloadBudget` across sequential
+downloads. Prefer raw provider endpoints over base64 metadata. Return the same file object in the
+declared `file` / `file[]` output and nested message associations; `FileToolProcessor` stores it once
+and replaces every alias with the same `UserFile` in both workflow and Copilot execution.
+
+Preserve stored `UserFile` fields (`id`, `key`, `url`, `context`, `type`, `name`, `size`) in transforms;
+rebuilding the old `{ name, mimeType, data, size }` shape discards the reference. File outputs do not
+need duplicate inline text/base64 aliases; the file system handles content materialization. When
+an existing tool explicitly exposes content aliases in its contract, preserve its legacy version and
+use the existing block/tool version pattern for a file-only output. Test a file over 10 MiB through
+executor admission, single persistence, trusted ownership, and the unchanged JSON cap. Avoid adding
+top-level filename, size, MIME type, URL, or success fields that merely repeat the canonical file or
+tool result; keep additional provider fields only when they convey distinct information.
+
 ### Output Types
 - `'string'`, `'number'`, `'boolean'` - Primitives
 - `'json'` - Complex objects (use this, NOT 'object')

.agents/skills/validate-integration/SKILL.md

@@ -345,6 +345,21 @@ If any tool lists, searches, exports, imports, downloads, uploads, paginates, ba
 - [ ] List/search tools expose API limits and do not auto-fetch every page into memory
 - [ ] Transform logic does not build unbounded arrays, maps, sets, or `Promise.all` fan-outs
 - [ ] File and HTTP body reads use explicit byte caps or existing stream-limit helpers
+- [ ] Internal file results reach `createInternalToolFileResult` / `createInternalToolFilesResult`
+      before JSON serialization; external raw downloads explicitly use `request.responseType: 'binary'`
+      and return a buffered `output.file`. Provider base64 JSON needs separate handling
+- [ ] Transforms retain stored `UserFile` identity/access fields, and tests cover a >10 MiB file
+      crossing executor admission without another upload. New file outputs contain references only,
+      without inline content aliases; preserve legacy versions when removing existing inline fields
+- [ ] Scan every file-producing path, including attachment fetches inside `transformResponse`, URL
+      descriptors, export operations, and old/new block versions; checking download-named tools alone
+      misses late reads that occur after the first response admission
+- [ ] Late attachment reads share a per-call byte budget, bound actual streamed bytes independently
+      of provider size metadata, and forward `ToolResponseContext.signal` through every fetch/read
+- [ ] Both workflow and Copilot tests produce compact `UserFile` outputs; nested message attachment
+      aliases reference the same stored files, with no duplicate upload or raw bytes left behind
+- [ ] New output contracts omit redundant copies of file name, MIME type, size, URL, and success;
+      retained provider metadata has a distinct purpose, and types match the stored-file runtime shape
 - [ ] Large result payloads are summarized, paginated, referenced, or capped rather than raw-dumped
 - [ ] Pagination and download tests cover caps, early stop behavior, or partial-result preservation when relevant
 

apps/docs/components/ui/icon-mapping.ts

@@ -339,6 +339,7 @@ export const blockTypeToIconMap: Record<string, IconComponent> = {
   azure_devops: AzureIcon,
   bitbucket: BitbucketIcon,
   box: BoxCompanyIcon,
+  box_v2: BoxCompanyIcon,
   brandfetch: BrandfetchIcon,
   brex: BrexIcon,
   brightdata: BrightDataIcon,
@@ -380,9 +381,11 @@ export const blockTypeToIconMap: Record<string, IconComponent> = {
   docusign: DocuSignIcon,
   downdetector: DowndetectorIcon,
   dropbox: DropboxIcon,
+  dropbox_v2: DropboxIcon,
   dropcontact: DropcontactIcon,
   dspy: DsPyIcon,
   dub: DubIcon,
+  dub_v2: DubIcon,
   duckduckgo: DuckDuckGoIcon,
   dynamodb: DynamoDBIcon,
   dynatrace: DynatraceIcon,
@@ -475,6 +478,7 @@ export const blockTypeToIconMap: Record<string, IconComponent> = {
   jotform: JotformIcon,
   jsm: JiraServiceManagementIcon,
   jupyter: JupyterIcon,
+  jupyter_v2: JupyterIcon,
   kalshi: KalshiIcon,
   kalshi_v2: KalshiIcon,
   ketch: KetchIcon,
@@ -506,6 +510,7 @@ export const blockTypeToIconMap: Record<string, IconComponent> = {
   memory: BrainIcon,
   microsoft_ad: AzureIcon,
   microsoft_dataverse: MicrosoftDataverseIcon,
+  microsoft_dataverse_v2: MicrosoftDataverseIcon,
   microsoft_dynamics_365: MicrosoftDataverseIcon,
   microsoft_excel: MicrosoftExcelIcon,
   microsoft_excel_v2: MicrosoftExcelIcon,
@@ -557,6 +562,7 @@ export const blockTypeToIconMap: Record<string, IconComponent> = {
   quartr: QuartrIcon,
   quickbooks: QuickBooksIcon,
   quiver: QuiverIcon,
+  quiver_v2: QuiverIcon,
   rabbitmq: RabbitmqIcon,
   railway: RailwayIcon,
   rb2b: RB2BIcon,
@@ -588,8 +594,10 @@ export const blockTypeToIconMap: Record<string, IconComponent> = {
   sentry: SentryIcon,
   serper: SerperIcon,
   servicenow: ServiceNowIcon,
+  servicenow_v2: ServiceNowIcon,
   ses: SESIcon,
   sftp: SftpIcon,
+  sftp_v2: SftpIcon,
   sharepoint: MicrosoftSharepointIcon,
   sharepoint_v2: MicrosoftSharepointIcon,
   shopify: ShopifyIcon,
@@ -607,6 +615,7 @@ export const blockTypeToIconMap: Record<string, IconComponent> = {
   sqs: SQSIcon,
   square: SquareIcon,
   ssh: SshIcon,
+  ssh_v2: SshIcon,
   ssm: SSMIcon,
   stagehand: StagehandIcon,
   start_trigger: StartIcon,

apps/docs/content/docs/integrations/box.mdx

@@ -6,7 +6,7 @@ description: Manage files, folders, and e-signatures with Box
 import { BlockInfoCard } from "@/components/ui/block-info-card"
 
 <BlockInfoCard 
-  type="box"
+  type="box_v2"
   color="#FFFFFF"
 />
 
@@ -63,7 +63,6 @@ Download a file from Box
 | Parameter | Type | Description |
 | --------- | ---- | ----------- |
 | `file` | file | Downloaded file stored in execution files |
-| `content` | string | Base64 encoded file content |
 
 ### Box Get File Info
 

apps/docs/content/docs/integrations/dropbox.mdx

@@ -6,7 +6,7 @@ description: Upload, download, share, and manage files in Dropbox
 import { BlockInfoCard } from "@/components/ui/block-info-card"
 
 <BlockInfoCard 
-  type="dropbox"
+  type="dropbox_v2"
   color="#0061FF"
 />
 
@@ -66,7 +66,7 @@ Upload a file to Dropbox
 
 ### Dropbox Download File
 
-Download a file from Dropbox with metadata and content
+Download a file from Dropbox with metadata
 
 #### Input
 
@@ -81,7 +81,6 @@ Download a file from Dropbox with metadata and content
 | `file` | file | Downloaded file stored in execution files |
 | `metadata` | json | The file metadata |
 | `temporaryLink` | string | Temporary link to download the file \(valid for ~4 hours\) |
-| `content` | string | Base64 encoded file content \(if fetched\) |
 
 ### Dropbox List Folder
 

apps/docs/content/docs/integrations/dub.mdx

@@ -6,7 +6,7 @@ description: Link management with Dub
 import { BlockInfoCard } from "@/components/ui/block-info-card"
 
 <BlockInfoCard 
-  type="dub"
+  type="dub_v2"
   color="#181C1E"
 />
 
@@ -474,7 +474,6 @@ Generate a customizable QR code (PNG) for a short link, with control over size,
 | Parameter | Type | Description |
 | --------- | ---- | ----------- |
 | `file` | file | Generated QR code image stored in execution files |
-| `content` | string | Base64-encoded PNG image data |
 
 ### Dub List Domains
 

apps/docs/content/docs/integrations/jupyter.mdx

@@ -6,7 +6,7 @@ description: Manage files, notebooks, kernels, and sessions on a Jupyter server
 import { BlockInfoCard } from "@/components/ui/block-info-card"
 
 <BlockInfoCard 
-  type="jupyter"
+  type="jupyter_v2"
   color="#FFFFFF"
 />
 
@@ -61,7 +61,7 @@ List files, notebooks, and subdirectories at a path on a Jupyter server
 
 ### Jupyter Get Content
 
-Read a file or notebook from a Jupyter server
+Download a file as a stored file, or read structured notebook and directory content
 
 #### Input
 
@@ -75,11 +75,11 @@ Read a file or notebook from a Jupyter server
 
 | Parameter | Type | Description |
 | --------- | ---- | ----------- |
-| `name` | string | File or notebook name |
-| `path` | string | Path relative to the server root |
-| `mimetype` | string | MIME type of the content |
-| `text` | string | Text content, for text files and notebooks \(JSON-stringified\) |
-| `file` | file | Binary content stored as a file, for base64-format content |
+| `file` | file | Downloaded file |
+| `text` | string | JSON-stringified notebook or directory content |
+| `name` | string | Notebook or directory name |
+| `path` | string | Notebook or directory path |
+| `mimetype` | string | Notebook or directory MIME type |
 
 ### Jupyter Create File
 

apps/docs/content/docs/integrations/microsoft_dataverse.mdx

@@ -6,7 +6,7 @@ description: Manage records in Microsoft Dataverse tables
 import { BlockInfoCard } from "@/components/ui/block-info-card"
 
 <BlockInfoCard 
-  type="microsoft_dataverse"
+  type="microsoft_dataverse_v2"
   color="#FFFFFF"
 />
 
@@ -136,7 +136,7 @@ Remove an association between two records in Microsoft Dataverse. For collection
 
 ### Download File from Microsoft Dataverse
 
-Download a file from a file or image column on a Dataverse record. Stores the file in execution storage and returns a file reference, plus the base64 content and metadata directly.
+Download a file from a Dataverse file or image column and return its stored file reference and metadata
 
 #### Input
 
@@ -152,12 +152,7 @@ Download a file from a file or image column on a Dataverse record. Stores the fi
 | Parameter | Type | Description |
 | --------- | ---- | ----------- |
 | `file` | file | Downloaded file stored in execution files |
-| `fileContent` | string | Base64-encoded file content |
-| `fileName` | string | Name of the downloaded file |
-| `fileSize` | number | File size in bytes |
-| `mimeType` | string | MIME type of the file |
 | `fileColumn` | string | File column the file was downloaded from |
-| `success` | boolean | Whether the file was downloaded successfully |
 
 ### Execute Microsoft Dataverse Action
 

apps/docs/content/docs/integrations/quiver.mdx

@@ -6,7 +6,7 @@ description: Generate and vectorize SVGs
 import { BlockInfoCard } from "@/components/ui/block-info-card"
 
 <BlockInfoCard 
-  type="quiver"
+  type="quiver_v2"
   color="#FFFFFF"
 />
 
@@ -57,16 +57,12 @@ Generate SVG images from text prompts using QuiverAI
 
 | Parameter | Type | Description |
 | --------- | ---- | ----------- |
-| `success` | boolean | Whether the SVG generation succeeded |
-| `output` | object | Generated SVG output |
-|   ↳ `file` | file | First generated SVG file |
-|   ↳ `files` | json | All generated SVG files \(when n &gt; 1\) |
-|   ↳ `svgContent` | string | Raw SVG markup content of the first result |
-|   ↳ `id` | string | Generation request ID |
-|   ↳ `usage` | json | Token usage statistics |
-|     ↳ `totalTokens` | number | Total tokens used |
-|     ↳ `inputTokens` | number | Input tokens used |
-|     ↳ `outputTokens` | number | Output tokens used |
+| `files` | file[] | All generated SVG files |
+| `id` | string | Request ID |
+| `usage` | json | Token usage statistics |
+|   ↳ `totalTokens` | number | Total tokens used |
+|   ↳ `inputTokens` | number | Input tokens used |
+|   ↳ `outputTokens` | number | Output tokens used |
 
 ### Quiver Image to SVG
 
@@ -90,15 +86,12 @@ Convert raster images into vector SVG format using QuiverAI
 
 | Parameter | Type | Description |
 | --------- | ---- | ----------- |
-| `success` | boolean | Whether the vectorization succeeded |
-| `output` | object | Vectorized SVG output |
-|   ↳ `file` | file | Generated SVG file |
-|   ↳ `svgContent` | string | Raw SVG markup content |
-|   ↳ `id` | string | Vectorization request ID |
-|   ↳ `usage` | json | Token usage statistics |
-|     ↳ `totalTokens` | number | Total tokens used |
-|     ↳ `inputTokens` | number | Input tokens used |
-|     ↳ `outputTokens` | number | Output tokens used |
+| `files` | file[] | All generated SVG files |
+| `id` | string | Request ID |
+| `usage` | json | Token usage statistics |
+|   ↳ `totalTokens` | number | Total tokens used |
+|   ↳ `inputTokens` | number | Input tokens used |
+|   ↳ `outputTokens` | number | Output tokens used |
 
 ### Quiver List Models