Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion apps/docs/content/docs/integrations/mistral_parse.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ Parse PDF documents using Mistral OCR API
| ↳ `pages_processed` | number | Total number of pages processed |
| ↳ `doc_size_bytes` | number | Document file size in bytes |
| `document_annotation` | string | Structured annotation data as JSON string \(when applicable\) |
| `images` | file[] | Images extracted from the document, named page-<index>-<image id>; empty unless image extraction is on |
| `images` | file[] | Images extracted from the document, named page-<index>-<image id> with the extension of the detected image type; empty unless image extraction is on |

</ReferenceDetails>

Expand Down
4 changes: 2 additions & 2 deletions apps/docs/content/docs/platform/self-hosting/networking.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -265,7 +265,7 @@ Most `/api` requests also pass through Next.js Proxy. The current app configurat
The app makes outbound calls to model providers, integration APIs, your email provider, object storage, and your telemetry backend. Whether `HTTP_PROXY` / `HTTPS_PROXY` apply depends on which of those paths a call takes — there is no single answer, and no global setting that covers all of them.
The server runs on Bun, and Bun's native `fetch` honors `$HTTP_PROXY`, `$HTTPS_PROXY`, and `$NO_PROXY`. Sim installs no global dispatcher. Default `fetch` uses the configured proxy unless `NO_PROXY` excludes the destination. The SSRF guard reads the same three variables for every request it sends, matching `NO_PROXY` the way curl does (see [Internal destinations and `NO_PROXY`](#internal-destinations-and-no_proxy)). The rest either build their own HTTP agent or speak a non-HTTP protocol.
The server runs on Bun, and Bun's native `fetch` honors `$HTTP_PROXY`, `$HTTPS_PROXY`, and `$NO_PROXY`. Sim installs no global dispatcher. Default `fetch` uses the configured proxy unless `NO_PROXY` excludes the destination. The SSRF guard reads the same three variables for every request it sends, matching `NO_PROXY` with the rules in [Internal destinations and `NO_PROXY`](#internal-destinations-and-no_proxy). The rest either build their own HTTP agent or speak a non-HTTP protocol.
Comment thread
waleedlatif1 marked this conversation as resolved.
Resolved
This depends on the runtime. The published images run Bun. If you build the standalone output and run it under Node instead, Node ignores these variables unless started with `NODE_USE_ENV_PROXY=1` (Node 22.21+ / 24.5+), and the `fetch`-based "Yes" rows stop being proxied — model providers, Resend, Gmail sending, the desktop update feed, and the telemetry relay. The Azure Blob, GCS and Azure Communication Services rows still hold: those SDKs read the proxy variables through their own agents rather than through `fetch`. See [Bun fetch](https://bun.sh/docs/runtime/networking/fetch) and [Node proxy configuration](https://nodejs.org/learn/http/enterprise-network-configuration).
Expand Down Expand Up @@ -301,7 +301,7 @@ A request through the SSRF guard keeps its checks when it is proxied. Before eac
Set `NO_PROXY` for every destination that is not on the public internet, not just model endpoints. The app reaches the realtime server (`SOCKET_SERVER_URL`), the Presidio PII service (`PII_URL`), and itself (`INTERNAL_API_BASE_URL`) over the same default `fetch`, alongside self-hosted Ollama, LiteLLM, and vLLM — so a proxy that cannot reach your internal network breaks live updates and PII redaction, not only inference. AWS clients and OTLP export follow the same rule, so also add an internal S3-compatible store such as MinIO, AWS VPC endpoints, and an in-cluster OpenTelemetry collector.
For the SSRF guard, AWS clients (including the boot-time Secrets Manager fetch), and OTLP export, Sim matches `NO_PROXY` entries the way curl does: `example.com`, `.example.com`, and `*.example.com` each cover the domain and all its subdomains; `host:port` and `[::1]:port` cover one port; IPv6 entries work bare (`::1`) or bracketed; and CIDR ranges such as `10.0.0.0/8` cover IP-literal destinations only, not hostnames that resolve into the range. Bun's default `fetch`, which carries internal service URLs such as the realtime server and PII service, applies Bun's own `NO_PROXY` rules, which may not honor CIDR or bare IPv6 entries, so list those destinations by hostname.
For the SSRF guard, AWS clients (including the boot-time Secrets Manager fetch), and OTLP export, Sim matches `NO_PROXY` entries with these rules: `example.com`, `.example.com`, and `*.example.com` each cover the domain and all its subdomains; `host:port` and `[::1]:port` cover one port; IPv6 entries work bare (`::1`) or bracketed; and CIDR ranges such as `10.0.0.0/8` cover IP-literal destinations only, not hostnames that resolve into the range. Bun's default `fetch`, which carries internal service URLs such as the realtime server and PII service, applies Bun's own `NO_PROXY` rules, which may not honor CIDR or bare IPv6 entries, so list those destinations by hostname.
Set it in the application environment, not your shell — under `app.env` on Helm, or the service's `environment:` on Compose. On Helm the suffixes alone are not enough: the chart wires `SOCKET_SERVER_URL`, `PII_URL`, and `OLLAMA_URL` to bare Service names, which no domain suffix matches. Add those names too — `helm template` prints the rendered ones, and the prefix is the release name unless it already contains `sim`, in which case it is the release name alone:
Expand Down
9 changes: 6 additions & 3 deletions apps/sim/lib/internal/mistral/operations.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -256,13 +256,16 @@ describe('Mistral ingestion authorization', () => {
)

it('returns extracted images as files, never as base64 in the output', async () => {
const png = Buffer.concat([Buffer.from([0xff, 0xd8, 0xff]), Buffer.from('synthetic jpeg body')])
const jpeg = Buffer.concat([
Buffer.from([0xff, 0xd8, 0xff]),
Buffer.from('synthetic jpeg body'),
])
submit.mockResolvedValue({
pages: [
{
index: 0,
images: [
{ id: 'img-0.png', image_base64: `data:image/png;base64,${png.toString('base64')}` },
{ id: 'img-0.png', image_base64: `data:image/png;base64,${jpeg.toString('base64')}` },
],
},
],
Expand All @@ -275,7 +278,7 @@ describe('Mistral ingestion authorization', () => {
expect(isInternalToolFileResult(result)).toBe(true)
if (!isInternalToolFileResult(result)) return
expect(result.files).toHaveLength(1)
expect(JSON.stringify(result.present([]))).not.toContain(png.toString('base64'))
expect(JSON.stringify(result.present([]))).not.toContain(jpeg.toString('base64'))
})

it('does not request images for tool versions that have no image output', async () => {
Expand Down
2 changes: 1 addition & 1 deletion apps/sim/scripts/saved-credential-contract.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ const DESTINATION_PARAM =
const OPERATION_PAYLOAD_PARAMS: Readonly<Record<string, readonly string[]>> = {
// The page the remote browser opens; the token goes to api.browser-use.com.
browser_use_run_task: ['startUrl'],
// A guild id in the discord.com API path.
// A guild id the operation never sends; it posts only to the channel's discord.com endpoint.
discord_send_message: ['serverId'],
// The audio Fireflies downloads; the token goes to api.fireflies.ai.
fireflies_upload_audio: ['audioUrl'],
Expand Down
2 changes: 1 addition & 1 deletion apps/sim/tools/generated/tool-outputs.ts

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion apps/sim/tools/mistral/parser.ts
Original file line number Diff line number Diff line change
Expand Up @@ -524,7 +524,7 @@ export const mistralParserV3Tool: InternalToolConfig<MistralParserV2Input, Mistr
images: {
type: 'file[]',
description:
'Images extracted from the document, named page-<index>-<image id>; empty unless image extraction is on',
'Images extracted from the document, named page-<index>-<image id> with the extension of the detected image type; empty unless image extraction is on',
},
},
params: {
Expand Down
Loading