Skip to content

Commit 8718b9c

Browse files
authored
feat(network): honor HTTP_PROXY, HTTPS_PROXY and NO_PROXY for guarded requests, AWS SDK clients and OTLP export (#9851)
* feat(network): honor HTTP_PROXY, HTTPS_PROXY and NO_PROXY for guarded outbound requests * fix(network): limit environment proxies to http and https, cover an https proxy * feat(network): route AWS SDK clients and OTLP exporters through the environment proxy * fix(network): record the proxy dependencies under apps/sim and register the proxy agent as egress infrastructure
1 parent e1a3f29 commit 8718b9c

41 files changed

Lines changed: 721 additions & 90 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎apps/docs/content/docs/platform/self-hosting/environment-variables.mdx‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -255,6 +255,7 @@ Configure at least one. Every configured provider stays active and is tried in o
255255
| `EGRESS_ALLOWED_HOSTS` | unset | Comma-separated hostnames outbound requests may reach on a private network. Leading wildcard allowed, e.g. `host.docker.internal,*.svc.cluster.local`. Not honored for URLs harvested from content or a third-party response, nor for an HTTP block's `proxyUrl` |
256256
| `EGRESS_ALLOWED_IP_RANGES` | unset | Comma-separated CIDRs or IPs outbound requests may reach on a private network, e.g. `10.0.0.0/8`. Same exclusions |
257257
| `ALLOW_PRIVATE_DATABASE_HOSTS` | unset | **Deprecated.** Vouches for the entire private address space, for database, cache, and mail connector hosts only. Replace it with the two settings above naming specific destinations |
258+
| `DOMAIN_VERIFICATION_NAMESERVERS` | `1.1.1.1,8.8.8.8` | Comma-separated nameserver IPs that SSO domain verification sends its TXT lookup to. Set it when the network only allows DNS to an internal resolver. Entries that are not IP addresses are ignored |
258259

259260
A malformed entry in either allowlist stops the app at startup with a message naming the setting. See [the SSRF boundary](/platform/self-hosting/security#the-ssrf-boundary) for the accepted syntax and for what the allowlist does and does not reach.
260261

‎apps/docs/content/docs/platform/self-hosting/networking.mdx‎

Lines changed: 20 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -265,7 +265,7 @@ Most `/api` requests also pass through Next.js Proxy. The current app configurat
265265
266266
The app makes outbound calls to model providers, integration APIs, your email provider, object storage, and your telemetry backend. Whether `HTTP_PROXY` / `HTTPS_PROXY` apply depends on which of those paths a call takes — there is no single answer, and no global setting that covers all of them.
267267
268-
The server runs on Bun, and Bun's native `fetch` honors `$HTTP_PROXY`, `$HTTPS_PROXY`, and `$NO_PROXY`. Sim installs no global dispatcher. Default `fetch` uses the configured proxy unless `NO_PROXY` excludes the destination. The rest either build their own HTTP agent or speak a non-HTTP protocol.
268+
The server runs on Bun, and Bun's native `fetch` honors `$HTTP_PROXY`, `$HTTPS_PROXY`, and `$NO_PROXY`. Sim installs no global dispatcher. Default `fetch` uses the configured proxy unless `NO_PROXY` excludes the destination. The SSRF guard reads the same three variables, with the same rules, for every request it sends. The rest either build their own HTTP agent or speak a non-HTTP protocol.
269269
270270
This depends on the runtime. The published images run Bun. If you build the standalone output and run it under Node instead, Node ignores these variables unless started with `NODE_USE_ENV_PROXY=1` (Node 22.21+ / 24.5+), and the `fetch`-based "Yes" rows stop being proxied — model providers, Resend, Gmail sending, the desktop update feed, and the telemetry relay. The Azure Blob, GCS and Azure Communication Services rows still hold: those SDKs read the proxy variables through their own agents rather than through `fetch`. See [Bun fetch](https://bun.sh/docs/runtime/networking/fetch) and [Node proxy configuration](https://nodejs.org/learn/http/enterprise-network-configuration).
271271
@@ -275,20 +275,31 @@ This depends on the runtime. The published images run Bun. If you build the stan
275275
| Email via Resend, Azure Communication Services, and Gmail sending | Yes |
276276
| The desktop update feed's calls to GitHub | Yes |
277277
| Object storage — Azure Blob and GCS | Yes — their SDK pipelines read the proxy variables |
278-
| Everything through the SSRF guard — the HTTP block, tools, connectors, outbound webhooks, content fetches, MCP servers | No |
279-
| Azure OpenAI, Azure Anthropic, vLLM | Only when the endpoint comes from `AZURE_OPENAI_ENDPOINT`, `AZURE_ANTHROPIC_ENDPOINT`, or `VLLM_BASE_URL`. An endpoint typed into the block is validated and pinned to its resolved IP, which bypasses the proxy |
280-
| Amazon Bedrock, and object storage on S3 | No — the AWS SDK uses its own request handler |
281-
| Email via SMTP | No — Nodemailer opens a raw TCP connection |
282-
| Email via Amazon SES | No — the AWS SDK transport, over HTTPS |
283-
| OTLP export from the server SDK | No |
278+
| Everything through the SSRF guard — SSO OIDC discovery, the HTTP block, tools, connectors, outbound webhooks, content fetches, MCP servers, and model endpoints typed into a block | Yes — see [Proxying guarded requests](#proxying-guarded-requests) |
279+
| SSO sign-in — the token, JWKS, and userinfo calls made while a user signs in | Yes — they use the default `fetch` |
280+
| SSO domain verification (TXT lookup) | No — DNS, sent to `1.1.1.1` and `8.8.8.8` unless `DOMAIN_VERIFICATION_NAMESERVERS` names your resolvers |
281+
| Amazon Bedrock, object storage on S3, and every AWS integration (DynamoDB, SQS, Lambda, Textract, and the rest) | Yes — each AWS client gets a proxy-aware agent while a proxy is set. Bedrock then uses HTTP/1.1 instead of HTTP/2 |
282+
| Email via SMTP, and the SMTP block | No — SMTP is not HTTP. Use a mail relay the app can reach directly, or Resend or SES |
283+
| Email via Amazon SES | Yes — the AWS SDK client above |
284+
| OTLP export from the server SDK | Yes — the exporters get the same proxy-aware agent |
284285
| The `/api/telemetry` relay that forwards browser events | Yes — it uses the default `fetch` |
285286
| Postgres and Redis | No — raw TCP |
286287
287-
The practical consequence: a mandatory-egress-proxy environment can route most LLM traffic, Resend mail, and Azure/GCS storage through the proxy, but guarded integration calls, S3, Bedrock, SMTP, telemetry, and datastore traffic still need a transparent proxy or NAT-based egress.
288+
The practical consequence: a mandatory-egress-proxy environment can route LLM traffic, SSO, integration and workflow HTTP calls, Resend mail, Azure/GCS storage, AWS services, SES, and OTLP export through the proxy. SMTP and datastore traffic (Postgres, Redis, and the database blocks) still need a transparent proxy or NAT-based egress.
289+
290+
### Proxying guarded requests
291+
292+
A request through the SSRF guard keeps its checks when it is proxied. Before each request, and again for each redirect hop, the app resolves the destination's hostname with its own DNS and refuses it if any policy in [Security](/platform/self-hosting/security#the-ssrf-boundary) blocks the address. It then hands the hostname, not the address, to the proxy, so a proxy that allowlists by domain sees the name it expects.
293+
294+
- **DNS must still resolve inside the container.** A destination that the app cannot resolve is refused before the proxy is asked.
295+
- **The proxy resolves the name again.** The app cannot pin the address across a proxy, so the proxy's own egress policy is the last check.
296+
- **`https` destinations are tunneled** with `CONNECT host:443`. **`http` destinations are forwarded** to the proxy as an absolute URL, as curl and Bun do.
297+
- **Proxy credentials** go in the URL, e.g. `http://user:pass@proxy.internal:3128`. They are sent only to the proxy. An `https://` proxy URL also works; SOCKS proxies are not supported.
298+
- **The HTTP block's `proxyUrl`** still takes precedence for that one request.
288299
289300
### Internal destinations and `NO_PROXY`
290301
291-
Set `NO_PROXY` for every destination that is not on the public internet, not just model endpoints. The app reaches the realtime server (`SOCKET_SERVER_URL`), the Presidio PII service (`PII_URL`), and itself (`INTERNAL_API_BASE_URL`) over the same default `fetch`, alongside self-hosted Ollama, LiteLLM, and vLLM — so a proxy that cannot reach your internal network breaks live updates and PII redaction, not only inference.
302+
Set `NO_PROXY` for every destination that is not on the public internet, not just model endpoints. The app reaches the realtime server (`SOCKET_SERVER_URL`), the Presidio PII service (`PII_URL`), and itself (`INTERNAL_API_BASE_URL`) over the same default `fetch`, alongside self-hosted Ollama, LiteLLM, and vLLM — so a proxy that cannot reach your internal network breaks live updates and PII redaction, not only inference. AWS clients and OTLP export follow the same rule, so also add an internal S3-compatible store such as MinIO, AWS VPC endpoints, and an in-cluster OpenTelemetry collector.
292303
293304
Set it in the application environment, not your shell — under `app.env` on Helm, or the service's `environment:` on Compose. On Helm the suffixes alone are not enough: the chart wires `SOCKET_SERVER_URL`, `PII_URL`, and `OLLAMA_URL` to bare Service names, which no domain suffix matches. Add those names too — `helm template` prints the rendered ones, and the prefix is the release name unless it already contains `sim`, in which case it is the release name alone:
294305

‎apps/sim/instrumentation-node.ts‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -176,6 +176,8 @@ async function initializeOpenTelemetry() {
176176
const { ATTR_SERVICE_NAME, ATTR_SERVICE_VERSION, ATTR_DEPLOYMENT_ENVIRONMENT } = await import(
177177
'@opentelemetry/semantic-conventions/incubating'
178178
)
179+
const { otlpExporterProxyOptions } = await import('@/lib/core/network/proxy-agent.server')
180+
const exporterProxyOptions = otlpExporterProxyOptions()
179181
const { OTLPTraceExporter } = await import('@opentelemetry/exporter-trace-otlp-http')
180182
const { OTLPMetricExporter } = await import('@opentelemetry/exporter-metrics-otlp-http')
181183
const { PeriodicExportingMetricReader } = await import('@opentelemetry/sdk-metrics')
@@ -236,6 +238,7 @@ async function initializeOpenTelemetry() {
236238
headers: otlpHeaders,
237239
timeoutMillis: Math.min(telemetryConfig.batchSettings.exportTimeoutMillis, 10000),
238240
keepAlive: false,
241+
...exporterProxyOptions,
239242
})
240243

241244
// Surface export failures (BatchSpanProcessor swallows them otherwise).
@@ -273,6 +276,7 @@ async function initializeOpenTelemetry() {
273276
headers: otlpHeaders,
274277
timeoutMillis: Math.min(telemetryConfig.batchSettings.exportTimeoutMillis, 10000),
275278
keepAlive: false,
279+
...exporterProxyOptions,
276280
}),
277281
exportIntervalMillis: 60000,
278282
})
@@ -289,6 +293,7 @@ async function initializeOpenTelemetry() {
289293
headers: otlpHeaders,
290294
timeoutMillis: Math.min(telemetryConfig.batchSettings.exportTimeoutMillis, 10000),
291295
keepAlive: false,
296+
...exporterProxyOptions,
292297
}),
293298
})
294299

‎apps/sim/lib/auth/sso/domain-verification.ts‎

Lines changed: 22 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,10 @@
11
import { Resolver } from 'node:dns/promises'
2+
import { isIP } from 'node:net'
23
import { createLogger } from '@sim/logger'
34
import { getErrorMessage } from '@sim/utils/errors'
45
import { generateShortId } from '@sim/utils/id'
56
import type { OrganizationDomain } from '@/lib/api/contracts/organization'
7+
import { env } from '@/lib/core/config/env'
68

79
const logger = createLogger('SSODomainVerification')
810

@@ -54,7 +56,24 @@ const TXT_VALUE_PREFIX = 'sim-domain-verification='
5456

5557
/** Public nameservers used for the challenge lookup, so verification does not
5658
* depend on (or get poisoned by) the host's local resolver/split-horizon DNS. */
57-
const VERIFICATION_NAMESERVERS = ['1.1.1.1', '8.8.8.8']
59+
const DEFAULT_VERIFICATION_NAMESERVERS = ['1.1.1.1', '8.8.8.8']
60+
61+
/**
62+
* `DOMAIN_VERIFICATION_NAMESERVERS` replaces the public resolvers for deployments whose
63+
* outbound DNS only reaches an internal resolver. An entry that is not an IP address is
64+
* skipped rather than failing every verification.
65+
*/
66+
function resolveVerificationNameservers(): string[] {
67+
const configured = (env.DOMAIN_VERIFICATION_NAMESERVERS ?? '')
68+
.split(',')
69+
.map((entry) => entry.trim())
70+
.filter(Boolean)
71+
const usable = configured.filter((entry) => isIP(entry) !== 0)
72+
if (usable.length < configured.length) {
73+
logger.error('Ignoring DOMAIN_VERIFICATION_NAMESERVERS entries that are not IP addresses')
74+
}
75+
return usable.length > 0 ? usable : DEFAULT_VERIFICATION_NAMESERVERS
76+
}
5877

5978
/**
6079
* Per-attempt timeout. c-ares multiplies this across servers and retries by
@@ -74,12 +93,12 @@ const DNS_TIMEOUT_MS = 2000
7493
const RECORD_ABSENT_DNS_CODES = new Set(['ENODATA', 'ENOTFOUND', 'NXDOMAIN'])
7594

7695
/**
77-
* Shared resolver pinned to the public nameservers. Its config is fully static
96+
* Shared resolver pinned to the verification nameservers. Its config is fully static
7897
* and `resolveTxt` is safe to call concurrently, so a single module-scope
7998
* instance avoids re-allocating one per verification.
8099
*/
81100
const verificationResolver = new Resolver({ timeout: DNS_TIMEOUT_MS, tries: 1 })
82-
verificationResolver.setServers(VERIFICATION_NAMESERVERS)
101+
verificationResolver.setServers(resolveVerificationNameservers())
83102

84103
/** The fully-qualified host an org must create the TXT record on. */
85104
export function buildChallengeHost(domain: string): string {

‎apps/sim/lib/billing/aws-marketplace/client.ts‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ import { SQSClient } from '@aws-sdk/client-sqs'
44
import { fromHttp } from '@aws-sdk/credential-providers'
55
import { getAwsCredentials } from '@/lib/core/config/aws'
66
import { env } from '@/lib/core/config/env'
7+
import { awsRequestHandler } from '@/lib/core/network/proxy-agent.server'
78

89
/**
910
* The Entitlement Service exists only here. Metering is multi-region, but a registration token is
@@ -38,7 +39,7 @@ function marketplaceClientConfig() {
3839
const fullUri = env.AWS_CONTAINER_CREDENTIALS_FULL_URI
3940
return {
4041
region: MARKETPLACE_REGION,
41-
requestHandler: REQUEST_HANDLER,
42+
requestHandler: awsRequestHandler(REQUEST_HANDLER),
4243
credentials:
4344
relativeUri || fullUri
4445
? fromHttp({

‎apps/sim/lib/core/config/appconfig.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ import { toRecordOrNull } from '@sim/utils/object'
1010
import { LRUCache } from 'lru-cache'
1111
import { getAwsCredentials } from '@/lib/core/config/aws'
1212
import { env } from '@/lib/core/config/env'
13+
import { awsRequestHandler } from '@/lib/core/network/proxy-agent.server'
1314

1415
const logger = createLogger('AppConfig')
1516

@@ -122,6 +123,7 @@ function getClient(): AppConfigDataClient {
122123
client = new AppConfigDataClient({
123124
region: env.AWS_REGION,
124125
credentials: getAwsCredentials(),
126+
requestHandler: awsRequestHandler(),
125127
})
126128
}
127129
return client

‎apps/sim/lib/core/config/env.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -134,6 +134,7 @@ export const env = createEnv({
134134
ALLOW_PRIVATE_DATABASE_HOSTS: z.boolean().optional(), // Deprecated alias for the egress allowlist, kept so existing self-hosted deployments keep working. Equivalent to allowing every private, reserved, and loopback destination. Prefer EGRESS_ALLOWED_HOSTS / EGRESS_ALLOWED_IP_RANGES, which name specific destinations.
135135
EGRESS_ALLOWED_HOSTS: z.string().optional(), // Comma-separated hostnames outbound requests may reach on a private network, leading wildcard allowed (e.g. "host.docker.internal,*.svc.cluster.local"). Self-hosted only; ignored on the hosted platform. Replaces ALLOW_PRIVATE_DATABASE_HOSTS.
136136
EGRESS_ALLOWED_IP_RANGES: z.string().optional(), // Comma-separated CIDRs or IPs outbound requests may reach on a private network (e.g. "10.0.0.0/8,192.168.65.254/32"). Self-hosted only; never lifts the cloud-metadata block.
137+
DOMAIN_VERIFICATION_NAMESERVERS: z.string().optional(), // Comma-separated nameserver IPs for the domain-verification TXT lookup (e.g. "10.0.0.2"). Defaults to 1.1.1.1 and 8.8.8.8; set it where outbound DNS is restricted to an internal resolver.
137138
ALLOWED_LOGIN_EMAILS: z.string().optional(), // Comma-separated list of allowed email addresses for login
138139
ALLOWED_LOGIN_DOMAINS: z.string().optional(), // Comma-separated list of allowed email domains for login
139140
BLOCKED_SIGNUP_DOMAINS: z.string().optional(), // Comma-separated list of email domains blocked from signing up (e.g., "gmail.com,yahoo.com")

0 commit comments

Comments
 (0)