Skip to content

Commit f19a3d8

Browse files
committed
feat(network): route AWS SDK clients and OTLP exporters through the environment proxy
1 parent b313325 commit f19a3d8

37 files changed

Lines changed: 314 additions & 99 deletions

File tree

‎apps/docs/content/docs/platform/self-hosting/networking.mdx‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -278,14 +278,14 @@ This depends on the runtime. The published images run Bun. If you build the stan
278278
| Everything through the SSRF guard — SSO OIDC discovery, the HTTP block, tools, connectors, outbound webhooks, content fetches, MCP servers, and model endpoints typed into a block | Yes — see [Proxying guarded requests](#proxying-guarded-requests) |
279279
| SSO sign-in — the token, JWKS, and userinfo calls made while a user signs in | Yes — they use the default `fetch` |
280280
| SSO domain verification (TXT lookup) | No — DNS, sent to `1.1.1.1` and `8.8.8.8` unless `DOMAIN_VERIFICATION_NAMESERVERS` names your resolvers |
281-
| Amazon Bedrock, and object storage on S3 | No — the AWS SDK uses its own request handler |
282-
| Email via SMTP | No — Nodemailer opens a raw TCP connection |
283-
| Email via Amazon SES | No — the AWS SDK transport, over HTTPS |
284-
| OTLP export from the server SDK | No |
281+
| Amazon Bedrock, object storage on S3, and every AWS integration (DynamoDB, SQS, Lambda, Textract, and the rest) | Yes — each AWS client gets a proxy-aware agent while a proxy is set. Bedrock then uses HTTP/1.1 instead of HTTP/2 |
282+
| Email via SMTP, and the SMTP block | No — SMTP is not HTTP. Use a mail relay the app can reach directly, or Resend or SES |
283+
| Email via Amazon SES | Yes — the AWS SDK client above |
284+
| OTLP export from the server SDK | Yes — the exporters get the same proxy-aware agent |
285285
| The `/api/telemetry` relay that forwards browser events | Yes — it uses the default `fetch` |
286286
| Postgres and Redis | No — raw TCP |
287287
288-
The practical consequence: a mandatory-egress-proxy environment can route LLM traffic, SSO, integration and workflow HTTP calls, Resend mail, and Azure/GCS storage through the proxy, but S3, Bedrock, SMTP, SES, OTLP export, and datastore traffic still need a transparent proxy or NAT-based egress.
288+
The practical consequence: a mandatory-egress-proxy environment can route LLM traffic, SSO, integration and workflow HTTP calls, Resend mail, Azure/GCS storage, AWS services, SES, and OTLP export through the proxy. SMTP and datastore traffic (Postgres, Redis, and the database blocks) still need a transparent proxy or NAT-based egress.
289289
290290
### Proxying guarded requests
291291
@@ -299,7 +299,7 @@ A request through the SSRF guard keeps its checks when it is proxied. Before eac
299299
300300
### Internal destinations and `NO_PROXY`
301301
302-
Set `NO_PROXY` for every destination that is not on the public internet, not just model endpoints. The app reaches the realtime server (`SOCKET_SERVER_URL`), the Presidio PII service (`PII_URL`), and itself (`INTERNAL_API_BASE_URL`) over the same default `fetch`, alongside self-hosted Ollama, LiteLLM, and vLLM — so a proxy that cannot reach your internal network breaks live updates and PII redaction, not only inference.
302+
Set `NO_PROXY` for every destination that is not on the public internet, not just model endpoints. The app reaches the realtime server (`SOCKET_SERVER_URL`), the Presidio PII service (`PII_URL`), and itself (`INTERNAL_API_BASE_URL`) over the same default `fetch`, alongside self-hosted Ollama, LiteLLM, and vLLM — so a proxy that cannot reach your internal network breaks live updates and PII redaction, not only inference. AWS clients and OTLP export follow the same rule, so also add an internal S3-compatible store such as MinIO, AWS VPC endpoints, and an in-cluster OpenTelemetry collector.
303303
304304
Set it in the application environment, not your shell — under `app.env` on Helm, or the service's `environment:` on Compose. On Helm the suffixes alone are not enough: the chart wires `SOCKET_SERVER_URL`, `PII_URL`, and `OLLAMA_URL` to bare Service names, which no domain suffix matches. Add those names too — `helm template` prints the rendered ones, and the prefix is the release name unless it already contains `sim`, in which case it is the release name alone:
305305

‎apps/sim/instrumentation-node.ts‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -176,6 +176,8 @@ async function initializeOpenTelemetry() {
176176
const { ATTR_SERVICE_NAME, ATTR_SERVICE_VERSION, ATTR_DEPLOYMENT_ENVIRONMENT } = await import(
177177
'@opentelemetry/semantic-conventions/incubating'
178178
)
179+
const { otlpExporterProxyOptions } = await import('./lib/core/network/proxy-agent.server')
180+
const exporterProxyOptions = otlpExporterProxyOptions()
179181
const { OTLPTraceExporter } = await import('@opentelemetry/exporter-trace-otlp-http')
180182
const { OTLPMetricExporter } = await import('@opentelemetry/exporter-metrics-otlp-http')
181183
const { PeriodicExportingMetricReader } = await import('@opentelemetry/sdk-metrics')
@@ -236,6 +238,7 @@ async function initializeOpenTelemetry() {
236238
headers: otlpHeaders,
237239
timeoutMillis: Math.min(telemetryConfig.batchSettings.exportTimeoutMillis, 10000),
238240
keepAlive: false,
241+
...exporterProxyOptions,
239242
})
240243

241244
// Surface export failures (BatchSpanProcessor swallows them otherwise).
@@ -273,6 +276,7 @@ async function initializeOpenTelemetry() {
273276
headers: otlpHeaders,
274277
timeoutMillis: Math.min(telemetryConfig.batchSettings.exportTimeoutMillis, 10000),
275278
keepAlive: false,
279+
...exporterProxyOptions,
276280
}),
277281
exportIntervalMillis: 60000,
278282
})
@@ -289,6 +293,7 @@ async function initializeOpenTelemetry() {
289293
headers: otlpHeaders,
290294
timeoutMillis: Math.min(telemetryConfig.batchSettings.exportTimeoutMillis, 10000),
291295
keepAlive: false,
296+
...exporterProxyOptions,
292297
}),
293298
})
294299

‎apps/sim/lib/billing/aws-marketplace/client.ts‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@ import { SQSClient } from '@aws-sdk/client-sqs'
44
import { fromHttp } from '@aws-sdk/credential-providers'
55
import { getAwsCredentials } from '@/lib/core/config/aws'
66
import { env } from '@/lib/core/config/env'
7+
import { awsRequestHandler } from '@/lib/core/network/proxy-agent.server'
78

89
/**
910
* The Entitlement Service exists only here. Metering is multi-region, but a registration token is
@@ -38,7 +39,7 @@ function marketplaceClientConfig() {
3839
const fullUri = env.AWS_CONTAINER_CREDENTIALS_FULL_URI
3940
return {
4041
region: MARKETPLACE_REGION,
41-
requestHandler: REQUEST_HANDLER,
42+
requestHandler: awsRequestHandler(REQUEST_HANDLER),
4243
credentials:
4344
relativeUri || fullUri
4445
? fromHttp({

‎apps/sim/lib/core/config/appconfig.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ import { toRecordOrNull } from '@sim/utils/object'
1010
import { LRUCache } from 'lru-cache'
1111
import { getAwsCredentials } from '@/lib/core/config/aws'
1212
import { env } from '@/lib/core/config/env'
13+
import { awsRequestHandler } from '@/lib/core/network/proxy-agent.server'
1314

1415
const logger = createLogger('AppConfig')
1516

@@ -122,6 +123,7 @@ function getClient(): AppConfigDataClient {
122123
client = new AppConfigDataClient({
123124
region: env.AWS_REGION,
124125
credentials: getAwsCredentials(),
126+
requestHandler: awsRequestHandler(),
125127
})
126128
}
127129
return client

‎apps/sim/lib/core/network/environment-proxy.server.test.ts‎

Lines changed: 97 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -9,8 +9,10 @@ import type { Duplex } from 'node:stream'
99
import { getCACertificates, setDefaultCACertificates } from 'node:tls'
1010
import { fileURLToPath } from 'node:url'
1111
import { promisify } from 'node:util'
12+
import { ListBucketsCommand, S3Client } from '@aws-sdk/client-s3'
1213
import { networkConfigMock } from '@sim/testing/mocks/network-config.mock'
13-
import { afterAll, afterEach, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest'
14+
import { NodeHttpHandler } from '@smithy/node-http-handler'
15+
import { afterAll, beforeAll, beforeEach, describe, expect, it, vi } from 'vitest'
1416

1517
vi.mock('@/lib/core/network/config.server', () => networkConfigMock)
1618
vi.mock('@sim/security/dns', async (importOriginal) => {
@@ -29,6 +31,8 @@ vi.mock('@sim/security/dns', async (importOriginal) => {
2931
}
3032
})
3133

34+
import { resolveEnvironmentProxy } from '@/lib/core/network/environment-proxy.server'
35+
import { awsRequestHandler } from '@/lib/core/network/proxy-agent.server'
3236
import {
3337
createPinnedFetchWithDispatcher,
3438
createSsrfGuardedFetchWithDispatcher,
@@ -53,6 +57,11 @@ const secureOrigin = httpsServer((req, res) => {
5357
res.end()
5458
return
5559
}
60+
if (String(req.headers.authorization).startsWith('AWS4-HMAC-SHA256')) {
61+
res.setHeader('content-type', 'application/xml')
62+
res.end('<ListAllMyBucketsResult><Buckets></Buckets></ListAllMyBucketsResult>')
63+
return
64+
}
5665
res.end('tls reached')
5766
})
5867

@@ -133,10 +142,6 @@ beforeEach(() => {
133142
originHeaders = {}
134143
})
135144

136-
afterEach(() => {
137-
vi.unstubAllEnvs()
138-
})
139-
140145
afterAll(async () => {
141146
for (const socket of sockets) socket.destroy()
142147
await Promise.all(
@@ -244,7 +249,7 @@ describe('guarded requests under HTTPS_PROXY', () => {
244249
])
245250
})
246251

247-
it('tunnels through the proxy under the actual Bun runtime', async () => {
252+
it('tunnels guarded fetches and AWS SDK requests under the actual Bun runtime', async () => {
248253
await promisify(execFile)(
249254
'bun',
250255
[
@@ -266,6 +271,91 @@ describe('guarded requests under HTTPS_PROXY', () => {
266271
},
267272
}
268273
)
269-
expect(tunnels).toEqual([{ authority: '1.1.1.1:443', authorization: PROXY_AUTHORIZATION }])
274+
expect(tunnels).toEqual([
275+
{ authority: '1.1.1.1:443', authorization: PROXY_AUTHORIZATION },
276+
{ authority: '1.1.1.1:443', authorization: PROXY_AUTHORIZATION },
277+
])
278+
})
279+
})
280+
281+
describe('proxy selection from the environment', () => {
282+
it.each([
283+
{
284+
case: 'an empty lowercase variable does not hide the uppercase one',
285+
env: { https_proxy: '', HTTPS_PROXY: 'http://upper:3128' },
286+
target: 'https://api.example.com/',
287+
proxy: 'http://upper:3128/',
288+
},
289+
{
290+
case: 'an https destination falls back to HTTP_PROXY',
291+
env: { HTTP_PROXY: 'http://plain:3128' },
292+
target: 'https://api.example.com/',
293+
proxy: 'http://plain:3128/',
294+
},
295+
{
296+
case: 'a bare host:port is read as an http proxy',
297+
env: { HTTPS_PROXY: 'proxy.internal:3128' },
298+
target: 'https://api.example.com/',
299+
proxy: 'http://proxy.internal:3128/',
300+
},
301+
{
302+
case: 'a NO_PROXY entry with a port only excludes that port',
303+
env: { HTTPS_PROXY: 'http://p:3128', NO_PROXY: 'api.example.com:8443' },
304+
target: 'https://api.example.com/',
305+
proxy: 'http://p:3128/',
306+
},
307+
{
308+
case: 'NO_PROXY=* excludes every destination',
309+
env: { HTTPS_PROXY: 'http://p:3128', NO_PROXY: '*' },
310+
target: 'https://api.example.com/',
311+
proxy: null,
312+
},
313+
])('$case', ({ env, target, proxy }) => {
314+
for (const name of PROXY_ENV) vi.stubEnv(name, '')
315+
for (const [name, value] of Object.entries(env)) vi.stubEnv(name, value)
316+
expect(resolveEnvironmentProxy(new URL(target))).toBe(proxy)
317+
})
318+
319+
it('fails closed instead of going direct when the proxy URL is unsupported', async () => {
320+
vi.stubEnv('HTTPS_PROXY', 'socks5://127.0.0.1:1080')
321+
await expect(secureFetchWithValidation('https://origin.invalid/', options)).rejects.toThrow(
322+
/http:\/\/ or https:\/\//
323+
)
324+
expect(tunnels).toEqual([])
325+
expect(originHeaders).toEqual({})
326+
})
327+
})
328+
329+
describe('AWS SDK clients under HTTPS_PROXY', () => {
330+
const listBuckets = (endpoint: string) =>
331+
new S3Client({
332+
region: 'us-east-1',
333+
endpoint,
334+
forcePathStyle: true,
335+
credentials: { accessKeyId: 'AKIDEXAMPLE', secretAccessKey: 'secret' },
336+
requestHandler: awsRequestHandler(),
337+
}).send(new ListBucketsCommand({}))
338+
339+
it('tunnels SDK requests by hostname and keeps proxy credentials off the origin', async () => {
340+
await expect(listBuckets('https://origin.invalid')).resolves.toMatchObject({ Buckets: [] })
341+
expect(tunnels).toEqual([
342+
{ authority: 'origin.invalid:443', authorization: PROXY_AUTHORIZATION },
343+
])
344+
expect(originHeaders['proxy-authorization']).toBeUndefined()
345+
})
346+
347+
it('connects directly when NO_PROXY names the endpoint', async () => {
348+
vi.stubEnv('NO_PROXY', '127.0.0.1')
349+
await expect(listBuckets(`https://127.0.0.1:${securePort}`)).resolves.toMatchObject({
350+
Buckets: [],
351+
})
352+
expect(tunnels).toEqual([])
353+
})
354+
355+
it('leaves the SDK defaults untouched when no proxy is configured', () => {
356+
vi.stubEnv('HTTPS_PROXY', '')
357+
vi.stubEnv('HTTP_PROXY', '')
358+
expect(awsRequestHandler({ requestTimeout: 10_000 })).not.toBeInstanceOf(NodeHttpHandler)
359+
expect(awsRequestHandler()).toBeUndefined()
270360
})
271361
})

‎apps/sim/lib/core/network/environment-proxy.server.ts‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,15 @@
11
const DEFAULT_PORTS: Record<string, number> = { 'http:': 80, 'https:': 443 }
2-
const SUPPORTED_PROXY_PROTOCOLS = new Set(['http:', 'https:'])
32

43
/** An empty lowercase variable counts as unset, so it cannot hide the uppercase one. */
54
function readEnv(lower: string, upper: string): string | undefined {
65
return process.env[lower]?.trim() || process.env[upper]?.trim() || undefined
76
}
87

8+
/** Whether any `HTTPS_PROXY` / `HTTP_PROXY` is set, before any destination is considered. */
9+
export function isEnvironmentProxyConfigured(): boolean {
10+
return Boolean(readEnv('https_proxy', 'HTTPS_PROXY') ?? readEnv('http_proxy', 'HTTP_PROXY'))
11+
}
12+
913
/** Matches `NO_PROXY` the way undici's `EnvHttpProxyAgent` does, so both stacks agree. */
1014
function isExcludedByNoProxy(hostname: string, port: number): boolean {
1115
const noProxy = readEnv('no_proxy', 'NO_PROXY')
@@ -35,9 +39,7 @@ export function resolveEnvironmentProxy(target: URL): string | null {
3539
const proxy =
3640
target.protocol === 'https:' ? (readEnv('https_proxy', 'HTTPS_PROXY') ?? httpProxy) : httpProxy
3741
if (!proxy) return null
38-
const hostname = target.host.replace(/:\d*$/, '').toLowerCase()
39-
const port = Number.parseInt(target.port, 10) || defaultPort
40-
if (isExcludedByNoProxy(hostname, port)) return null
42+
if (isExcludedByNoProxy(target.hostname, Number(target.port) || defaultPort)) return null
4143
return normalizeProxyUrl(proxy)
4244
}
4345

@@ -49,7 +51,7 @@ function normalizeProxyUrl(value: string): string {
4951
} catch {
5052
throw new Error('The configured HTTP(S)_PROXY is not a valid URL')
5153
}
52-
if (!SUPPORTED_PROXY_PROTOCOLS.has(parsed.protocol)) {
54+
if (DEFAULT_PORTS[parsed.protocol] === undefined) {
5355
throw new Error('The configured HTTP(S)_PROXY must be an http:// or https:// URL')
5456
}
5557
return parsed.href

‎apps/sim/lib/core/network/fixtures/environment-proxy-runtime.fixture.ts‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,6 @@
11
/** Real runtime probe invoked by environment-proxy.server.test.ts against its local proxy. */
2+
import { ListBucketsCommand, S3Client } from '@aws-sdk/client-s3'
3+
import { awsRequestHandler } from '@/lib/core/network/proxy-agent.server'
24
import { secureFetchWithValidation } from '@/lib/core/security/input-validation.server'
35

46
const response = await secureFetchWithValidation(
@@ -7,3 +9,13 @@ const response = await secureFetchWithValidation(
79
)
810
if (response.headers.get('x-via-proxy') !== 'yes') throw new Error('Guarded fetch bypassed proxy')
911
if ((await response.text()) !== 'tls reached') throw new Error('Guarded fetch response mismatch')
12+
13+
const buckets = await new S3Client({
14+
region: 'us-east-1',
15+
endpoint: 'https://1.1.1.1',
16+
forcePathStyle: true,
17+
credentials: { accessKeyId: 'AKIDEXAMPLE', secretAccessKey: 'secret' },
18+
requestHandler: awsRequestHandler(),
19+
}).send(new ListBucketsCommand({}))
20+
if (!Array.isArray(buckets.Buckets)) throw new Error('AWS SDK response mismatch')
21+
process.exit(0)
Lines changed: 82 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,82 @@
1+
import http from 'node:http'
2+
import https from 'node:https'
3+
import { isIP } from 'node:net'
4+
import { NodeHttpHandler } from '@smithy/node-http-handler'
5+
import { Agent, type AgentConnectOpts } from 'agent-base'
6+
import { HttpProxyAgent } from 'http-proxy-agent'
7+
import { HttpsProxyAgent } from 'https-proxy-agent'
8+
import {
9+
isEnvironmentProxyConfigured,
10+
resolveEnvironmentProxy,
11+
} from '@/lib/core/network/environment-proxy.server'
12+
13+
/** Matches the AWS SDK's own default agent, so swapping it in changes routing and nothing else. */
14+
const AGENT_OPTIONS: http.AgentOptions = { keepAlive: true, maxSockets: 50 }
15+
16+
/**
17+
* A Node agent for SDKs that bring their own HTTP stack (the AWS SDK, the OTLP exporters) and so
18+
* bypass Bun's proxy-aware `fetch`. Each connection takes the `HTTPS_PROXY` / `HTTP_PROXY` that
19+
* applies to its destination, or a direct keep-alive connection when `NO_PROXY` excludes it.
20+
*/
21+
class EnvironmentProxyAgent extends Agent {
22+
private readonly direct = {
23+
http: new http.Agent(AGENT_OPTIONS),
24+
https: new https.Agent(AGENT_OPTIONS),
25+
}
26+
private readonly proxies = new Map<string, http.Agent>()
27+
28+
connect(_req: http.ClientRequest, options: AgentConnectOpts): http.Agent {
29+
const secure = options.secureEndpoint
30+
const host = options.host ?? 'localhost'
31+
const authority = isIP(host) === 6 ? `[${host}]` : host
32+
const target = new URL(`${secure ? 'https' : 'http'}://${authority}:${options.port}`)
33+
const proxyUrl = resolveEnvironmentProxy(target)
34+
if (!proxyUrl) return secure ? this.direct.https : this.direct.http
35+
const key = `${target.protocol}${proxyUrl}`
36+
let agent = this.proxies.get(key)
37+
if (!agent) {
38+
agent = secure
39+
? new HttpsProxyAgent(proxyUrl, AGENT_OPTIONS)
40+
: new HttpProxyAgent(proxyUrl, AGENT_OPTIONS)
41+
this.proxies.set(key, agent)
42+
}
43+
return agent
44+
}
45+
}
46+
47+
let sharedAgent: EnvironmentProxyAgent | null = null
48+
49+
/** The process-wide {@link EnvironmentProxyAgent}; only meaningful while a proxy is configured. */
50+
function environmentProxyAgent(): EnvironmentProxyAgent {
51+
sharedAgent ??= new EnvironmentProxyAgent(AGENT_OPTIONS)
52+
return sharedAgent
53+
}
54+
55+
interface AwsRequestHandlerOptions {
56+
connectionTimeout?: number
57+
requestTimeout?: number
58+
throwOnRequestTimeout?: boolean
59+
}
60+
61+
/**
62+
* `requestHandler` for an AWS SDK v3 client. The SDK's Node handlers ignore `HTTPS_PROXY`, so while
63+
* a proxy is configured this returns an HTTP/1.1 handler on {@link environmentProxyAgent} — which
64+
* also replaces the HTTP/2 handler some clients default to, since HTTP/2 cannot use an agent.
65+
* Without a proxy it returns `options` unchanged and the SDK keeps its own defaults.
66+
*/
67+
export function awsRequestHandler(
68+
options?: AwsRequestHandlerOptions
69+
): NodeHttpHandler | AwsRequestHandlerOptions | undefined {
70+
if (!isEnvironmentProxyConfigured()) return options
71+
const agent = environmentProxyAgent()
72+
return new NodeHttpHandler({ ...options, httpAgent: agent, httpsAgent: agent })
73+
}
74+
75+
/**
76+
* Agent settings for an OTLP HTTP exporter, which sends through `node:http` and so ignores
77+
* `HTTPS_PROXY`. Empty without a proxy, leaving the exporter's own agent in place.
78+
*/
79+
export function otlpExporterProxyOptions(): { httpAgentOptions?: () => http.Agent } {
80+
if (!isEnvironmentProxyConfigured()) return {}
81+
return { httpAgentOptions: environmentProxyAgent }
82+
}

0 commit comments

Comments
 (0)