Repository navigation
feat(netproxy): HTTP CONNECT proxy dialer + registry client WithDialer - #50
Merged
Merged
Conversation
Add package netproxy so Pilot components can reach their :443 endpoints
from hosts whose only egress is an authenticating HTTP proxy that allows
CONNECT host:443 (hosted agent sandboxes: UDP blocked, direct TCP killed,
local DNS for the Pilot hostnames poisoned).
- Resolver picks the proxy per target: Explicit(url), FromEnvironment()
(HTTPS_PROXY/https_proxy, fallback ALL_PROXY/all_proxy; HTTP_PROXY for
plain http:// requests; NO_PROXY/no_proxy; loopback never proxied) or
Off(). Parse("auto"|"off"|URL) maps a -proxy style setting. The same
decision is exposed as ProxyForAddr (raw TCP) and ProxyForRequest
(http.Transport.Proxy), so TCP dials and HTTP clients agree.
- Dialer.DialContext tunnels through the chosen proxy (plain TCP for
http://, TLS to the proxy for https://), sends CONNECT host:port with
Host and Basic Proxy-Authorization, parses the reply with
http.ReadResponse, keeps bytes read ahead of the tunnel, honours ctx
cancellation/deadline plus a dial timeout, and closes the conn on every
error path. The target is never resolved locally.
- Credentials never reach errors or logs: non-2xx replies become
ConnectError ("proxy CONNECT host:port: 407 Proxy Authentication
Required", status text from net/http, never the proxy's reason phrase);
Redact and Resolver.String render http://***@host:port.
- NO_PROXY matching follows x/net/http/httpproxy, reimplemented to keep
the module dependency-free.
Tests run an in-process CONNECT proxy that only knows made-up .invalid
host names: success, Basic auth header, 407 with credential-echoing
reason phrase, NO_PROXY bypass, https:// proxy, ctx cancel/deadline and
Timeout, read-ahead bytes preserved, hostname-only CONNECT, and an
http.Transport driven by ProxyForRequest.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Add DialOption / WithDialer(DialContextFunc) as a trailing variadic option on Dial, DialPool, DialTLS, DialTLSPool, DialTLSPinned and DialBinary, plus DialTLSPinnedPool (already referenced by the DialTLSPool error message). When set, the dialer opens every connection: the primary, each pool conn, Client.reconnect, reconnectEntry and BinaryClient.reconnect. The registry address reaches the dialer unresolved, so a netproxy.Dialer asks the proxy to CONNECT by host name. For TLS the handshake runs on top of the dialer's conn via tls.Client; ServerName defaults to the address host (as crypto/tls.Dial does) on a clone, so SNI, CA verification and pinned-fingerprint verification are unchanged, and the caller's tls.Config is never mutated. The dial and handshake are bounded by the existing 5s dialTimeout. Without WithDialer every path performs exactly the direct dials it did before (same timeouts and ctx handling); existing call sites compile unchanged. Tests reach a TLS registry stand-in only through a CONNECT proxy under registry.pilot.invalid (never resolvable locally): DialTLS (SNI and CA check), untrusted cert, DialTLSPinned (match and mismatch), DialTLSPool / DialTLSPinnedPool with concurrent sends, reconnect on the primary, pinned and pool paths, plain Dial / DialPool, DialBinary with reconnect, and the WithDialer contract (unresolved addr, bounded ctx, error propagation, conn closed on handshake failure, nil keeps direct). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
FromEnvironment parsed every proxy variable and failed as a whole when any of them was unusable, so an http_proxy Go cannot use (socks5h://, stray whitespace) or a socks ALL_PROXY threw away a valid HTTPS_PROXY. Callers then fell back to direct dials, which a proxy-only sandbox (Meta Muse) kills. Each variable is now parsed on its own: an unusable HTTP_PROXY, http_proxy, ALL_PROXY or all_proxy is skipped (the next variable in its list applies) and reported per variable by the new Resolver.Warnings as *EnvError; String() names the skipped variables. Only an unusable HTTPS_PROXY / https_proxy, which explicitly names the TLS proxy, is still an error (an *EnvError naming it). parseProxyURL used url.Parse, which ends the authority at the first '/', '?' or '#'. Credentials holding those characters unescaped (base64 tokens) had part of the secret parsed as the proxy host, which then appeared in String()/Redact, in dial errors, and in a DNS lookup. The userinfo is now split off at the LAST '@' and percent-decoded by hand; only the part after it goes through url.Parse. Unusable values are rejected with errors that never echo credentials (including a "scheme" that may be a user name). Redact also withholds everything before the last '@' when handed a URL that url.Parse misparsed this way. Tests cover both findings at the resolver level and end to end through the in-process CONNECT proxy (Proxy-Authorization, dialed addresses, errors). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
The required govulncheck check has failed on every PR since mid-August: go1.25.12's standard library carries GO-2026-6218 (net/url), GO-2026-6090 (crypto/tls), GO-2026-5972 (encoding/asn1) and GO-2026-5026 (net/http idna), all fixed in go1.25.13 and reachable from code already on main (registry/wire, decision, driver). CI takes its Go version from go.mod. pilotprotocol already requires go 1.25.13, so this adds no new requirement for it. With 1.25.13: govulncheck reports no vulnerabilities. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two additions so pilot components can run where the only way out is an authenticating HTTP CONNECT proxy:
netproxy: proxy resolution plus an HTTP CONNECT dialer.registry/clientWithDialeroption: every registry connection (primary, pool members, reconnects,BinaryClient) can be opened through a caller-supplied dial function, for example anetproxy.Dialer.Why
In Meta Muse agent VMs the only egress is an authenticating
HTTPS_PROXYthat allowsCONNECT host:443and nothing else. UDP is blocked, direct TCP is killed, and DNS for*.pilotprotocol.networkis poisoned to198.18.x.x. The released v1.13.9 daemon installs fine there, but it never registers: it dials the registry directly. This PR gives web4 the pieces to send the registry (TLS over TCP) and HTTP/WSS traffic through the proxy. The target is kept as a host name all the way to the proxy and is never resolved locally, and TLS stays end-to-end with the real server.API
Behaviour notes:
Proxy-Authorization. Errors,String(),RedactandConnectErrornever include it, andConnectErroralso leaves out the proxy's reason phrase. The userinfo is split at the last@, so tokens with unescaped/,?,#or@(for example base64) decode intact. They are never parsed as the proxy host, so they never reach logs, errors or DNS.HTTP_PROXY,http_proxy,ALL_PROXYorall_proxy(malformed, orsocks5://) is skipped and reported throughWarnings(). It never disables a validHTTPS_PROXY. Only an unusableHTTPS_PROXY/https_proxyis an error.x/net/http/httpproxy. With noHTTP_PROXY, plainhttp://andws://requests use the TLS proxy, because direct egress is killed in the target sandbox. Explicit mode proxies everything.ServerNamedefaulting to the address host. SNI, CA verification and pinned-fingerprint checks are unchanged, and the caller'stls.Configis never mutated. The existing 5s per-attemptdialTimeoutnow covers proxy connect + CONNECT + TLS handshake.Also:
go 1.25.13The required
govulncheckcheck has failed on every PR since mid-August. The go1.25.12 standard library has GO-2026-6218 (net/url), GO-2026-6090 (crypto/tls), GO-2026-5972 (encoding/asn1) and GO-2026-5026 (net/http idna). All are fixed in go1.25.13 and are reachable from code already on main. This PR bumps thegodirective to 1.25.13, which pilotprotocol already requires.govulncheck: no vulnerabilities.Tests
netproxy: an in-process CONNECT proxy maps.invalidhost names to loopback, which proves targets are sent by name and never resolved locally. Coverage includes Basic auth, 407/403/5xx, an unreachable proxy, https:// proxies, context deadline/cancel, read-ahead bytes after the 200, NO_PROXY and loopback bypass, and anhttp.Transportdriven byProxyForRequest. Every failure path asserts that no credential appears.TestFromEnvVariablesAreIndependentandTestFromEnvUnusableTLSProxyIsAnError: a valid HTTPS_PROXY next to a socks or garbageHTTP_PROXY/http_proxystill proxiesregistry.pilotprotocol.network:443. A socksALL_PROXYalone is only a warning.TestUnescapedDelimitersInUserinfo,TestUnusableUserinfoNeverLeaks,TestRedactMisparsedURL: credentials with/,?,#or@are handled.TestDialerUnescapedCredentialsBesideUnusableHTTPProxy: end to end. Checks the Proxy-Authorization the proxy receives, that the only address dialed is the real proxy (so no DNS lookup of a secret), and that the error text is clean.registry/client: all constructors and reconnect paths go throughWithDialer. This covers pinned TLS through a CONNECT proxy, SNI and CA verification over the tunnel, and that thetls.Configis not mutated.GOWORK=off go test -race ./... -count=1,go vet ./...,govulncheck@v1.1.4 ./...(go1.25.13): pass.🤖 Generated with Claude Code