Feat: Transparent inbound interception for the Go proxy (proxy-sidecar) — SO_ORIGINAL_DST, no Envoy

[huang195]

Summary

Implement a transparent inbound listener for the Go proxy (proxy-sidecar mode): capture iptables-REDIRECTed inbound connections, recover the app's original destination port via SO_ORIGINAL_DST, run the inbound auth pipeline (JWT validation) inline, and forward to the app on its real port over loopback — no Envoy, no ext_proc, and no port stealing.

This is the inbound half of the original "transparent-proxy mode" idea. Since this issue was filed, the outbound transparent path has landed (see Status below); the remaining gap is inbound.

Revised 2026-08-17 after grounding every claim against the code. Three claims in the original body were wrong or stale and have been corrected — most importantly the "requires Service rewiring" gap, which described a mechanism the operator does not use. Corrections are called out inline.

Status of the original proposal (what already landed)

The Go-native transparent-interception core was built and shipped — but as the outbound enforce-redirect egress guard inside proxy-sidecar mode, not as a standalone Envoy-replacement mode:

• SO_ORIGINAL_DST getsockopt recovery, IPv4 + IPv6 — authlib/listener/transparentproxy/origdst_linux.go:54,60
• TLS ClientHello passthrough + SNI/Host recovery — authlib/listener/forwardproxy/sniff.go, transparent.go:74-84
• Shares the forward proxy's CONNECT auth pipeline via HandleTransparentConn (transparent.go:58; wired at cmd/authbridge-proxy/main.go:605)
• Landed in bb15d20 (+ follow-ups); bound to :8082 (authlib/config/presets.go:29); outbound only

The broader "drop the ~87 MB Envoy binary to shrink the image" motivation is also already met: proxy-sidecar is now the default mode and bundles no Envoy (cmd/authbridge-proxy/Dockerfile), and the authbridge-lite image variant shrinks it further.

What's still missing: there is no transparent inbound listener. Valid modes are envoy-sidecar, waypoint, proxy-sidecar (authlib/config/config.go:493-497, validated in validate.go:19-26) — there is no standalone transparent-proxy mode. proxy-init has a PREROUTING REDIRECT chain (PROXY_INBOUND, init-iptables.sh:565,602) but MODE=enforce-redirect calls setup_enforce_redirect and then exit 0 at init-iptables.sh:426, so it never reaches the inbound block. enforce-redirect installs only nat OUTPUT (AB_REDIRECT) and mangle OUTPUT (AB_NOTCP).

Current inbound mechanism and its limits

proxy-sidecar inbound today is a reverse proxy to a single fixed backend — httputil.NewSingleHostReverseProxy against reverse_proxy_backend (authlib/listener/reverseproxy/server.go:95, authlib/config/config.go:373). Traffic hits the listener, the inbound pipeline validates the JWT, and it forwards to one known app URL.

The operator wires this up by port stealing, not by patching the Service. From internal/webhook/injector/pod_mutator.go:499-501:

"Port-stealing: the reverse proxy takes over the agent's original port so the Service doesn't need patching. The agent is moved to a free port."
Service → :8000 → reverse proxy (validates JWT) → :8002 → agent

Concretely (pod_mutator.go:530-613): AuthBridge binds the app's original port, the app's Ports[0].ContainerPort is rewritten to originalPort+1, PORT is set to match, and reverse_proxy_backend becomes http://127.0.0.1:<newPort>.

That works for a single-port, PORT-honoring app, but it has four gaps:

1. Bypassable. Validation is enforced only for traffic that reaches AuthBridge's listener. Anything that reaches the app's relocated port directly — another pod dialing podIP:<newPort>, a co-located sidecar, a mis-scoped NetworkPolicy — skips validation entirely. The relocated port is declared in the pod spec, so it is discoverable. This is the inbound twin of the problem the outbound enforce-redirect guard already solves.

   Note: no NetworkPolicy ships for operator-managed agent workloads (the only ones in-tree are under deployments/openshell/ and deployments/sandbox/), so this bypass is unmitigated by default.

2. Depends on the app honoring PORT. pod_mutator.go:580-583 concedes it: "Go agents that hardcode their listen port won't be affected by this env var." Such an app keeps binding the original port and collides with AuthBridge's reverse proxy — a hard bind failure, and one that no amount of config can fix from the platform side. Transparent inbound removes the relocation entirely, so this failure mode disappears.

3. Single port only. Only Ports[0] of the first container that declares ports is relocated (pod_mutator.go:530-542); a second declared port stays put and is never proxied. Undeclared ports are worse — they aren't in the usedPorts map at all (:522-527), so findFreePort can hand the app a port it is already listening on.

4. Requires Service rewiring. (Corrected: the original body claimed the operator must point targetPort at :8080. It does not — see the port-stealing quote above. The Service is untouched.)

What the transparent inbound listener would do

• Listen on an inbound transparent port that proxy-init PREROUTING-REDIRECTs to. :8083 is the natural pick — 8080 reverse, 8081 forward, 8082 transparent-out are taken (authlib/config/presets.go:21-30).
• Recover the original destination (the app's port) via SO_ORIGINAL_DST — reuse the existing transparentproxy package.
• Run the inbound pipeline (JWT validation) inline — no ext_proc gRPC indirection.
• Forward to the app on its recovered original port over loopback, replacing the single-fixed-backend assumption.
• TLS: terminate or passthrough per the top-level mtls config, consistent with the reverse proxy's tlssniff behavior.
• Drop port stealing for this shape: no PORT mutation, no Ports[0] rewrite, no reverse_proxy_backend.

Inbound is not a mirror of outbound

The outbound transparent path gates on destination host and then blind-tunnels (forwardproxy/transparent.go:157) — policy is host-based and the bytes stay opaque. Inbound cannot work that way:

• jwtvalidation reads pctx.Headers.Get("Authorization") and pctx.Path (plugins/jwtvalidation/plugin.go:389-390), and mint() rewrites the Authorization header to a placeholder before forwarding (:368-383).
• The reverse proxy buffers the request body when InboundPipeline.NeedsBody() (reverseproxy/server.go:240).

So the inbound listener must be a full HTTP server over the recovered connection, not a gate-and-tunnel. transparentproxy supplies the listener; the handler is new work.

Loopback forwarding is load-bearing

Dialing 127.0.0.1:<recoveredPort> is exempted from the egress guard three ways — -o lo, -d 127.0.0.0/8, and --uid-owner $PROXY_UID (init-iptables.sh:321-324) — so there is no re-capture loop. Dialing podIP:<port> instead would be caught by AB_REDIRECT's TCP catch-all. Consequence: an app bound only to podIP rather than 0.0.0.0 will not work under this listener.

Why transparent inbound (in priority order)

1. Enforcement / no-bypass — a hard inbound boundary at pod granularity, which is the unit both Kubernetes NetworkPolicy and ztunnel enforce at. Directly symmetric to the outbound enforce-redirect guard. See "Why NetworkPolicy is not a substitute" below.
2. Removes the port-stealing hack — no PORT-env dependency, no pod-spec port mutation, no relocated port to discover.
3. Multi-port apps — SO_ORIGINAL_DST recovers whichever port each connection targeted; no per-port listener config, and no "only Ports[0] is protected" gap.
4. Envoy drop-in parity — makes the Go proxy transparent on both directions, so migrating a deployment off envoy-sidecar needs no manifest surgery.

(Removed from this list: multi-protocol / gRPC. See "Not in scope" below.)

Why NetworkPolicy is not a substitute

The obvious objection is "why take NET_ADMIN when a NetworkPolicy closes most of this for free?" Two reasons it does not:

• It can only allow/deny, not validate. The bypass is not "an unauthorized pod reached us" — it is "an authorized caller reached the app without JWT validation." Agent-to-agent traffic is legitimate; it just has to go through the pipeline. NetworkPolicy cannot express that.
• Port-level ingress rules do not survive HBONE. Under Istio ambient, mesh traffic arrives at ztunnel on :15008 and the real destination port exists only inside the tunnel. A policy of ingress: ports: [8000] either blocks all mesh traffic or — once 15008 is allowed — allows everything, including the relocated port. The CNI sees 15008; it never sees 8001.

The iptables approach works precisely because it runs after ztunnel decapsulates, which is also why the ambient-path DNAT rule is mandatory rather than optional.

Intra-pod loopback is inside the boundary

A container sharing the pod's network namespace can dial 127.0.0.1:<appPort> and is not intercepted. This is not a gap: containers in a pod share a netns and are a single entity to every network enforcement layer — NetworkPolicy selects pods, not containers, and ztunnel enforces at the same granularity. It is also not capturable without breaking AuthBridge's own forward hop, which is loopback by design. envoy-sidecar has the identical property.

Scope this issue must cover

Two items are not "reuse the existing rules" and were missing from the original body.

1. Istio ambient inbound would silently bypass the listener

Ambient inbound never traverses PREROUTING. ztunnel terminates HBONE and delivers plaintext via the OUTPUT chain, which redirect mode handles with a dedicated DNAT-to-POD_IP rule (init-iptables.sh:482). Under enforce-redirect, AB_REDIRECT's first rule is -m mark --mark 0x539 -j RETURN (init-iptables.sh:318), so mesh-delivered traffic sails straight past.

Fixing this needs the equivalent mark-based DNAT rule and POD_IP in the pod env — which the operator currently injects for redirect mode only (container_builder.go:513; the enforce-redirect branch at :477-481 sets just MODE, PROXY_UID, TRANSPARENT_PORT). init-iptables.sh likewise requires POD_IP only in redirect mode. Left unaddressed this is a fail-open on the mesh path.

2. PROXY_INBOUND's port exclusions are Envoy-shaped

The existing chain excludes 15123, 15124, 9090, 9901 (init-iptables.sh:571-575) — Envoy's ports. Under proxy-sidecar they must become 8080/8081/8082/8083 plus 9091 (health, cmd/authbridge-proxy/main.go:508), 9093 (stats), and 9094 (session events API). Miss 9091 and a readiness probe gets JWT-gated. kubectl port-forward is safe by accident — it originates inside the netns and hits -o lo -j RETURN — but Service-routed traffic to :9094 would not be.

3. Test coverage

proxy-init/test-enforce-redirect.sh is a manual unshare --net harness and is not wired into CI — .github/workflows/build.yaml:28-29 only builds the proxy-init image. Adding a PREROUTING chain doubles an untested surface, so a Kind e2e gate should land with it.

Implementation sketch

The Go side is small. startReverseProxyServer already constructs the http.Server (cmd/authbridge-proxy/main.go:568-573), so per-connection destination resolution is roughly:

• an inbound listener that wraps accepted conns to carry originalDst();
• http.Server.ConnContext to stash that destination into the request context;
• swap NewSingleHostReverseProxy(target) for a &httputil.ReverseProxy{Rewrite: …} that resolves http://127.0.0.1:<port(dst)> per request, falling back to reverse_proxy_backend when absent.

That keeps the entire inbound pipeline, session recording, tlssniff mTLS, and streaming behavior unchanged. The real cost is the iptables work (items 1–2 above) and the operator changes (drop port stealing for this shape, inject POD_IP, add the role and port).

One reuse caveat: transparentproxy/server.go:79-99 drops any loopback dst on the stated premise that "a genuinely REDIRECTed connection's original destination is always some external host — never this listener itself." That premise is outbound-specific. For inbound, dst is the pod's own IP, so the guard passes and the dst == LocalAddr arm usefully catches a self-redirect on the transparent port — but the comment needs revising.

Trade-offs / notes

• Needs the privileged proxy-init container (NET_ADMIN) for inbound and is Linux-only — same constraints as enforce-redirect and envoy-sidecar.
• Should be opt-in and off by default, per the feature-flag rule. For a single-port PORT-honoring agent on a cluster where the pod-to-pod bypass is acceptable, the existing reverse proxy remains simpler and needs no privileges.
• envoy-sidecar, waypoint, and the existing reverse-proxy inbound path remain unchanged.

Not in scope

gRPC / non-HTTP inbound. The original body listed multi-protocol (HTTP API + gRPC) as a motivation. That is not reachable: there is no h2c anywhere in the inbound path, and authlib/tls sets no NextProtos, so no ALPN h2 is advertised even under mTLS — the reverse proxy is HTTP/1.1 only. SO_ORIGINAL_DST solves port multiplexing, not protocol support. Multi-port here means HTTP API + admin/metrics, and admin/metrics ports carry no JWT anyway. gRPC inbound is a separate issue.

Implementation status

Implemented across four commits (branches feat/transparent-inbound in cortex and the operator repo, feat/transparent-inbound-e2e in rossoctl):

Part	What
authbridge	transparentproxy.InboundListener (SO_ORIGINAL_DST) + per-connection backend in the reverse proxy + listener.inbound_interception
proxy-init	INBOUND_TRANSPARENT_PORT -> AB_INBOUND PREROUTING chain + ambient mark-based DNAT at the head of AB_REDIRECT; harness wired into CI
operator	namespace inboundInterception switch, proxy.allowedInboundInterception allowlist, POD_IP injection, port stealing skipped
e2e	pod-to-pod bypass regression test plus shape and "don't break the neighbours" coverage

Draft PRs: #776 (listener + iptables), rossoctl/operator#511 (the switch), rossoctl/rossoctl#2393 (e2e).

Verified: authbridge unit tests (46 packages), operator make test (21 packages), and the proxy-init netns harness (45/45 — including that the ambient DNAT precedes the ztunnel-mark RETURN, the ordering that decides whether mesh traffic is validated). Operator-side injection verified live on Kind: agent keeps its port with no PORT override, sidecar declares transparent-in=8083, proxy-init receives INBOUND_TRANSPARENT_PORT/POD_IP/SIDECAR_PORTS_EXCLUDE, and the per-agent ConfigMap correctly omits reverse_proxy_*.

Not yet verified — the gate for closing this issue: live traffic assertions (pod-to-pod → 401). The dev cluster used lacks the k8s.keycloak.org/v2alpha1 CRD, so per-agent Keycloak credential Secrets are never created and any new agent stays Pending on FailedMount, independent of this feature. Needs a run on a freshly provisioned cluster.

Two design decisions worth flagging for review:

• No AgentRuntime spec field. Mirroring spec.egressEnforcement was the obvious move, but that field is dead surface — the pod mutator has no access to the CR and resolves everything from the namespace ConfigMap, so nothing reads it. An enum-validated spec field that silently does nothing is worse than no field. The switch is the namespace authbridge-runtime-config.
• transparent requires egressEnforcement: enforce-redirect and falls back to reverse-proxy otherwise, since the two share one proxy-init container.

Related

• Original outbound transparent listener + enforce-redirect: bb15d20 feat: Add transparent listener + enforce-redirect for proxy-sidecar egress
• Prior art / possible duplicate: authbridge/docs/superpowers/specs/2026-07-30-proxy-shim-layer-design.md (branch docs/proxy-shim-layer-design, not yet merged) already specs this as PR6 of 6 — INBOUND_TRANSPARENT_PORT env, listener.roles gains transparent-inbound, :8083, off by default (:437-465, :560). Its risk table assumes the existing PROXY_INBOUND chain can be reused as-is; the ambient and port-exclusion items above say that is optimistic. These two should be reconciled before work starts so Feat: Transparent inbound interception for the Go proxy (proxy-sidecar) — SO_ORIGINAL_DST, no Envoy #330 isn't planned twice.
• Image size reduction (UBI9-micro for the Envoy variant): ⚡ Shrink authbridge images: UBI9-micro for Envoy variant, distroless for lightweight #324 (merged)
• Modular token exchange library: Modular token exchange library: shared core for AuthBridge, Klaviger, and Waypoint #279 (closed) — the original body cited this as "Authbridge unified binary", which is not what it is
• Operator upgrade / authbridge-unified swap: ⬆️ Upgrade operator to v0.2.0-alpha.26, swap to authbridge-unified rossoctl#1257 (merged PR) — the original body cited this as an "operator–authbridge coupling analysis"