fix(gateway): support delayed podman bridge binds

Signed-off-by: Evan Lezar <elezar@nvidia.com>

Author: elezar

architecture/gateway.md

@@ -66,6 +66,24 @@ reflection, non-callback inference APIs, and HTTP routes before normal request
 authentication. The operator-configured primary listener retains the full
 multiplexed API surface.
 
+Rootful Podman can report a private bridge gateway before netavark assigns that
+address. If an exact built-in Podman callback bind fails with
+`EADDRNOTAVAIL`, the Linux gateway retries the same address with
+`IP_FREEBIND`. The listener remains callback-only and becomes reachable when
+the first sandbox materializes the bridge. If delayed exact binding also fails
+while the gateway itself runs in a container, the gateway replaces the
+loopback primary and missing bridge sockets with one IPv4 wildcard socket. The
+wildcard defaults to callback-only authorization; only traffic addressed to
+the configured loopback endpoint receives primary scope. This keeps user and
+administrator APIs off the container's non-loopback interfaces, but it does
+make the sandbox-callable gRPC surface reachable on every IPv4 interface in
+that container namespace for the lifetime of the gateway process. Sandbox
+mTLS/JWT authentication and the RPC allowlist remain mandatory defenses.
+Delayed binding does not apply to rootless Podman, an explicit
+`host_gateway_ip`, Docker or external drivers, public callback addresses, or
+bind errors other than `EADDRNOTAVAIL`. A host gateway never uses the wildcard
+fallback.
+
 The `rpc_auth` classification is also the source of truth for negotiated
 listener exposure: marking an RPC as `sandbox` or `dual` makes it callable on
 these listeners. Review such changes as both authorization and network-surface

crates/openshell-driver-docker/src/lib.rs

@@ -1979,6 +1979,7 @@ impl ComputeDriver for DockerComputeDriver {
                             DockerGatewayRoute::HostGateway => "docker host-gateway IPv4 loopback",
                         }
                         .to_string(),
+                        allow_delayed_bind: false,
                         selector: Some(Selector::ExactBindAddress(bind_address.to_string())),
                     }]
                 });

crates/openshell-driver-podman/NETWORKING.md

@@ -281,6 +281,25 @@ the supervisor's RPCs. Otherwise, it creates an additional listener that
 exposes only the gateway's sandbox-callable gRPC methods. Operator, health,
 reflection, and HTTP requests must use the primary listener.
 
+Netavark may not assign a rootful managed bridge gateway until the first
+sandbox joins the network. If that exact private callback address fails to bind
+with `EADDRNOTAVAIL`, the Linux gateway uses `IP_FREEBIND` to bind the same
+address before the interface exists. The listener remains callback-only and
+becomes reachable when netavark materializes the bridge. Only the in-process
+built-in Podman driver can mark its discovered rootful managed-bridge address
+as eligible; rootless Podman, explicit `host_gateway_ip` values, and external
+drivers cannot activate delayed binding.
+
+If delayed exact binding fails while both the gateway and rootful Podman run
+inside another Linux container, the gateway uses one scoped IPv4 wildcard
+listener for that process. Connections addressed to loopback retain primary
+scope; connections addressed to the Podman bridge or any other IPv4 interface
+are callback-only. This exposes the sandbox-callable gRPC surface on every
+IPv4 interface in the outer container namespace, so deployments should still
+restrict that namespace at the container-network boundary. A gateway running
+directly on a host never uses the wildcard fallback. Neither strategy exposes
+operator, health, reflection, or HTTP routes on non-loopback interfaces.
+
 ### Layer 3 Inner Sandbox Network Namespace
 
 Inside the container, the supervisor creates another network namespace for the

crates/openshell-driver-podman/src/driver.rs

@@ -561,6 +561,7 @@ impl PodmanComputeDriver {
                         reason:
                             "Podman rootless pasta callback uses the host default-route interface"
                                 .to_string(),
+                        allow_delayed_bind: false,
                         selector: Some(Selector::DefaultRouteInterface(
                             GatewayDefaultRouteInterfaceRequirement {},
                         )),
@@ -586,6 +587,10 @@ impl PodmanComputeDriver {
             })?;
             Ok(vec![GatewayListenerRequirement {
                 reason: format!("Podman network '{}' host gateway", self.config.network_name),
+                // A rootful managed bridge can be created after gateway
+                // startup. An explicit override is operator-owned, and
+                // rootless networking must never broaden the listener.
+                allow_delayed_bind: !self.rootless && self.config.host_gateway_ip.trim().is_empty(),
                 selector: Some(Selector::ExactBindAddress(
                     SocketAddr::new(gateway_ip, callback_port).to_string(),
                 )),
@@ -596,6 +601,7 @@ impl PodmanComputeDriver {
             Ok(vec![GatewayListenerRequirement {
                 reason: "Podman machine callback forwarding terminates on gateway loopback"
                     .to_string(),
+                allow_delayed_bind: false,
                 selector: Some(Selector::LoopbackInterface(
                     GatewayLoopbackInterfaceRequirement {},
                 )),

crates/openshell-server/Cargo.toml

@@ -34,7 +34,7 @@ k8s-openapi = { workspace = true }
 
 # Async runtime
 tokio = { workspace = true }
-socket2 = { workspace = true }
+socket2 = { workspace = true, features = ["all"] }
 nix = { workspace = true }
 
 # gRPC

crates/openshell-server/src/compute/mod.rs

@@ -138,6 +138,7 @@ pub enum GatewayListenerRequirement {
         address: SocketAddr,
         driver_name: String,
         reason: String,
+        allow_delayed_bind: bool,
     },
     DefaultRouteInterface {
         driver_name: String,
@@ -149,6 +150,12 @@ pub enum GatewayListenerRequirement {
     },
 }
 
+#[derive(Clone, Copy, Debug, Eq, PartialEq)]
+pub enum GatewayListenerBindPolicy {
+    Deny,
+    TrustedBuiltinPodman,
+}
+
 impl GatewayListenerRequirement {
     pub fn driver_name(&self) -> &str {
         match self {
@@ -587,6 +594,7 @@ impl ComputeRuntime {
         driver_name: String,
         driver: SharedComputeDriver,
         driver_process: Option<Arc<ManagedDriverProcess>>,
+        listener_bind_policy: GatewayListenerBindPolicy,
         store: Arc<Store>,
         sandbox_index: SandboxIndex,
         sandbox_watch_bus: SandboxWatchBus,
@@ -642,6 +650,9 @@ impl ComputeRuntime {
                                 address,
                                 driver_name: driver_name.clone(),
                                 reason: requirement.reason,
+                                allow_delayed_bind: listener_bind_policy
+                                    == GatewayListenerBindPolicy::TrustedBuiltinPodman
+                                    && requirement.allow_delayed_bind,
                             })
                         }
                         Selector::DefaultRouteInterface(_) => {
@@ -724,6 +735,7 @@ impl ComputeRuntime {
             endpoint.name,
             driver,
             endpoint.driver_process,
+            GatewayListenerBindPolicy::Deny,
             store,
             sandbox_index,
             sandbox_watch_bus,
@@ -10487,6 +10499,7 @@ mod tests {
             "test-driver".to_string(),
             Arc::new(TestDriver::default()),
             None,
+            GatewayListenerBindPolicy::Deny,
             store,
             SandboxIndex::new(),
             SandboxWatchBus::new(),
@@ -10674,6 +10687,7 @@ mod tests {
                 address: "172.19.0.1:17670".parse().unwrap(),
                 driver_name: "docker".to_string(),
                 reason: "external driver managed bridge".to_string(),
+                allow_delayed_bind: false,
             }]
         );