network: publish guest listeners to the host's private networks

.github/actions/validate-nvx/action.yml

@@ -10,7 +10,7 @@ runs:
       run: |
         set -euo pipefail
         python3 -m compileall -q scripts
-        python3 -m unittest scripts/test_performance.py scripts/test_nvx_tools.py scripts/test_microvm_tests.py scripts/test_development_release.py scripts/test_egress_policy.py scripts/test_control_session.py scripts/test_managed_agent.py scripts/test_guest_time.py scripts/test_host_telemetry.py scripts/test_time_abi.py scripts/test_tool_versions.py -v
+        python3 -m unittest scripts/test_performance.py scripts/test_nvx_tools.py scripts/test_microvm_tests.py scripts/test_development_release.py scripts/test_egress_policy.py scripts/test_control_session.py scripts/test_managed_agent.py scripts/test_guest_time.py scripts/test_guest_ingress.py scripts/test_host_telemetry.py scripts/test_time_abi.py scripts/test_tool_versions.py -v
         python3 scripts/test_adversarial.py -v
         python3 .github/skills/nvx-host-connect/scripts/test_hosts.py -v
         python3 scripts/nvx.py --help
@@ -29,7 +29,7 @@ runs:
         $ErrorActionPreference = "Stop"
         python -m compileall -q scripts
         if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
-        python -m unittest scripts/test_performance.py scripts/test_nvx_tools.py scripts/test_microvm_tests.py scripts/test_development_release.py scripts/test_egress_policy.py scripts/test_control_session.py scripts/test_managed_agent.py scripts/test_guest_time.py scripts/test_host_telemetry.py scripts/test_time_abi.py scripts/test_tool_versions.py -v
+        python -m unittest scripts/test_performance.py scripts/test_nvx_tools.py scripts/test_microvm_tests.py scripts/test_development_release.py scripts/test_egress_policy.py scripts/test_control_session.py scripts/test_managed_agent.py scripts/test_guest_time.py scripts/test_guest_ingress.py scripts/test_host_telemetry.py scripts/test_time_abi.py scripts/test_tool_versions.py -v
         if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }
         python scripts\test_adversarial.py -v
         if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE }

aci_edge_sandboxes/README.md

@@ -347,8 +347,9 @@ provisioned again to accept the change.
   behind the gateway's counterpart, such as `fd00::a00:1`. The guest names the gateway as its
   DNS server in `/etc/resolv.conf` when the policy allows TCP or UDP port 53 to it, and
   otherwise the gateway's IPv6 address when the policy allows DNS to that one. Choose a guest
-  network that contains no address the guest must reach. Ingress must be `deny`, and
-  host-loopback access must be `deny` unless ports are forwarded, as described next.
+  network that contains no address the guest must reach. Unlike with the
+  [OpenVMM backend](#network-rules), ingress must be `deny`, and host-loopback access must be
+  `deny` unless ports are forwarded, as described next.
 
 ### Native proxy and forwarded ports
 
@@ -530,7 +531,7 @@ terminate the VM when the caller exits.
 | `filesystem.deniedPaths` | hidden inside mapped paths | n/a |
 | `network.egress.default` | applied (`allow` or `deny`) | n/a |
 | `network.egress.allow`, `deny` | applied; see [Network rules](#network-rules) | n/a |
-| `network.ingress.default` | `deny` applied; `allow` rejected | n/a |
+| `network.ingress.default` | applied (`allow` or `deny`); see [Network rules](#network-rules) | n/a |
 | `network.ingress.hostLoopback` | `deny` applied (also when omitted); `allow` rejected | n/a |
 | `microvm.provision.memoryMib` | applied | n/a |
 | `process.commandLine` | n/a | run as `/bin/sh -c <commandLine>`; at most 4096 bytes |
@@ -714,6 +715,17 @@ otherwise the guest gets `10.0.0.2/24` behind the profile's NAT gateway
 `10.0.0.1`, which also serves DNS, and `fd00::a00:2/120` behind the gateway's
 IPv6 address `fd00::a00:1`.
 
+With `network.ingress.default: allow`, OpenVMM listens on every host address at
+each TCP and UDP port on which the workload listens, within about a second of
+the workload starting to listen, and relays clients on the host's private
+networks, whose addresses are not globally reachable, to the guest. It refuses
+other clients, clients inside the guest network, which the guest would take for
+its neighbors, IPv6 link-local clients, and the host itself, because host
+loopback stays denied. Replies to admitted clients pass whatever the egress
+policy. A port that a host process already holds is not published, and clients
+on other hosts need the host's firewall to admit them to OpenVMM; see
+[Private-network ingress](../doc/run.md#private-network-ingress).
+
 ### Idempotence and concurrency
 
 - Every provision creates a new sandbox.
@@ -794,7 +806,8 @@ python3 scripts/nvx.py test-aci-edge-sandboxes --backend kvm   # real VM (reposi
   discovered artifacts.
 - `nvx.py test-aci-edge-sandboxes` runs the ignored `openvmm_e2e` tests against a real
   hypervisor with the repository's kernel and Alpine initramfs: the lifecycle,
-  working directories, host path mapping, network rules, and exec environments.
+  working directories, host path mapping, network rules, private-network
+  ingress, and exec environments.
   CI runs them on
   Linux/KVM, Linux/MSHV, and Windows/WHP.
 

aci_edge_sandboxes/src/openvmm/mod.rs

@@ -32,7 +32,8 @@
 //! | `filesystem.readonlyPaths`, `readwritePaths` | mapped at their [`guest_path`] | n/a |
 //! | `filesystem.deniedPaths` | hidden inside mapped paths | n/a |
 //! | `network.egress` | default and IPv4 and IPv6 rules applied | n/a |
-//! | `network.ingress`, `hostLoopback` | `deny` only | n/a |
+//! | `network.ingress.default` | `allow` and `deny` applied; see below | n/a |
+//! | `network.ingress.hostLoopback` | `deny` only | n/a |
 //! | `microvm.provision.memoryMib` | applied | n/a |
 //! | `process.commandLine` | n/a | run as `/bin/sh -c <commandLine>`, at most 4096 bytes |
 //! | `process.cwd` | n/a | an absolute guest path of at most 4095 bytes; `/` when omitted |
@@ -56,7 +57,12 @@
 //! Egress rules are expanded
 //! into OpenVMM's IPv4 and IPv6 rules exactly; rules that would need more than 256 OpenVMM rules
 //! are rejected. Egress denied without allow rules, with ingress denied,
-//! attaches no network device. Workloads run as the fixed non-root identity of
+//! attaches no network device. With `network.ingress.default: allow`, OpenVMM listens on every
+//! host address at the TCP and UDP ports where the workload listens, and admits clients on the
+//! host's private networks, whose addresses are not globally reachable, but not clients inside
+//! the guest network, IPv6 link-local clients, or the host itself, because host loopback stays
+//! denied; replies to their connections pass whatever the egress
+//! policy. Workloads run as the fixed non-root identity of
 //! [`OpenVmmConfig::workload_uid`] and [`OpenVmmConfig::workload_gid`] (see
 //! [`OpenVmmConfig::map_host_identity`] for Linux hosts) with no capabilities, read end-of-file on
 //! standard input, and may produce at most 1 MiB of combined output.
@@ -460,6 +466,7 @@ impl Backend for OpenVmmBackend {
         capabilities.exec.max_output_bytes = Some(MAX_OUTPUT_BYTES as u64);
         capabilities.network.egress_allow = true;
         capabilities.network.egress_deny = true;
+        capabilities.network.ingress_allow = true;
         capabilities.network.ingress_deny = true;
         capabilities.network.host_loopback_deny = true;
         capabilities.network.egress_rules = true;

aci_edge_sandboxes/src/openvmm/network.rs

@@ -18,7 +18,7 @@ const MAX_RULES: usize = 256;
 ///
 /// Without a network section, or when egress is denied without allow rules and ingress is
 /// denied, no network device is attached. Otherwise the portable profile applies the egress
-/// posture with ingress and host loopback denied; other postures are rejected before
+/// posture and the ingress default with host loopback denied; other postures are rejected before
 /// provisioning by the backend's capabilities.
 pub(crate) fn network_arguments(
     policy: Option<&NetworkPolicy>,
@@ -42,7 +42,7 @@ pub(crate) fn network_arguments(
         "--network-egress",
         egress.default.as_str(),
         "--network-ingress",
-        "deny",
+        policy.ingress.default.as_str(),
         "--host-loopback",
         "deny",
     ]
@@ -225,6 +225,25 @@ mod tests {
                 .unwrap()
                 .is_empty()
         );
+
+        // Ingress needs the device even when egress is denied.
+        let mut ingress = NetworkPolicy::deny_all();
+        ingress.ingress.default = Access::Allow;
+        assert_eq!(
+            network_arguments(Some(&ingress), "10.0.0.2/24").unwrap(),
+            [
+                "--net",
+                "10.0.0.2/24",
+                "--network-profile",
+                "portable",
+                "--network-egress",
+                "deny",
+                "--network-ingress",
+                "allow",
+                "--host-loopback",
+                "deny",
+            ]
+        );
     }
 
     #[test]

aci_edge_sandboxes/tests/openvmm_e2e.rs

@@ -1239,6 +1239,76 @@ fn ipv6_network_policies_are_enforced() {
     }
 }
 
+/// Returns whether a host socket holds TCP `port` on every IPv4 address, as OpenVMM's listener
+/// for a published guest port does.
+fn published(port: u16) -> bool {
+    std::net::TcpListener::bind(("0.0.0.0", port)).is_err()
+}
+
+fn wait_until(mut condition: impl FnMut() -> bool, description: &str) {
+    let deadline = Instant::now() + Duration::from_secs(30);
+    while !condition() {
+        assert!(
+            Instant::now() < deadline,
+            "timed out waiting until {description}"
+        );
+        thread::sleep(Duration::from_millis(50));
+    }
+}
+
+#[test]
+#[ignore = "requires a hypervisor and NVX guest artifacts; run scripts/nvx.py test-aci-edge-sandboxes"]
+fn private_network_ingress_publishes_workload_listeners() {
+    use std::io::Read as _;
+    use std::io::Write as _;
+
+    let (nvx, backend) = client("ingress");
+    let port = std::net::TcpListener::bind(("0.0.0.0", 0))
+        .unwrap()
+        .local_addr()
+        .unwrap()
+        .port();
+    let listen = || ExecRequest::command_line(format!("timeout 10 nc -l -p {port}"));
+    let mut ingress = NetworkPolicy::deny_all();
+    ingress.ingress.default = Access::Allow;
+    let sandbox = started(
+        &nvx,
+        &backend,
+        &ProvisionRequest::new().with_network(ingress),
+    );
+    let listener = nvx.exec(id(&sandbox), &listen()).unwrap();
+    wait_until(|| published(port), "the workload's port is published");
+    // The host is not a private-network client: the published listener completes its
+    // connection and then resets it, so nothing reaches the workload. A marker that the
+    // workload printed would reveal a connection that reached it.
+    let mut stream = std::net::TcpStream::connect(("127.0.0.1", port)).unwrap();
+    let _ = stream.write_all(b"NVX-HOST-PROBE\n");
+    let _ = stream.shutdown(std::net::Shutdown::Write);
+    stream
+        .set_read_timeout(Some(Duration::from_secs(10)))
+        .unwrap();
+    let mut buffer = [0u8; 16];
+    if let Ok(count) = stream.read(&mut buffer) {
+        assert_eq!(count, 0, "the host reached the workload");
+    }
+    let output = listener.wait_with_output().unwrap();
+    assert!(output.stdout.is_empty(), "{output:?}");
+    wait_until(|| !published(port), "the closed listener is withdrawn");
+    drop(sandbox);
+
+    // With ingress denied, nothing is published.
+    let sandbox = started(
+        &nvx,
+        &backend,
+        &ProvisionRequest::new().with_network(NetworkPolicy::egress(Access::Allow)),
+    );
+    let listener = nvx.exec(id(&sandbox), &listen()).unwrap();
+    thread::sleep(Duration::from_secs(2));
+    assert!(!published(port));
+    listener.canceller().cancel().unwrap();
+    let _ = listener.wait();
+}
+
 #[test]
 #[ignore = "requires a hypervisor and NVX guest artifacts; run scripts/nvx.py test-aci-edge-sandboxes"]
 fn network_port_ranges_are_enforced() {

aci_edge_sandboxes/tests/openvmm_fake.rs

@@ -896,10 +896,12 @@ fn unsupported_requests_are_rejected_before_anything_runs() {
             .code(),
         ErrorCode::PolicyValidation
     );
-    let mut ingress = NetworkPolicy::deny_all();
-    ingress.ingress.default = Access::Allow;
+    // The backend publishes listeners to private networks, but never to the host itself.
+    let mut host_loopback = NetworkPolicy::deny_all();
+    host_loopback.ingress.default = Access::Allow;
+    host_loopback.ingress.host_loopback = Some(Access::Allow);
     assert_eq!(
-        nvx.provision(&ProvisionRequest::new().with_network(ingress))
+        nvx.provision(&ProvisionRequest::new().with_network(host_loopback))
             .unwrap_err()
             .code(),
         ErrorCode::PolicyValidation
@@ -1372,6 +1374,22 @@ fn launch_arguments_select_the_managed_alpine_guest() {
     );
     nvx.stop(&sandbox_id).unwrap();
     nvx.deprovision(&sandbox_id).unwrap();
+
+    // Ingress attaches the device under denied egress and keeps host loopback denied.
+    let mut ingress = NetworkPolicy::deny_all();
+    ingress.ingress.default = Access::Allow;
+    let sandbox_id = nvx
+        .provision(&ProvisionRequest::new().with_network(ingress))
+        .unwrap()
+        .sandbox_id;
+    nvx.start(&sandbox_id).unwrap();
+    let arguments = fixture.launch_arguments(&sandbox_id);
+    assert!(has_pair(&arguments, "--net", "10.0.0.2/24"));
+    assert!(has_pair(&arguments, "--network-egress", "deny"));
+    assert!(has_pair(&arguments, "--network-ingress", "allow"));
+    assert!(has_pair(&arguments, "--host-loopback", "deny"));
+    nvx.stop(&sandbox_id).unwrap();
+    nvx.deprovision(&sandbox_id).unwrap();
 }
 
 #[test]

doc/design/cold-boot.md

@@ -99,8 +99,8 @@ tokens. A fresh boot then appends host-owned tokens in this order:
    slot, boot console, sandbox blocks, and the control console
    followed by `nvx_control_tty=hvc2`; and
 5. network bootstrap tokens, including gateway DNS only when the egress
-   policy permits it, which names the IPv4 gateway or else the IPv6 one,
-   followed by
+   policy permits it, which names the IPv4 gateway or else the IPv6 one, and
+   `virtnet_ingress=allow` last when ingress is allowed, followed by
    [filesystem bootstrap tokens](machine-and-device-abi.md#filesystem) for
    the attached HostFs export or aggregate.
 
@@ -111,7 +111,7 @@ fails with `E_CMDLINE_CLOCK_TOKEN`.
 
 Callers may not supply `earlycon=`, `console=`, `virtio_mmio.device=`,
 `virtnet_ip=`, `virtnet_mask=`, `virtnet_gw=`, `virtnet_dns=`, `virtnet_ip6=`,
-`virtnet_gw6=`, `virtfs_dir=`, `virtfs_tag=`, `virtfs_mode=`,
+`virtnet_gw6=`, `virtnet_ingress=`, `virtfs_dir=`, `virtfs_tag=`, `virtfs_mode=`,
 `virtfs_aggregate=`, `nvx_snapshot_tier=`, `nr_cpus=`, `nvx_workload_uid=`,
 `nvx_workload_gid=`, or `nvx_lifecycle=` tokens. A machine with a control
 console also applies the

doc/design/current-limits.md

@@ -20,9 +20,10 @@ The current ABI family intentionally does not provide:
    hard-link support;
 - sandbox-block, network, or control-console construction through the
    management RPC, or its restore of snapshots that contain them;
-- host networking other than the in-process portable endpoint, unrestricted
-   ingress, generic bidirectional host-loopback connectivity, or live
-   host-loopback port forwards in snapshots;
+- host networking other than the in-process portable endpoint, ingress from
+   globally reachable addresses, generic bidirectional host-loopback
+   connectivity without private-network ingress, or live host-loopback port
+   forwards or private-network ingress in snapshots;
 - a production control-session RPC protocol beyond the managed lifecycle
    subset;
 - replaceable sandbox configuration, a production OCI agent, or a public sandbox snapshot workflow;

doc/design/integrated-implementation-status.md

@@ -10,7 +10,7 @@ The implementation and remaining proposals separate as follows:
 | Snapshot and restore | Guest-requested version-5 publication, same-backend restore, exact saved inventory, private COW RAM, optional fresh RAM expansion, fresh/paired scratch, three sandbox tiers, input gating, single-use resume claims, a single-use restore readiness event, and management exclusion at the capture boundary. The management RPC drives the same capture and restore for blockless machines without a NIC or control console. Older supported manifests remain readable subject to their recorded capabilities. |
 | Resource activation | Opt-in CPU-prefix and 128-MiB-aligned RAM targets within immutable capacity. Explicit MSHV targets materialize only that VP prefix and cannot be saved again; KVM, WHP, and untargeted MSHV retain full VP capacity. |
 | Console | Boot virtio-console with private RX/TX state and reconnect policies; bounded portb/host-relay drain at process exit, with guest exit status propagated to management-RPC waiters. The separate control console has authenticated Linux and Windows endpoints, bounded framing with receive credits, per-instance epochs, broker saved state, and managed workload RPC. |
-| Network | Static dual-stack identity, fixed transport, the portable in-process Consomme endpoint, directional egress `allow`/`deny`, fixed ingress `deny`, IPv4 and IPv6 L3/L4 egress rules, host-loopback deny or explicit port publishing, an exact proxy exception, and quiesced restore are implemented. Stateful replies remain available; unrestricted ingress is rejected before launch. Capture drains packet ownership instead of serializing arbitrary pending packets or host flow state. |
+| Network | Static dual-stack identity, fixed transport, the portable in-process Consomme endpoint, directional egress `allow`/`deny`, ingress `deny` or private-network ingress that publishes every port on which the guest listens to clients that are not globally reachable, IPv4 and IPv6 L3/L4 egress rules, host-loopback deny or explicit port publishing, an exact proxy exception, and quiesced restore are implemented. Stateful replies remain available under both ingress actions; ingress from globally reachable addresses is refused, and snapshots reject private-network ingress. Capture drains packet ownership instead of serializing arbitrary pending packets or host flow state. |
 | Guest filesystem | Kernel features and shell bootstrap for EROFS over ext4 scratch, overlayfs, namespace/cgroup isolation, capability stripping, supervision, and low-level snapshot hooks. Not a complete OCI agent or public sandbox restore workflow. |
 | Host filesystem | Fixed no-DAX HostFs in a slot that command-line and management-RPC cold boots always expose and that stays dormant without an attachment, live attachment revalidation, server-side denied subtrees with alias-identity enforcement, allowed paths that expose subtrees of denied paths again through traverse-only directories, write narrowing to declared writable paths of a read-write export, guest-created symbolic links on read-write exports with exact targets that the host never follows, and optional caller ownership on Linux hosts, which performs each request as its guest caller's UID and GID with root squashed to the export owner and fails closed with `EPERM`. Provider-backed immutable filesystem generations remain outside the current profile. |
 | Workload policy and operations | Host-owned non-root workload identity, one-shot or managed lifecycle selection, bounded local outcome reports, and opt-in lifecycle profiling that does not change ordinary runs. |