Skip to content

fix(windows): repair OpenClaw ProcessContainer startup (NVBug 6782898) - #3475

Open
prekshivyas wants to merge 1 commit into
NVIDIA:windowsfrom
prekshivyas:nvbug/6782898-openclaw-process-container
Open

prekshivyas wants to merge 1 commit into
NVIDIA:windowsfrom
prekshivyas:nvbug/6782898-openclaw-process-container

Conversation

@prekshivyas

Copy link
Copy Markdown
Contributor

Summary

Fixes the two observable defects tracked by NVBug 6782898 in the supported Windows OpenClaw/MXC flow:

  1. prevent Node's pre-entrypoint CommonJS main-module realpath from traversing to C:\ under process_container; and
  2. report an early target crash immediately, with its actual exit/stderr diagnostic, instead of surfacing a generic provisioning timeout about five minutes later.

Base: origin/windows at fb2980e077288b61ef03a2e6187e162d158526aa.

Related Issue

NVBug 6782898. No GitHub issue required; this is an internally tracked Windows bug.

Root cause

  • Node resolves a CommonJS main module through toRealPath() before preload/application code can run. In the reported ProcessContainer, that attempts lstat("C:\\") and fails with EPERM; the existing capture script's later realpath shim cannot affect this pre-entrypoint lookup.
  • The relay awaited tokio::net::TcpStream::connect() and only called Child::try_wait() afterward. The ProcessContainer loopback connect can remain pending despite the Tokio timeout, starving the later exit check until the full ~300-second readiness budget expires. The relay then closed without an error event, so the driver replaced the useful error with spawner exited before its target became ready.
  • The qualification script then performed a second log-text readiness poll and did not include the caught provisioning error in summary.txt.

Changes

  • Start staged Node with --preserve-symlinks-main before the capture main module. This avoids only Node's pre-entrypoint main-module realpath; it does not add an ACL, capability, filesystem grant, or root traversal permission.
  • Race the target's Child::wait() directly against port readiness and shutdown. Isolate each loopback connect on Tokio's blocking pool and give the socket its own two-second deadline, avoiding process-handle polling behind a potentially stuck connect.
  • Add protocol-v3 target_failed events carrying the exit status plus a bounded stderr tail (20 lines, 1,024 characters per line). Flush the event before relay exit and route the diagnostic into the sandbox failure.
  • Make the harness trust the lifecycle target_ready result, preserve the caught error in its summary, synchronously check sandbox deletion, and wait for forward/gateway teardown.
  • Add the current [openshell] version = 2 header to the three OpenClaw example configs.
  • Add regression coverage for an early child crash, Node argument ordering/no broad grant, diagnostic routing, deterministic harness cleanup, and denial of an unrelated root-level file read.

Before / after evidence

Identical credential-redacted qualification shape on Yukon Windows ARM64, native ARM64 wxc-exec 0.8.0, Node v22.22.3, OpenClaw 2026.7.1, process_container:

pwsh -File .\run-openclaw-forward-test.ps1 `
  -Backend process_container `
  -WxcExecPath <arm64-wxc-exec> `
  -NodeExePath <node-v22.22.3-arm64> `
  -OpenClawInstallDir <openclaw-2026.7.1> `
  -GatewayToken <redacted>
Before (fb2980e0) After (e369cf9f)
Target launch 11 args; child exited before binding 12 args including --preserve-symlinks-main; child exited before binding
Exit detected launch 19:15:01.635; useful target status logged 19:20:01.125 (~299.5 s) readiness wait 20:14:29.137; real failure routed 20:14:29.156 (~19 ms)
User-visible result after 333,773 ms: generic sandbox provisioning timed out after 300s; lifecycle reduced this to spawner exited before its target became ready after 36,891 ms including staging: ProvisionFailed: target process exited before port 18889 came up: exit code: 0xc0000142
Summary omitted the caught provisioning failure preserves the complete credential-redacted provisioning diagnostic

The pinned ARM64 Node binary fails in this host's ProcessContainer loader with STATUS_DLL_INIT_FAILED (0xc0000142) before JavaScript starts. Therefore this ARM64 machine cannot re-demonstrate the report's later EPERM lstat C:\ or complete the OpenClaw health check. The Node-side regression is covered by the durable argument-order/no-grant test and the fix follows Node's documented main-module resolution switch; a native x64 Windows qualification run remains required to validate the original EPERM path end to end. The early-exit defect is reproduced and fixed directly on ARM64.

Security impact

  • No new root/share ACLs, capabilities, policy exceptions, local-network access, or filesystem grants.
  • The existing staged share remains the only OpenClaw workload grant.
  • The real ProcessContainer security test creates an ungranted root-level sentinel beneath C:\, grants only an unrelated workload directory, and verifies the sentinel read fails with Access is denied and never exposes its contents.
  • Failure payloads are bounded and contain only target stderr already forwarded to the gateway log; launch arguments and environment remain excluded.

Testing

  • mise run pre-commit passes (not claimed: full test linking exhausted the host disk, and full Clippy reaches a pre-existing driver.rs:819 clippy::unused_self warning)
  • Unit tests added/updated
  • E2E tests added/updated

Passed:

  • mise run --skip-tools windows:build:arm64 (full release build)
  • mise run --skip-tools windows:check:arm64 (with an exact pinned Z3 4.16.0 local cache after the GitHub binary download returned HTTP 403)
  • mise run --skip-tools windows:test:unsupported:arm64 (all five unsupported-feature matrices)
  • cargo test --release -p openshell-supervisor-relay --test control_channel_contract (10 passed; includes prompt exit-23/stderr propagation)
  • cargo test --release -p openshell-driver-mxc --test openclaw_appcontainer_compat (6 passed)
  • focused openshell-driver-mxc control_channel tests (12 passed, 82 filtered)
  • cargo test --release -p openshell-driver-mxc --test wxc_exec_real pc_oneshot_unrelated_root_path_read_denied -- --ignored --exact --nocapture (passed on real ProcessContainer)
  • targeted all-target Clippy for openshell-supervisor-relay and openshell-driver-mxc with only the two confirmed base-tree lint allowances (clippy::unused_self, clippy::needless_update); all other warnings denied
  • cargo fmt --all -- --check
  • git diff --check
  • PowerShell AST parse of run-openclaw-forward-test.ps1

Limitations observed:

  • mise run --skip-tools windows:test:arm64 could not finish linking test binaries because the host ran out of disk (LNK1140 / OS error 112); no test assertion failed.
  • mise run --skip-tools windows:test:mxc-real:arm64 passed its eight dry-run cases, then the existing probe_processcontainer_proxy helper hung inside wxc-exec even for cmd /c exit 0; the new root-denial case was run and passed independently.
  • Full windows:lint:arm64 reaches a pre-existing, untouched clippy::unused_self warning at driver.rs:819; the affected crates pass targeted Clippy with that base lint and the existing needless_update base lint allowed.

Residual risk

  • Protocol v3 intentionally requires staging a matching relay and driver; a stale relay fails immediately with the existing version-mismatch diagnostic.
  • Native x64 qualification is still needed for the exact reported Node EPERM lstat C:\ path and successful OpenClaw health response because this Yukon host is ARM64 and the available x64 qualification inputs cannot run natively here.

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (not applicable; wire protocol and lifecycle behavior are documented inline)

Signed-off-by: Prekshi Vyas <prekshiv@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Sep 18, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@prekshivyas

Copy link
Copy Markdown
Contributor Author

Qualification before/after summary

NVBug: 6782898 — OpenClaw ProcessContainer startup failure and delayed error reporting

Before: The target exited before binding its port, but the useful failure was surfaced only after approximately 299.5 seconds. The final result was a generic provisioning timeout / spawner exited before its target became ready, and the summary omitted the original diagnostic. The reported x64 path failed earlier in Node startup with EPERM: lstat C:\.

After: The relay races child exit directly against readiness and routes a bounded stderr/exit-status target_failed event. On the same ARM64 host, failure was detected and routed in approximately 19 ms; the full run, including staging, ended in 36.9 seconds with the real 0xc0000142 diagnostic preserved. Node is launched with --preserve-symlinks-main, avoiding pre-entrypoint main-module realpath without adding root ACLs, capabilities, or filesystem grants. A real ProcessContainer test also confirms an unrelated C:\ sentinel remains unreadable.

Qualification status: Early-exit/fail-fast scenario PASS; exact x64 OpenClaw EPERM end-to-end qualification pending. ARM64 build/check, five unsupported-feature matrices, relay contract tests (10), OpenClaw compatibility tests (6), focused control-channel tests (12), and the real root-denial test passed. The available ARM64 Node fails earlier with STATUS_DLL_INIT_FAILED, so a native x64 Windows run is still required to prove the original EPERM lstat C:\ path and a successful OpenClaw health response. The full ARM64 test lane was interrupted by host disk exhaustion; no test assertion failure was observed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant