refactor(sandbox)!: default to Alpine and drop community image resolution - #3236
Draft
akram wants to merge 9 commits into
Draft
refactor(sandbox)!: default to Alpine and drop community image resolution#3236akram wants to merge 9 commits into
akram wants to merge 9 commits into
Conversation
akram
requested review from
a team,
derekwaynecarr,
mrunalp and
sjenning
as code owners
September 9, 2026 10:08
akram
marked this pull request as draft
September 9, 2026 10:29
akram
force-pushed
the
refactor/sandbox-alpine-default
branch
from
September 10, 2026 16:21
1344ed3 to
e3e2878
Compare
akram
changed the title
akram
force-pushed
the
refactor/sandbox-alpine-default
branch
5 times, most recently
from
September 10, 2026 20:15
23d2a69 to
ceb6dcc
Compare
… and gateway Add multi-stage Dockerfiles that build the OpenShell supervisor and gateway images entirely inside an OpenShift/Buildah cluster, for environments without the upstream Nix CI pipeline that stages prebuilt binaries under deploy/docker/.build/prebuilt-binaries. Both reproduce the exact upstream artifacts by running the project's own Nix devShells in a builder stage, then assembling a runtime stage identical to the existing Dockerfile.supervisor / Dockerfile.gateway: - Dockerfile.supervisor.multistage: builds the static musl openshell-sandbox binary via the musl devShell; runtime is alpine:3.22 with nftables/iptables and COPY --chmod=0555. - Dockerfile.gateway.multistage: builds openshell-gateway via the glibc-2-28 devShell, normalizes the ELF interpreter with patchelf and asserts z3 is statically embedded; runtime is distroless cc-debian13. Each builder collapses build and cleanup into a single RUN so the Nix store never enters the committed layer, keeping the intermediate commit small and within the node's ephemeral-storage budget. The upstream Dockerfiles and CI binary pipeline are unchanged. Signed-off-by: Akram Signed-off-by: Akram <akram.benaissi@gmail.com>
default_sandbox_image() now returns docker.io/library/alpine:3.22, a generic version-qualified official image, so a fresh install no longer depends on the community sandbox image catalog. All compute drivers (docker, podman, kubernetes, vm) inherit this fallback. Part of NVIDIA#3116. Signed-off-by: Akram Signed-off-by: Akram <akram.benaissi@gmail.com>
akram
force-pushed
the
refactor/sandbox-alpine-default
branch
from
September 10, 2026 20:19
ceb6dcc to
813092f
Compare
Remove DEFAULT_COMMUNITY_REGISTRY, resolve_community_image, and the OPENSHELL_COMMUNITY_REGISTRY override. Bare --from values are no longer expanded into the OpenShell Community registry; the CLI and TUI now pass explicit OCI image references through to the gateway unchanged. The openshell-core image module is reduced to default_sandbox_image(). BREAKING CHANGE: community sandbox shorthand names and OPENSHELL_COMMUNITY_REGISTRY are no longer supported; pass a full OCI image reference to --from. Part of NVIDIA#3116. Signed-off-by: Akram Signed-off-by: Akram <akram.benaissi@gmail.com>
The restrictive default policy granted read-only access to /app, a directory that only existed in the community base image. A generic Alpine default has no /app, so remove it. Landlock best-effort already ignores absent paths; this just stops advertising a community-specific layout in the default. Part of NVIDIA#3116. Signed-off-by: Akram Signed-off-by: Akram <akram.benaissi@gmail.com>
akram
changed the title
Docker and Podman cleared SANDBOX_UID/GID and forwarded the image's OCI Config.User, so the supervisor rejected any image without a USER (e.g. a plain Alpine base) with 'OCI USER is required'. With the default sandbox image now Alpine, such images must start. When the image declares no OCI USER, the Docker and Podman drivers now supply a numeric non-root identity (DEFAULT_SANDBOX_UID/GID = 1000) instead of an empty declaration, mirroring the Kubernetes and VM drivers. The supervisor's existing resolved-identity path synthesizes the passwd/group entry, so the sandbox runs non-root without the account existing in the image. Images that declare a USER keep the OCI resolution path unchanged. The identity normalization already prefers a resolved numeric pair over an empty OCI declaration (covered by driver_identity_inputs_are_mutually_exclusive_and_complete), so no supervisor logic change is needed. Part of NVIDIA#3116. Signed-off-by: Akram Signed-off-by: Akram <akram.benaissi@gmail.com>
…ox image Update the shared gateway default_image, Helm chart values, the standalone Kubernetes manifest, and the dev gateway task scripts to use docker.io/library/alpine:3.22 instead of the community base image, consistent with default_sandbox_image(). GPU e2e image-build base is left unchanged (CUDA needs a glibc base). Part of NVIDIA#3116. Signed-off-by: Akram Signed-off-by: Akram <akram.benaissi@gmail.com>
Introduce netns/netlink.rs, which creates and tears down the sandbox network namespace, veth pair, addresses, and routes entirely in-process over route netlink (rtnetlink) plus unshare/setns/mount syscalls, with no external ip/nsenter binary. The namespace is FD-owned: a short-lived thread unshares CLONE_NEWNET and bind-mounts it onto netns_path via mount(2) (not `ip netns add`), so bare Alpine works and the nsenter-based nft path still reaches it. rtnetlink is async and confined here, driven from synchronous callers via a local current-thread runtime; namespace-scoped operations run on a dedicated setns thread. Refs NVIDIA#3281 Signed-off-by: Akram <akram.benaissi@gmail.com>
…senter Rewire NetworkNamespace::create(), Drop, install_transparent_tcp_rules, and validate_synthetic_pool_routes onto the new netlink module, and change first_route_overlap to work on parsed route prefixes. Remove the now-dead run_ip/run_ip_netns/run_ip_netns_output helpers and IP_SEARCH_PATHS, and drop the iproute2 hint from the proxy-mode error. nsenter and the nft helpers stay: the nft-in-namespace path is converted separately (NVIDIA#3282). Refs NVIDIA#3281 Signed-off-by: Akram <akram.benaissi@gmail.com>
…ntime NetworkNamespace::create() is invoked from within a tokio runtime on the podman driver, so building a local current-thread runtime on the calling thread panicked with "Cannot start a runtime from within a runtime". Route all host-side netlink work through a fresh std::thread (on_thread), matching the setns path, so block_on never nests in the caller's runtime. Refs NVIDIA#3281 Signed-off-by: Akram <akram.benaissi@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
First code increment toward #3116 (retire community images, default to Alpine): change the default sandbox image to a generic official Alpine image, remove the built-in community catalog resolution, make the container drivers work on a USER-less base image, and point the deployment configs at Alpine. Also adds OpenShift in-cluster build tooling. Upstream Dockerfiles and explicit image-reference behavior are preserved.
Related Issue
Part of #3116. Depends on #2750 for proxy-mode networking on a bare image (see Follow-up).
Changes
default_sandbox_image()→docker.io/library/alpine:3.22, inherited by all compute drivers.DEFAULT_COMMUNITY_REGISTRY,resolve_community_image,OPENSHELL_COMMUNITY_REGISTRY; explicit--fromOCI references pass through. BREAKING./apppath from the default policy.USER(like K8s/VM), instead of rejecting withOCI USER is required.Dockerfile.{supervisor,gateway}.multistagereproducing the upstream Nix build inside OpenShift/Buildah.Testing
CreateSandboxwith no image →docker.io/library/alpine:3.22.driver=podman) starts an Alpine sandbox — supervisor boots (noOCI USER is required), interoperates with existing supervisors. Fails only later on proxy-mode networking (see below).cargo testgreen for modified crates.Follow-up
ip(×32),nsenter(×9),nft(×6) anddmesg, which bare Alpine lacks (busyboxiphas nonetns). feat(sandbox): make privileged supervisor independent of workload-image code #2750 proposes replacing these with direct namespace syscalls / route netlink / nf_tables netlink / NFLOG, making the supervisor independent of the workload image. That work is a prerequisite for a bare-Alpine default in proxy mode.0.1.0migration guide, community repo decommission.