Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

42 changes: 37 additions & 5 deletions architecture/sandbox.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,20 +63,52 @@ replacement from granting authority.
## Startup Flow

1. The driver resolves the immutable workload identity, installs the outer
network fence, and starts `openshell-sandbox` with one-use bootstrap state.
network fence, validates its native evidence, and starts `openshell-sandbox`
with one-use bootstrap state. Docker inspects container networking,
Kubernetes verifies its NetworkPolicy, and VM drivers inspect the guest
device model; those native schemas remain in their driver crates.
2. The sandbox consumes and unlinks bootstrap material, proves the admitted
runtime posture, and listens on the protected driver channel. It does not
run untrusted code yet.
3. `openshell-supervisor` loads policy and runtime settings from the gateway,
attaches to the sandbox, and verifies the driver's generation and evidence.
4. The sandbox installs its seccomp notification broker and Landlock baseline,
then reports measured confirmation. The supervisor must accept that evidence
before it sends the launch permit.
validates its mechanism-specific audit evidence, and reports backend-neutral
enforcement properties. The supervisor must accept those properties and
their immutable session and resource binding before it sends the launch
permit. Other isolation backends may establish the same properties with
different mechanisms and retain their detailed evidence in backend-owned
audit data.
5. The sandbox starts the canonical process through its single workload
launcher. The supervisor starts SSH and registers its gateway session.
6. Exec, signaling, PTY, DNS, TCP, and loopback-forwarding operations cross the
authenticated channel for the lifetime of the sandbox generation.

The shared isolation contract receives only normalized outer-fence guarantees:
egress is default-deny, there is no unmanaged egress path, the evidence is bound
to the sandbox generation, revocation has been verified, and controller loss
fails closed. A digest commits those guarantees to the native
evidence without teaching the shared contract about container networks,
Kubernetes objects, VM devices, or accelerator resources.

The component that owns the outer fence also validates its native evidence and
makes that projection explicitly. In the current Docker, Podman, Kubernetes,
and VM placements, that component is the compute driver. A delegated isolation
backend may own the fence and make the same projection instead. Non-empty native
evidence alone does not establish a guarantee:

| Current enforcement owner | Native evidence | Guarantees projected by the owner |
|---|---|---|
| Docker | Pinned container ID, `network_mode=none`, and no unexpected network attachments | No workload route establishes default-deny, revocation, and controller-loss behavior; the attachment inspection establishes that no unmanaged route exists. |
| Podman | Pinned container ID, `--network=none`, and no unexpected network attachments | The same container-network facts establish the same four guarantees. |
| Kubernetes | NetworkPolicy UID and resource version, ingress and egress isolation, and zero workload egress rules | The persisted, selecting policy establishes default-deny and continued denial after revocation or controller loss; zero egress rules establish that no unmanaged route is permitted. |
| VM | Generation and zero guest network devices | The absent NIC establishes all four guarantees; approved traffic uses the separate supervisor-owned channel. |

The shared contract checks that all four guarantees are present, that the
projection names the admitted generation, and that its evidence digest matches
the value passed to the workload-side runtime. It does not infer guarantees or
interpret the native fields.

When the admitted main process exits, its status and retained terminal output
remain available. The confirmed sandbox and supervisor-owned access plane continue
to serve policy-authorized exec and loopback forwarding until explicit stop or
Expand All @@ -100,7 +132,7 @@ OpenShell uses overlapping controls rather than a single sandbox primitive:
| Filesystem policy | Landlock restricts the paths the agent can read or write. |
| Process policy | Sandbox and children run as one immutable non-root identity with zero capabilities. |
| Seccomp notification | Virtualizes supported INET sockets and sends DNS/TCP decisions to the supervisor without nftables or proxy environment variables. |
| Driver outer fence | Docker `network_mode=none`, a NIC-less VM, or Kubernetes NetworkPolicy prevents any missed or unsupported kernel path from escaping. |
| Outer network fence | The component that owns network enforcement prevents any missed or unsupported kernel path from escaping. Current examples are Docker `network_mode=none`, a NIC-less VM, and Kubernetes NetworkPolicy. |
| Policy proxy | Evaluates destination, binary identity, TLS/L7 rules, SSRF checks, and inference interception. |

The supervisor may enrich baseline filesystem allowances for runtime-required
Expand Down Expand Up @@ -201,7 +233,7 @@ cannot transfer that approval to another socket.

The outer fence remains mandatory. If notification handling misses a syscall,
loses the supervisor, exceeds a bound, or encounters an unsupported socket
type, the request fails and the driver-owned fence still blocks direct egress.
type, the request fails and the outer fence still blocks direct egress.

CONNECT and absolute-form forward HTTP are explicit-proxy adapters over the same
egress pipeline. Each adapter normalizes its request into an egress intent, and
Expand Down
100 changes: 83 additions & 17 deletions crates/openshell-driver-docker/src/isolation.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,52 @@ use std::collections::{BTreeMap, HashMap};
use std::net::IpAddr;
use std::path::PathBuf;

use openshell_isolation_interface::contract::{DriverFenceEvidence, ResolvedWorkloadIdentity};
use openshell_isolation_interface::contract::{
BackendError, OuterFenceGuarantee, OuterFenceGuarantees, ResolvedWorkloadIdentity,
};
use openshell_sandbox_backend::GPU_RESOURCE_CLAIM;
use openshell_sandbox_backend::boundary_protocol::{
BoundaryConfig, BoundaryListener, GatewayVerificationKey, SandboxRuntimeDescriptor,
SandboxTlsClientConfig, SandboxTlsServerConfig, SandboxTransport,
};
use serde::Serialize;

#[derive(Serialize)]
struct DockerOuterFenceEvidence<'a> {
container_id: &'a str,
network_mode: &'static str,
unexpected_networks: &'a [String],
}

impl DockerOuterFenceEvidence<'_> {
fn project(&self, generation: &str) -> Result<OuterFenceGuarantees, BackendError> {
if self.container_id.is_empty() {
return Err(BackendError::Descriptor(
"Docker outer fence evidence is incomplete".to_string(),
));
}
let mut established = Vec::new();
if self.network_mode == "none" {
// With no container network namespace attachment, workload egress
// remains denied both after revocation and if the supervisor exits.
established.extend([
OuterFenceGuarantee::DefaultDenyEgress,
OuterFenceGuarantee::RevocationVerified,
OuterFenceGuarantee::ControllerLossFailsClosed,
]);
}
if self.unexpected_networks.is_empty() {
established.push(OuterFenceGuarantee::NoUnmanagedEgressPath);
}
let encoded = serde_json::to_vec(self).map_err(|error| {
BackendError::Descriptor(format!("encode Docker outer fence evidence: {error}"))
})?;
let projection =
OuterFenceGuarantees::from_enforcement_evidence(generation, established, &encoded)?;
projection.validate(generation)?;
Ok(projection)
}
}

/// Driver-owned inputs that bind one Docker container to one boundary.
pub struct DockerBoundarySpec {
Expand Down Expand Up @@ -48,21 +88,22 @@ pub struct DockerBoundaryProvisioning {
impl DockerBoundarySpec {
/// Produce both sides of the common protocol from the same immutable
/// Docker coordinates so attach cannot bind a different container.
#[must_use]
pub fn provision(self) -> DockerBoundaryProvisioning {
pub fn provision(self) -> Result<DockerBoundaryProvisioning, BackendError> {
let mut resource_claims = BTreeMap::from([
("docker.container_id".to_string(), self.container_id),
("docker.image_identity".to_string(), self.image_identity),
]);
if self.gpu_requested {
resource_claims.insert(GPU_RESOURCE_CLAIM.to_string(), "true".to_string());
}
let driver_fence = DriverFenceEvidence::Docker {
container_id: resource_claims["docker.container_id"].clone(),
network_mode: "none".to_string(),
unexpected_networks: Vec::new(),
};
DockerBoundaryProvisioning {
let unexpected_networks = Vec::new();
let outer_fence = DockerOuterFenceEvidence {
container_id: &resource_claims["docker.container_id"],
network_mode: "none",
unexpected_networks: &unexpected_networks,
}
.project(&self.generation)?;
Ok(DockerBoundaryProvisioning {
boundary_config: BoundaryConfig {
boundary_id: self.boundary_id.clone(),
generation: self.generation.clone(),
Expand All @@ -78,7 +119,7 @@ impl DockerBoundarySpec {
resource_claims: resource_claims.clone(),
resource_claim_files: BTreeMap::new(),
workload_identity: self.workload_identity.clone(),
driver_fence: driver_fence.clone(),
outer_fence: outer_fence.clone(),
child_env: self.child_env,
},
runtime_descriptor: SandboxRuntimeDescriptor {
Expand All @@ -92,16 +133,40 @@ impl DockerBoundarySpec {
tls: self.supervisor_tls,
host_gateway_ip: self.host_gateway_ip,
resource_claims,
driver_fence,
outer_fence,
},
}
})
}
}

#[cfg(test)]
mod tests {
use super::*;

#[test]
fn outer_fence_projection_rejects_each_missing_native_fact() {
let unexpected_networks = vec!["bridge".to_string()];
for evidence in [
DockerOuterFenceEvidence {
container_id: "",
network_mode: "none",
unexpected_networks: &[],
},
DockerOuterFenceEvidence {
container_id: "container",
network_mode: "bridge",
unexpected_networks: &[],
},
DockerOuterFenceEvidence {
container_id: "container",
network_mode: "none",
unexpected_networks: &unexpected_networks,
},
] {
assert!(evidence.project("generation-1").is_err());
}
}

#[test]
fn provisioning_binds_container_and_image_claims() {
let session_id = openshell_core::SandboxSessionId::new();
Expand Down Expand Up @@ -143,7 +208,8 @@ mod tests {
.unwrap(),
child_env: HashMap::new(),
}
.provision();
.provision()
.unwrap();

assert_eq!(
provisioned.boundary_config.resource_claims,
Expand All @@ -158,14 +224,14 @@ mod tests {
"true"
);
assert_eq!(
provisioned.boundary_config.driver_fence,
provisioned.runtime_descriptor.driver_fence
provisioned.boundary_config.outer_fence,
provisioned.runtime_descriptor.outer_fence
);
assert!(
provisioned
.runtime_descriptor
.driver_fence
.validate()
.outer_fence
.validate("generation-1")
.is_ok()
);
}
Expand Down
3 changes: 2 additions & 1 deletion crates/openshell-driver-docker/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4433,7 +4433,8 @@ async fn prepare_docker_boundary_files(
workload_identity: workload_identity.clone(),
child_env: docker_child_environment(sandbox),
}
.provision();
.provision()
.map_err(|error| Status::failed_precondition(error.to_string()))?;
let boundary_config = provisioning
.boundary_config
.encode()
Expand Down
6 changes: 4 additions & 2 deletions crates/openshell-driver-kubernetes/src/driver.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2374,7 +2374,8 @@ impl KubernetesComputeDriver {
workload_identity,
child_env,
}
.provision();
.provision()
.map_err(|error| KubernetesDriverError::Message(error.to_string()))?;
let descriptor = provisioned
.runtime_descriptor
.backend_descriptor()
Expand Down Expand Up @@ -2612,7 +2613,8 @@ impl KubernetesComputeDriver {
workload_identity,
child_env,
}
.provision();
.provision()
.map_err(|error| KubernetesDriverError::Message(error.to_string()))?;
let descriptor = provisioned
.runtime_descriptor
.backend_descriptor()
Expand Down
Loading
Loading