From 0b74308b67f0647fdd261d4eb648c4858efd7e5c Mon Sep 17 00:00:00 2001 From: Piotr Mlocek Date: Wed, 9 Sep 2026 17:05:01 -0700 Subject: [PATCH 1/2] docs(middleware): reorganize protocol guides Signed-off-by: Piotr Mlocek --- architecture/sandbox-limits.md | 2 +- architecture/sandbox.md | 2 +- docs/extensibility/supervisor-middleware.mdx | 246 ------------------ .../supervisor-middleware/configure.mdx | 124 +++++++++ .../supervisor-middleware/http.mdx | 168 ++++++++++++ .../supervisor-middleware/index.mdx | 75 ++++++ .../supervisor-middleware/operate.mdx | 67 +++++ .../supervisor-middleware/websocket.mdx | 109 ++++++++ 8 files changed, 545 insertions(+), 248 deletions(-) delete mode 100644 docs/extensibility/supervisor-middleware.mdx create mode 100644 docs/extensibility/supervisor-middleware/configure.mdx create mode 100644 docs/extensibility/supervisor-middleware/http.mdx create mode 100644 docs/extensibility/supervisor-middleware/index.mdx create mode 100644 docs/extensibility/supervisor-middleware/operate.mdx create mode 100644 docs/extensibility/supervisor-middleware/websocket.mdx diff --git a/architecture/sandbox-limits.md b/architecture/sandbox-limits.md index 9635bc1c30..a14f999996 100644 --- a/architecture/sandbox-limits.md +++ b/architecture/sandbox-limits.md @@ -73,7 +73,7 @@ Middleware also validates every non-body envelope component. Important examples include 64 KiB service config, 4 KiB request context, 32 KiB target data, 128 request headers totaling 64 KiB, 64 header mutations, 32 findings per stage, and 64 metadata entries. The detailed external contract lives in -[Supervisor Middleware](../docs/extensibility/supervisor-middleware.mdx). +[Supervisor Middleware](../docs/extensibility/supervisor-middleware/index.mdx). The work semaphore bounds aggregate buffered middleware input to approximately `32 × 4 MiB`, plus bounded envelope and parser overhead. It is a concurrency diff --git a/architecture/sandbox.md b/architecture/sandbox.md index 7a2d707bf1..5c5725fd51 100644 --- a/architecture/sandbox.md +++ b/architecture/sandbox.md @@ -240,7 +240,7 @@ against body-aware L7 policy before later stages or the upstream can observe them. Requests, results, chain length, execution time, and diagnostics are bounded; external free-form diagnostic text is not exposed in responses or security logs. See -[Supervisor Middleware](../docs/extensibility/supervisor-middleware.mdx) for +[Supervisor Middleware](../docs/extensibility/supervisor-middleware/index.mdx) for configuration and protocol details. `https://inference.local` is special. It bypasses OPA network policy and is diff --git a/docs/extensibility/supervisor-middleware.mdx b/docs/extensibility/supervisor-middleware.mdx deleted file mode 100644 index b70f4c7d5f..0000000000 --- a/docs/extensibility/supervisor-middleware.mdx +++ /dev/null @@ -1,246 +0,0 @@ ---- -# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. -# SPDX-License-Identifier: Apache-2.0 -title: "Supervisor Middleware" -sidebar-title: "Supervisor Middleware" -description: "Configure and operate built-in and operator-run middleware for sandbox HTTP requests and WebSocket messages." -keywords: "Generative AI, Cybersecurity, AI Agents, Supervisor Middleware, Extensibility, Request Filtering" ---- - -Supervisor middleware adds ordered processing stages to allowed HTTP and WebSocket egress. Middleware runs after network and L7 policy admit traffic and before OpenShell injects provider credentials. A stage can allow or deny an HTTP request or client WebSocket text message, replace its payload, add approved HTTP headers, and report audit-safe findings. - -Middleware selection is independent of the network policy rule that admitted the request. OpenShell matches middleware by destination host, so the same middleware applies consistently across broad, specific, user-authored, and provider-derived network policies. - -## Request Flow - -For each inspected HTTP request, the supervisor: - -1. Evaluates network and L7 policy. -2. Selects middleware whose host selectors match the admitted destination. -3. Buffers the request body using the largest body limit in the selected chain. -4. Runs matching middleware by ascending `order`. Policy validation rejects duplicate order values. -5. Re-checks body-aware protocol policy (GraphQL, JSON-RPC, MCP) after each stage that replaces the body. Every middleware receives a payload the policy admits, and a transformation cannot smuggle a denied or unparseable operation to a later stage or the upstream. -6. Applies allowed transformations, injects provider credentials, and forwards the request. - -For an RFC 6455 upgrade over `ws://` or `wss://`, the supervisor first finds every host-matched attachment, then selects only implementations that advertise `WEBSOCKET_MESSAGE/PRE_CREDENTIALS`. It opens one ordered, phase-specific `EvaluateWebSocketSession` stream per selected stage. OpenShell sends `WebSocketSessionEvent` values, while the service returns `WebSocketSessionEventResult` values only for preflight and message events; session start and end are notifications. Future upstream-to-client inspection uses the same RPC with `PRE_RETURN`; an implementation that advertises both phases receives two independent streams for the WebSocket session. An attachment without the selected binding can still inspect the HTTP upgrade request when it advertises the HTTP binding, but it is not a failed WebSocket stage. OpenShell allows post-upgrade traffic and emits an informational `binding_not_selected` coverage event for that attachment. - -1. A preflight before the upgrade is sent upstream. The stage chooses `INSPECT`, voluntary `SKIP`, or authoritative `DENY` and may return a bounded diagnostic reason, stable reason code, findings, and metadata. OpenShell runs selected preflights concurrently; any `DENY` rejects the upgrade regardless of `on_error`. -2. A session-start event after the upstream accepts the upgrade, including the negotiated subprotocol. -3. Complete client-to-upstream text messages in sequence order. OpenShell reassembles fragmented messages and decompresses negotiated `permessage-deflate` messages before evaluation. -4. A best-effort session-end event when the stage stream remains writable. OpenShell attempts at most one terminal event for each opened stream, including streams opened during a preflight that rejects the upgrade before session start. It then half-closes the request stream and briefly drains the response stream so the terminal event can leave the local transport before the RPC closes. Middleware services should finish their response stream after the request stream reaches EOF. - -The protobuf represents each logical message with a `text` or `binary` payload variant. Text uses the protobuf `string` type, so invalid UTF-8 cannot enter the middleware contract. Results use an optional matching replacement variant: absence preserves the input, while presence represents a replacement even when its content is empty. OpenShell rejects attempts to change the message type. Allowed replacements are re-framed, re-compressed when required, and forwarded. Binary messages, control frames, and upstream-to-client traffic remain uninspected. Binary messages pass through under both `on_error` modes. For each active selected stage, OpenShell emits an informational `unsupported_message_type` coverage event and advances the session-global sequence; the next text message can therefore reach the stage with a valid sequence gap. - -The network supervisor reserves process-wide assembly capacity before buffering every parsed WebSocket text message, even when no middleware is selected. At most 32 assemblies run while 64 additional callers wait without buffering payload bytes. When both bounds are full, OpenShell closes the WebSocket with code `1013` before reading the new message payload. A text message may contain at most 4,096 fragments, must make input progress within 30 seconds, and must finish assembly within 2 minutes. Forwarding the completed text frame must finish within another 2 minutes. The assembly budget lasts for the supervisor process lifetime, so policy reloads do not reset its capacity. - -Active middleware sessions additionally reserve shared middleware capacity before buffering WebSocket text, and HTTP middleware reserves the same capacity before buffering request bodies; at most 32 evaluations run and 64 additional unbuffered callers wait for capacity. When both middleware bounds are full, OpenShell sheds an HTTP request with `503 Service Unavailable` before reading its body. Persistent middleware streams use a separate process-wide budget of 32 sessions. WebSocket session admission does not wait: if the budget is full, OpenShell applies each selected config's `on_error` behavior before opening a stream. - -Because each transformed body is re-checked before the next stage runs, a middleware hook always receives a request that satisfies the sandbox policy. A stage whose output the policy rejects stops the chain; under `enforcement: audit` the rejection is logged and the request proceeds. - -If post-transformation policy evaluation itself fails, OpenShell denies the request and emits a high-severity detection finding. This failure is separate from middleware `on_error` because the middleware completed successfully; the sandbox policy could not validate its output. - -Middleware receives the request before credential injection. Operator-run services cannot inspect OpenShell-managed credentials. Middleware-visible request headers are delivered in wire order and repeated header names are preserved as separate entries. OpenShell filters credential, routing, framing, and hop-by-hop headers before invoking middleware. It rejects malformed request headers and unsupported transfer-coding sequences before middleware or policy dispatch. Headers named by a request's `Connection` field are omitted from middleware input and removed before forwarding, except for the validated WebSocket upgrade pair. - -The request context identifies the originating sandbox to operator-run services. It carries the sandbox ID (`sandbox_id`), the sandbox name (`sandbox_name`), and the workspace (`workspace`), letting audit and approval interfaces show a human-readable name and its workspace instead of an opaque ID. `sandbox_name` and `workspace` are for display and logging only: names are workspace-scoped and may be reused for different sandbox instances, so services must use `sandbox_id` for authorization, persistence, durable correlation, and identity. `sandbox_id` is always present on middleware requests. `sandbox_name` and `workspace` are best-effort: a supervisor that cannot resolve a value, or an older supervisor that predates a field, sends an empty string. Services should fall back to the sandbox ID when the name or workspace is empty. - -## Choose a Middleware Type - -| Type | Registration | Payload limit | Deployment | -| --- | --- | --- | --- | -| Built-in | None | Defined by OpenShell | Runs inside the supervisor | -| Operator-run service | Required in gateway TOML | Set by the operator, up to the service capability | Runs as a separate service reachable by the gateway and supervisors | - -`openshell/regex` is an example built-in middleware. It replaces only simple, self-contained token patterns in UTF-8 HTTP bodies and client WebSocket text messages; the initial pattern recognizes `sk-` tokens. It does not infer values from keyword assignments such as JSON `password` fields. This best-effort text transformation is not parser-aware and does not guarantee that it will detect or fully remove sensitive values. Its `config` accepts one field, `mode: redact`, which is also the default when the field is omitted. Unknown config fields and non-string values are rejected at policy validation. Custom expressions are not configurable yet. - -Operator-run services expose bindings for supported operation and phase pairs. A binding is identified by its operation and phase. V1 supports `HttpRequest/pre_credentials` and `WebSocketMessage/pre_credentials`; a service may expose either or both. Policies attach the complete middleware by its operator-owned gateway registration name. - -## Register a Middleware Service - -Start an operator-run service before starting the gateway, then add a registration to the local gateway TOML: - -```toml -[[openshell.supervisor.middleware]] -name = "local-content-guard" -grpc_endpoint = "https://content-guard.example:50051" -tls_ca_cert_path = "/etc/openshell/content-guard-ca.pem" -audience = "urn:example:content-guard" -max_payload_bytes = 262144 -timeout = "500ms" -``` - -| Field | Description | -| --- | --- | -| `name` | Operator-owned registration name used by policy attachments and diagnostics. Names must be unique, and `openshell/` is reserved for built-ins. | -| `grpc_endpoint` | Service address reachable from both the gateway and sandbox supervisors. Authenticated extensions use TLS `https://`. | -| `tls_ca_cert_path` | Optional PEM trust roots for a private HTTPS service. Custom roots replace platform roots and retain hostname verification. | -| `audience` | Exact audience expected by the service. Defaults to `urn:openshell:extension:middleware:`. | -| `allow_insecure_transport` | Opt this registration out of extension authentication, permitting a plaintext `http://` endpoint with no bearer credential. Defaults to `false`. Development and trusted-network deployments only. | -| `max_payload_bytes` | Shared operator limit applied to inspectable logical payloads across every binding exposed by the service, up to the 4 MiB platform maximum. It caps HTTP bodies and complete WebSocket text messages. | -| `timeout` | Optional service-wide RPC timeout using an integer with an `ms` or `s` suffix. Defaults to `500ms`; valid values range from `10ms` through `30s`. | - -Each binding returned by `Describe` may advertise a shorter `timeout` using the same syntax and bounds. The operator-configured service timeout is a ceiling: OpenShell uses the smaller of the binding and service values. An omitted binding timeout inherits the service setting, and an omitted service setting uses the 500 ms platform default. OpenShell rejects an invalid timeout before accepting the manifest. The operator-configured service timeout applies to `Describe` and `ValidateConfig`. The effective binding timeout applies only to `EvaluateHttpRequest`, WebSocket preflight, and each WebSocket message. WebSocket streams have no connection-wide deadline. - -The gateway connects to every registered service and verifies its capabilities before accepting traffic. Gateway startup fails when a service is unavailable, reports an invalid capability, or exposes more than one binding for the same operation and phase. The manifest `name` is diagnostic metadata and does not need to match the operator registration name. Operator-run registration names cannot claim the reserved `openshell/` namespace. - -Registration is static. Restart the gateway after adding, removing, or changing a service. See [Gateway Configuration](/reference/gateway-config#supervisor-middleware-services) for the complete gateway TOML context. - -### Authenticate OpenShell Callers - -When gateway JWT signing is configured, OpenShell attaches a short-lived EdDSA bearer token to every remote middleware RPC. Gateway calls use `caller_kind: gateway`; sandbox supervisor calls use `caller_kind: supervisor` and include the sandbox ID. Supervisors request credentials by registration name through `RefreshSandboxToken`. The gateway derives the audience from operator-owned configuration and authorizes each name against the sandbox's effective policy. - -Return your expected audience in the `expected_audience` field of your `Describe` manifest. After authenticated `Describe` succeeds, OpenShell compares the advertised value with its operator-configured audience and refuses to start when they differ. This is a post-authentication consistency assertion, not audience discovery: a strict verifier may reject an incorrect audience before returning the manifest, in which case startup reports an authentication failure. Leave the field empty to skip the consistency check. - -Provision the trusted gateway URL, expected gateway ID, and public key or JWKS through the deployment. This operator-provisioned key material is the authoritative cold-start trust anchor. The expected issuer is exactly `openshell-gateway:`; fetching JWKS does not establish that identity by itself. After initial trust is established, `GET /.well-known/openid-configuration` and its `jwks_uri` provide steady-state key refresh and operational convenience. The document is OIDC-shaped rather than OIDC-compliant: `issuer` is the gateway identity, not the URL serving the document, so compare `iss` against the configured value and fetch updates only over authenticated TLS at the trusted gateway URL. - -Cache keys by `kid`. Validate, at minimum: - -- `typ` is exactly `openshell-ext+jwt`. Extension tokens and sandbox-to-gateway bootstrap tokens share a signing key and differ only in audience; this header is a second, independent discriminator. -- `alg` is pinned to `EdDSA`. Never select the algorithm from the token. -- Signature, expected issuer, exact audience, and positive expiry. -- `caller_kind`, and the sandbox identity when your service scopes behavior per sandbox. - -A sandbox-to-gateway JWT is not an extension credential even though both token types use the same signing key. - -Each token carries a unique `jti` that identifies that token instance for correlation and future explicit revocation. OpenShell reuses a token across calls until rotation and does not track `jti`, so rejecting a repeated `jti` would reject legitimate requests. Per-request replay resistance requires a request nonce or signature, channel binding, or another proof-of-possession mechanism. - -### Run Without Extension Authentication - -Set `allow_insecure_transport = true` on a registration to keep a plaintext `http://` endpoint working. OpenShell then attaches no credential to that service, supervisors do not request one, and the gateway refuses to mint one if asked. The gateway logs a warning naming the registration at every startup. - -The service cannot distinguish OpenShell from any other client that can reach it. Use this only where the network already provides that guarantee, and prefer `https://` everywhere else. - -## Apply Middleware with Policy - -Add middleware configs to the top-level `network_middlewares` map. Each key is the policy-local config name: - -```yaml -network_middlewares: - regex-redactor: - name: Redact API tokens - middleware: openshell/regex - order: 10 - config: - mode: redact - on_error: fail_closed - endpoints: - include: ["*.example.com"] - exclude: ["trusted.example.com"] -``` - -Each config has a stable policy-local identity from its map key, an optional human-readable `name` that defaults to that key, a built-in or operator-owned registration name in `middleware`, an integer `order`, implementation-owned `config`, failure behavior, and host selectors. The optional name does not replace the map key for attachment or future keyed updates. A policy accepts at most 10 middleware configs. - -`include` selects destination hosts. `exclude` takes precedence and removes hosts from that selection. Each config accepts at most 32 combined include and exclude patterns. Matching is case-insensitive and uses the same exact-host and DNS glob behavior as network policy endpoints: `*` matches exactly one DNS label, `**` matches one or more labels, and intra-label patterns like `*-api.example.com` work. Brace alternates such as `{prod,staging}` are rejected at validation; list each host pattern separately. - -Matching configs run once each by ascending `order`; lower values run first. Order values must be unique across the complete policy, even when endpoint selectors do not overlap. The default order is `0`, so policies with multiple configs normally set explicit values. Different map keys may attach the same middleware and run as separate stages. Map keys are structurally unique. Runtime selection defensively rejects chains with more than 10 stages. - -See [Policy Schema](/reference/policy-schema#network-middleware) for the complete field reference. - -## Configure Failure Behavior - -`on_error` controls what happens after an operation binding is selected and middleware is unavailable, rejects its configuration, returns an invalid result, or exceeds the selected binding's payload limit. It does not turn an unadvertised operation or an unsupported WebSocket message class into a middleware failure. - -| Value | Behavior | -| --- | --- | -| `fail_closed` | Denies the HTTP request or closes the WebSocket when the stage fails. This is the default. | -| `fail_open` | Skips the failed HTTP stage. For a broken WebSocket stage stream, disables that stage for the rest of the connection and continues the remaining chain. | - -A valid upstream response can exceed the middleware envelope limits or contain header bytes that the middleware protocol cannot represent. OpenShell relays the original response when every selected response stage uses `fail_open`; any selected `fail_closed` stage causes the canonical 502 delivery failure. Malformed or unsafe HTTP does not qualify for this bypass. - -Use `fail_open` only when bypassing the middleware preserves the intended security policy. OpenShell emits a detection finding when a failed stage is bypassed and a separate state-change finding when a WebSocket stage is disabled for the session. - -Capability coverage is separate from failure handling. A host-matched HTTP-only attachment does not join the WebSocket chain, regardless of `on_error`. Binary messages are outside the V1 text-message binding and pass through even when a selected stage is `fail_closed`. OpenShell records both states as informational coverage events so operators do not mistake pass-through traffic for inspected traffic. If a deployment requires all WebSocket message classes to be inspected, V1 cannot express that requirement. - -An explicit deny decision always stops the chain and denies the request or WebSocket upgrade, regardless of `on_error`. A WebSocket preflight `DENY` is a successful policy decision, not a middleware failure; OpenShell rejects the upgrade before upstream contact and ends each still-writable stream opened by a successful preflight decision with `MIDDLEWARE_DENIAL`. The HTTP response uses `error: middleware_denied`, identifies the policy-local middleware config, and omits policy-advisor remediation because the network and L7 allow rules already matched. OpenShell never copies the free-form middleware `reason` into the response or security logs. HTTP results, WebSocket preflight decisions, and WebSocket message results can instead return an optional stable `reason_code`: 1–64 bytes, starting with a lowercase ASCII letter and containing only lowercase ASCII letters, digits, and underscores. Invalid codes make the result a middleware failure governed by `on_error`. Preflight findings and metadata use the same bounds and audit-safe handling as message results. - -```json -{ - "error": "middleware_denied", - "detail": "Request rejected by configured middleware", - "policy": "api-policy", - "middleware": "prototype-content-guard", - "reason_code": "content_match" -} -``` - -A failed `fail_closed` stage uses `error: middleware_failed` and a platform-owned `detail`. It also omits `rule_missing`, `next_steps`, and `agent_guidance`: the failure did not result from a missing network or L7 policy rule, and changing policy cannot repair it. Runtime diagnostic text is available only through sanitized operator telemetry. - -Middleware decisions are enforced regardless of the endpoint's `enforcement` mode. `enforcement: audit` applies to an endpoint's network and L7 policy rules and does not bypass middleware: a middleware deny, or a failed `fail_closed` stage, blocks the request even on an audit endpoint. A middleware service that needs to observe traffic without blocking should return an allow decision with findings, which OpenShell emits as detection findings. - -## Set Payload Limits - -Every middleware binding declares the largest logical payload or replacement it supports through `max_payload_bytes`. For `HTTP_REQUEST`, that payload is one request body. For `WEBSOCKET_MESSAGE`, it is one complete message rather than the whole session. - -- Built-in middleware uses its OpenShell-defined limit. -- Each operator-run registration sets one `max_payload_bytes` ceiling no higher than any binding's advertised `max_payload_bytes` capability. -- A selected chain buffers using its largest stage limit, so every stage that can process the body receives it. -- The same per-stage limit applies to request bodies and replacement bodies. - -The gateway rejects a registration whose operator limit exceeds the service capability or the 4 MiB platform maximum instead of silently clamping it. OpenShell also bounds the non-payload protobuf components: 64 KiB for service config, 4 KiB for request context, 32 KiB for the target, and 128 request header lines totaling at most 64 KiB encoded. Results allow a 4 KiB discarded free-form reason, a 64-byte validated reason code, 64 header mutations totaling at most 64 KiB encoded, 32 findings of at most 4 KiB encoded each, and 64 metadata entries totaling at most 32 KiB. Middleware gRPC servers should configure request and response message limits to at least 4 MiB plus 293 KiB so every platform-valid envelope fits. - -At request time, exceeding a selected stage's limit is a middleware failure for that stage alone and follows that config's `on_error` behavior; other stages in the chain still run against their own limits. OpenShell can apply `fail_open` to an oversized `Content-Length` before consuming body bytes. A chunked body can cross the limit only after bytes have been consumed, so OpenShell denies that request because it cannot safely resume the original stream. - -For a WebSocket binding, `max_payload_bytes` covers complete client text messages and replacements. Exceeding a selected stage's effective text-message limit follows that stage's `on_error`. The 4 MiB parsed-text platform cap and other protocol-safety limits are independent of middleware failure policy. Binary messages are not delivered to middleware, so the operator ceiling does not become a binary relay limit; individual raw binary frames retain the 16 MiB relay-safety bound. Oversized parsed text closes the connection with code `1009`; invalid UTF-8 uses `1007`; protocol errors use `1002`; middleware or policy denials use `1008`; and policy reload uses `1012`. - -## Mutate Request Headers - -A middleware result can return ordered header mutations before OpenShell injects credentials. A `write` mutation adds a value when the case-insensitive header name is absent and selects one behavior when it is already present: - -- `append` adds another field value. -- `overwrite` removes every existing value before adding the new value. -- `skip` leaves existing values unchanged. - -A `remove` mutation removes every value for a case-insensitive header name. OpenShell applies each successful stage's mutations before invoking the next middleware, so later stages observe the accumulated header state. - -Writes and removals may target middleware-visible end-to-end request headers. Protected credential, routing, framing, and hop-by-hop headers are always rejected. Header values must not contain control characters or OpenShell credential placeholder syntax. Middleware runs before credential injection, but it cannot introduce a value that the later injection step would resolve. - -OpenShell validates and applies each stage's mutations atomically. An invalid operation discards every mutation from that stage and follows its `on_error` behavior. Built-in failures can name the offending header. Operator-run failures use a platform-owned error code so request-derived header text cannot reach logs or denied responses. - -## Operate Middleware Services - -Plan startup and updates around these boundaries: - -- Start registered services before the gateway. The gateway validates every registration during startup. -- Keep service endpoints reachable from both the gateway and sandbox supervisors. The supervisors call operator-run services directly on the request path. -- Restart the gateway after changing registrations. -- Keep required services available before creating or updating policies. The gateway validates implementation-owned config before persisting a policy. -- Treat `fail_open` as an explicit availability-over-enforcement decision. - -When the effective sandbox configuration changes, a running supervisor validates the new service registry before installing it. If the reload fails, the supervisor keeps its last-known-good registry and emits a configuration failure event. - -## Observe Middleware - -Middleware activity is emitted through OpenShell's OCSF logging: - -- Each invocation records its policy-local config name, attached middleware name, decision, transformation state, and failure state. -- A denied invocation records a platform-owned reason derived from the policy-local config name and optional validated reason code. OpenShell does not record service-provided free-form reason text. -- A bypass under `fail_open` emits a detection finding. -- A required stage that fails closed emits a high-severity detection finding. -- A host-matched attachment without a WebSocket binding emits an informational `binding_not_selected` coverage event. -- A binary message encountered by an active WebSocket stage emits an informational `unsupported_message_type` coverage event with message type, sequence, and byte count. It is not reported as an invocation or failure. -- Built-in findings include their type, label, and aggregate count. Operator-run findings use the operator-owned registration name and a platform label plus the aggregate count; OpenShell does not log service-provided finding text or diagnostic metadata. A stage can return at most 32 findings. Exceeding the per-stage cap is an invalid response handled through `on_error`. A maximum 10-stage chain retains and emits up to 320 findings without silently dropping findings from later stages. -- Registry reload success and failure are emitted as configuration state changes. - -See [Logging](/observability/logging) for log access and [OCSF JSON Export](/observability/ocsf-json-export) for structured export. - -## Runnable example - -The [content guard example](https://github.com/NVIDIA/OpenShell/tree/main/examples/supervisor-middleware-content-guard) -matches configured literal terms in UTF-8 request bodies, complete response -bodies, and client WebSocket text messages. It supports redaction and denial. -Responses require whole-body inspection; unavailable inspection or invalid -UTF-8 invokes the configured `on_error` policy. It is not a general PII detector. - -The example includes a policy, local fixture, and smoke launcher. - -## Current Limitations - -- Middleware applies only through operation bindings advertised by each implementation. For protocols that have no supported middleware operation at all, such as HTTP/2 prior knowledge or non-HTTP TCP, the existing uninspectable-traffic gate denies a host match containing `fail_closed` and relays an all-`fail_open` match with a detection finding. -- The typed operation and phase pairs are `HTTP_REQUEST/PRE_CREDENTIALS`, `HTTP_RESPONSE/PRE_RETURN`, and `WEBSOCKET_MESSAGE/PRE_CREDENTIALS`. -- A host match does not imply every advertised operation: an HTTP-only attachment can inspect the upgrade GET, then post-upgrade traffic passes with `binding_not_selected` coverage. -- The V1 WebSocket binding inspects complete client text messages only. Binary messages pass with `unsupported_message_type` coverage for active stages; control frames and upstream-to-client messages remain outside the middleware operation. -- Selection uses destination host include and exclude patterns. -- A fail-closed middleware cannot cover `tls: skip` endpoints because OpenShell cannot inspect that traffic. An all-`fail_open` match may cover the endpoint; OpenShell bypasses the middleware and emits a detection finding. -- Operator-run services use TLS `https://` when gateway JWT signing is enabled, unless the registration sets `allow_insecure_transport`. Certificates must chain to the configured custom CA or platform roots, and the endpoint hostname must match. -- Extension tokens and sandbox-to-gateway tokens are signed by the same key. They are separated by audience and by `typ`, but the extension credential path cannot yet be rotated or revoked independently of sandbox admission. -- OpenShell does not track or revoke `jti`; bearer tokens can be replayed until expiry. Per-request replay resistance requires proof of possession or request binding. -- mTLS client authentication, health checks, runtime registration, and overlapping signing-key rotation are not available. diff --git a/docs/extensibility/supervisor-middleware/configure.mdx b/docs/extensibility/supervisor-middleware/configure.mdx new file mode 100644 index 0000000000..91285b7929 --- /dev/null +++ b/docs/extensibility/supervisor-middleware/configure.mdx @@ -0,0 +1,124 @@ +--- +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +title: "Configure Supervisor Middleware" +sidebar-title: "Configure Middleware" +description: "Choose, register, and attach supervisor middleware to sandbox traffic." +keywords: "Supervisor Middleware, Configuration, Network Policy, Extension Authentication" +position: 2 +--- + +Configure middleware in two places. Register operator-run services in gateway TOML, then attach built-in or registered middleware to destination hosts in sandbox policy. + +## Choose a middleware type + +| Type | Registration | Payload limit | Deployment | +| --- | --- | --- | --- | +| Built-in | None. | Defined by OpenShell. | Runs inside the supervisor. | +| Operator-run service | Required in gateway TOML. | Set by the operator, up to the service capability. | Runs as a separate service reachable by the gateway and supervisors. | + +`openshell/regex` is an example built-in middleware. It replaces simple, self-contained token patterns in UTF-8 HTTP bodies and client WebSocket text messages. The initial pattern recognizes `sk-` tokens. It does not infer values from fields such as a JSON `password` property, and it does not guarantee that it will detect or remove every sensitive value. + +Its `config` accepts `mode: redact`, which is also the default. Policy validation rejects unknown fields and non-string values. Custom expressions are not configurable. + +Operator-run services advertise the operation and phase pairs they support. V1 defines `HTTP_REQUEST/PRE_CREDENTIALS`, `HTTP_RESPONSE/PRE_RETURN`, and `WEBSOCKET_MESSAGE/PRE_CREDENTIALS`. A service may advertise any combination. + +## Register an operator-run service + +Start the service before the gateway, then add its registration to the local gateway TOML: + +```toml +[[openshell.supervisor.middleware]] +name = "local-content-guard" +grpc_endpoint = "https://content-guard.example:50051" +tls_ca_cert_path = "/etc/openshell/content-guard-ca.pem" +audience = "urn:example:content-guard" +max_payload_bytes = 262144 +timeout = "500ms" +``` + +| Field | Description | +| --- | --- | +| `name` | Operator-owned name used by policy attachments and diagnostics. Names must be unique. The `openshell/` namespace is reserved for built-ins. | +| `grpc_endpoint` | Service address reachable from the gateway and sandbox supervisors. Authenticated extensions use TLS `https://`. | +| `tls_ca_cert_path` | Optional PEM trust roots for a private HTTPS service. Custom roots replace platform roots and retain hostname verification. | +| `audience` | Exact audience expected by the service. Defaults to `urn:openshell:extension:middleware:`. | +| `allow_insecure_transport` | Allows plaintext `http://` without a bearer credential. Defaults to `false`. Use it only on a network that already authenticates callers. | +| `max_payload_bytes` | Operator ceiling for inspectable logical payloads across all advertised bindings, up to the 4 MiB platform maximum. | +| `timeout` | Optional service-wide RPC timeout from `10ms` through `30s`. Defaults to `500ms`. | + +Each binding returned by `Describe` may advertise a shorter timeout. OpenShell uses the smaller of the binding and service values. The service timeout applies to `Describe` and `ValidateConfig`. The effective binding timeout applies to request evaluation, response preflight and unit results, WebSocket preflight, and each WebSocket message. Response and WebSocket streams have no connection-wide deadline. + +The gateway calls `Describe` and validates every registration before it accepts traffic. Startup fails if a service is unavailable, returns an invalid capability, or exposes duplicate bindings for one operation and phase. The manifest `name` is diagnostic metadata and does not need to match the operator registration name. + +Registration is static. Restart the gateway after you add, remove, or change a service. See [Gateway Configuration](/reference/gateway-config#supervisor-middleware-services) for the full TOML context. + +## Authenticate OpenShell callers + +When gateway JWT signing is configured, OpenShell attaches a short-lived EdDSA bearer token to every remote middleware RPC. Gateway calls use `caller_kind: gateway`. Sandbox supervisor calls use `caller_kind: supervisor` and include the sandbox ID. Supervisors request credentials by registration name through `RefreshSandboxToken`; the gateway authorizes that name against the sandbox's effective policy. + +Return the expected audience in the `expected_audience` field of the `Describe` manifest. After authentication succeeds, OpenShell compares this field with the operator-configured audience and refuses to start on a mismatch. A strict service may reject the token before returning its manifest, in which case startup reports an authentication failure. Leave the field empty to skip the consistency check. + +Provision the trusted gateway URL, expected gateway ID, and public key or JWKS with the service. The expected issuer is exactly `openshell-gateway:`. After initial trust is established, `GET /.well-known/openid-configuration` and its `jwks_uri` provide key refresh. This document is OIDC-shaped, but its `issuer` is the gateway identity rather than the URL that serves the document. Compare `iss` with the configured value and fetch updates only over authenticated TLS from the trusted gateway URL. + +Cache keys by `kid` and validate: + +- `typ` is exactly `openshell-ext+jwt`. +- `alg` is pinned to `EdDSA` rather than selected from the token. +- The signature, expected issuer, exact audience, and positive expiry. +- `caller_kind`, plus the sandbox identity when the service scopes behavior per sandbox. + +Extension tokens and sandbox-to-gateway bootstrap tokens use the same signing key but have different audiences and `typ` values. Each extension token also carries a `jti`. OpenShell reuses a token until rotation and does not track `jti`, so do not reject a repeated value as a replay. Per-request replay resistance requires proof of possession or request binding. + +Set `allow_insecure_transport = true` only when the surrounding network authenticates callers. OpenShell then sends no credential to the service, and the gateway refuses to mint one if asked. The gateway logs a warning for the registration at every startup. + +## Attach middleware in policy + +Add configurations to the top-level `network_middlewares` map. Each map key is a stable policy-local identity: + +```yaml +network_middlewares: + regex-redactor: + name: Redact API tokens + middleware: openshell/regex + order: 10 + config: + mode: redact + on_error: fail_closed + endpoints: + include: ["*.example.com"] + exclude: ["trusted.example.com"] +``` + +The optional `name` defaults to the map key. `middleware` names a built-in or operator-run registration. A policy accepts at most 10 configurations. + +`include` selects destination hosts. `exclude` takes precedence. Each configuration accepts at most 32 combined patterns. Matching is case-insensitive and uses the same exact-host and DNS glob behavior as network policy endpoints. `*` matches one DNS label, `**` matches one or more labels, and an intra-label pattern such as `*-api.example.com` works. Policy validation rejects brace alternates such as `{prod,staging}`. + +Matching configurations run once each by ascending `order`. Order values must be unique across the complete policy, even when selectors do not overlap. The default is `0`, so set explicit values when a policy has multiple configurations. + +See [Policy Schema](/reference/policy-schema#network-middleware) for the complete field reference. + +## Choose failure behavior + +`on_error` applies after OpenShell selects a supported binding and that stage fails. Failures include an unavailable service, rejected configuration, invalid result, timeout, or payload over the stage limit. + +| Value | Request behavior | Response behavior | WebSocket behavior | +| --- | --- | --- | --- | +| `fail_closed` | Denies the request. This is the default. | Returns `502 response_delivery_failed` before commitment or aborts delivery after commitment. | Rejects the upgrade or closes the connection. | +| `fail_open` | Skips the failed stage. | Disables the stage and continues with the retained input. | Disables a broken stage for the rest of the connection and continues the remaining chain. | + +A valid upstream response can still exceed middleware envelope limits or contain header bytes that the middleware protocol cannot represent. OpenShell relays the original response when every selected response stage uses `fail_open`. Any selected `fail_closed` stage causes the canonical `502` delivery failure. Malformed or unsafe HTTP does not qualify for this bypass. + +An unsupported binding or message class is a coverage gap, not a middleware failure. `on_error` does not make an HTTP-only service inspect WebSocket messages, and it does not make V1 inspect binary messages. + +Use `fail_open` only when bypassing the stage preserves the intended security policy. OpenShell emits a detection finding for a bypass and a separate state-change finding when it disables a WebSocket stage. + +An explicit deny result always stops the chain, regardless of `on_error`. Middleware decisions also remain enforced when the endpoint uses `enforcement: audit`. To observe traffic without blocking it, return an allow decision with findings. + +Continue with the [HTTP request guide](/extensibility/supervisor-middleware/http) or [WebSocket session guide](/extensibility/supervisor-middleware/websocket) for protocol-specific results and limits. + +## Run the content guard example + +The [content guard example](https://github.com/NVIDIA/OpenShell/tree/main/examples/supervisor-middleware-content-guard) implements request, response, and WebSocket bindings in one service. It matches configured literal terms in UTF-8 request bodies, complete response bodies, and client WebSocket text messages. It supports redaction and denial but is not a general PII detector. + +The example includes a policy, local fixture, and smoke launcher. diff --git a/docs/extensibility/supervisor-middleware/http.mdx b/docs/extensibility/supervisor-middleware/http.mdx new file mode 100644 index 0000000000..9d953e9e50 --- /dev/null +++ b/docs/extensibility/supervisor-middleware/http.mdx @@ -0,0 +1,168 @@ +--- +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +title: "HTTP Supervisor Middleware" +sidebar-title: "HTTP Requests" +description: "Understand how supervisor middleware inspects HTTP requests and transforms upstream responses." +keywords: "Supervisor Middleware, HTTP, Request Body, Header Mutation, Payload Limits" +position: 3 +--- + +HTTP middleware evaluates admitted requests through `HTTP_REQUEST/PRE_CREDENTIALS` and final upstream responses through `HTTP_RESPONSE/PRE_RETURN`. Request stages run before credential injection. Response stages run before OpenShell delivers the result to the sandbox. + +## Request flow + +```mermaid +sequenceDiagram + participant App as Sandbox process + participant Supervisor as Network supervisor + participant Policy as Network and L7 policy + participant Chain as Ordered middleware chain + participant Upstream as Upstream service + + App->>Supervisor: HTTP request + Supervisor->>Policy: Evaluate destination, method, and path + Policy-->>Supervisor: Admit + Supervisor->>Supervisor: Select host matches and buffer body + loop Each stage by ascending order + Supervisor->>Chain: EvaluateHttpRequest + Chain-->>Supervisor: Allow, deny, or replace + opt Body replaced + Supervisor->>Policy: Re-check body-aware protocol policy + Policy-->>Supervisor: Admit or deny transformed body + end + end + alt A stage denies, fails closed, or produces a denied body + Supervisor-->>App: Deny before credential injection + else All selected stages complete + Supervisor->>Supervisor: Inject provider credentials + Supervisor->>Upstream: Forward request + end +``` + +The supervisor follows this sequence: + +1. It evaluates network and L7 policy. +2. It selects middleware whose host patterns match the admitted destination. +3. It buffers the body using the largest payload limit in the selected chain. +4. It runs stages by ascending `order`. +5. It re-checks GraphQL, JSON-RPC, or MCP policy after each body replacement. +6. It applies allowed changes, injects provider credentials, and forwards the request. + +Each stage receives a request that the current policy admits. A replacement cannot carry a denied or unparseable operation to a later stage or the upstream. If policy rejects a transformed body, the chain stops. With `enforcement: audit`, OpenShell logs the policy rejection and continues the request. + +If the post-transformation policy evaluation itself fails, OpenShell denies the request and emits a high-severity detection finding. This is not a middleware failure because the stage completed successfully. + +## Response flow + +```mermaid +sequenceDiagram + participant Upstream as Upstream service + participant Supervisor as Network supervisor + participant Chain as Response middleware chain + participant App as Sandbox process + + Upstream-->>Supervisor: Final HTTP response + loop Each stage by ascending order + Supervisor->>Chain: Preflight with status and safe headers + Chain-->>Supervisor: Skip or inspect with body mode + opt Inspect body + loop Normalized body units + Supervisor->>Chain: Body unit + Chain-->>Supervisor: Pass, transform, stop, or skip remaining + end + opt Response has trailers + Supervisor->>Chain: Normalized trailers + Chain-->>Supervisor: Trailer mutations + end + end + end + alt A stage stops delivery or fails closed + Supervisor--xApp: Return 502 before commitment or abort after commitment + else Response chain completes + Supervisor->>Supervisor: Repair framing and stale metadata + Supervisor-->>App: Deliver response + end +``` + +The supervisor relays interim `1xx` responses unchanged and retains the final non-`1xx` response. A `101` protocol upgrade bypasses generic response processing. + +For the final response, OpenShell runs `HttpResponsePreReturn.Evaluate` preflight once per matching stage in policy order. Each stage sees earlier response-header mutations and returns `SKIP` or `INSPECT`. An inspecting stage selects one body mode: + +| Mode | Behavior | +| --- | --- | +| `HEADERS_ONLY` | Inspects the status and safe response headers without receiving body or trailer events. | +| `WHOLE_BODY_BYTES` | Buffers the normalized body and sends one bounded body unit before committing the response head. | +| `STREAM_BYTES` | Sends ordered normalized units of at most 64 KiB. Each result accounts for its complete input unit before OpenShell reads more data. | + +Bodyless responses, partial responses, non-identity content encodings, and responses with `Cache-Control: no-transform` allow headers-only inspection but reject body inspection according to the stage's `on_error` policy. + +Body-inspecting stages receive a final body marker followed by normalized trailers. A stage may add only trailer names it declared during preflight. After transformation, OpenShell removes stale range and integrity metadata and repairs downstream framing. Buffered output uses a recalculated `Content-Length` unless trailers require chunked framing. Streaming output uses middleware-owned chunked framing. + +OpenShell keeps one request ID across the request and response hooks for an exchange. Response streams have no whole-response deadline. Preflight and each unit result use the effective binding timeout, and each unit's complete middleware chain is capped at 30 seconds. + +## Middleware input + +Middleware receives the request body, target, filtered headers, and request context. The context always includes `sandbox_id`. It also includes best-effort `sandbox_name` and `workspace` values. + +Use `sandbox_id` for authorization, persistence, durable correlation, and identity. Names are workspace-scoped and may be reused. Use `sandbox_name` and `workspace` only for display or logging, and fall back to the ID when either value is empty. + +OpenShell delivers visible request headers in wire order and preserves repeated names as separate entries. It removes credential, routing, framing, and hop-by-hop headers before evaluation. It also removes headers named by `Connection`, except for a validated WebSocket upgrade pair. Malformed headers and unsupported transfer-coding sequences fail before middleware or policy dispatch. + +Response middleware receives the final status and safe end-to-end headers. OpenShell removes framing, hop-by-hop, and `Connection`-nominated fields. It does not expose upstream transfer coding, transfer chunks, or socket-read boundaries. + +## Request results and body replacement + +A stage may allow or deny the request, return a replacement body, mutate approved headers, and report findings or metadata. The same per-stage limit applies to the input and replacement body. + +An explicit denial returns a structured response such as: + +```json +{ + "error": "middleware_denied", + "detail": "Request rejected by configured middleware", + "policy": "api-policy", + "middleware": "prototype-content-guard", + "reason_code": "content_match" +} +``` + +OpenShell does not copy a service-provided free-form `reason` into the response or security logs. A result may provide a stable `reason_code` of 1 through 64 bytes. It must start with a lowercase ASCII letter and contain only lowercase ASCII letters, digits, and underscores. An invalid code makes the result a middleware failure governed by `on_error`. + +A failed `fail_closed` stage returns `error: middleware_failed` with a platform-owned `detail`. Middleware errors omit policy-advisor remediation because changing network policy cannot repair the stage. + +## Response delivery failures + +OpenShell retains each response input until the corresponding stage returns a valid result. A failed `fail_open` stage can therefore be disabled without losing bytes, and the retained input continues through later stages. + +Before response commitment, a failed `fail_closed` stage returns `502 Bad Gateway` with `error: response_delivery_failed`. A `HEAD` response carries the same headers and no body. After commitment, OpenShell aborts delivery without appending an error body, terminating chunk, or error trailer. It also does not reuse the upstream connection. Retrying the request may repeat upstream side effects. + +## Header mutations + +A request result or response preflight may return ordered header mutations. OpenShell applies successful mutations before the next stage, so later middleware sees the accumulated header state. + +A `write` mutation selects one behavior when a case-insensitive header name already exists: + +- `append` adds another value. +- `overwrite` removes all existing values before writing the new one. +- `skip` leaves the existing values unchanged. + +A `remove` mutation removes all values for a case-insensitive name. Writes and removals may target middleware-visible end-to-end fields without a required prefix. OpenShell rejects request credential and routing fields. For responses it also protects status, framing, connection control, authentication challenges, content coding, range metadata, and security policy fields. Hop-by-hop and `Connection`-nominated fields remain protected in every profile. + +Header values cannot contain control characters. Request middleware also cannot write OpenShell credential placeholder syntax. Response trailer mutations use the same validation rules. Middleware may change an existing safe trailer, but it may add a new trailer only when the response preflight declared that name. + +OpenShell validates and applies each stage's mutations atomically. If one operation is invalid, it discards every mutation from that stage and applies `on_error`. + +## Payload and capacity limits + +Each binding declares `max_payload_bytes`. An operator-run registration sets a ceiling no higher than the advertised capability or the 4 MiB platform maximum. A selected request chain buffers with its largest stage limit so that every stage that can process the body receives it. + +Exceeding one stage's limit fails that stage and follows its `on_error`; other stages keep their own limits. OpenShell can skip an oversized `Content-Length` request under `fail_open` before it consumes body bytes. If a chunked body crosses the limit after consumption begins, OpenShell denies the request because it cannot safely resume the original stream. + +HTTP bodies share process-wide middleware evaluation capacity with active WebSocket message evaluations. At most 32 evaluations run while 64 additional callers wait without buffering payload bytes. When both bounds are full, OpenShell returns `503 Service Unavailable` before reading the request body. + +For `HTTP_RESPONSE`, `max_payload_bytes` limits a complete body under `WHOLE_BODY_BYTES`, each replacement, and the largest unit a streaming stage accepts. The platform caps stream input units at 64 KiB. Whole-body inspection delays commitment. Headers-only and streaming inspection commit streaming-compatible framing after preflight. + +Response body work shares the same 32 active and 64 waiting evaluation budget. Response streams also share the separate process-wide budget of 32 persistent middleware sessions with WebSocket stages. Response stream admission does not wait when that budget is full. OpenShell applies each selected configuration's `on_error` before opening its stream. + +See [Configure middleware](/extensibility/supervisor-middleware/configure) for registration limits and `on_error`, and [Operate middleware](/extensibility/supervisor-middleware/operate) for logging and service lifecycle. diff --git a/docs/extensibility/supervisor-middleware/index.mdx b/docs/extensibility/supervisor-middleware/index.mdx new file mode 100644 index 0000000000..aff8cbb69a --- /dev/null +++ b/docs/extensibility/supervisor-middleware/index.mdx @@ -0,0 +1,75 @@ +--- +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +title: "Supervisor Middleware" +sidebar-title: "Overview" +slug: "extensibility/supervisor-middleware" +description: "Understand where supervisor middleware runs and choose the guide for HTTP requests, WebSocket sessions, configuration, or operations." +keywords: "Generative AI, Cybersecurity, AI Agents, Supervisor Middleware, Extensibility, Request Filtering" +position: 1 +--- + +Supervisor middleware adds ordered processing stages to allowed sandbox egress. Request middleware runs after network and application-layer policy admit traffic and before OpenShell injects provider credentials. Response middleware runs after the upstream returns a final response and before OpenShell delivers it to the sandbox. + +Use middleware when an admitted request or message needs another decision or transformation. A stage can allow or deny traffic, replace an inspected payload, add approved HTTP headers, and report audit-safe findings. + +```mermaid +flowchart LR + A["Sandbox process"] --> B["Network and L7 policy"] + B -->|Admitted| C["Request middleware chain"] + C --> D["Provider credential injection"] + D --> E["Upstream service"] + E --> H["Response middleware chain"] + H --> A + B -->|Denied| F["Block before middleware"] + C -->|Denied or failed closed| G["Block before credentials"] + H -->|Failed closed| I["Stop response delivery"] +``` + +Middleware selection is independent of the network policy rule that admitted the traffic. OpenShell matches middleware by destination host, then runs matching configurations by ascending `order`. This keeps the same middleware attached when the admitting rule comes from a user policy, a provider profile, or a broader endpoint definition. + +## Choose a guide + + + + + +Choose a built-in or operator-run implementation, register remote services, attach middleware to hosts, and set failure behavior. + + + + +Understand request and response evaluation, body transformation, policy re-checks, header mutation, payload limits, and delivery failures. + + + + +Understand upgrade preflight, session streams, text-message inspection, pass-through traffic, capacity limits, and close codes. + + + + +Plan service startup and reloads, read OCSF events, and account for authentication and platform limits. + + + + +## Supported operations + +| Protocol path | Binding | Inspected unit | Supported changes | +| --- | --- | --- | --- | +| HTTP | `HTTP_REQUEST/PRE_CREDENTIALS` | One admitted request before credential injection. | Allow, deny, replace the body, mutate approved headers, and report findings. | +| HTTP | `HTTP_RESPONSE/PRE_RETURN` | One final upstream response before delivery to the sandbox. | Skip or inspect the response, transform selected body units, mutate approved headers and trailers, stop delivery, and report findings. | +| WebSocket | `WEBSOCKET_MESSAGE/PRE_CREDENTIALS` | Upgrade preflight and complete client-to-upstream text messages. | Inspect or skip the session, deny the upgrade, allow or deny text messages, replace text, and report findings. | + +An implementation may advertise either binding or both. A destination-host match does not imply protocol coverage. For example, HTTP-only middleware can inspect a WebSocket upgrade request but does not inspect messages after the upgrade. + +## Boundaries + +- Middleware sees traffic only after network and L7 policy allow it. +- Request middleware runs before OpenShell injects provider credentials. Operator-run services cannot inspect those credentials. Response middleware runs after the upstream call and before delivery to the sandbox. +- An explicit middleware denial always blocks traffic. Endpoint `enforcement: audit` does not bypass middleware decisions. +- V1 WebSocket middleware inspects complete client text messages only. Binary messages, control frames, and upstream-to-client messages are outside the binding. +- Protocols without a supported middleware operation use the existing uninspectable-traffic behavior. A matching `fail_closed` configuration blocks that traffic; an all-`fail_open` match bypasses middleware and emits a detection finding. + +See [Policy Schema](/reference/policy-schema#network-middleware) for the complete policy fields and [Gateway Configuration](/reference/gateway-config#supervisor-middleware-services) for the gateway TOML reference. diff --git a/docs/extensibility/supervisor-middleware/operate.mdx b/docs/extensibility/supervisor-middleware/operate.mdx new file mode 100644 index 0000000000..d7f9335b98 --- /dev/null +++ b/docs/extensibility/supervisor-middleware/operate.mdx @@ -0,0 +1,67 @@ +--- +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +title: "Operate Supervisor Middleware" +sidebar-title: "Operate Middleware" +description: "Run and observe operator middleware services and understand shared platform limits." +keywords: "Supervisor Middleware, Operations, OCSF, Logging, Service Lifecycle" +position: 5 +--- + +Operator-run middleware sits on the request path. Plan service availability, gateway restarts, sandbox configuration reloads, and alerting around the failure behavior in each policy attachment. + +## Start and update services + +- Start registered services before the gateway. The gateway validates every registration during startup. +- Keep service endpoints reachable from the gateway and sandbox supervisors. Supervisors call services directly on the request path. +- Restart the gateway after changing registrations. +- Keep required services available before creating or updating policies. The gateway validates implementation-owned configuration before persisting a policy. +- Treat `fail_open` as an explicit choice to favor availability over enforcement. + +When a sandbox's effective configuration changes, its running supervisor validates the new service registry before installing it. If validation fails, the supervisor keeps the last-known-good registry and emits a configuration failure event. + +## Observe middleware + +OpenShell emits middleware activity through OCSF logging: + +- Each invocation records the policy-local configuration name, attached middleware name, decision, transformation state, and failure state. +- A denied invocation records a platform-owned reason built from the policy-local name and optional validated reason code. It does not record free-form reason text from the service. +- A bypass under `fail_open` emits a detection finding. +- A required stage that fails closed emits a high-severity detection finding. +- An HTTP-only host match on a WebSocket session emits an informational `binding_not_selected` coverage event. +- A binary message encountered by an active WebSocket stage emits an informational `unsupported_message_type` event with message type, sequence, and byte count. It is not an invocation or failure. +- Registry reload success and failure emit configuration state changes. + +Built-in findings include their type, label, and aggregate count. Operator-run findings use the registration name, a platform label, and the aggregate count. OpenShell does not log service-provided finding text or diagnostic metadata. + +A stage can return at most 32 findings. A 10-stage chain can retain and emit up to 320 findings. Exceeding the per-stage cap makes the response invalid and applies `on_error`. + +See [Logging](/observability/logging) for log access and [OCSF JSON Export](/observability/ocsf-json-export) for structured export. + +## Size gRPC messages + +The 4 MiB platform payload maximum does not include the rest of the protobuf envelope. OpenShell also bounds: + +| Component | Limit | +| --- | --- | +| Service configuration | 64 KiB. | +| Request context | 4 KiB. | +| Target | 32 KiB. | +| Request headers | 128 lines and 64 KiB encoded. | +| Discarded free-form reason | 4 KiB. | +| Validated reason code | 64 bytes. | +| Header mutations | 64 operations and 64 KiB encoded. | +| Findings | 32 entries of at most 4 KiB encoded each. | +| Metadata | 64 entries and 32 KiB total. | + +Configure middleware gRPC servers to accept at least 4 MiB plus 293 KiB for requests and responses so they can process every platform-valid envelope. + +## Security and deployment limits + +- A `fail_closed` selector cannot cover a `tls: skip` endpoint because OpenShell cannot inspect that traffic. An all-`fail_open` match may cover it; OpenShell bypasses middleware and emits a detection finding. +- Operator-run services use TLS `https://` when gateway JWT signing is enabled unless their registration sets `allow_insecure_transport`. Certificates must chain to the configured custom CA or platform roots, and the hostname must match. +- Extension and sandbox admission tokens use the same signing key. Audience and `typ` separate them, but the extension credential path cannot rotate or revoke independently. +- OpenShell does not track or revoke `jti`. Bearer tokens can be replayed until expiry unless the service adds proof of possession or request binding. +- mTLS client authentication, health checks, runtime registration, and overlapping signing-key rotation are not available. + +Protocol-specific runtime limits are documented in [HTTP requests](/extensibility/supervisor-middleware/http#payload-and-capacity-limits) and [WebSocket sessions](/extensibility/supervisor-middleware/websocket#payload-assembly-and-capacity-limits). diff --git a/docs/extensibility/supervisor-middleware/websocket.mdx b/docs/extensibility/supervisor-middleware/websocket.mdx new file mode 100644 index 0000000000..fdbde4ea19 --- /dev/null +++ b/docs/extensibility/supervisor-middleware/websocket.mdx @@ -0,0 +1,109 @@ +--- +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +title: "WebSocket Supervisor Middleware" +sidebar-title: "WebSocket Sessions" +description: "Understand WebSocket middleware preflight, session events, message inspection, limits, and close behavior." +keywords: "Supervisor Middleware, WebSocket, RFC 6455, Session Stream, Text Messages" +position: 4 +--- + +WebSocket middleware uses the `WEBSOCKET_MESSAGE/PRE_CREDENTIALS` binding to evaluate an upgrade preflight and complete client-to-upstream text messages. OpenShell opens one ordered `EvaluateWebSocketSession` stream for each selected stage. + +## Session flow + +```mermaid +sequenceDiagram + participant App as Sandbox process + participant Supervisor as Network supervisor + participant Stages as Selected middleware stages + participant Upstream as Upstream service + + App->>Supervisor: WebSocket upgrade request + Supervisor->>Supervisor: Admit HTTP upgrade and select bindings + Supervisor->>Stages: Open streams and send preflights concurrently + Stages-->>Supervisor: Inspect, skip, or deny + Note over Supervisor,Stages: SKIP ends that stage stream + alt Any stage denies + Supervisor-->>App: Reject upgrade + Supervisor->>Stages: Session end for each writable stream + else No stage denies + Supervisor->>Upstream: Forward upgrade + Upstream-->>Supervisor: 101 Switching Protocols + Supervisor->>Stages: Session start for inspecting stages + loop Each client text message + App->>Supervisor: Frames + Supervisor->>Supervisor: Reassemble and decompress + Supervisor->>Stages: Complete text message + Stages-->>Supervisor: Allow, deny, or replace + alt Message allowed + Supervisor->>Upstream: Re-frame and forward text + else Message denied or a stage fails closed + Supervisor-->>App: Close with code 1008 + end + end + Supervisor->>Stages: Best-effort session end + end +``` + +The supervisor first finds host-matched middleware configurations, then keeps only implementations that advertise the WebSocket binding. An HTTP-only attachment may still inspect the upgrade request through its HTTP binding, but it does not join the post-upgrade message chain. OpenShell emits `binding_not_selected` coverage for that attachment. + +## Session events + +OpenShell sends these events over each selected stage stream: + +1. A preflight before contacting the upstream. The stage returns `INSPECT`, voluntary `SKIP`, or authoritative `DENY`, plus optional bounded findings and metadata. Selected preflights run concurrently. Any `DENY` rejects the upgrade regardless of `on_error`. +2. A session-start notification after the upstream accepts the upgrade. It includes the negotiated subprotocol. +3. Complete client-to-upstream text messages in sequence order. OpenShell reassembles fragmented messages and decompresses negotiated `permessage-deflate` messages before evaluation. +4. A best-effort session-end notification while the stream remains writable. + +Preflight and message events require `WebSocketSessionEventResult` responses. Session start and end are notifications. OpenShell attempts one terminal event for every opened stream, including a stream opened for a preflight that later rejects the upgrade. It half-closes the request stream and briefly drains the response stream. Middleware services should finish their response stream after request EOF. + +The protobuf also reserves `PRE_RETURN` for future upstream-to-client inspection. A service that eventually advertises both phases receives two independent streams for one WebSocket session. + +## Message handling + +The protobuf represents each logical message with a `text` or `binary` variant. Text uses the protobuf `string` type, so invalid UTF-8 cannot enter the middleware contract. + +A result may omit its replacement to preserve the input or return a text replacement, including an empty string. OpenShell rejects a replacement that changes the message type. It re-frames an allowed replacement, re-compresses it when required, and forwards it. + +V1 does not deliver binary messages to middleware. Binary messages pass through under both `on_error` modes. OpenShell emits `unsupported_message_type` coverage for each active stage and advances the session-wide sequence number, so the next text event can contain a valid sequence gap. Control frames and upstream-to-client messages also remain uninspected. + + + +If your deployment requires inspection of every WebSocket message class or both directions, V1 cannot enforce that requirement. + + + +## Failure behavior + +A preflight `DENY` is a successful policy decision, not a middleware failure. OpenShell rejects the upgrade before contacting the upstream and sends `MIDDLEWARE_DENIAL` session-end notifications to streams opened by successful preflights. + +If a selected stage fails under `fail_open`, OpenShell disables it for the rest of the connection and continues the remaining chain. It emits both a bypass finding and a state-change finding. Under `fail_closed`, OpenShell rejects the upgrade or closes the connection. + +A message result may include the same validated `reason_code` used by HTTP results. It must be 1 through 64 bytes, start with a lowercase ASCII letter, and contain only lowercase ASCII letters, digits, and underscores. An invalid code is a middleware failure governed by `on_error`. + +## Payload, assembly, and capacity limits + +For a WebSocket binding, `max_payload_bytes` covers one complete client text message and its replacement. It does not cover the whole session or binary relay traffic. Exceeding a selected stage's effective limit follows that stage's `on_error`. + +The parsed-text platform maximum is 4 MiB. A text message may contain at most 4,096 fragments, must make input progress within 30 seconds, and must finish assembly within 2 minutes. Forwarding the completed message must finish within another 2 minutes. + +The supervisor allows at most 32 concurrent text assemblies and 64 additional callers waiting without buffered payload bytes. If both bounds are full, it closes the connection with code `1013` before reading the new payload. This process-wide assembly budget applies even when no middleware is selected and persists across policy reloads. + +Active message evaluations share the middleware budget with HTTP bodies. At most 32 evaluations run and 64 additional callers wait. Persistent middleware streams have a separate process-wide limit of 32 sessions. Session admission does not wait when that limit is full. OpenShell applies each selected configuration's `on_error` before it opens a stream. + +## Close codes + +| Code | Meaning in the parsed relay | +| --- | --- | +| `1002` | WebSocket protocol error. | +| `1007` | Invalid UTF-8 text. | +| `1008` | Middleware or policy denial. | +| `1009` | Parsed text exceeds the platform limit. | +| `1012` | Policy reload makes the pinned generation stale. | +| `1013` | Assembly capacity is full. | + +Raw binary frames retain the 16 MiB relay-safety bound. The middleware payload limit does not change that bound because middleware never receives binary messages. + +See [Configure middleware](/extensibility/supervisor-middleware/configure) for attachment and failure settings, and [Operate middleware](/extensibility/supervisor-middleware/operate) for coverage events and reload behavior. From 5698564c4d1f43d5d82c8cc4cb43e92fcebd53df Mon Sep 17 00:00:00 2001 From: Piotr Mlocek Date: Wed, 9 Sep 2026 17:30:19 -0700 Subject: [PATCH 2/2] docs(extensibility): clarify overview and middleware navigation Signed-off-by: Piotr Mlocek --- architecture/sandbox.md | 3 +- docs/extensibility/gateway-interceptors.mdx | 1 + docs/extensibility/overview.mdx | 29 ++++++++ .../supervisor-middleware/configure.mdx | 2 + .../supervisor-middleware/index.mdx | 70 ++++++++----------- 5 files changed, 64 insertions(+), 41 deletions(-) create mode 100644 docs/extensibility/overview.mdx diff --git a/architecture/sandbox.md b/architecture/sandbox.md index 5c5725fd51..017df3b3d5 100644 --- a/architecture/sandbox.md +++ b/architecture/sandbox.md @@ -241,7 +241,8 @@ them. Requests, results, chain length, execution time, and diagnostics are bounded; external free-form diagnostic text is not exposed in responses or security logs. See [Supervisor Middleware](../docs/extensibility/supervisor-middleware/index.mdx) for -configuration and protocol details. +an introduction, or the [configuration guide](../docs/extensibility/supervisor-middleware/configure.mdx) +for service registration and policy attachment. `https://inference.local` is special. It bypasses OPA network policy and is handled by the inference interception path: diff --git a/docs/extensibility/gateway-interceptors.mdx b/docs/extensibility/gateway-interceptors.mdx index bf9656a5ed..bce7c31455 100644 --- a/docs/extensibility/gateway-interceptors.mdx +++ b/docs/extensibility/gateway-interceptors.mdx @@ -5,6 +5,7 @@ title: "Gateway Interceptors" sidebar-title: "Gateway Interceptors" description: "Extend OpenShell gateway operations with deployment-specific governance and business logic." keywords: "Generative AI, Cybersecurity, AI Agents, Gateway Interceptors, Extensibility, Governance" +position: 2 --- Gateway interceptors let operators add deployment-specific governance to OpenShell control-plane operations without modifying the gateway. An external gRPC service can modify or validate selected API writes before the gateway handles them, then observe successful responses after commit. diff --git a/docs/extensibility/overview.mdx b/docs/extensibility/overview.mdx new file mode 100644 index 0000000000..be9f14fe80 --- /dev/null +++ b/docs/extensibility/overview.mdx @@ -0,0 +1,29 @@ +--- +# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +title: "Extensibility" +sidebar-title: "Overview" +description: "Add custom checks to sandbox traffic and gateway operations." +keywords: "Supervisor Middleware, Gateway Interceptors, Extensibility" +position: 0 +--- + +OpenShell lets you add custom checks and transformations to sandbox traffic and gateway API operations. You can use these extensions to connect your organization's content checks, policy rules, or audit services to OpenShell. + +Choose an extension based on what you need to control. Supervisor middleware handles network traffic between an agent and external services. Gateway interceptors handle selected API operations that create or change OpenShell resources. You can use both in the same deployment. + +## Supervisor middleware + +Use supervisor middleware when you need to check the content an agent sends or receives. For example, you might redact recognized API tokens from outgoing requests, block prohibited content, or remove sensitive content from an HTTP response before the agent sees it. + +Middleware runs in the sandbox's network request and response flow. You select destination hosts in sandbox policy and choose a built-in implementation or a service you operate. + +Start with [Supervisor Middleware](/extensibility/supervisor-middleware) to learn how it works and choose a setup or protocol guide. + +## Gateway interceptors + +Use gateway interceptors when you need to enforce rules on how people and applications manage OpenShell resources. For example, you might apply an approved policy to new sandboxes, reject unauthorized policy changes, or report completed operations to an audit service. + +Interceptors run as external services that the gateway calls for selected API operations. They can modify or reject an operation before it takes effect, or observe it after it succeeds. + +Start with [Gateway Interceptors](/extensibility/gateway-interceptors) to choose operations and connect an interceptor service. diff --git a/docs/extensibility/supervisor-middleware/configure.mdx b/docs/extensibility/supervisor-middleware/configure.mdx index 91285b7929..a69d2a8f08 100644 --- a/docs/extensibility/supervisor-middleware/configure.mdx +++ b/docs/extensibility/supervisor-middleware/configure.mdx @@ -111,6 +111,8 @@ A valid upstream response can still exceed middleware envelope limits or contain An unsupported binding or message class is a coverage gap, not a middleware failure. `on_error` does not make an HTTP-only service inspect WebSocket messages, and it does not make V1 inspect binary messages. +For protocols without a supported middleware operation, OpenShell uses its uninspectable-traffic behavior. A matching `fail_closed` configuration blocks that traffic. If all matching configurations use `fail_open`, OpenShell bypasses middleware and emits a detection finding. + Use `fail_open` only when bypassing the stage preserves the intended security policy. OpenShell emits a detection finding for a bypass and a separate state-change finding when it disables a WebSocket stage. An explicit deny result always stops the chain, regardless of `on_error`. Middleware decisions also remain enforced when the endpoint uses `enforcement: audit`. To observe traffic without blocking it, return an allow decision with findings. diff --git a/docs/extensibility/supervisor-middleware/index.mdx b/docs/extensibility/supervisor-middleware/index.mdx index aff8cbb69a..1d11c3d516 100644 --- a/docs/extensibility/supervisor-middleware/index.mdx +++ b/docs/extensibility/supervisor-middleware/index.mdx @@ -2,31 +2,39 @@ # SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. # SPDX-License-Identifier: Apache-2.0 title: "Supervisor Middleware" -sidebar-title: "Overview" +sidebar-title: "Supervisor Middleware" slug: "extensibility/supervisor-middleware" -description: "Understand where supervisor middleware runs and choose the guide for HTTP requests, WebSocket sessions, configuration, or operations." +description: "Inspect, block, or change sandbox requests and responses with supervisor middleware." keywords: "Generative AI, Cybersecurity, AI Agents, Supervisor Middleware, Extensibility, Request Filtering" position: 1 --- -Supervisor middleware adds ordered processing stages to allowed sandbox egress. Request middleware runs after network and application-layer policy admit traffic and before OpenShell injects provider credentials. Response middleware runs after the upstream returns a final response and before OpenShell delivers it to the sandbox. +Supervisor middleware lets you inspect, block, or change the content that an agent sends and receives over the network. The sandbox's supervisor, which enforces its network policy, runs middleware as traffic passes between the agent and an external service. -Use middleware when an admitted request or message needs another decision or transformation. A stage can allow or deny traffic, replace an inspected payload, add approved HTTP headers, and report audit-safe findings. +## Why use middleware -```mermaid -flowchart LR - A["Sandbox process"] --> B["Network and L7 policy"] - B -->|Admitted| C["Request middleware chain"] - C --> D["Provider credential injection"] - D --> E["Upstream service"] - E --> H["Response middleware chain"] - H --> A - B -->|Denied| F["Block before middleware"] - C -->|Denied or failed closed| G["Block before credentials"] - H -->|Failed closed| I["Stop response delivery"] -``` +A network policy can allow an agent to call an API, but you may also need to check what the agent sends to that API or what the API returns. Middleware lets you add those checks without changing the agent's code. -Middleware selection is independent of the network policy rule that admitted the traffic. OpenShell matches middleware by destination host, then runs matching configurations by ascending `order`. This keeps the same middleware attached when the admitting rule comes from a user policy, a provider profile, or a broader endpoint definition. +For example, you can use middleware to: + +- Redact recognized API token patterns from outgoing request bodies. +- Reject requests that contain content your organization prohibits. +- Remove sensitive content from HTTP responses before the agent receives it. +- Check outgoing WebSocket text messages during a session. + +The checks depend on the middleware you choose. OpenShell includes a basic token-pattern redactor, and you can run your own service for custom checks and transformations. See [Configure middleware](/extensibility/supervisor-middleware/configure) for the built-in redactor's limits and an example content guard service. + +## How it fits into a request + +OpenShell checks network and application-layer policy first. If policy blocks a request, it never reaches middleware. + +For an allowed request, OpenShell selects middleware by destination host. It runs the selected checks in the order you configure, then adds provider credentials and sends the request to the external service. Request middleware cannot see the provider credentials that OpenShell adds afterward. + +When the service returns an HTTP response, response middleware can inspect or change it before OpenShell delivers it to the agent. An outgoing request check and an incoming response check are separate capabilities, so choose middleware that supports the direction you need. + +Middleware can also inspect outgoing WebSocket text messages. It does not inspect binary messages or messages returning from the service over a WebSocket connection. + +If middleware explicitly denies a request or message, OpenShell blocks it even when network policy allows it. You also choose what happens if a check fails or its service is unavailable. By default, OpenShell stops the affected traffic. You can configure it to continue without the failed check when that is acceptable. ## Choose a guide @@ -34,42 +42,24 @@ Middleware selection is independent of the network policy rule that admitted the -Choose a built-in or operator-run implementation, register remote services, attach middleware to hosts, and set failure behavior. +Start here to choose middleware, connect your own service, and select which destinations it checks. -Understand request and response evaluation, body transformation, policy re-checks, header mutation, payload limits, and delivery failures. +Check or change outgoing requests and incoming responses. Learn what middleware can read and modify, and how failures affect delivery. -Understand upgrade preflight, session streams, text-message inspection, pass-through traffic, capacity limits, and close codes. +Check outgoing text messages during a connection. Learn which messages middleware sees and when OpenShell closes a session. -Plan service startup and reloads, read OCSF events, and account for authentication and platform limits. +Run middleware services, apply configuration changes, and use logs to understand decisions and failures. -## Supported operations - -| Protocol path | Binding | Inspected unit | Supported changes | -| --- | --- | --- | --- | -| HTTP | `HTTP_REQUEST/PRE_CREDENTIALS` | One admitted request before credential injection. | Allow, deny, replace the body, mutate approved headers, and report findings. | -| HTTP | `HTTP_RESPONSE/PRE_RETURN` | One final upstream response before delivery to the sandbox. | Skip or inspect the response, transform selected body units, mutate approved headers and trailers, stop delivery, and report findings. | -| WebSocket | `WEBSOCKET_MESSAGE/PRE_CREDENTIALS` | Upgrade preflight and complete client-to-upstream text messages. | Inspect or skip the session, deny the upgrade, allow or deny text messages, replace text, and report findings. | - -An implementation may advertise either binding or both. A destination-host match does not imply protocol coverage. For example, HTTP-only middleware can inspect a WebSocket upgrade request but does not inspect messages after the upgrade. - -## Boundaries - -- Middleware sees traffic only after network and L7 policy allow it. -- Request middleware runs before OpenShell injects provider credentials. Operator-run services cannot inspect those credentials. Response middleware runs after the upstream call and before delivery to the sandbox. -- An explicit middleware denial always blocks traffic. Endpoint `enforcement: audit` does not bypass middleware decisions. -- V1 WebSocket middleware inspects complete client text messages only. Binary messages, control frames, and upstream-to-client messages are outside the binding. -- Protocols without a supported middleware operation use the existing uninspectable-traffic behavior. A matching `fail_closed` configuration blocks that traffic; an all-`fail_open` match bypasses middleware and emits a detection finding. - -See [Policy Schema](/reference/policy-schema#network-middleware) for the complete policy fields and [Gateway Configuration](/reference/gateway-config#supervisor-middleware-services) for the gateway TOML reference. +For field definitions, see [Policy Schema](/reference/policy-schema#network-middleware) and [Gateway Configuration](/reference/gateway-config#supervisor-middleware-services).