feat(middleware): non-blocking observe binding for HTTP request and response traffic #3782

User Story

As a platform operator running agents under OpenShell, I need to attach a service that receives the request and response bodies of every model call after the call completes, for audit and conversation-level analysis. The service must not be able to delay, deny or alter any call, and its availability must not affect the sandbox.

Consumers with this requirement already in this repo's issues: post-completion audit for SOC2 / FedRAMP (@jordigilh, #1733) and a request that middleware be able to report what it could not observe (@eeee2345, #1733). The third is ours: a conversation-diligence instrument that re-asks each party to an agent-to-agent exchange what it understood, from a copy of its context, and compares the accounts.

Disclosure: I work at Mossrake, which builds that instrument. The binding serves any post-completion consumer; we will bind to it when it lands. Runtime interface statement: https://github.com/mossrake/meaning-slip/blob/main/meaning-slip-tracker-runtime-interface-statement.pdf. Framework paper: https://github.com/mossrake/meaning-slip/blob/main/meaning-slip-framework.pdf.

Problem Statement

Supervisor middleware has two bindings, HTTP_REQUEST / PRE_CREDENTIALS and HTTP_RESPONSE / PRE_RETURN. Both are gates: the relay waits for each stage's result before forwarding. There is no binding for a consumer that only receives traffic. The documented alternative is a middleware that allows everything and reports findings, which puts a non-enforcing consumer on the critical path with the timeouts, failure handling and payload caps of an enforcing one.

Impact / Why This Matters

Without this feature, users must run a post-completion consumer as a pass-through gating middleware, and remove Accept-Encoding at the request stage on every call, since compressed responses cannot be inspected.

This results in:

• One middleware round trip per response unit on the path to the sandbox. Streaming responses flush small units, so this is per flush.
• A choice between fail_closed, where a consumer outage is a sandbox outage, and fail_open, where it emits a High-severity bypass finding.
• Lost coverage: request bodies over 4 MiB follow on_error, and WHOLE_BODY_BYTES is unusable for streaming.
• A 500 ms per-stage, 30 s per-chain time budget that a binding cannot extend.

This matters because the consumers are audit and governance functions. They either accept latency and alert noise on production traffic, or run outside OpenShell as a separate proxy and lose the correlation with policy decisions. RFC 0009 rejected an audit-only mode as insufficient for prompt injection and exfiltration; that applies to gates, and this proposal leaves the gating bindings unchanged.

Proposed Design

A middleware advertises an observe binding for the HTTP request and response operations in its Describe manifest, and the operator attaches its config as is done today. No new policy syntax is proposed. OpenShell then:

1. Never waits for an observe stage. Requests are delivered after they go upstream; response units after they go to the sandbox. Sandbox-visible latency is unchanged whether the stage is fast, slow or absent.
2. Accepts only findings and metadata from it. Mutations, deny and block are ignored and counted.
3. Treats unavailability as a coverage gap, not a failure. on_error does not apply; missed deliveries are recorded at informational severity.
4. Delivers the same material the gating stages receive, with the same request_id, including compressed and over-cap bodies.
5. Bounds delivery: drops rather than buffers without limit, and reports the count dropped per request.
6. Emits the usual OCSF event per delivery (body hash, duration, request_id, no body).

The consumer's own outbound calls are ordinary egress; running it in a sandbox (#2684) puts them under policy.

Acceptance Criteria

• An operator can attach an observe-bound middleware per endpoint with no new policy syntax.
• Measured sandbox-visible latency on a streamed inference response is unchanged whether the stage is attached, stopped, or responding slower than any configured middleware timeout.
• The stage receives the same request and response material as gating stages, including compressed and over-cap bodies.
• A mutation, deny or block from the stage has no effect on traffic.
• Stopping the stage produces no sandbox-visible error and no High-severity bypass finding.
• Policy validation warns on on_error for an observe-only config.
• Docs describe the binding, and the "observe without blocking" guidance refers to it.

Alternatives Considered

• Pass-through PRE_RETURN middleware. Works today; costs are listed above.
• An audit flag on existing bindings. Changes what a result may do, not whether the relay waits.
• The OCSF stream. It carries hashes, not bodies.
• A separate proxy outside OpenShell. A second component on the live path, with no correlation to policy decisions.

The observe binding is the only option in which the consumer cannot affect traffic.

Agent Investigation

• Surveyed proto/supervisor_middleware.proto, crates/openshell-supervisor-middleware/, RFC 0009 and the supervisor-middleware docs on a clone of main
• Found that response units are processed in lockstep: response.rs process_units_from awaits each stage's result per 64 KiB unit before forwarding, so a pass-through stage adds a round trip per flush
• Found that on_error, the 4 MiB request cap, the 500 ms / 30 s timeouts and the no-compressed-bodies rule all apply to a pass-through stage as they do to an enforcing one
• Found an existing case of recording absence as coverage rather than failure (binding_not_selected_coverage in websocket.rs), which the proposed binding would follow
• An observe stage that reuses the existing evaluation messages and adds one phase value would leave the policy model and SDKs unchanged; the per-flush cost is inferred from the loop, not measured

Checklist

• I've reviewed existing issues and the architecture docs
• This is a design proposal, not a "please build this" request