feat(middleware): make the 4 MiB middleware body cap configurable; multimodal model requests exceed it #4944

User Story

I run OpenShell 0.1.4 on a home server to sandbox a fleet of always-on agents, using the Podman compute driver and provider-credentialed model endpoints.
I need a deterministic request middleware on those model endpoints that checks the tools[] list of each model request against an allowlist. My agents also read images, and a request carrying one image is over the 4 MiB middleware cap.
This means I cannot put a body-inspecting guard on the model endpoints of an agent that reads images.

Problem Statement

MAX_MIDDLEWARE_PAYLOAD_BYTES is a compile-time 4 MiB (supervisor-middleware lib.rs#L293 @ v0.1.4). Registrations and bindings above it are rejected at load (#L802-L806, #L854-L858 @ v0.1.4), and there is no config knob. RFC 0009 says "The gateway rejects an invalid registration rather than silently clamping it" (README.md#L351 @ v0.1.4).

Impact / Why This Matters

A single 5 MiB image is about 7 MB once base64-encoded into a JSON request, so an agent that reads images can't sit behind a middleware that inspects bodies. Our use is a deterministic tools[] allowlist on model endpoints, which a separate issue we are filing (on provider-side server tools bypassing network policy) asks about. A security guard has to deny what it cannot inspect, so today it denies any image-carrying request over the cap; the alternative is no guard on that endpoint.

Proposed Design

1. Prioritise streaming hooks (feat(middleware): add streaming v2 HTTP hooks and deprecate the unary request hook #3307 / feat(middleware): introduce improved HTTP request/response hooks for the middleware #4359), so a guard can inspect the start of the body (tools[]) without buffering images. From the operator's side, a request middleware would declare that it needs only the first part of the body, and requests of any size would reach it.

2. Failing that, make the platform cap operator-configurable with a memory guard. Notional:

   # gateway config
   [middleware]
   max_payload_bytes = 16777216   # platform cap; default 4194304 (4 MiB)

   A registration's own max_payload_bytes could then go up to the operator's cap, and above it the gateway still rejects the registration at load, as RFC 0009 says. We know RFC 0009 treats 4 MiB as a deliberate limit (#L168, #L295, #L534 @ v0.1.4), so the default would stay 4 MiB.

Acceptance Criteria

• A body-inspecting request middleware bound to a model endpoint can inspect a request carrying a 5 MiB image (about 7 MB of JSON), either whole or as a stream whose start it can read, instead of the request being denied as over cap.
• Any larger cap is set explicitly by the operator; without it, the limit stays 4 MiB.
• A registration above the effective cap is still rejected at load.
• The docs state the cap and how an operator changes it.

Alternatives Considered

• Streaming hooks (feat(middleware): add streaming v2 HTTP hooks and deprecate the unary request hook #3307, feat(middleware): introduce improved HTTP request/response hooks for the middleware #4359). This is our preferred route, listed first above. As far as we can tell from its diff, feat(middleware): introduce improved HTTP request/response hooks for the middleware #4359 still caps the withheld request body at 4 MiB or the route's own limit (for example 10 MiB for SigV4), so on its own it does not cover large multimodal requests yet.
• A fail-open middleware. The guard then skips exactly the requests it cannot read, which defeats a security check.
• A separate inspecting proxy outside OpenShell. It duplicates the L7 inspection OpenShell already does and sits outside OpenShell's policy and audit.

Agent Investigation

• Related: feat(middleware): add streaming v2 HTTP hooks and deprecate the unary request hook #3307 (open: streaming hooks), feat(supervisor): add streaming HTTP request middleware evaluation #2431, feat(middleware): introduce improved HTTP request/response hooks for the middleware #4359 (open PR: per its diff, the withheld request body is capped at 4 MiB or the route's own limit, e.g. 10 MiB for SigV4), and a separate issue we are filing on over-cap denials being hard to see (they are logged as allowed HTTP requests).
• Code read at v0.1.4; unchanged on main 1d880c9.

Note: this content was drafted by Claude, an AI assistant, working on behalf of Brian Bouterse and in collaboration with him. It is AI generated and human reviewed.

Checklist

• I've reviewed existing issues and the published docs
• This is a design proposal, not a "please build this" request