feat(server): bind guarded streams to HTTP

[Pouyanpi]

Summary

Adds an injected streaming HTTP lifecycle bridge around the provider-neutral runner, including response validation and reliable source closure.

What Changed

• Validates SSE media type, inspected identity encoding, and stream framing before headers.
• Closes the stream body and source in shielded HTTP response cleanup, even before iteration or when body closure raises; source closure must be idempotent.
• Reuses the existing kernel response-header policy and buffered renderer for provider HTTP errors.
• Preserves valid uninspected response lengths, detects relay length mismatches, and removes lengths when guarded output can change.
• Tests duplicate safe headers, connection-specific filtering, bodyless responses, limits, and source closure.
• Exercises mixed endings, BOMs, byte preservation, and truncation through the real OpenAI stream classifier.

Review Notes

Review pre-header validation, framing, transparent relay, and closure. Dispatch remains injected; concrete outbound HTTP and application client lifecycle remain separate work.

AI Assistance

• No AI tools were used.
• AI tools were used (Codex assisted with commit reconstruction, HTTP/error reconciliation, regression tests, and PR drafting). Human review of the new diff is pending.

Checklist

• I've read the CONTRIBUTING guidelines.
• This is maintainer-directed work.
• The PR title follows the project commit convention.
• Applicable declarations and tests are included.
• Generated changelog files were not edited manually.
• Automated and human review comments are addressed or answered.
• The responsible reviewer or team is mentioned.

Stack Position

Part 3 of 4.

• Previous: feat(server): add OpenAI Chat stream projections and hooks #2435
• Next: feat(server): integrate OpenAI Chat streaming #2437

Stack Context

Adds OpenAI Chat streaming above #2414 in four parts: provider-neutral execution, typed OpenAI stream declarations and handwritten hooks, injected HTTP lifecycle handling, and integration with the buffered request pipeline.

The open chain is #2411 → #2413 → #2414 → #2434 → #2435 → #2436 → #2437. #2412 is closed; its provider provenance and boundary documentation are included in #2413.

Python declarations own field and event policy. Hooks are handwritten and created for each stream by the integration. The shared exporter currently emits buffered policy; streaming contract export remains separate work. Concrete outbound HTTP, deployment configuration, and replacement execution land separately.

Review each PR against its listed base branch.

Order	PR	Branch	Base
1	#2434	pouyanpi/transparent-proxy-streaming-kernel-1	pouyanpi/openai-chat-buffered-integration-4
2	#2435	pouyanpi/openai-chat-streaming-contract-2	pouyanpi/transparent-proxy-streaming-kernel-1
3	#2436	pouyanpi/transparent-proxy-streaming-http-3	pouyanpi/openai-chat-streaming-contract-2
4	#2437	pouyanpi/openai-chat-streaming-integration-4	pouyanpi/transparent-proxy-streaming-http-3

Summary by CodeRabbit

• New Features
  • Added support for relaying server-sent event streams over HTTP, with optional safeguards that inspect and limit streamed content.
  • Streaming responses preserve their event data and are closed cleanly when finished or interrupted.
• Bug Fixes
  • Improved HTTP response handling by filtering connection-specific headers and ensuring downstream framing reflects the response body.
  • Invalid or unsafe streams are rejected rather than relayed.

[codecov]

Codecov Report

❌ Patch coverage is 94.82759% with 6 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...oguardrails/server/experimental/_streaming_http.py	94.59%	6 Missing ⚠️

📢 Thoughts on this report? Let us know!

[greptile-apps]

Confidence Score: 5/5

[High impact] The PR appears safe to merge; the remaining client identity-encoding preference issue is non-blocking.

Findings

1.  Stale headers on guarded streams ▶
2.  Identity encoding against client preference ▶

Fix with agent prompt

### Issue 1
nemoguardrails/server/experimental/_streaming_http.py:113-117
When a guarded stream blocks content, emits an error event, or drops an unfinished tail, its bytes differ from the provider response. This code removes `Content-Length` but still forwards headers such as `ETag`, `Content-MD5`, and `Digest` unchanged. A client verifying the body can reject it, and a cache can associate a validator with bytes it does not describe. Remove or recompute headers that depend on the original body whenever the stream may change.

### Issue 2
nemoguardrails/server/experimental/_streaming_http.py:undefined-193
If a streaming client explicitly forbids an identity response, for example with `Accept-Encoding: gzip, identity;q=0`, this line still asks the provider for identity. The inspected stream is then returned without a content encoding, so the client may receive a response it said it cannot accept. Reject that request or negotiate an acceptable downstream encoding.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

The PR adds an injected HTTP bridge for guarded SSE streams and shares response-header filtering with the buffered path.

• It validates streaming responses before sending headers, preserves uninspected framing, and closes stream resources on completion or interruption.
• The latest changes bound shielded cleanup and log when a close times out.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Prepared request] --> B[Injected dispatch]
  B --> C{Response}
  C -->|Buffered error| D[Shared buffered renderer]
  C -->|Stream| E[Validate SSE and framing]
  E -->|Rejected| F[Close source and render error]
  E -->|Uninspected| G[Relay bytes]
  E -->|Inspected| H[Guard provider stream]
  G --> I[HTTP response cleanup]
  H --> I

Loading

_{Reviews (6) · Last reviewed commit: "fix(server): bound shielded stream clean..." · Reviewed by Greptile}

[greptile-apps]

Stale headers on guarded streams When a guarded stream blocks content, emits an error event, or drops an unfinished tail, its bytes differ from the provider response. This code removes Content-Length but still forwards headers such as ETag, Content-MD5, and Digest unchanged. A client verifying the body can reject it, and a cache can associate a validator with bytes it does not describe. Remove or recompute headers that depend on the original body whenever the stream may change.

Prompt To Fix With AI

This is a comment left during a code review.
Path: nemoguardrails/server/experimental/_streaming_http.py
Line: 99-103

Comment:
**Stale headers on guarded streams** When a guarded stream blocks content, emits an error event, or drops an unfinished tail, its bytes differ from the provider response. This code removes `Content-Length` but still forwards headers such as `ETag`, `Content-MD5`, and `Digest` unchanged. A client verifying the body can reject it, and a cache can associate a validator with bytes it does not describe. Remove or recompute headers that depend on the original body whenever the stream may change.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[Pouyanpi]

@greptileai this is resolved in the latest commit. Do you confirm?

[greptile-apps]

Yes, this is resolved. The latest implementation defines _BODY_DEPENDENT_HEADERS with content-length, content-md5, content-digest, digest, etag, and repr-digest, and _streaming_response(..., may_modify=True) filters all of them before returning the guarded response. The uninspected relay keeps the original headers (may_modify=False), so valid transparent responses still preserve their validators. I confirm the previous stale-validator finding is addressed.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Repository: NVIDIA-NeMo/Guardrails/.coderabbit.yaml
• Review profile: CHILL
• Plan: Enterprise
• Run ID: 6ff5c615-0638-4baf-bfed-9c773cbaf822

---

📥 Commits

Reviewing files that changed from the base of the PR and between 969832f and fe7408e.

---

📒 Files selected for processing (3)

• nemoguardrails/server/experimental/_http_kernel.py
• nemoguardrails/server/experimental/_streaming_http.py
• tests/server/experimental/test_streaming_http.py

---

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 8 remain after this review.

---

---

📝 Walkthrough

Walkthrough

The PR adds an experimental HTTP streaming adapter. It validates streamed responses, relays or guards their bodies, filters headers, and closes response bodies and provider sources. It also updates shared response header filtering.

Changes

Streaming HTTP response handling

Layer / File(s)	Summary
Header filtering and framing nemoguardrails/server/experimental/_http_kernel.py, nemoguardrails/server/experimental/_streaming_http.py, tests/server/experimental/test_streaming_http.py	Shared response rendering filters hop-by-hop headers and omits upstream Content-Length before downstream framing. Streaming header helpers filter body-dependent headers when inspection may modify the body. Tests cover header filtering and framing validation.
Streaming dispatch and body lifecycle nemoguardrails/server/experimental/_streaming_http.py, tests/server/experimental/test_streaming_http.py	The adapter validates dispatch results and SSE framing, then relays uninspected streams or guards streams when a buffering policy is configured. It enforces declared content lengths and closes response bodies and provider sources. Tests cover dispatch failures, stream validation, errors, cancellation, and cleanup.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant execute_streaming_http
  participant dispatch
  participant checker
  Client->>execute_streaming_http: Submit request
  execute_streaming_http->>dispatch: Dispatch prepared request
  dispatch-->>execute_streaming_http: Return streaming response
  opt Buffering policy configured
    execute_streaming_http->>checker: Guard provider stream
  end
  execute_streaming_http-->>Client: Return streaming response

Loading

---

Merge Risk: ⚪ Minimal · up to fe740

No confirmed issue prevents merging after normal checks.

🚥 Pre-merge checks | ✅ 5 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 5.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 80 functions across 3 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly and concisely describes the main change: connecting guarded streaming responses to the HTTP lifecycle.
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Test Results For Major Changes		The PR is major: it adds a streaming HTTP feature and 857 changed lines, including 602 test lines. The description documents testing information for headers, framing, closure, limits, lengths, SSE par…

---

---

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Commit to this branch
• Create a new PR

---

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

---

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

_{Comment @coderabbitai help to get the list of available commands.}

[greptile-apps]

Identity encoding against client preference If a streaming client explicitly forbids an identity response, for example with Accept-Encoding: gzip, identity;q=0, this line still asks the provider for identity. The inspected stream is then returned without a content encoding, so the client may receive a response it said it cannot accept. Reject that request or negotiate an acceptable downstream encoding.

Prompt To Fix With AI

This is a comment left during a code review.
Path: nemoguardrails/server/experimental/_streaming_http.py
Line: 176

Comment:
**Identity encoding against client preference** If a streaming client explicitly forbids an identity response, for example with `Accept-Encoding: gzip, identity;q=0`, this line still asks the provider for identity. The inspected stream is then returned without a content encoding, so the client may receive a response it said it cannot accept. Reject that request or negotiate an acceptable downstream encoding.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.