feat(server): integrate OpenAI Chat streaming

[Pouyanpi]

Summary

Integrates the preceding OpenAI Chat stream modules and HTTP bridge with the existing guarded endpoint. Buffered and streaming requests share one input projection and check.

What Changed

• Selects buffered or streaming response handling after one shared request preparation and input check.
• Binds the OpenAI stream adapter through the endpoint and preserves request target state across preparation and dispatch.
• Keeps the shared contract export buffered-only while binding both buffered and streaming response handling.
• Extends the OpenAI error mapping for routing failures, pre-header errors, in-stream failures, and OpenAPI documentation.
• Covers native route/framing rejection, unsafe policies, hidden blocked content, source closure, and buffered regressions.
• Documents both JSON and SSE success responses for the guarded Chat endpoint.

Review Notes

Review target-state continuity, one input check, request-local hooks, error disclosure, and buffered regressions. The stream projection, classifier, and hooks are supplied by part 2; the HTTP bridge is supplied by part 3. Concrete outbound HTTP, application configuration, live recordings, and replacement execution remain separate work.

AI Assistance

• No AI tools were used.
• AI tools were used (Codex assisted with commit reconstruction, HTTP/error reconciliation, regression tests, and PR drafting). Human review of the new diff is pending.

Checklist

• I've read the CONTRIBUTING guidelines.
• This is maintainer-directed work.
• The PR title follows the project commit convention.
• Applicable declarations and tests are included.
• Generated changelog files were not edited manually.
• Automated and human review comments are addressed or answered.
• The responsible reviewer or team is mentioned.

Stack Position

Part 4 of 4.

• Previous: feat(server): bind guarded streams to HTTP #2436
• Next: none, final stack PR

Stack Context

Adds OpenAI Chat streaming above #2414 in four parts: provider-neutral execution, typed OpenAI stream declarations and handwritten hooks, injected HTTP lifecycle handling, and integration with the buffered request pipeline.

The open chain is #2411 → #2413 → #2414 → #2434 → #2435 → #2436 → #2437. #2412 is closed; its provider provenance and boundary documentation are included in #2413.

Python declarations own field and event policy. Hooks are handwritten and created for each stream by the integration. The shared exporter currently emits buffered policy; streaming contract export remains separate work. Concrete outbound HTTP, deployment configuration, and replacement execution land separately.

Review each PR against its listed base branch.

Order	PR	Branch	Base
1	#2434	pouyanpi/transparent-proxy-streaming-kernel-1	pouyanpi/openai-chat-buffered-integration-4
2	#2435	pouyanpi/openai-chat-streaming-contract-2	pouyanpi/transparent-proxy-streaming-kernel-1
3	#2436	pouyanpi/transparent-proxy-streaming-http-3	pouyanpi/openai-chat-streaming-contract-2
4	#2437	pouyanpi/openai-chat-streaming-integration-4	pouyanpi/transparent-proxy-streaming-http-3

Summary by CodeRabbit

• New Features
  • Added streaming support for OpenAI Chat Completions, including request and response inspection, with streaming routes documented in the API.
  • Streaming requests can use an optional streaming dispatcher; requests that require unsupported output inspection are rejected before dispatch.
• Bug Fixes
  • Unsupported stream inspection and upstream stream failures now return clear, provider-compatible error responses instead of uncaught errors.

[codecov]

Codecov Report

❌ Patch coverage is 97.40260% with 4 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...moguardrails/server/experimental/_guarded_proxy.py	96.77%	1 Missing ⚠️
...ls/server/experimental/provider/contract_export.py	98.11%	1 Missing ⚠️
...oguardrails/server/experimental/provider/stream.py	93.75%	1 Missing ⚠️
...ils/server/experimental/providers/openai/errors.py	85.71%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge; no new actionable issue or outstanding previous finding was identified.

Summary

The PR connects the OpenAI Chat stream adapter to the guarded HTTP endpoint while retaining buffered handling through a shared input check.

• Adds provider-native stream error handling and documents JSON and SSE responses.
• Exports the bound stream classifier alongside buffered contract policy.
• The latest changes refine stream binding, keepalive delivery, and bounded cleanup.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Chat request] --> B[Prepare and project input]
  B --> C[Input check]
  C -->|Stopped| D[Render guarded error]
  C -->|Allowed| E{Streaming requested?}
  E -->|No| F[Buffered dispatch and output check]
  E -->|Yes| G[Create request-local stream hooks]
  G --> H[Streaming HTTP dispatch]
  H --> I[Classify and check SSE events]
  I --> J[Release permitted events]

Loading

_{Reviews (6) · Last reviewed commit: "feat(server): export bound streaming gua..." · Reviewed by Greptile}

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Repository: NVIDIA-NeMo/Guardrails/.coderabbit.yaml
• Review profile: CHILL
• Plan: Enterprise
• Run ID: 049cda59-b0d1-4fc1-a476-6524dbdc815c

---

📥 Commits

Reviewing files that changed from the base of the PR and between fe7408e and 6269303.

---

📒 Files selected for processing (13)

• nemoguardrails/server/experimental/_buffered_kernel.py
• nemoguardrails/server/experimental/_guarded_proxy.py
• nemoguardrails/server/experimental/_guarded_stream.py
• nemoguardrails/server/experimental/_http_kernel.py
• nemoguardrails/server/experimental/_streaming_http.py
• nemoguardrails/server/experimental/provider/endpoint.py
• nemoguardrails/server/experimental/provider/errors.py
• nemoguardrails/server/experimental/providers/openai/chat_completions/README.md
• nemoguardrails/server/experimental/providers/openai/chat_completions/endpoint.py
• nemoguardrails/server/experimental/providers/openai/errors.py
• nemoguardrails/server/experimental/providers/openai/integration.py
• tests/server/experimental/test_openai_chat_streaming_integration.py
• tests/server/experimental/test_streaming_http.py

---

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 9 remain after this review.

---

---

📝 Walkthrough

Walkthrough

The guarded HTTP pipeline now separates request preparation from execution. OpenAI Chat Completions can route requests through streaming dispatch when configured, while retaining buffered dispatch for non-streaming requests.

Changes

Guarded streaming HTTP

Layer / File(s)	Summary
Shared request preparation and execution nemoguardrails/server/experimental/_buffered_kernel.py, nemoguardrails/server/experimental/_http_kernel.py	Input preparation now returns reusable prepared input. The HTTP handler can pass it to a prepared-request handler or continue with buffered execution.
Streaming adapter and guarded execution nemoguardrails/server/experimental/provider/endpoint.py, nemoguardrails/server/experimental/_guarded_stream.py, nemoguardrails/server/experimental/_guarded_proxy.py, nemoguardrails/server/experimental/_streaming_http.py, nemoguardrails/server/experimental/provider/errors.py	Endpoints can create and validate stream adapters. The guarded proxy routes prepared streaming requests to streaming execution when configured. Unsupported stream inspection is rendered as an outcome.
OpenAI Chat Completions streaming wiring nemoguardrails/server/experimental/providers/openai/chat_completions/endpoint.py, nemoguardrails/server/experimental/providers/openai/errors.py, nemoguardrails/server/experimental/providers/openai/integration.py, nemoguardrails/server/experimental/providers/openai/chat_completions/README.md	The endpoint binds a stream adapter, and the router accepts optional stream dispatch. The error renderer maps stream outcomes to OpenAI-compatible errors. The README describes runtime support and the buffered-only contract export.
Streaming behavior and routing tests tests/server/experimental/test_openai_chat_streaming_integration.py, tests/server/experimental/test_streaming_http.py	Tests cover dispatch selection, stream inspection, error handling, route and request-framing validation, adapter creation, and OpenAPI response media types.

Priority: ⬇️ Low

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant GuardedHttpOperation
  participant prepare_operation_input
  participant prepared_request_handler
  participant execute_streaming_http
  participant StreamingHttpDispatch
  Client->>GuardedHttpOperation: Send request
  GuardedHttpOperation->>prepare_operation_input: Project and check input
  prepare_operation_input-->>GuardedHttpOperation: Return prepared input
  GuardedHttpOperation->>prepared_request_handler: Select response mode
  prepared_request_handler->>execute_streaming_http: Execute streaming request
  execute_streaming_http->>StreamingHttpDispatch: Dispatch provider request

Loading

---

Merge Risk: ⚪ Minimal · up to 62693

No identified streaming integration issue blocks merging after normal checks.

🚥 Pre-merge checks | ✅ 5 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 68 functions across 12 files. (1 skipped:…	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly and concisely describes the main change: integrating OpenAI Chat streaming into the server.
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Test Results For Major Changes		The PR is a major feature change, but its description includes testing information. It states that coverage includes route/framing rejection, unsafe policies, hidden blocked content, source closure, a…

---

Full details: Docstring Coverage

Explanation

Docstring coverage is 25.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 68 functions across 12 files. (1 skipped: 1 unsupported.)

---

---

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Commit to this branch
• Create a new PR

---

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

---

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

_{Comment @coderabbitai help to get the list of available commands.}