feat(server): execute guarded provider streams

[Pouyanpi]

Summary

Adds provider-neutral SSE parsing, event classification, bounded buffering, and guarded stream execution. Content-bearing events are checked before release, and allowed events retain their original bytes.

What Changed

• Declares stream roles, shape coverage, adapters, and declarative event classifiers.
• Frames mixed SSE line endings incrementally, preserves raw bytes, and handles one initial BOM.
• Rejects unfinished field blocks and drops unfinished comment tails; error hooks must encode complete SSE blocks.
• Bounds events and unchecked pending bytes, verifies snapshots, and rejects modification requests.
• Keeps the async-generator interface and closes upstream on iteration end or rejected arguments; callers own cleanup if iteration never starts.
• Requires matching discriminator value types, consistent with buffered projections.

Review Notes

Review release ordering, ambiguity, bounds, byte preservation, and cancellation. HTTP dispatch, provider grammar, and request preparation are supplied by later PRs.

AI Assistance

• No AI tools were used.
• AI tools were used (Codex assisted with commit reconstruction, HTTP/error reconciliation, regression tests, and PR drafting). Human review of the new diff is pending.

Checklist

• I've read the CONTRIBUTING guidelines.
• This is maintainer-directed work.
• The PR title follows the project commit convention.
• Applicable declarations and tests are included.
• Generated changelog files were not edited manually.
• Automated and human review comments are addressed or answered.
• The responsible reviewer or team is mentioned.

Stack Position

Part 1 of 4.

• Previous: none, based on pouyanpi/openai-chat-buffered-integration-4
• Next: feat(server): add OpenAI Chat stream projections and hooks #2435

Stack Context

Adds OpenAI Chat streaming above #2414 in four parts: provider-neutral execution, typed OpenAI stream declarations and handwritten hooks, injected HTTP lifecycle handling, and integration with the buffered request pipeline.

The open chain is #2411 → #2413 → #2414 → #2434 → #2435 → #2436 → #2437. #2412 is closed; its provider provenance and boundary documentation are included in #2413.

Python declarations own field and event policy. Hooks are handwritten and created for each stream by the integration. The shared exporter currently emits buffered policy; streaming contract export remains separate work. Concrete outbound HTTP, deployment configuration, and replacement execution land separately.

Review each PR against its listed base branch.

Order	PR	Branch	Base
1	#2434	pouyanpi/transparent-proxy-streaming-kernel-1	pouyanpi/openai-chat-buffered-integration-4
2	#2435	pouyanpi/openai-chat-streaming-contract-2	pouyanpi/transparent-proxy-streaming-kernel-1
3	#2436	pouyanpi/transparent-proxy-streaming-http-3	pouyanpi/openai-chat-streaming-contract-2
4	#2437	pouyanpi/openai-chat-streaming-integration-4	pouyanpi/transparent-proxy-streaming-http-3

Summary by CodeRabbit

• New Features
  • Added support for inspecting provider-streamed responses before releasing buffered content, with configurable chunk and context sizes.
  • Unsafe or uninspectable streamed content can be withheld, while provider errors and stream-processing failures are handled as distinct outcomes.
  • Added support for parsing streamed events across varied chunk boundaries and validating provider event formats.

[codecov]

Codecov Report

❌ Patch coverage is 91.19850% with 47 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
.../server/experimental/provider/stream_classifier.py	79.62%	33 Missing ⚠️
...oguardrails/server/experimental/_guarded_stream.py	96.31%	6 Missing ⚠️
...oguardrails/server/experimental/provider/stream.py	93.25%	6 Missing ⚠️
...guardrails/server/experimental/_content_checker.py	94.11%	1 Missing ⚠️
nemoguardrails/server/experimental/provider/sse.py	99.02%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge based on the reviewed stream behavior.

Summary

The PR adds provider-neutral SSE framing, stream-event classification, and guarded execution that checks buffered text before releasing its original provider bytes. Since the previous review, it also releases validated data-less keepalives promptly when no payload is pending and clarifies the input-context invariant. No new actionable issue was identified.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Provider bytes] --> B[SSE framing]
  B --> C[Classify and observe event]
  C -->|Data-less opaque event; nothing pending| D[Release original bytes]
  C -->|Guarded text| E[Buffer text window]
  E --> F[Output check]
  F -->|Allowed| G[Release original event bytes]
  F -->|Blocked or failed| H[Encode terminal outcome]

Loading

_{Reviews (6) · Last reviewed commit: "docs(server): state the single user inpu..." · Reviewed by Greptile}

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Repository: NVIDIA-NeMo/Guardrails/.coderabbit.yaml
• Review profile: CHILL
• Plan: Enterprise
• Run ID: 78bd4143-092c-4fcc-818f-56da00e60ccc

---

📥 Commits

Reviewing files that changed from the base of the PR and between 4512d08 and 558d602.

---

📒 Files selected for processing (9)

• nemoguardrails/server/experimental/_content_checker.py
• nemoguardrails/server/experimental/_guarded_stream.py
• nemoguardrails/server/experimental/provider/sse.py
• nemoguardrails/server/experimental/provider/stream.py
• nemoguardrails/server/experimental/provider/stream_classifier.py
• tests/server/experimental/test_content_checker.py
• tests/server/experimental/test_guarded_stream.py
• tests/server/experimental/test_sse.py
• tests/server/experimental/test_stream_classifier.py

---

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 8 remain after this review.

---

---

📝 Walkthrough

Walkthrough

This change adds configuration for buffering provider streams, parses and classifies server-sent events, and checks buffered text before releasing its original events. It also adds validation for stream contracts, limits, snapshots, failure outcomes, and source cleanup.

Changes

Guarded provider stream inspection

Layer / File(s)	Summary
Stream buffering policy nemoguardrails/server/experimental/_content_checker.py, tests/server/experimental/test_content_checker.py	Adds validated chunk and context sizes, a release setting, and optional buffering configuration on ContentInspectionPolicy. Tests cover valid and invalid policy values.
Byte-preserving SSE parsing nemoguardrails/server/experimental/provider/sse.py, tests/server/experimental/test_sse.py	Adds event framing across byte chunks, preserves raw event bytes, exposes parsed data and event fields, and handles size limits and truncated input. Tests cover line endings, BOMs, comments, and large inputs.
Provider stream contracts and classification nemoguardrails/server/experimental/provider/stream.py, nemoguardrails/server/experimental/provider/stream_classifier.py, tests/server/experimental/test_stream_classifier.py	Adds event roles, projection contracts, adapter protocols, and rule-based classification. Tests cover recognized events, text selection, discriminator types, and contract mismatches.
Guarded stream buffering and release nemoguardrails/server/experimental/_guarded_stream.py, tests/server/experimental/test_guarded_stream.py	Adds event validation, buffered text-window checks, snapshot history checks, and outcome rendering. Tests cover event release, blocked output, error handling, limits, and source closure.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant Source as Async byte source
  participant Guard as guard_provider_stream
  participant Parser as iter_sse_events
  participant Adapter as ProviderStreamAdapter
  participant Checker as ContentChecker
  participant Renderer as StreamOutcomeRenderer
  Source->>Guard: yield byte chunks
  Guard->>Parser: pass source chunks
  Parser-->>Guard: return parsed SSE events
  Guard->>Adapter: classify and validate events
  Adapter-->>Guard: return guarded stream events
  Guard->>Checker: check buffered text windows
  Checker-->>Guard: return check result
  Guard->>Renderer: render blocked or failed outcome
  Guard-->>Source: close async source

Loading

---

Merge Risk: ⚪ Minimal · up to 558d6

This change adds experimental guarded stream parsing and release logic that nothing yet uses, and no concrete merge-blocking issue was found.

🚥 Pre-merge checks | ✅ 5 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 25.24% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 103 functions across 9 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (5 passed)

Check name	Status	Explanation
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.
Test Results For Major Changes		The PR contains major new stream execution and parsing features, but the description includes testing information: it states that applicable tests are included and notes that regression tests were par…
Description Check		Check skipped - CodeRabbit’s high-level summary is enabled.
Title check		The title clearly identifies the main change: adding guarded provider-stream execution. It is concise and directly related to the pull request objectives and implementation.

---

---

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Commit to this branch
• Create a new PR

---

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

---

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

_{Comment @coderabbitai help to get the list of available commands.}