Skip to content

feat(server): define OpenAI Chat contract - #2412

Closed
Pouyanpi wants to merge 3 commits into
pouyanpi/guard-contract-foundation-1from
pouyanpi/openai-chat-contract-2
Closed

Pouyanpi wants to merge 3 commits into
pouyanpi/guard-contract-foundation-1from
pouyanpi/openai-chat-contract-2

Conversation

@Pouyanpi

@Pouyanpi Pouyanpi commented Sep 26, 2026 •

Copy link
Copy Markdown
Collaborator

Superseded

This standalone PR is folded into #2413. Its remaining OpenAI source pin, boundary documentation, and provenance tests are already included there; #2413 now targets #2411 directly.

The authoritative Python models remain in #2413. The buffered contract is exported from the endpoint-bound models in #2414 at contracts/openai/_generated/chat-completions.buffered.guard.yaml; it is not a separately maintained YAML policy. Streaming and its contract export remain separate work.

No branch or commits are deleted. The review discussion is preserved. The logprobs, explicit plain-text/no-tools values, conversation-cardinality, and streaming-null questions are linked from #2413 and are not marked resolved by this consolidation.

Revised Stack

The Python projection models and runtime bindings are authoritative. This stack makes their policy explicit through typed declarations and exports a guard contract for inspection and documentation, not runtime configuration or a separately maintained policy.

#2412's remaining provider provenance, boundary documentation, and metadata tests are folded into #2413. Streaming implementation and its contract export are outside this buffered stack.

Review each PR against its parent branch; only #2411 targets develop.

Order PR Branch Base
1 #2411 pouyanpi/guard-contract-foundation-1 develop
2 #2413 pouyanpi/openai-chat-buffered-projections-3 pouyanpi/guard-contract-foundation-1
3 #2414 pouyanpi/openai-chat-buffered-integration-4 pouyanpi/openai-chat-buffered-projections-3

AI Assistance

  • No AI tools were used.
  • AI tools were used; a human reviewed and can explain every change.

Checklist

  • I've read the CONTRIBUTING guidelines.
  • This is maintainer-led work tracked internally.
  • The PR title follows the project commit convention.
  • Applicable documentation and tests are included.
  • Generated changelog files were not edited manually.
  • Automated and human review comments are addressed or answered.
  • The responsible reviewer or team is mentioned.

@github-actions github-actions Bot added the status: needs triage New issues that have not yet been reviewed or categorized. label Sep 26, 2026
@Pouyanpi Pouyanpi self-assigned this Sep 26, 2026
@Pouyanpi Pouyanpi added this to the v0.25.0 milestone Sep 26, 2026
@Pouyanpi Pouyanpi added status: triaged Triaged by a maintainer; eligible for automated review (CodeRabbit/Greptile). and removed status: needs triage New issues that have not yet been reviewed or categorized. labels Sep 26, 2026
@Pouyanpi
Pouyanpi marked this pull request as ready for review September 26, 2026 20:34
@Pouyanpi Pouyanpi added status: triaged Triaged by a maintainer; eligible for automated review (CodeRabbit/Greptile). and removed status: triaged Triaged by a maintainer; eligible for automated review (CodeRabbit/Greptile). labels Sep 26, 2026
@greptile-apps

greptile-apps Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

[Low risk] Documents OpenAI Chat API contract and adds validation test.

The PR is not ready to merge as stacked because the later streaming PR still requires the contract removed here.

Findings

  1. P1 Streaming contract is missing ▶
Fix with agent prompt
### Issue 1
nemoguardrails/server/experimental/contracts/openai/README.md:31-33
This revision removes `chat-completions.guard.yaml` and says the machine-readable contract belongs with the integration, but PR #2435 modifies that file rather than adding it. Its streaming test reads the file directly, so stacking that PR on this head leaves the file missing and the test fails. The buffered export in PR #2414 does not provide the stream section the test needs.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

The PR pins OpenAI Chat provider-source metadata and documents a narrow single-text boundary, replacing the previously authored operation contract with a descriptive README and offline provenance tests.

  • The revised boundary no longer supplies the streaming contract expected by a higher PR in the stack.

Reviews (7) · Last reviewed commit: "docs(server): align OpenAI contract docs..." · Reviewed by Greptile

Comment thread tests/server/experimental/test_openai_chat_contract.py Outdated
@Pouyanpi
Pouyanpi force-pushed the pouyanpi/openai-chat-contract-2 branch from 99dc816 to 8dc08bf Compare September 26, 2026 20:53
@codecov

codecov Bot commented Sep 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@Pouyanpi
Pouyanpi force-pushed the pouyanpi/guard-contract-foundation-1 branch from 66e9cb1 to a95dc3b Compare October 5, 2026 12:58
@Pouyanpi
Pouyanpi force-pushed the pouyanpi/openai-chat-contract-2 branch 2 times, most recently from 7e78b93 to 9cdf959 Compare October 5, 2026 14:58
@Pouyanpi
Pouyanpi force-pushed the pouyanpi/guard-contract-foundation-1 branch from a95dc3b to bb80c4e Compare October 5, 2026 14:58
@Pouyanpi
Pouyanpi force-pushed the pouyanpi/openai-chat-contract-2 branch from 9cdf959 to 138d75a Compare October 6, 2026 11:05
Signed-off-by: Pouyanpi <13303554+Pouyanpi@users.noreply.github.com>
Signed-off-by: Pouyanpi <13303554+Pouyanpi@users.noreply.github.com>
@Pouyanpi
Pouyanpi force-pushed the pouyanpi/guard-contract-foundation-1 branch from afe73c2 to d565e48 Compare October 7, 2026 08:54
@Pouyanpi
Pouyanpi force-pushed the pouyanpi/openai-chat-contract-2 branch from 138d75a to 91bea76 Compare October 7, 2026 09:00

@tgasser-nv tgasser-nv left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, marking as comment only until the YAML format question in #2411 is answered

index:
x-nemo-guardrails:
classification: opaque
logprobs:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR allows logprobs on the request-side but disable the logprobs response from the model. This is true for non-streaming and streaming. Recommend constraining logprobs with const: false and making top_logprobs a null-only disabled gate

gate: disabled
extension: true
content:
type: string

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't Streaming content could be null as well?

type: array
x-nemo-guardrails:
classification: guarded
modalities:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The request projection rejects values that this supports:

  • modalities: ["text"] (lines 61-67)
  • response_format: {"type": "text"} (88-94)
  • tool_choice: "none" (100-106)
  • parallel_tool_calls: false (74-80)

Recommend declaring these as constrained with the expected value allowed

x-nemo-guardrails:
unknown_fields: configurable
source: '#/components/schemas/ChatCompletionRequestUserMessage'
maxItems: 1

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would this block a request with both system/developer and user entries?

Merge the updated contract foundation without rewriting the stack. Replace the handwritten full-operation YAML with a boundary summary and provider provenance checks; integration exports remain separate.

Signed-off-by: Pouyanpi <13303554+Pouyanpi@users.noreply.github.com>
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown
Contributor

PR merge guidance

@Pouyanpi thanks for the PR. GitHub is currently blocking merge for one or more repository requirements:

  • 1 commit does not have a verified signature (4e59192). Please sign the commits and force-push the updated branch.

Relevant guide:

@github-actions github-actions Bot removed the size: L label Oct 8, 2026
Comment on lines +31 to +33
The Python projections, bindings, and endpoint determine exact acceptance and
runtime behavior. Their machine-readable contract belongs with the integration,
not in a separately maintained handwritten policy here. Recognizing the request

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Streaming contract is missing

This revision removes chat-completions.guard.yaml and says the machine-readable contract belongs with the integration, but PR #2435 modifies that file rather than adding it. Its streaming test reads the file directly, so stacking that PR on this head leaves the file missing and the test fails. The buffered export in PR #2414 does not provide the stream section the test needs.

Prompt To Fix With AI
This is a comment left during a code review.
Path: nemoguardrails/server/experimental/contracts/openai/README.md
Line: 31-33

Comment:
**Streaming contract is missing**

This revision removes `chat-completions.guard.yaml` and says the machine-readable contract belongs with the integration, but PR #2435 modifies that file rather than adding it. Its streaming test reads the file directly, so stacking that PR on this head leaves the file missing and the test fails. The buffered export in PR #2414 does not provide the stream section the test needs.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

@Pouyanpi

Pouyanpi commented Oct 8, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by #2413, which now includes this PR's remaining OpenAI provenance, boundary documentation, and metadata tests and targets #2411 directly. The endpoint-bound buffered contract export remains in #2414. The existing branch and commits are preserved. Review concerns about logprobs, explicit plain-text/no-tools values, message cardinality, and nullable stream content are linked from #2413 and are not resolved by closing this standalone PR. Revised stack: #2411 → #2413 → #2414.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs: signing size: M status: triaged Triaged by a maintainer; eligible for automated review (CodeRabbit/Greptile).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants