Skip to content

feat(vgr): add bounded verification runtime - #718

Open
gburachas wants to merge 1 commit into
vgr/review/02-policyfrom
vgr/review/03-runtime
Open

gburachas wants to merge 1 commit into
vgr/review/02-policyfrom
vgr/review/03-runtime

Conversation

@gburachas

@gburachas gburachas commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

What

Adds the executable Vgr routing algorithm, connecting capability derivation from #716 and decision policy from #717 to model calls.

The runtime buffers a local attempt, gathers verification evidence, and either returns that response or routes to the capable tier. Local generation and verification share one deadline, defaulting to 30 seconds.

Why

Verification needs to finish before a local response reaches the caller. This runtime controls that boundary, limits verification time, and handles endpoint failures. It also distinguishes final-answer verification from judging whether an ongoing tool-using session needs escalation.

Notes for reviewers

Review this against #717. Start with Vgr::new and Algorithm::route in crates/libsy/src/algorithms/vgr.rs, then follow VgrClassifier::score in vgr/runtime.rs.

  • Existing APIs: Composes FallThrough<State>, the Classifier interface, Driver::call_model, and AffinityRouter. It reuses buffer_response to inspect a completed response and replay its original stream events, preserving provider metadata rather than reconstructing the stream.
  • Final answers: Verification starts with the local probability readout, then deliberation, followed by capable-tier checks where the policy allows them. It stops once acceptance is established. Failed verifier calls supply no affirmative evidence.
  • Tool calls: Complete tool calls follow a separate trajectory check. Two escalation votes at or above 0.5 latch the session to the capable tier. Missing or inconclusive trajectory scores allow local continuation; local-judge endpoint failures escalate. Truncated calls and string-valued arguments also escalate.
  • Fallback and state: Affinity retains capable-tier routing between user turns. The trajectory latch is separate and persists across user turns. Once cloud is selected, the fallback list excludes local so a downstream failure cannot bypass that decision.
  • Controls: The default mode is off. Shadow runs verification but serves cloud; evaluate serves decisions; active requires an approval attestation. A kill switch, local-endpoint circuit breaker, and image-support check can bypass local generation. Eligible local failures escalate; other errors propagate.
  • Tests: Use the existing test_drive_with_models harness to cover tool-call handling, session latching, truncated-attempt verification, recovery confirmation, and local failure handling.

This PR exposes the Rust algorithm and configuration types. #719 adds the runner configuration and public integration.

@gburachas
gburachas requested a review from a team as a code owner September 16, 2026 02:07
@github-actions

github-actions Bot commented Sep 16, 2026 •

Copy link
Copy Markdown
PR Preview Action v1.8.1

🚀 View preview at
https://NVIDIA-NeMo.github.io/Switchyard/pr-preview/pr-718/

Built to branch gh-pages at 2026-09-30 19:22 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@afourniernv afourniernv reopened this Sep 27, 2026
Signed-off-by: adhaile <adhaile@nvidia.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants