Route /v1/models to ai-gateway-discovery for per-key filtering - #799
Closed
johnl-amd wants to merge 2 commits into
Closed
Route /v1/models to ai-gateway-discovery for per-key filtering#799johnl-amd wants to merge 2 commits into
johnl-amd wants to merge 2 commits into
Conversation
Envoy AI Gateway's built-in /v1/models returns every model to any valid key. Add an exact-path GET /v1/models HTTPRoute on the ai-gateway that routes the call to ai-gateway-discovery's models handler, which returns only the models the caller's key can use. A more-specific HTTPRoute overrides the gateway's built-in handling; the call stays authenticated by the existing gateway-scoped default-deny + ext_authz. Includes the cross-namespace ReferenceGrant for the discovery Service backendRef. Both render only under aiGateway.enabled + routeHostname, so they ship and retract atomically with the rest of the AI gateway.
The gateway-scoped ai-gateway-default-deny applies both extAuth and an authorization RBAC deny to every route on the ai-gateway. The static /v1/models HTTPRoute has no per-model SecurityPolicy to override it, so a valid key passed extAuth but was then refused by the RBAC deny (HTTP 403, verified on app-dev). This route-scoped policy overrides the gateway one for /v1/models: it keeps extAuth (missing/invalid key still 401'd, valid key forwarded with x-api-key-id) but carries no authorization block, so the request reaches the models handler. Verified end-to-end on app-dev: a key bound to one of two deployed models now gets a 200 filtered to that model.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
GET /v1/modelsHTTPRoute on theai-gatewayGateway that routes the OpenAI model-list call toai-gateway-discovery's new per-key models handler (Service:8082), plus the cross-namespace ReferenceGrant for that backendRef./v1/modelsaggregator, so the caller gets a list scoped to the models their key can use instead of the whole fleet.Why: the built-in
/v1/modelsreturns every declared model to any valid key. The companionai-gateway-discoverychange serves a per-key list; this routes the endpoint to it.Non-obvious decisions:
ai-gateway-default-denySecurityPolicy + body-aware ext_authz (which target the wholeai-gatewayGateway) — no new auth wiring needed; the handler always sees a valid key.aiGateway.enabled+routeHostname, so they ship and retract atomically with the rest of the AI gateway.8082is hardcoded (like the ext_authz8083in the default-deny SP) and must match the discovery chart'scontroller.modelsPort.Base branch
Targets `bump_version_envoy_and_ai_gateway` — the AI-gateway templates (`ai-gateway` Gateway, default-deny SP) live on that line, not main.
Risk
Low. Gated behind `aiGateway.enabled`; no change to existing routes/policies.
Test plan
Dependencies