gateway: block bifrost realtime routes at the wrapper (default off) - #1695
Merged
Merged
Conversation
Bifrost's realtime WebSocket / WebRTC handlers dial the upstream provider — with the account's real API key — during connection setup, which runs before the per-turn virtual-key check. So an unauthenticated caller who just opens `wss://<gateway>/v1/realtime?model=gpt-realtime` makes the gateway open a provider socket on the org's account (verified on swarm38: 101 upgrade + a genuine OpenAI `session.created`). The mandatory-VK check still blocks token generation (`response.create` → 401), so this is not free inference, but it is an unauthenticated upstream-connection / quota-exhaustion / key-validity vector, and the gateway's macaroon plugin never sees realtime at all (it hooks PreLLMHook; realtime runs its own turn pipeline). Refuse the whole realtime family at the wrapper, before it reaches bifrost-http, so the provider socket is never dialed. Match on the `realtime` path segment so every prefix variant is covered (`/realtime`, `/v1/realtime`, `/openai/**/realtime`, and their `/calls`, `/client_secrets`, `/sessions` subpaths). Hive's agents are text-only; set BIFROST_ENABLE_REALTIME=1 to opt back in. Split the routing decision into newRouter() so it is unit-tested with fake upstreams (no live loopback dial). Blocked requests get 403 + a JSON body and are logged for abuse visibility.
`gateway/wrapper/wrapper` is a compiled Go binary (~8.7MB, host-specific) that was checked into the repo. It is a build output, not source: the Docker image builds the wrapper fresh in the Dockerfile's wrapper-builder stage (`go build -o /out/wrapper .`) and never uses the committed copy. Any local `go build ./...` in wrapper/ overwrites it and produces a spurious multi-MB diff. Remove it from tracking and gitignore `wrapper/wrapper` so a local build no longer shows up as a change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two commits:
gateway/wrapper/wrapperbuild artifact.1. Block realtime routes
What
Blocks bifrost's realtime route family at the wrapper, before it can reach
bifrost-http. Off by default; opt back in withBIFROST_ENABLE_REALTIME=1.Why
Bifrost's realtime WebSocket / WebRTC handlers dial the upstream provider — with the account's real API key — during connection setup, which runs before the per-turn virtual-key check. So an unauthenticated caller who just opens a socket makes the gateway open a provider connection on the org's account.
Verified live on swarm38 (public
:8181, port-based SSL):Sending
response.createon that socket returns401 virtual key is required, so this is not free token generation — the mandatory-VK gate (enforce_auth_on_inference: true) still holds. But it is an unauthenticated upstream-connection / concurrent-session-quota-exhaustion / key-validity-oracle vector, and it is independent ofenforce_macaroons(the gateway's macaroon plugin hooksPreLLMHook; realtime runs its own turn pipeline and never reaches it).This is upstream bifrost behavior (
transports/v1.6.2), so the fix lives in our wrapper rather than waiting on an upstream change.How
isRealtimePathmatches on therealtimepath segment, so every prefix variant bifrost registers is covered:/realtime,/v1/realtime,/openai/realtime,/openai/v1/realtime, and the/calls,/client_secrets,/sessionssubpaths. Exact-segment match means/v1/realtimelessis not blocked.403+ a small JSON body and a log line (abuse visibility). Because the wrapper returns before proxying,bifrost-http's realtime handler never runs and no provider socket is dialed.BIFROST_ENABLE_REALTIME(truthy:1/true/yes/on) re-enables the routes if a swarm ever needs voice. Hive's agents are text-only, so default-off is safe.newRouter()so it is unit-tested against fake upstreams (no live loopback dial).2. Remove the tracked wrapper binary
gateway/wrapper/wrapper(~8.7MB, host-specific compiled Go binary) was checked into the repo. It's a build output, not source — the Docker image builds the wrapper fresh in thewrapper-builderstage (go build -o /out/wrapper .) and never uses the committed copy. Any localgo build ./...overwrites it and produces a spurious multi-MB diff (it did while building this PR). Removed from tracking and addedwrapper/wrappertogateway/.gitignore.Test
go test ./...ingateway/wrapperpasses, including the newrealtime_test.go:isRealtimePathblocks the full realtime family (all prefixes, case-insensitive) and does not misfire on substrings.realtimeEnabledenv parsing.newRouter: realtime blocked → 403 and bifrost upstream never hit; realtime allowed when enabled → reaches bifrost; normal inference and/_plugin/*unaffected; no-plugin-server → 503.Also confirmed a local
go build .inwrapper/no longer shows up ingit status(ignored).Notes