diff --git a/docs-site/src/content/docs/fr/reference/configuration/providers.md b/docs-site/src/content/docs/fr/reference/configuration/providers.md index ca22b571c3..ed6658c5e3 100644 --- a/docs-site/src/content/docs/fr/reference/configuration/providers.md +++ b/docs-site/src/content/docs/fr/reference/configuration/providers.md @@ -68,6 +68,7 @@ sauvegarde dont le contenu diffère, puis réécrit en identifiants sans préfix | `baseUrl` | `string` | URL de base de l'API en amont. La plupart des points de terminaison fixes intégrés ignorent une valeur incompatible ; les préréglages de clés protégés contre les collisions préservent une ancienne destination personnalisée portant le même nom. | | `requestPacing?` | `{ enabled, requestsPerMinute?, minIntervalMs?, models? }` | Cadencement facultatif du démarrage des requêtes sortantes côté client, distinct de l’utilisation, de la facturation et des indicateurs de limitation en amont. Le nombre de requêtes par minute est converti en intervalle régulier ; `minIntervalMs` peut imposer un intervalle plus long. Les limites du fournisseur s’appliquent à tous ses modèles, tandis que les entrées `models` ciblent les identifiants exacts des modèles en amont, par exemple `nvidia/llama-3.1-nemotron-ultra-253b-v1`, et ne peuvent qu’ajouter du délai. L’attente dans la file ne consomme pas le délai d’expiration des en-têtes de réponse en amont. Les requêtes HTTP, Responses WebSocket et les distributions explicites `fetchResponse`/`runTurn` des adaptateurs sont couvertes. | | `responsesPath?` | `string` | Chemin de ressource relatif pour les requêtes d'authentification par clé `openai-responses`. Il doit commencer par `/` et ne contenir aucun schéma, requête ou fragment. | +| `upstreamWebsocket?` | `boolean` | Active le transport Responses WebSocket en amont pour les requêtes `openai-responses` (désactivé par défaut). Lorsque le service en amont prend en charge ce protocole, les requêtes POST en streaming utilisent le chemin Responses configuré (par défaut `/v1/responses`) via WSS avec une base HTTPS, puis sont reconverties en SSE. Les fournisseurs en mode forward utilisent `{baseUrl}/responses` ; les fournisseurs avec clé utilisent `responsesPath`, ou le repli historique `/v1/responses`. Une base HTTP reste en SSE ; les chemins qui ne sont pas Responses et les requêtes `openai-chat` restent en HTTP. | | `supportsServiceTier?` | `boolean` | Repli à trois états pour la capacité `service_tier`. `true` : le mode rapide peut injecter le champ et les valeurs de l’appelant sont conservées. `false` : le champ est retiré et jamais injecté, et aucune déclaration précise de modèle ne peut le réactiver. Absent : le fournisseur n’est pas classé ; les valeurs de l’appelant sont conservées intactes et le mode rapide n’injecte rien, sauf pour un modèle exact activé. Le registre classe OpenAI canonique comme `true`, et DeepSeek ainsi que Volcengine Ark comme `false`. Ne le définissez explicitement que pour les passerelles personnalisées qui prennent réellement en charge les niveaux. Les routes Chat exigent en plus une autorisation globale ou propre au modèle. | | `modelSupportsServiceTier?` | `Record` | Remplacements de capacité par identifiant exact de modèle en amont. La valeur exacte `true` autorise ce modèle Chat même sans `chatServiceTier` ; `false` restreint les valeurs globales et l’autorisation Chat. Une valeur globale explicite `supportsServiceTier: false` reste fermée et ne peut pas être réactivée. Les modèles non déclarés suivent le comportement global. La requête de gestion `PATCH /api/providers` fusionne les entrées et accepte `null` pour en supprimer une. | | `chatServiceTier?` | `boolean` | Active globalement la sérialisation de `service_tier` sur `/chat/completions`. Des modèles exacts peuvent aussi l’activer avec `modelSupportsServiceTier` ; les modèles non déclarés restent bloqués lorsque ce champ est absent ou faux. | diff --git a/docs-site/src/content/docs/ja/reference/configuration/providers.md b/docs-site/src/content/docs/ja/reference/configuration/providers.md index 1467e36f83..e93782cff8 100644 --- a/docs-site/src/content/docs/ja/reference/configuration/providers.md +++ b/docs-site/src/content/docs/ja/reference/configuration/providers.md @@ -58,6 +58,7 @@ account を削除しても mapping は保持され、同じ id を再追加す | `baseUrl` | `string` |アップストリーム API のベース URL。ほとんどの組み込み固定エンドポイントは不一致を無視します。衝突安全キー プリセットは、古い同じ名前のカスタム宛先を保持します。 | | `requestPacing?` | `{ enabled, requestsPerMinute?, minIntervalMs?, models? }` | 上流の使用量、請求、レート制限表示とは別の、クライアント側の送信開始間隔調整です。プロバイダー制限は全モデルに適用され、`models` は上流の正確なモデル ID に一致し、遅延を増やす場合のみ有効です。キュー待機は応答ヘッダーのタイムアウトを消費しません。HTTP、Responses WebSocket、明示的なアダプターの `fetchResponse`/`runTurn` 送信を対象にします。 | | `responsesPath?` | `string` |キー認証 `openai-responses` リクエストの相対リソース パス。 `/` で始まり、スキーム、クエリ、またはフラグメントが含まれていない必要があります。 | +| `upstreamWebsocket?` | `boolean` | `openai-responses` リクエストで使用するアップストリーム Responses WebSocket トランスポート（既定値は無効）。アップストリームがこのプロトコルに対応している場合、ストリーミング POST は設定済みの Responses パス（既定値 `/v1/responses`）へ HTTPS の WSS で接続し、通常の処理向けに SSE へ再エンコードされます。forward プロバイダーは `{baseUrl}/responses`、キー認証プロバイダーは `responsesPath`（未設定時は従来の `/v1/responses`）を使用します。HTTP のベース URL は SSE のままとなり、Responses 以外のパスと `openai-chat` リクエストは HTTP を使用します。 | | `supportsServiceTier?` | `boolean` | `service_tier` ケイパビリティの 3 状態です。`true`: fast モードが注入でき、呼び出し元の値も保持されます。`false`: フィールドは削除され、注入もされません (非対応と文書化されたアップストリームには送りません)。未設定: 未分類 — 呼び出し元の値はそのまま保持され、fast モードは注入しません。レジストリは正規 OpenAI (`true`)、DeepSeek、Volcengine Ark (`false`) を分類します。実際にティアをサポートするカスタム ゲートウェイにのみ明示的に設定してください。 | | `preserveResponsesReasoningContent?` | `boolean` | リプレイされる Responses reasoning アイテムの平文 reasoning コンテンツを消去せずに保持します (消去は ChatGPT バックエンドのルールです)。DeepSeek のように reasoning リプレイを受け入れるアップストリームで有効にしてください。プロキシ生成の `ocxr1` エンベロープは常に削除されます。 | | `disabled?` | `boolean` |プロバイダーをディスク上に保持しますが、ルーティングおよびモデル/カタログのリストからは除外します。 | diff --git a/docs-site/src/content/docs/ko/reference/configuration/providers.md b/docs-site/src/content/docs/ko/reference/configuration/providers.md index ba5980d1f8..d6429bd523 100644 --- a/docs-site/src/content/docs/ko/reference/configuration/providers.md +++ b/docs-site/src/content/docs/ko/reference/configuration/providers.md @@ -58,6 +58,7 @@ managed map을 활성화하면 privacy-safe selector를 만들고, 이후 계정 | `baseUrl` | `string` | 상위 API 기본 URL입니다. 대부분의 내장 고정 엔드포인트는 불일치를 무시합니다. 충돌 안전 키 프리셋은 같은 이름의 이전 사용자 지정 목적지를 보존합니다. | | `requestPacing?` | `{ enabled, requestsPerMinute?, minIntervalMs?, models? }` | 업스트림 사용량, 과금, rate-limit 지표와 별개인 선택적 클라이언트 측 아웃바운드 요청 시작 속도 조절입니다. Provider 제한은 모든 모델에 적용되고 `models` 항목은 정확한 업스트림 모델 ID와 일치하며 지연을 더 늘릴 때만 적용됩니다. 큐 대기는 응답 헤더 타임아웃을 소모하지 않습니다. HTTP, Responses WebSocket, 명시적 어댑터 `fetchResponse`/`runTurn` 전송을 포함합니다. | | `responsesPath?` | `string` | 키 인증 `openai-responses` 요청의 상대 리소스 경로입니다. 반드시 `/`로 시작해야 하며 스킴, query, fragment를 포함하면 안 됩니다. | +| `upstreamWebsocket?` | `boolean` | `openai-responses` 요청에 대한 업스트림 Responses WebSocket 전송을 선택적으로 활성화합니다(기본값 `false`). 업스트림이 이 프로토콜을 지원하면 스트리밍 POST가 설정된 Responses 경로(기본값 `/v1/responses`)로 HTTPS 기반 WSS를 사용하고, 일반 파이프라인을 위해 SSE로 다시 인코딩됩니다. forward 공급자는 `{baseUrl}/responses`를 사용하고, key-auth 공급자는 `responsesPath`를 사용하며 미설정 시 기존 `/v1/responses`로 대체됩니다. HTTP 기본 URL은 SSE를 유지하고, Responses가 아닌 경로와 `openai-chat` 요청은 HTTP를 사용합니다. | | `supportsServiceTier?` | `boolean` | `service_tier` 케이퍼빌리티 3상태입니다. `true`: fast 모드가 주입할 수 있고 호출자 값도 보존합니다. `false`: 필드를 제거하고 절대 주입하지 않습니다(미지원으로 문서화된 업스트림에는 볼 수 없습니다). 미설정: 미분류 — 호출자가 준 값은 그대로 보존하고 fast 모드는 주입하지 않습니다. 레지스트리는 정식 OpenAI(`true`), DeepSeek, Volcengine Ark(`false`)를 분류하며, 실제로 티어를 지원하는 커스텀 게이트웨이에만 명시적으로 설정하세요. | | `preserveResponsesReasoningContent?` | `boolean` | 리플레이되는 Responses reasoning 항목의 평문 reasoning 내용을 지우지 않고 유지합니다(지우는 것은 ChatGPT 백엔드 규칙입니다). DeepSeek처럼 reasoning 리플레이를 허용하는 업스트림에 켜세요. 프록시가 만든 `ocxr1` 봉투는 항상 제거됩니다. | | `disabled?` | `boolean` | 공급자를 디스크에는 남기되, 라우팅과 모델/카탈로그 목록에서는 제외합니다. | diff --git a/docs-site/src/content/docs/reference/configuration/providers.md b/docs-site/src/content/docs/reference/configuration/providers.md index 7cd68216ef..65c6f1f322 100644 --- a/docs-site/src/content/docs/reference/configuration/providers.md +++ b/docs-site/src/content/docs/reference/configuration/providers.md @@ -69,6 +69,7 @@ differing backup and rewrites known legacy namespaced selected ids to bare ids. | `requestPacing?` | `{ enabled, requestsPerMinute?, minIntervalMs?, models? }` | Optional client-side outbound request-start pacing, separate from upstream usage, billing, and rate-limit indicators. RPM is converted to an even interval; `minIntervalMs` may impose a longer interval. Provider limits apply across all models, while `models` entries use exact upstream model IDs (for example `nvidia/llama-3.1-nemotron-ultra-253b-v1`) and can only add delay. Queue waits do not consume the upstream response-header timeout. HTTP, Responses WebSocket, and explicit adapter `fetchResponse`/`runTurn` dispatches are covered. | | `upstreamHttpVersion?` | `"auto" \| "http1.1" \| "h1" \| "http2" \| "h2"` | Pin the HTTP version used for upstream requests to this provider. Defaults to `auto`, which lets Bun negotiate. An explicit pin requires an HTTPS target and fails locally when it cannot be honored. Set `http1.1` when a provider's HTTP/2 SSE stream stalls instead of delivering events — the symptom is a long-running streaming request that produces nothing and eventually times out. For Cursor, `http1.1`/`h1` selects its `RunSSE` + `BidiAppend` compatibility transport for inference and also pins live model discovery. Management `POST`/`PATCH` accept `null` to clear it back to `auto`. | | `responsesPath?` | `string` | Relative resource path for key-auth `openai-responses` requests. It must start with `/` and contain no scheme, query, or fragment. | +| `upstreamWebsocket?` | `boolean` | Opt-in upstream Responses WebSocket transport for `openai-responses` requests (default false). When the upstream supports the Responses WebSocket protocol, streaming POST requests to the configured Responses path (default `/v1/responses`) are dialed as WSS over an HTTPS base URL and re-encoded to SSE for the usual pipeline. Forward providers use `{baseUrl}/responses`; key-auth providers use `responsesPath`, or the legacy `/v1/responses` fallback. This mirrors the canonical ChatGPT backend optimization for OpenAI-compatible gateways (for example sub2api) whose WebSocket ingress is measurably faster than its SSE queue. Plain HTTP remains on SSE; non-Responses paths and `openai-chat` requests stay on HTTP. | | `supportsServiceTier?` | `boolean` | Tri-state canonical Fast capability fallback. `true` publishes Fast in the catalog, satisfies service-tier routing requirements, contributes a supported fingerprint, and lets fast mode inject the provider's canonical wire value on a compatible final adapter. `false` strips the field and never injects, and exact model declarations cannot reopen it. Absent leaves the provider unclassified: fast mode does not inject or normalize a canonical caller value, and caller values obey the final wire's forwarding permission (`chatServiceTier` on Chat; passthrough on Responses). The registry classifies canonical OpenAI (`true`), DeepSeek, and Volcengine Ark (`false`); set it explicitly only for custom gateways that genuinely support tiers. | | `modelSupportsServiceTier?` | `Record` | Exact upstream model capability overrides. Exact `true` enables canonical Fast for that model; exact `false` narrows provider defaults. An explicit provider-level `supportsServiceTier: false` remains fail-closed and cannot be reopened. Exact `true` does not authorize foreign caller-tier forwarding on Chat. Undeclared models fall back to provider-wide behavior. Management `PATCH /api/providers` merges entries and accepts `null` to clear one. | | `chatServiceTier?` | `boolean` | Provider-wide Chat-wire opt-in for forwarding caller `service_tier` values. On a classified route it governs foreign values such as `flex`, not proxy-owned canonical Fast after capability validation; on an unclassified route it governs every caller value because no Fast capability has been validated. Exact model capability does not authorize foreign forwarding. Responses routes retain their capability-based caller forwarding behavior. | diff --git a/docs-site/src/content/docs/ru/reference/configuration/providers.md b/docs-site/src/content/docs/ru/reference/configuration/providers.md index fc76d5a456..d4d0a11ba4 100644 --- a/docs-site/src/content/docs/ru/reference/configuration/providers.md +++ b/docs-site/src/content/docs/ru/reference/configuration/providers.md @@ -71,6 +71,7 @@ cross-route credential fallback не существует. Строки API GPT- | `baseUrl` | `string` | Базовый URL API upstream'а. Большинство built-in fixed-endpoint'ов игнорируют несовпадение; collision-safe key-preset'ы сохраняют старый custom destination с тем же именем. | | `requestPacing?` | `{ enabled, requestsPerMinute?, minIntervalMs?, models? }` | Опциональное клиентское выравнивание начала исходящих запросов, отдельное от учёта использования, биллинга и индикаторов rate limit апстрима. Лимит провайдера действует на все модели, а `models` сопоставляется с точными ID моделей апстрима и может только увеличить задержку. Ожидание очереди не расходует таймаут заголовков ответа. Поддерживаются HTTP, Responses WebSocket и явные вызовы адаптеров `fetchResponse`/`runTurn`. | | `responsesPath?` | `string` | Relative resource path для key-auth запросов `openai-responses`. Должен начинаться с `/` и не может содержать scheme, query или fragment. | +| `upstreamWebsocket?` | `boolean` | Необязательный upstream Responses WebSocket для запросов `openai-responses` (по умолчанию `false`). Если upstream поддерживает этот протокол, потоковые POST-запросы используют настроенный путь Responses (по умолчанию `/v1/responses`), подключаются по WSS через HTTPS и перекодируются обратно в SSE для обычного конвейера. Провайдеры в режиме forward используют `{baseUrl}/responses`; провайдеры с ключом используют `responsesPath` или исторический fallback `/v1/responses`. Для HTTP остаётся SSE; пути, не относящиеся к Responses, и запросы `openai-chat` остаются на HTTP. | | `supportsServiceTier?` | `boolean` | Три состояния поддержки `service_tier`. `true`: fast mode может подставлять поле, значения вызывающего сохраняются. `false`: поле удаляется и никогда не подставляется (апстрим, для которого задокументировано отсутствие поддержки, не должен его получать). Не задано: провайдер не классифицирован — значения вызывающего сохраняются без изменений, fast mode не подставляет. Registry классифицирует canonical OpenAI (`true`), DeepSeek и Volcengine Ark (`false`); задавайте явно только для custom gateway'ев, реально поддерживающих tier'ы. | | `preserveResponsesReasoningContent?` | `boolean` | Сохранять plaintext reasoning content в replay'нутых Responses reasoning item'ах вместо очистки (очистка — правило ChatGPT backend'а). Включайте для upstream'ов, чей контракт принимает reasoning replay, например DeepSeek. Proxy-minted `ocxr1` envelope'ы удаляются всегда. | | `disabled?` | `boolean` | Сохранить провайдера на диске, но исключить его из routing'а и из model/catalog-listing'ов. | diff --git a/docs-site/src/content/docs/tr/reference/configuration/providers.md b/docs-site/src/content/docs/tr/reference/configuration/providers.md index f12229afb9..b84e783c22 100644 --- a/docs-site/src/content/docs/tr/reference/configuration/providers.md +++ b/docs-site/src/content/docs/tr/reference/configuration/providers.md @@ -77,6 +77,7 @@ alanlı seçilmiş kimlikleri yalın kimliklere yeniden yazar. | `adapter` | `string` | `openai-chat`, `openai-responses`, `anthropic`, `google`, `kiro`, `cursor`, `ollama-native`, `azure-openai` (veya takma ad `azure`) seçeneklerinden biri. | | `baseUrl` | `string` | Yukarı akış API temel URL'si. Çoğu yerleşik sabit uç nokta uyumsuzluğu yok sayar; çakışma güvenli anahtar önayarları aynı adlı daha eski özel bir hedefi korur. | | `responsesPath?` | `string` | Anahtar kimlik doğrulamalı `openai-responses` istekleri için göreli kaynak yolu. `/` ile başlamalı ve şema, sorgu veya parça içermemelidir. | +| `upstreamWebsocket?` | `boolean` | `openai-responses` istekleri için isteğe bağlı upstream Responses WebSocket aktarımıdır (varsayılan `false`). Upstream bu protokolü desteklediğinde, akışlı POST istekleri yapılandırılmış Responses yolunu (varsayılan `/v1/responses`) HTTPS tabanında WSS ile kullanır ve normal işlem hattı için SSE'ye yeniden kodlanır. Forward sağlayıcılar `{baseUrl}/responses`, anahtar kimlik doğrulamalı sağlayıcılar `responsesPath` veya eski `/v1/responses` geri dönüşünü kullanır. Düz HTTP SSE olarak kalır; Responses dışı yollar ve `openai-chat` istekleri HTTP'de kalır. | | `supportsServiceTier?` | `boolean` | Üç durumlu `service_tier` yeteneği. `true`: hızlı mod enjekte edebilir ve arayan değerleri korunur. `false`: alan kaldırılır ve asla enjekte edilmez (desteklemediği belgelenen yukarı akış bunu almamalıdır). Yok: sağlayıcı sınıflandırılmamıştır — arayan tarafından sağlanan değerler dokunulmadan korunur ve hızlı mod asla enjekte etmez. Kayıt defteri kurallı OpenAI'yi (`true`), DeepSeek'i ve Volcengine Ark'ı (`false`) sınıflandırır; bunu yalnızca katmanları gerçekten destekleyen özel ağ geçitleri için açıkça ayarlayın. | | `preserveResponsesReasoningContent?` | `boolean` | Düz metin akıl yürütme içeriğini boşaltmak yerine (boşaltma ChatGPT arka ucunun kuralıdır) tekrarlanan Responses akıl yürütme öğelerinde tutun. DeepSeek gibi sözleşmesi akıl yürütme tekrarını kabul eden yukarı akışlar için etkinleştirin. Proxy tarafından basılan `ocxr1` zarfları her zaman kaldırılır. | | `disabled?` | `boolean` | Sağlayıcıyı diskte tutun ancak yönlendirmeden ve model/katalog listelerinden hariç tutun. | diff --git a/docs-site/src/content/docs/zh-cn/reference/configuration/providers.md b/docs-site/src/content/docs/zh-cn/reference/configuration/providers.md index 229f6c205d..be56cbd00f 100644 --- a/docs-site/src/content/docs/zh-cn/reference/configuration/providers.md +++ b/docs-site/src/content/docs/zh-cn/reference/configuration/providers.md @@ -58,6 +58,7 @@ selector，而不是分配一个新名称。 | `baseUrl` | `string` | 上游 API 基础 URL。大多数内置固定端点会忽略不匹配的值；具备冲突安全键的预设会保留一个更早、同名的自定义目标。 | | `requestPacing?` | `{ enabled, requestsPerMinute?, minIntervalMs?, models? }` | 可选的客户端出站请求启动节流，与上游用量、计费和限流指标相互独立。提供商限制适用于所有模型，`models` 按上游模型精确 ID 匹配且只能增加延迟。排队等待不计入响应头超时。覆盖 HTTP、Responses WebSocket 以及显式适配器 `fetchResponse`/`runTurn` 调用。 | | `responsesPath?` | `string` | 用于 key-auth `openai-responses` 请求的相对资源路径。必须以 `/` 开头，且不能包含 scheme、query 或 fragment。 | +| `upstreamWebsocket?` | `boolean` | 为 `openai-responses` 请求选择性启用上游 Responses WebSocket 传输（默认 `false`）。当上游支持该协议时，流式 POST 请求会使用配置的 Responses 路径（默认 `/v1/responses`），通过 HTTPS 基础 URL 以 WSS 连接，并重新编码为常规流程使用的 SSE。forward 提供者使用 `{baseUrl}/responses`；key-auth 提供者使用 `responsesPath`，未设置时回退到传统的 `/v1/responses`。普通 HTTP 仍使用 SSE；非 Responses 路径和 `openai-chat` 请求仍使用 HTTP。 | | `supportsServiceTier?` | `boolean` | `service_tier` 能力的三态。`true`：fast 模式可以注入，调用方提供的值也会被保留。`false`：剥离该字段且绝不注入（已明确不支持的上游不会收到它）。未设置：未分类——调用方提供的值原样保留，fast 模式绝不注入。注册表已对官方 OpenAI（`true`）、DeepSeek 和 Volcengine Ark（`false`）分类；仅对真正支持分层的自定义网关显式设置。 | | `preserveResponsesReasoningContent?` | `boolean` | 在重放的 Responses reasoning 项中保留明文 reasoning 内容，而不是清空（清空是 ChatGPT 后端的规则）。对接受 reasoning 重放的上游（如 DeepSeek）启用。代理生成的 `ocxr1` 信封始终会被剥离。 | | `disabled?` | `boolean` | 将提供者保留在磁盘上，但从路由和模型/目录列表中排除。 | diff --git a/docs-site/src/content/docs/zh-tw/reference/configuration/providers.md b/docs-site/src/content/docs/zh-tw/reference/configuration/providers.md index d15c5a3f31..509d4c88c4 100644 --- a/docs-site/src/content/docs/zh-tw/reference/configuration/providers.md +++ b/docs-site/src/content/docs/zh-tw/reference/configuration/providers.md @@ -42,6 +42,7 @@ description: 供應商項目、認證、端點、模型目錄、配額、context | `baseUrl` | `string` | 上游 API base URL。多數內建固定端點忽略不符；碰撞安全的金鑰預設保留較舊的同名自訂目的地。 | | `requestPacing?` | `{ enabled, requestsPerMinute?, minIntervalMs?, models? }` | 選用的用戶端出站請求啟動節流，與上游用量、計費及限流指標彼此獨立。供應商限制適用於所有模型，`models` 依上游模型精確 ID 比對且只能增加延遲。排隊等待不計入回應標頭逾時。涵蓋 HTTP、Responses WebSocket 及明確的適配器 `fetchResponse`/`runTurn` 呼叫。 | | `responsesPath?` | `string` | Key-auth `openai-responses` 請求的相對資源路徑。必須以 `/` 開頭且不含 scheme、query 或 fragment。 | +| `upstreamWebsocket?` | `boolean` | 為 `openai-responses` 請求選用上游 Responses WebSocket 傳輸（預設 `false`）。當上游支援此協定時，串流 POST 請求會使用設定的 Responses 路徑（預設 `/v1/responses`），透過 HTTPS 基礎 URL 以 WSS 連線，再重新編碼為一般流程使用的 SSE。forward 供應商使用 `{baseUrl}/responses`；key-auth 供應商使用 `responsesPath`，未設定時回退到傳統的 `/v1/responses`。一般 HTTP 仍使用 SSE；非 Responses 路徑與 `openai-chat` 請求仍使用 HTTP。 | | `disabled?` | `boolean` | 將供應商保留在磁碟上但排除於路由與模型／目錄清單。 | | `apiKey?` | `string` | API 金鑰，或在請求時解析的 `${ENV_VAR}` / `$ENV_VAR` 參考。 | | `apiKeyTransport?` | `"x-api-key" \| "bearer"` | Anthropic 金鑰標頭風格。預設為原生 `x-api-key`；僅對 key-auth `anthropic` 供應商有效。 | diff --git a/src/config.ts b/src/config.ts index 11d88af91d..78e8b26f2b 100644 --- a/src/config.ts +++ b/src/config.ts @@ -524,6 +524,10 @@ const providerConfigSchema = z.object({ upstreamHttpVersion: z.enum(UPSTREAM_HTTP_VERSION_VALUES) .nullish() .transform(value => value ?? undefined), + // Opt-in upstream Responses WebSocket for OpenAI-compatible providers (e.g. + // aggregators whose WebSocket ingress is measurably faster than SSE). The + // canonical ChatGPT backend WS selection is independent of this flag. + upstreamWebsocket: z.boolean().optional(), directGeminiWireRenames: z.boolean().optional(), noStructuredOutputModels: z.array(z.string().min(1)) .transform(normalizeNonBlankStringArray) diff --git a/src/server/management/provider-routes.ts b/src/server/management/provider-routes.ts index 626148fd0c..bb42428385 100644 --- a/src/server/management/provider-routes.ts +++ b/src/server/management/provider-routes.ts @@ -241,6 +241,11 @@ function applyProviderPatchFields( } touched = true; } + if (Object.hasOwn(rawBody, "upstreamWebsocket")) { + if (typeof rawBody.upstreamWebsocket !== "boolean") return { error: "upstreamWebsocket must be a boolean" }; + next.upstreamWebsocket = rawBody.upstreamWebsocket; + touched = true; + } // The Models page edits the catalog hints in place; keep them on the existing // provider mutation path so validation, cache invalidation, and convergence stay unified (#1073). if (Object.hasOwn(rawBody, "contextWindow")) { @@ -469,6 +474,7 @@ export async function handleProviderRoutes(ctx: ManagementContext): Promise; + if (rawProvider.upstreamWebsocket !== undefined && typeof rawProvider.upstreamWebsocket !== "boolean") { + return jsonResponse({ error: "upstreamWebsocket must be a boolean" }, 400); + } const serviceTierError = providerServiceTierConfigError(name, body.provider); if (serviceTierError) return jsonResponse({ error: serviceTierError }, 400); const prov = body.provider ? stripCodexRuntimeProviderFields(body.provider as OcxProviderConfig) : undefined; @@ -604,6 +614,7 @@ export async function handleProviderRoutes(ctx: ManagementContext): Promise[0], init?: RequestInit) => { - if (typeof input === "string" && init && shouldUseCodexWsUpstream(input, init, runtime)) { + const upstreamWebsocket = provider.upstreamWebsocket === true; + if (typeof input === "string" && init && shouldUseCodexWsUpstream(input, init, runtime, upstreamWebsocket)) { // The fallback has to be the same HTTP fetch the non-WS branch would have // used, protocol pin included: a WS turn that falls back is serving the // request over HTTP, and dropping the provider's `upstreamHttpVersion` diff --git a/src/server/responses/ws-upstream.ts b/src/server/responses/ws-upstream.ts index 275eff6dfb..38e3642f60 100644 --- a/src/server/responses/ws-upstream.ts +++ b/src/server/responses/ws-upstream.ts @@ -18,6 +18,36 @@ import { compareBunVersions } from "../../lib/bun-stream-caps"; const CODEX_RESPONSES_HTTP_URL = "https://chatgpt.com/backend-api/codex/responses"; const CODEX_RESPONSES_WS_URL = "wss://chatgpt.com/backend-api/codex/responses"; const WS_BETA = "responses_websockets=2026-02-06"; + +/** + * Dial URL for a request URL. The canonical ChatGPT backend keeps its constant; + * an operator-opted OpenAI-compatible upstream swaps https for wss on the same + * path so gateways that serve the Responses WebSocket protocol on their + * /v1/responses path get the same fast lane. Plain HTTP remains on SSE because + * a provider WS handshake would otherwise send credentials and request data + * without transport encryption. + */ +function wsUpstreamUrlFor(httpUrl: string): string { + if (httpUrl === CODEX_RESPONSES_HTTP_URL) return CODEX_RESPONSES_WS_URL; + return httpUrl.replace(/^http(s?):/, "ws$1:"); +} + +/** + * An operator-opted OpenAI-compatible upstream only joins the WS lane for + * Responses endpoints: the WebSocket path speaks the Responses event protocol, + * and every downstream consumer (adapter parsers, usage sniffing, SSE relay) + * assumes that wire. Other paths (chat completions, images, search) stay HTTP. + */ +function isResponsesWebsocketEligibleUrl(url: string): boolean { + let parsed: URL; + try { + parsed = new URL(url); + } catch { + return false; + } + return parsed.protocol === "https:" + && parsed.pathname.endsWith("/responses"); +} // If the 101 never arrives (network black hole), give SSE a chance well before // the caller's connect timeout (default 200s) would fire. const UPGRADE_DEADLINE_MS = 10_000; @@ -102,9 +132,11 @@ export function shouldUseCodexWsUpstream( url: string, init?: RequestInit, runtime: BunRuntimeGateInput = currentBunRuntimeIdentity(), + upstreamWebsocketConfigured = false, ): boolean { if (!bunSupportsBoundedCodexWsRelay(runtime)) return false; - if (url !== CODEX_RESPONSES_HTTP_URL) return false; + if (url !== CODEX_RESPONSES_HTTP_URL && !upstreamWebsocketConfigured) return false; + if (upstreamWebsocketConfigured && !isResponsesWebsocketEligibleUrl(url)) return false; if ((init?.method ?? "GET").toUpperCase() !== "POST") return false; const body = init?.body; if (typeof body !== "string") return false; @@ -123,6 +155,50 @@ export function shouldUseCodexWsUpstream( const CLOSED_BEFORE_TERMINAL = "codex websocket closed before a Responses terminal event"; +type ResponsesWsRelayEvent = { + type: string; + text: string; +}; + +/** + * Responses WebSocket uses `response.done` as its terminal event, while the + * SSE Responses surface uses status-specific terminal events. Normalize the + * WS-only discriminator before relaying so the existing SSE consumers can + * settle the turn and the socket close cannot be mistaken for a drop. Unknown + * or missing status values fail closed instead of being reported as success. + */ +function normalizeResponsesWsRelayEvent(text: string): ResponsesWsRelayEvent | null { + let payload: unknown; + try { + payload = JSON.parse(text); + } catch { + return null; + } + if (!payload || typeof payload !== "object" || Array.isArray(payload)) return null; + const record = payload as Record; + if (typeof record.type !== "string") return null; + if (record.type !== "response.done") return { type: record.type, text }; + + const response = record.response; + const status = response && typeof response === "object" && !Array.isArray(response) + ? (response as Record).status + : undefined; + const type = status === "completed" + ? "response.completed" + : status === "failed" + ? "response.failed" + : status === "incomplete" || status === "cancelled" + ? "response.incomplete" + : "response.failed"; + const normalizedRecord: Record = { ...record, type }; + if (type === "response.failed" && status !== "failed") { + normalizedRecord.response = response && typeof response === "object" && !Array.isArray(response) + ? { ...(response as Record), status: "failed" } + : { status: "failed" }; + } + return { type, text: JSON.stringify(normalizedRecord) }; +} + /** * The close code is the only thing that separates "the backend refused this * payload" from "the network dropped", and both used to reach the caller as the @@ -221,7 +297,7 @@ export function codexWsUpstreamFetch( let ws: WebSocket; try { // Bun accepts per-handshake headers; the DOM lib types only list protocol arrays. - ws = new WebSocket(CODEX_RESPONSES_WS_URL, { headers } as unknown as string[]); + ws = new WebSocket(wsUpstreamUrlFor(url), { headers } as unknown as string[]); } catch { resolve(sseFallback(url, init)); return; @@ -311,14 +387,19 @@ export function codexWsUpstreamFetch( failStream("codex websocket frame exceeds the response size limit"); return; } - const encodedText = encoder.encode(text); + const rawEncodedText = encoder.encode(text); + if (rawEncodedText.byteLength > MAX_CODEX_WS_FRAME_BYTES) { + failStream("codex websocket frame exceeds the response size limit"); + return; + } + const normalized = normalizeResponsesWsRelayEvent(text); + if (!normalized) return; + const { type } = normalized; + const encodedText = normalized.text === text ? rawEncodedText : encoder.encode(normalized.text); if (encodedText.byteLength > MAX_CODEX_WS_FRAME_BYTES) { failStream("codex websocket frame exceeds the response size limit"); return; } - let type: unknown; - try { type = (JSON.parse(text) as { type?: unknown }).type; } catch { return; } - if (typeof type !== "string") return; // Relay only the event surface the SSE path produces today. WS-only // frames (codex.rate_limits, responsesapi.websocket_timing) are dropped // so downstream clients see exactly the stream shape they always got. diff --git a/src/types/provider.ts b/src/types/provider.ts index d1ef45722b..37e749e145 100644 --- a/src/types/provider.ts +++ b/src/types/provider.ts @@ -275,6 +275,18 @@ export interface OcxProviderConfig { * (current behavior unchanged). Only meaningful for https: base URLs. */ upstreamHttpVersion?: UpstreamHttpVersion; + /** + * Opt-in upstream Responses WebSocket transport for `openai-responses` requests. When true, + * streaming POST turns use the configured Responses path (default `/v1/responses`): forward + * providers use `{baseUrl}/responses`, while key-auth providers use `responsesPath` or the + * legacy `/v1/responses` fallback. HTTPS providers use wss and are re-encoded to SSE; HTTP + * providers continue using SSE, and `openai-chat` requests stay on HTTP. This mirrors the + * canonical ChatGPT backend optimization for any OpenAI-compatible gateway that speaks the + * Responses WebSocket protocol (for example an aggregator like sub2api whose WS ingress is + * measurably faster than its SSE queue). Default false. Canonical ChatGPT backend WS selection + * is independent of this flag. + */ + upstreamWebsocket?: boolean; /** * Google only. When `false`, the AI Studio (direct) path sends Gemini Flash ids * unchanged to the wire instead of applying the `-tiered` suffix (`gemini-3.7-flash` diff --git a/tests/management-provider-validation.test.ts b/tests/management-provider-validation.test.ts index ac976fc785..106cdcd3d0 100644 --- a/tests/management-provider-validation.test.ts +++ b/tests/management-provider-validation.test.ts @@ -3850,7 +3850,7 @@ describe("provider management validation", () => { }); }); -describe("provider upstreamHttpVersion management contract (#1668)", () => { +describe("provider transport option management contract (#1668, #2816)", () => { function makeConfig(): OcxConfig { return { port: 0, @@ -4073,4 +4073,136 @@ describe("provider upstreamHttpVersion management contract (#1668)", () => { upstreamHttpVersion: 42, })).toContain("upstreamHttpVersion"); }); + + test("upstreamWebsocket round-trips through POST, GET, and PATCH", async () => { + if (existsSync(TEST_DIR)) rmSync(TEST_DIR, { recursive: true }); + mkdirSync(TEST_DIR, { recursive: true }); + process.env.OPENCODEX_HOME = TEST_DIR; + const liveConfig = makeConfig(); + saveConfig(liveConfig); + await withRequest(liveConfig, async (request) => { + const created = await request("/api/providers", { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ + name: "ws-provider", + provider: { + adapter: "openai-responses", + baseUrl: "https://api.example.test/v1", + upstreamWebsocket: true, + }, + }), + }); + expect(created?.status).toBe(200); + expect(liveConfig.providers["ws-provider"]?.upstreamWebsocket).toBe(true); + expect(loadConfig().providers["ws-provider"]?.upstreamWebsocket).toBe(true); + + const list = await request("/api/providers"); + expect(await list?.json()).toContainEqual(expect.objectContaining({ + name: "ws-provider", + upstreamWebsocket: true, + })); + + const invalid = await request("/api/providers?name=ws-provider", { + method: "PATCH", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ upstreamWebsocket: "true" }), + }); + expect(invalid?.status).toBe(400); + expect(liveConfig.providers["ws-provider"]?.upstreamWebsocket).toBe(true); + + const cleared = await request("/api/providers?name=ws-provider", { + method: "PATCH", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ upstreamWebsocket: false }), + }); + expect(cleared?.status).toBe(200); + expect(liveConfig.providers["ws-provider"]?.upstreamWebsocket).toBe(false); + expect(loadConfig().providers["ws-provider"]?.upstreamWebsocket).toBe(false); + + const invalidPost = await request("/api/providers", { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ + name: "invalid-ws-provider", + provider: { + adapter: "openai-responses", + baseUrl: "https://api.example.test/v1", + upstreamWebsocket: "true", + }, + }), + }); + expect(invalidPost?.status).toBe(400); + expect(liveConfig.providers["invalid-ws-provider"]).toBeUndefined(); + }); + }); + + test("POST overwrite preserves omitted upstreamWebsocket and honors explicit false", async () => { + if (existsSync(TEST_DIR)) rmSync(TEST_DIR, { recursive: true }); + mkdirSync(TEST_DIR, { recursive: true }); + process.env.OPENCODEX_HOME = TEST_DIR; + const liveConfig = makeConfig(); + saveConfig(liveConfig); + await withRequest(liveConfig, async (request) => { + const create = await request("/api/providers", { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ + name: "ws-overwrite", + provider: { + adapter: "openai-responses", + baseUrl: "https://api.example.test/v1", + upstreamWebsocket: true, + }, + }), + }); + expect(create?.status).toBe(200); + + const omitted = await request("/api/providers", { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ + name: "ws-overwrite", + provider: { + adapter: "openai-responses", + baseUrl: "https://api.example.test/v1", + }, + }), + }); + expect(omitted?.status).toBe(200); + expect(liveConfig.providers["ws-overwrite"]?.upstreamWebsocket).toBe(true); + expect(loadConfig().providers["ws-overwrite"]?.upstreamWebsocket).toBe(true); + + const explicitFalse = await request("/api/providers", { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ + name: "ws-overwrite", + provider: { + adapter: "openai-responses", + baseUrl: "https://api.example.test/v1", + upstreamWebsocket: false, + }, + }), + }); + expect(explicitFalse?.status).toBe(200); + expect(liveConfig.providers["ws-overwrite"]?.upstreamWebsocket).toBe(false); + expect(loadConfig().providers["ws-overwrite"]?.upstreamWebsocket).toBe(false); + + const omittedAfterDisable = await request("/api/providers", { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ + name: "ws-overwrite", + provider: { + adapter: "openai-responses", + baseUrl: "https://api.example.test/v1", + }, + }), + }); + expect(omittedAfterDisable?.status).toBe(200); + expect(liveConfig.providers["ws-overwrite"]?.upstreamWebsocket).toBe(false); + expect(loadConfig().providers["ws-overwrite"]?.upstreamWebsocket).toBe(false); + }); + }); }); diff --git a/tests/ws-upstream.test.ts b/tests/ws-upstream.test.ts index c16060c9f5..975f9ad215 100644 --- a/tests/ws-upstream.test.ts +++ b/tests/ws-upstream.test.ts @@ -32,8 +32,8 @@ const BOUNDED_WS_RUNTIME = "1.4.0"; // constant that only held before the backfill landed. const EAGER_RELAY_FORCED_BY_PLATFORM = isWin32EagerRewrite(process.platform, true); -function shouldUseCodexWsUpstream(url: string, init?: RequestInit): boolean { - return rawShouldUseCodexWsUpstream(url, init, BOUNDED_WS_RUNTIME); +function shouldUseCodexWsUpstream(url: string, init?: RequestInit, upstreamWebsocket = false): boolean { + return rawShouldUseCodexWsUpstream(url, init, BOUNDED_WS_RUNTIME, upstreamWebsocket); } function codexWsUpstreamFetch( @@ -136,6 +136,27 @@ describe("shouldUseCodexWsUpstream", () => { // Malformed JSON stays on HTTP. expect(shouldUseCodexWsUpstream(CODEX_URL, { method: "POST", body: "{\"stream\":true" })).toBe(false); }); + + test("opt-in upstream WebSocket only for configured OpenAI-compatible Responses endpoints", () => { + // The canonical backend ignores the flag. + expect(shouldUseCodexWsUpstream(CODEX_URL, streamingInit(), false)).toBe(true); + // Configured providers join the WS lane on their own /v1/responses path. + expect(shouldUseCodexWsUpstream("https://sub2api.example.com/v1/responses", streamingInit(), true)).toBe(true); + // Plain HTTP stays on SSE; never send credentials or request data through ws://. + expect(shouldUseCodexWsUpstream("http://10.0.0.5:8080/v1/responses", streamingInit(), true)).toBe(false); + expect(shouldUseCodexWsUpstream("https://sub2api.example.com/v1/responses", streamingInit(), false)).toBe(false); + // Non-Responses paths on a configured provider stay on HTTP. + expect(shouldUseCodexWsUpstream("https://sub2api.example.com/v1/chat/completions", streamingInit(), true)).toBe(false); + expect(shouldUseCodexWsUpstream("https://sub2api.example.com/v1/images", streamingInit(), true)).toBe(false); + expect(shouldUseCodexWsUpstream("https://sub2api.example.com/v1/alpha/search", streamingInit(), true)).toBe(false); + // The usual streaming/body rules still apply to configured providers. + expect(shouldUseCodexWsUpstream("https://sub2api.example.com/v1/responses", { method: "GET" }, true)).toBe(false); + expect(shouldUseCodexWsUpstream("https://sub2api.example.com/v1/responses", { + method: "POST", + body: JSON.stringify({ model: "m" }), + }, true)).toBe(false); + expect(shouldUseCodexWsUpstream("not a url", streamingInit(), true)).toBe(false); + }); }); type Listener = (event: unknown) => void; @@ -241,6 +262,34 @@ describe("providerFetch routing", () => { expect(baseCalls).toHaveLength(3); expect(FakeWebSocket.instances).toHaveLength(1); }); + + test("routes an opt-in provider's Responses streams over its upstream WS", async () => { + installFake(ws => { + ws.emit("open", {}); + ws.emit("message", { data: JSON.stringify({ type: "response.completed", response: { id: "r1" } }) }); + }); + const baseCalls: string[] = []; + const sentinel = new Response("base"); + const provider = { + upstreamWebsocket: true, + fetch: (async (input: unknown) => { + baseCalls.push(String(input)); + return sentinel.clone(); + }) as unknown as typeof fetch, + } as unknown as OcxProviderConfig; + const wrapped = providerFetch(provider, BOUNDED_WS_RUNTIME); + + const wsResponse = await wrapped("https://sub2api.example.com/v1/responses", streamingInit()); + expect(wsResponse.headers.get("content-type")).toContain("text/event-stream"); + expect(baseCalls).toHaveLength(0); + expect(FakeWebSocket.instances).toHaveLength(1); + expect(FakeWebSocket.instances[0]!.url).toBe("wss://sub2api.example.com/v1/responses"); + + // The same provider's non-Responses paths (images/search/chat) stay on the base fetch. + await wrapped("https://sub2api.example.com/v1/images", streamingInit()); + expect(baseCalls).toHaveLength(1); + expect(FakeWebSocket.instances).toHaveLength(1); + }); }); describe("handleResponses Codex WS relay selection", () => { @@ -472,6 +521,56 @@ describe("codexWsUpstreamFetch", () => { expect(FakeWebSocket.instances[0].closed).toBe(true); }); + test("normalizes the Responses WebSocket response.done terminal to SSE", async () => { + installFake(ws => { + ws.emit("open", {}); + ws.emit("message", { + data: JSON.stringify({ + type: "response.done", + response: { id: "r-done", status: "completed", output: [] }, + }), + }); + ws.emit("close", { code: 1000, reason: "normal" }); + }); + const response = await codexWsUpstreamFetch(CODEX_URL, streamingInit(), (() => { + throw new Error("fallback must not run after open"); + }) as unknown as typeof fetch); + + const text = await response.text(); + expect(text).toContain("event: response.completed"); + expect(text).toContain('"type":"response.completed"'); + expect(text).not.toContain("response.done"); + expect(FakeWebSocket.instances[0]!.closed).toBe(true); + }); + + test("fails closed when response.done has no recognized terminal status", async () => { + const cases: Array<{ id: string; status?: string }> = [ + { id: "r-missing" }, + { id: "r-queued", status: "queued" }, + { id: "r-unknown", status: "provider_future_state" }, + ]; + for (const response of cases) { + installFake(ws => { + ws.emit("open", {}); + ws.emit("message", { + data: JSON.stringify({ type: "response.done", response }), + }); + }); + const upstream = await codexWsUpstreamFetch(CODEX_URL, streamingInit(), (() => { + throw new Error("fallback must not run after open"); + }) as unknown as typeof fetch); + + const text = await upstream.text(); + expect(text).toContain("event: response.failed"); + const payload = text + .split("\n") + .filter(line => line.startsWith("data: ") && line !== "data: [DONE]") + .map(line => JSON.parse(line.slice("data: ".length))) + .find(event => event.type === "response.failed"); + expect(payload?.response?.status).toBe("failed"); + } + }); + test("falls back to the HTTP fetch when the upgrade is rejected before open", async () => { installFake(ws => ws.close()); const sentinel = new Response("sse-fallback", { status: 429 }); @@ -836,4 +935,21 @@ describe("oversized Codex create frames", () => { await expect(response.text()).rejects.toThrow("closed before a Responses terminal event (close 1006)"); }); + + test("dials the configured provider's own wss URL for an opt-in upstream", async () => { + installFake(ws => { + ws.emit("open", {}); + ws.emit("message", { data: JSON.stringify({ type: "response.completed", response: { id: "r-ws" } }) }); + }); + const sentinel = new Response("fallback"); + const response = await codexWsUpstreamFetch( + "https://sub2api.example.com/v1/responses", + streamingInit(), + (async () => sentinel) as typeof fetch, + ); + expect(FakeWebSocket.instances).toHaveLength(1); + expect(FakeWebSocket.instances[0]!.url).toBe("wss://sub2api.example.com/v1/responses"); + expect(response.headers.get("content-type")).toContain("text/event-stream"); + expect(await response.text()).toContain("response.completed"); + }); });