feat(anthropic): add quota-window account pool routing (carry of #2560) #2982
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -159,8 +159,9 @@ affinity を維持します。これらの戦略は provider enforcement を回 | |
| |キー |タイプ |デフォルト |説明 | | ||
| | --- | --- | --- | --- | | ||
| | `anthropicAccountPool.enabled?` | `boolean` | `false` |スティッキー アフィニティと 429 クールダウン フェイルオーバーを有効にします。 | | ||
| | `anthropicAccountPool.autoSwitchThreshold?` | `number` | `80` |新しいセッションでは、アクティブなアカウントがこのしきい値に達すると、設定した期間で既知のキャッシュ使用量が最も低いアカウントを選択します。 `0` はクォータ選択を無効にします。 | | ||
| | `anthropicAccountPool.strategy?` | `"quota" \| "round-robin" \| "fill-first"` | `"quota"` |新しいセッション戦略。`quota` は `quotaWindow` で指定した期間（既定は 5 時間足）でアカウントを順位付けし、`fill-first` も同じ期間で使い切りのしきい値を判定します。 | | ||
| | `anthropicAccountPool.quotaWindow?` | `"five-hour" \| "weekly" \| "max-utilization"` | `"five-hour"` |使用量ベースのアカウント選択で使う、プロバイダー報告のキャッシュ済み使用率です。`five-hour` は従来の動作を維持します。`weekly` は週次使用量を使い、他に対象アカウントが残る間だけ 5 時間使用量が上限に達したアカウントを除外し、残らない場合はそれらへフォールバックします。`max-utilization` は判明している値のうち最も高いものを使うため、週次使用量が未取得でも 5 時間使用量を利用できます。どちらも不明なら unknown の順位付けに従います。既知の使用量は unknown より先ですが、対象がすべて unknown でも対象順の先頭を選択します。記載した 5 時間使用量による同点判定後も完全に同点なら、対象順を維持します。正常な affinity セッションを先回りして再配置することはありません。新規セッションの割り当てと、対象となる 429 代替後のルーティング復旧では、`quota` はこの期間で対象候補を直接順位付けし、`fill-first` はこの期間のしきい値と上限到達ルールを使って安定順に進み、`round-robin` はこの設定を無視します。クールダウン、フェイルオーバー上限、再認証の適格性は別のローカル状態です。アカウント別の週次使用量は、ダッシュボードのプロバイダーページで取得した後にのみ利用できます。 | | ||
| | `anthropicAccountPool.stickyLimit?` | `number` | `1` |成功した新しいセッションのバインドは 1 つのラウンドロビン選択で保持されます。範囲は 1 ～ 100。 | | ||
|
|
||
| 有効にすると、429 レコードは `Retry-After` またはデフォルトのバックオフからの制限されたクールダウンを記録し、リクエスト内でローテーションする可能性があります。アフィニティはプロセスローカルであり、サイズ制限があります。資格情報 401/403 は、アカウントに再認証が必要であることをマークします。すべての対象となるアカウントが冷却されている場合、クライアントは、既知の場合、認証エラーではなく、`Retry-After` を含む 429 を受け取ります。 | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -163,8 +163,9 @@ affinity 초기화 뒤의 기존 작업도 포함될 수 있습니다. 출력 | |
| | 키 | 타입 | 기본값 | 설명 | | ||
| | --- | --- | --- | --- | | ||
| | `anthropicAccountPool.enabled?` | `boolean` | `false` | sticky 결속과 429 쿨다운 failover를 켭니다. | | ||
| | `anthropicAccountPool.autoSwitchThreshold?` | `number` | `80` | 새 세션에서는 활성 계정이 이 임계값에 도달하면 설정된 창의 알려진 캐시 사용량이 가장 낮은 계정을 고릅니다. `0`이면 quota 선택을 끕니다. | | ||
| | `anthropicAccountPool.strategy?` | `"quota" \| "round-robin" \| "fill-first"` | `"quota"` | 새 세션 전략입니다. `quota`는 `quotaWindow`로 지정한 창(기본값은 5시간 막대)으로 계정 순위를 매기고, `fill-first`도 같은 창에서 소진 임계값을 판정합니다. | | ||
| | `anthropicAccountPool.quotaWindow?` | `"five-hour" \| "weekly" \| "max-utilization"` | `"five-hour"` | 사용량 기반 계정 선택에 사용하는, 공급자가 보고한 캐시 사용률 막대입니다. `five-hour`는 기존 동작을 유지합니다. `weekly`는 주간 막대를 사용하며 다른 사용 가능한 계정이 남아 있을 때만 5시간 막대가 소진된 계정을 건너뛰고, 아무 계정도 남지 않으면 해당 계정으로 폴백합니다. `max-utilization`은 알려진 값 중 가장 높은 값을 사용하므로 주간 사용량을 알기 전에도 5시간 사용량을 쓸 수 있고, 둘 다 모르면 unknown 순서를 따릅니다. 알려진 사용량은 unknown보다 앞서지만, 사용 가능한 계정이 모두 unknown이어도 사용 가능한 순서의 계정을 선택합니다. 앞서 설명한 5시간 사용량 동점 판정 뒤에도 완전히 같으면 사용 가능한 순서를 유지합니다. 정상 affinity 세션을 선제적으로 재배치하지 않습니다. 새 세션 배정과 가능한 429 대체 이후 라우팅 복구에서 `quota`는 이 창으로 사용 가능한 후보의 순위를 직접 매기고, `fill-first`는 이 창의 임계값과 소진 규칙에 따라 안정 순서로 이동하며, `round-robin`은 이 설정을 무시합니다. 쿨다운, failover 한도, 재인증 가능 여부는 별도의 로컬 상태로 유지됩니다. 계정별 주간 막대는 대시보드의 프로바이더 페이지에서 조회한 뒤에만 알 수 있습니다. | | ||
| | `anthropicAccountPool.stickyLimit?` | `number` | `1` | 성공한 새 세션 결속이 한 번의 라운드로빈 선택에 유지되는 횟수입니다. 범위는 1–100입니다. | | ||
|
|
||
| 활성화되면 429 레코드가 `Retry-After` 또는 기본 backoff에서 제한된 쿨다운을 기록하고, 요청 안에서 회전할 수 있습니다. 결속은 프로세스 로컬이며 크기가 제한됩니다. 자격 증명 401/403은 해당 계정이 재인증이 필요함을 표시합니다. 적격한 계정이 모두 쿨다운 중이면, 클라이언트는 인증 오류가 아니라 알려진 경우 `Retry-After`가 포함된 429를 받습니다. | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -283,8 +283,9 @@ rotation may trigger provider restrictions. | |
| | Key | Type | Default | Description | | ||
| | --- | --- | --- | --- | | ||
| | `anthropicAccountPool.enabled?` | `boolean` | `false` | Enable sticky affinity and 429 cooldown failover. | | ||
| | `anthropicAccountPool.autoSwitchThreshold?` | `number` | `80` | For new sessions, when the active account reaches this threshold, choose the lowest known cached usage in the configured window; the account chosen does not itself have to be at or above the threshold. `0` disables **proactive** usage-based switching only — new-session selection and routing recovery after an eligible 429 still consult `quotaWindow`. | | ||
|
Contributor
There was a problem hiding this comment. 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win 🔎 Supported by static analysis🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- repository convention and learning files ---'
find /tmp/coderabbit-repo-knowledge/lidge-jun-opencodex-7afea732 \
-maxdepth 2 -type f -name '*.md' -print
printf '%s\n' '--- declaration and targeted test excerpts ---'
rg -n -C 8 'autoSwitchThreshold|quotaWindow|quota|eligible|429' \
src/oauth/anthropic-routing.ts tests/anthropic-account-pool.test.ts
printf '%s\n' '--- documentation and locale references ---'
rg -n -C 3 'autoSwitchThreshold|quotaWindow|quota selection|account pool' \
docs-site/src/content/docs/reference/configuration/providers.md \
docs-site/src/content/docs --glob '*providers.md' --glob '*configuration*' || trueRepository: lidge-jun/opencodex Length of output: 50376 🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- routing control flow ---'
sed -n '450,540p' src/oauth/anthropic-routing.ts
printf '%s\n' '--- threshold-zero and quota-window tests ---'
rg -n -C 10 'threshold.?0|threshold.?=.?0|autoSwitchThreshold: 0|quota.*429|429.*quota|quota.*selection' \
tests/anthropic-account-pool.test.ts
printf '%s\n' '--- documentation convention and scoped learnings ---'
cat /tmp/coderabbit-repo-knowledge/lidge-jun-opencodex-7afea732/conventions/docs-site.md
cat /tmp/coderabbit-repo-knowledge/lidge-jun-opencodex-7afea732/learnings/docs-site-src-content-docs.md
cat /tmp/coderabbit-repo-knowledge/lidge-jun-opencodex-7afea732/learnings/docs-site-src-content-docs-ru.md
printf '%s\n' '--- all provider configuration locale copies ---'
git ls-files 'docs-site/src/content/docs/**' | rg 'configuration/providers\.md$|providers\.md$'Repository: lidge-jun/opencodex Length of output: 12339 🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- exact scoring and failover implementations ---'
sed -n '245,345p' src/oauth/anthropic-routing.ts
sed -n '565,603p' src/oauth/anthropic-routing.ts
printf '%s\n' '--- exact threshold-zero tests and surrounding assertions ---'
sed -n '545,595p' tests/anthropic-account-pool.test.ts
printf '%s\n' '--- English and localized provider configuration entries ---'
for f in \
docs-site/src/content/docs/reference/configuration/providers.md \
docs-site/src/content/docs/ja/reference/configuration/providers.md \
docs-site/src/content/docs/ko/reference/configuration/providers.md \
docs-site/src/content/docs/ru/reference/configuration/providers.md \
docs-site/src/content/docs/zh-cn/reference/configuration/providers.md
do
printf '\n### %s\n' "$f"
rg -n -C 2 'autoSwitchThreshold|quotaWindow|anthropicAccountPool' "$f" || true
doneRepository: lidge-jun/opencodex Length of output: 38712 Align With 🤖 Prompt for AI AgentsSource: Path instructions |
||
| | `anthropicAccountPool.strategy?` | `"quota" \| "round-robin" \| "fill-first"` | `"quota"` | New-session strategy; `quota` ranks accounts by the window set by `quotaWindow`, and `fill-first` evaluates its drain threshold in that same window. | | ||
| | `anthropicAccountPool.quotaWindow?` | `"five-hour" \| "weekly" \| "max-utilization"` | `"five-hour"` | The cached provider-reported utilization bar used for usage-aware account selection. `five-hour` keeps the original behavior. `weekly` scores the weekly bar and skips accounts whose 5-hour bar is exhausted while another eligible account remains, but falls back to exhausted candidates when none do. `max-utilization` scores the highest known bar, so it can use 5-hour usage before weekly usage is available; if neither is known, the account follows unknown-usage ordering. Known usage ranks before unknown usage under the opt-in `weekly` and `max-utilization` windows only; an omitted or explicit `five-hour` preserves the legacy ordering. If every eligible account is unknown, selection still returns one in eligible order. After the documented lower-5-hour tie-break, exact ties preserve eligible order. A healthy affinity-bound session is not proactively rebalanced. For new-session assignment and routing recovery after an eligible 429 replacement, `quota` ranks eligible candidates directly with this window; `fill-first` advances in stable order using this window's threshold and exhaustion rules; `round-robin` ignores it. Cooldown, failover limits, and reauthentication eligibility remain separate local state. Per-account weekly bars are only known once the dashboard Providers page has polled them. | | ||
| | `anthropicAccountPool.stickyLimit?` | `number` | `1` | Successful new-session binds retained on one round-robin selection. Range 1–100. | | ||
|
|
||
| When enabled, 429 records bounded cooldown from `Retry-After` or a default backoff and may rotate | ||
|
|
||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
Limit known-before-unknown ordering to opt-in quota windows.
The translated reference rows describe the opt-in ordering rule as if it also applies to the default
five-hoursetting. State thatweeklyandmax-utilizationuse known-before-unknown ordering, while omitted or explicitfive-hourpreserves legacy ordering.docs-site/src/content/docs/fr/reference/configuration/providers.md#L198-L198: Scope the known-before-unknown sentence to opt-in windows.docs-site/src/content/docs/ja/reference/configuration/providers.md#L164-L164: Scope the known-before-unknown sentence to opt-in windows.docs-site/src/content/docs/zh-tw/reference/configuration/providers.md#L131-L131: Scope the known-before-unknown sentence to opt-in windows.As per path instructions, translated documentation must stay synchronized with actual CLI/API behavior. The supplied PR objective states that known-before-unknown ordering is restricted to opt-in windows and that explicit
five-hourpreserves legacy ordering.🧰 Tools
🪛 LanguageTool
[typographical] ~198-~198: Caractère d’apostrophe incorrect.
Context: ... pour la sélection selon l'utilisation.
five-hourconserve le comportement actu...(APOS_INCORRECT)
[typographical] ~198-~198: Caractère d’apostrophe incorrect.
Context: ...-hour
conserve le comportement actuel.weekly` utilise la barre hebdomadaire et...(APOS_INCORRECT)
[style] ~198-~198: Les chiffres s’écrivent généralement en lettres.
Context: ...et ignore les comptes dont la barre sur 5 heures est épuisée tant qu'un autre com...
(NOMBRES_EN_LETTRES_2)
[typographical] ~198-~198: Caractère d’apostrophe incorrect.
Context: ...mais y revient si aucun autre ne reste.
max-utilizationutilise la valeur connu...(APOS_INCORRECT)
[style] ~198-~198: Un autre mot peut être plus précis.
Context: ... ne reste.
max-utilizationutilise la valeur connue la plus élevée et peut donc employer la...(CELEBRE)
[style] ~198-~198: Les chiffres s’écrivent généralement en lettres.
Context: ...evée et peut donc employer la barre sur 5 heures avant que la barre hebdomadaire ...
(NOMBRES_EN_LETTRES_2)
[style] ~198-~198: Un autre mot peut être plus précis.
Context: ...l'ordre des utilisations inconnues. Les utilisations connues précèdent les inconnues, mais si tous l...
(CELEBRE)
[style] ~198-~198: Les chiffres s’écrivent généralement en lettres.
Context: ...enté par la plus faible utilisation sur 5 heures, une égalité exacte conserve cet...
(NOMBRES_EN_LETTRES_2)
[style] ~198-~198: Un verbe peut dynamiser votre phrase.
Context: ...rééquilibrée de manière proactive. Pour l'affectation des nouvelles sessions et la reprise du rou...
(PROP_NOMINALES_EN_VERBALES)
📍 Affects 3 files
docs-site/src/content/docs/fr/reference/configuration/providers.md#L198-L198(this comment)docs-site/src/content/docs/ja/reference/configuration/providers.md#L164-L164docs-site/src/content/docs/zh-tw/reference/configuration/providers.md#L131-L131🤖 Prompt for AI Agents
Source: Path instructions