-
Notifications
You must be signed in to change notification settings - Fork 960
fix(responses): keep the terminal-guard continuation on the shared transient budget #2998
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -6144,7 +6144,15 @@ async function handleResponsesInner( | |
| { | ||
| abortSignal: upstream.signal, | ||
| label: safeHostLabel(builtContinuationRequest.url), | ||
| ...(continuationTransientPolicy ? { attempts: continuationTransientPolicy.attempts } : {}), | ||
| // Same request-scoped budget as the initial send and the 429/rotation refetches: | ||
| // a terminal-guard continuation is another leg of ONE request, so handing it a | ||
| // fresh `attempts` would let one request exceed the configured total-send ceiling. | ||
| ...(continuationTransientPolicy | ||
| ? { | ||
| attempts: remainingTransientSendBudget(continuationTransientPolicy.attempts), | ||
| onSendsConsumed: noteTransientSends, | ||
| } | ||
| : {}), | ||
|
Comment on lines
+6150
to
+6155
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win 🔎 Supported by static analysis🏁 Script executed: #!/usr/bin/env bash
set -euo pipefail
rg -n -C 10 \
'transientRetryPolicyFor|fetchWithTransientRetry|adapter === "google"|emptyCompletionRetryEnabled|fetchGuardedEmptyCompletionRetry' \
src testsRepository: lidge-jun/opencodex Length of output: 50375 🏁 Script executed: #!/usr/bin/env bash
set -euo pipefail
printf '%s\n' '--- src/server/responses/core.ts: relevant call sites ---'
sed -n '5525,5730p' src/server/responses/core.ts
sed -n '6085,6175p' src/server/responses/core.ts
printf '%s\n' '--- src/providers/key-failover.ts: policy contract ---'
sed -n '120,150p' src/providers/key-failover.ts
printf '%s\n' '--- src/lib/upstream-retry.ts: option defaults and callback ---'
sed -n '330,455p' src/lib/upstream-retry.tsRepository: lidge-jun/opencodex Length of output: 22129 🏁 Script executed: #!/usr/bin/env bash
set -euo pipefail
printf '%s\n' '--- shared transient-budget definitions in src/server/responses/core.ts ---'
rg -n -C 8 \
'remainingTransientSendBudget|noteTransientSends|transientSend|TRANSIENT' \
src/server/responses/core.ts
printf '%s\n' '--- source-level regression contract ---'
sed -n '1,80p' tests/transient-budget-scope-source.test.ts
printf '%s\n' '--- applicable repository convention and learning files ---'
find /tmp/coderabbit-repo-knowledge/lidge-jun-opencodex-7afea732 -maxdepth 2 -type f -name '*.md' -printRepository: lidge-jun/opencodex Length of output: 8411 Preserve the request-wide transient budget for direct Google retries.
Apply the shared budget whenever 🤖 Prompt for AI Agents |
||
| }, | ||
| ); | ||
| } finally { | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,52 @@ | ||
| import { describe, expect, test } from "bun:test"; | ||
| import { readFileSync } from "node:fs"; | ||
| import { join } from "node:path"; | ||
|
|
||
| const source = (relative: string): string => | ||
| readFileSync(join(import.meta.dir, "..", "src", ...relative.split("/")), "utf8"); | ||
|
|
||
| /** | ||
| * `transientRetryOn5xx.attempts` is ONE request-wide total-send budget, not a per-leg | ||
| * allowance. A Responses request can reach upstream on several legs — the initial send, a | ||
| * 429/account-rotation refetch, and the terminal-guard continuation — and each leg calls | ||
| * `fetchWithTransientRetry` separately. The budget only holds if every leg draws from the | ||
| * shared request-scoped counter. | ||
| * | ||
| * The continuation leg shipped on the raw policy value instead, so a request that reached it | ||
| * received a fresh full `attempts` allowance: with `attempts: 3` an initial send that had | ||
| * already spent its budget could still emit three more upstream sends. Runtime coverage in | ||
| * `tests/upstream-transient-retry.test.ts` proves the helper reports and honors a remainder; | ||
| * it cannot prove that every call site asks for one, because a site that forgets simply | ||
| * passes a larger number. This asserts the wiring at the source, which is the only place the | ||
| * omission is visible. | ||
| */ | ||
| describe("transient send budget stays request-scoped", () => { | ||
| test("every transient-retry call site draws from the shared counter", () => { | ||
| const core = source("server/responses/core.ts"); | ||
|
|
||
| // One owner per request, declared before any leg can send. | ||
| expect(core.match(/let transientSendsUsed = 0;/g)).toHaveLength(1); | ||
| expect(core.match(/const remainingTransientSendBudget = \(budget: number\): number =>/g)).toHaveLength(1); | ||
|
|
||
| // Initial send, 429/rotation refetch, and terminal-guard continuation: three legs, three | ||
| // reports into the same counter. | ||
| expect(core.match(/onSendsConsumed: noteTransientSends/g)).toHaveLength(3); | ||
|
|
||
| // The refetch and continuation legs must ask for the REMAINDER. Only the initial send may | ||
| // pass a policy value directly, because nothing has been spent yet. | ||
| expect(core.match(/attempts: remainingTransientSendBudget\(/g)).toHaveLength(2); | ||
| expect(core).toContain("attempts: remainingTransientSendBudget(refetchTransientPolicy.attempts)"); | ||
| expect(core).toContain("attempts: remainingTransientSendBudget(continuationTransientPolicy.attempts)"); | ||
|
|
||
| // The regressed shape: a leg handing itself a fresh full budget. | ||
| expect(core).not.toContain("attempts: continuationTransientPolicy.attempts }"); | ||
| expect(core).not.toContain("attempts: refetchTransientPolicy.attempts }"); | ||
| }); | ||
|
|
||
| test("the helper still exposes the seam those call sites depend on", () => { | ||
| const retry = source("lib/upstream-retry.ts"); | ||
| expect(retry).toContain("onSendsConsumed?: (sends: number) => void;"); | ||
| // Reported in `finally` so every exit path — return, throw, abort — feeds the counter. | ||
| expect(retry).toMatch(/} finally \{\n\s*opts\.onSendsConsumed\?\.\(sent\);/); | ||
| }); | ||
| }); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When the initial leg consumes every configured attempt but succeeds on its final send and then triggers the terminal guard—for example,
503, 503, 200withattempts: 3—remainingTransientSendBudgetfloors the remainder at 1, so this call still makes a fourth upstream request. Withattempts: 1, every guarded response similarly produces two sends. This still violates the documented request-wide total-send ceiling; allow an exhausted budget to prevent the continuation fetch, and add a focused server regression covering this exact sequence.AGENTS.md reference: src/AGENTS.md:L24-L25
Useful? React with 👍 / 👎.