Repository navigation
fix(mcp): Retry rate-limited GETs using Sentry rate-limit reset headers - #1673
Conversation
The hosted MCP saw about 32,600 Sentry API 429s in 7 days, nearly all on
GETs ("Limit is 40 requests in 1 seconds"). request() only retried
502/503/504, so every one failed the tool call, and ApiRateLimitError's
retryAfter read body fields Sentry never sends.
Add a pure retry policy in toolkit-core/api-retry (OpenSpec task 5.0 in
#1669). It retries only GET/HEAD/OPTIONS unless a request opts in as
idempotent, waits for Retry-After or X-Sentry-Rate-Limit-Reset with a
Date-header clock-skew guard and per-wait/total caps, falls back to
jittered backoff for concurrency 429s and gateway errors, and sleeps
abortably.
SentryApiService.request() now retries 429 and 502/503/504 on reads, at most
2 times, 2s per wait, and 3s in total. A reset further out fails at once with
an ApiRateLimitError whose retryAfterMs and message say when to retry. Writes
are still never retried.
Co-Authored-By: Burak Yigit Kaya <126780+BYK@users.noreply.github.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit a4f9a98. Configure here.
A local clock up to 2s behind the response Date was treated as in sync, so a 1s Sentry window could look 2s+ away and exceed the MCP's 2s per-wait cap, dropping a retry that would have succeeded. Clamp the local estimate into the range the Date header allows instead of switching clocks at a threshold.
…back off Sentry's rate-limit middleware rejects a 429 in process_request, before the endpoint runs, so replaying it is safe for any method. Retry 429s for writes too; 5xx responses still retry only for GET/HEAD since the endpoint may have run. Add 500 to the MCP's retryable statuses; 501, 505, and other 5xx codes stay out because they are deterministic. Format rate-limit tool errors as "Rate Limited" instead of "Input Error", with the retry time and advice to stop calling tools in parallel.
…uses Sentry's RatelimitMiddleware rejects a 429 in process_request, before the view runs, so replaying one cannot repeat a write. The sentry-api-transport spec now retries 429s for any method, and retries 5xx, timeouts, and network errors only for GET/HEAD/OPTIONS or an explicit idempotent opt-in. The retried statuses are 429, 500, 502, 503, and 504. Other 5xx codes are deterministic. The clock-skew wording now matches the Date-bounded delay from #1673. The long requirements are split so `openspec validate --strict` passes.
BYK
left a comment
There was a problem hiding this comment.
We should then abstract this logic out either into toolkit-core or @sentry/api (ideally) and make it shared across CLI and MCP
| `SentryApiService.request()` transparently retries rate limits (`429`) and | ||
| transient upstream gateway failures (`502`/`503`/`504`), using the shared | ||
| policy in `@sentry/toolkit-core/api-retry`. Retries are: | ||
| transient server failures (`500`/`502`/`503`/`504`), using the shared policy in |
There was a problem hiding this comment.
We should link this information to the code or from the code. Ideally the code should have something like:
const RETRIABLES = {
429: true,
500: ["GET", "HEAD", "OPTIONS"],
};or other way around:
const RETRIABLES = {
"GET": [429, 500, 502, 503, 504],
"POST": [429],
}etc. Or some rule. Also ideally these will be Map and Set instances yadda yadda.
There was a problem hiding this comment.
Done in abc41b8. packages/toolkit-core/src/api-retry.ts now has RETRYABLE_STATUSES: ReadonlyMap<number, "any" | "idempotent">, which maps 429 to any and 500/502/503/504 to idempotent. IDEMPOTENT_METHODS is a ReadonlySet. The any/idempotent rationale lives on the map. RetryPolicy lost retryableStatuses, so a product only sets caps and can't drift from the rule. This doc section now shows the same table and links to the file, and the code comment points back here.
Replace the per-product retryableStatuses list and the 429 special case
with RETRYABLE_STATUSES, a Map from status to the requests it may replay
("any" for 429, "idempotent" for 500/502/503/504). Products now set only
caps, so the CLI and MCP share one rule. The contributor docs show the
table and link to the code.
|
On sharing this between CLI and MCP: the rule now lives only in I haven't switched the CLI in this PR. #1677 (
|
The contributor table and caps copied RETRYABLE_STATUSES and REQUEST_RETRY_POLICY by hand, so they could drift. Point to the code instead, and add a short user-facing note on rate limits to the README.
…reset on 429 (#1677) ## Problem `packages/cli/src/lib/sentry-client.ts` retried **every** HTTP method, including POST/PUT/DELETE, on 408/429/500/502/503/504 and on network errors and timeouts. That can replay writes. A 429 also only got the generic 1s/2s backoff: the client never read Sentry's `X-Sentry-Rate-Limit-Reset` header, and Sentry doesn't send `Retry-After` on API rate limits. ## Behavior changes - **5xx, timeouts and network errors** now retry only for GET, HEAD and OPTIONS, or when a request sets the new `idempotent: true` opt-in on `getSdkConfig` / `ApiRequestOptions`. - **429 retries for any method.** Sentry's rate-limit middleware rejects the request in `process_view`, before the endpoint runs, so nothing was applied. - **Retried statuses stay 408/429/500/502/503/504.** Deterministic 5xx like 501 and 505 are not retried. - **PUT and DELETE are deliberately not retried.** Sentry PUTs have side effects (merge, activity), and a replayed DELETE returns 404. - **The 401-refresh retry still covers every method**, and no longer waits 1s first. - **A timed-out write** now gets a "may still have been applied" hint. - **The 429 wait** is the longer of `X-Sentry-Rate-Limit-Reset` (read only when Remaining is 0) and `Retry-After` (seconds or HTTP date), both measured against the response `Date` to guard against clock skew. Concurrency-limit 429s, or 429s with no hint, use jittered backoff: the upper half of min(1s·2^n, 10s). - **Caps are unchanged:** `MAX_RETRIES` is 2 and each wait is at most 10s. - **New `RateLimitError`** (extends `ApiError`, status 429). It's thrown when the wait would exceed the cap or retries run out. The message is agent-friendly: server detail, then "Retry in N seconds; do not repeat the request before then. Make fewer requests in parallel." - **The policy is pure**, with no CLI imports, in `packages/cli/src/lib/retry-policy.ts`. That way the CLI can switch to the shared `@sentry/toolkit-core/api-retry` from #1673 without a rewrite. ## POST callers given the `idempotent` opt-in Each was checked against its getsentry/sentry handler: - **Chunk uploads** (`chunk-upload.ts`): blobs are stored by SHA-1 (`FileBlob.from_files`). - **Artifact-bundle assemble and `pollAssembly`** (`sourcemaps.ts`). - **DIF assemble** in `debug-files.ts`, `proguard.ts` and `dart-symbols.ts`: the server checks the cached assemble status by checksum. **Not opted in:** preprod assemble (each call creates a new `PreprodArtifact`), Seer `/autofix/`, snapshot create and archive trigger, and SDK and `sentry api` writes. ## Tests - The POST-retry body-cloning tests now go through the 429 and opt-in paths. - New cases: - a rate-limited POST is resent with the same body - a 501 is not retried - a 503 on a non-idempotent method is not retried - the reset-header and `Retry-After` wait, clock skew, and the cap leading to `RateLimitError` - the exact rate-limit message text - `api-client.coverage` used 429 as a sample error and now uses 400. ## Checks - `tsc --noEmit` and `pnpm run lint` pass. - The sentry-client, retry-policy, api-client and api-client.coverage tests pass: 204/204. - Full CLI suite on `bf3b710`: 11,482 passed and 6 failed. The same 6 fail on `main` (`37c7193`) in this sandbox. They're chmod-based permission tests (completions ×2, code scanner, init file-changes, telemetry readonly, upgrade lock), and the sandbox runs with full capabilities, so they can't fail the way they expect. - After the 429-any-method change (`7099e2b`), all other CLI test files that mention 429 pass (1,195), apart from the 2 known sandbox permission failures. ## Relation to #1669 This matches the retry policy in #1669 (task 5.0 and the `sentry-api-transport` spec). Task 5.1 will move this behavior into toolkit-core's shared transport unchanged. <!-- junior-request-attribution:start --> via **BYK**. <!-- junior-request-attribution:end --> <!-- junior-session-footer:start --> <!-- junior-conversation-id:slack%3AC08J1NSPU6S%3A1792772794.175999 --> -- [View Junior Session](https://junior-prod.sentry.dev/conversations/slack%3AC08J1NSPU6S%3A1792772794.175999) [[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AC08J1NSPU6S%3A1792772794.175999/?project=4510944073809921) <!-- junior-session-footer:end --> --------- Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com> Co-authored-by: Burak Yigit Kaya <126780+BYK@users.noreply.github.com>

The hosted MCP gets rate limited by the Sentry API a lot. In the last 7 days
sentry/mcp-serverlogged about 32.6Khttp.client429s, nearly all on GETs, usually "Limit is 40 requests in 1 seconds", plus about 72 concurrency-limit hits.SentryApiService.request()only retried 502/503/504, so every 429 failed the tool call.ApiRateLimitError.retryAfterwas read from body fields Sentry never sends, so nothing could say when to try again.Sentry's 429 body is just
{"detail": ...}. The timing comes in headers set bysrc/sentry/middleware/ratelimit.py:X-Sentry-Rate-Limit-Resetis the Unix second when the next window starts, andRemainingis0only when the fixed window is used up. Concurrency 429s carry a reset that doesn't apply to them.This adds
@sentry/toolkit-core/api-retry, a pure retry policy with no Node APIs, so it is safe on Workers. It is task 5.0 in #1669. The MCP adopts it here:idempotentopt-in exists for the CLI's content-addressed uploads, but the MCP doesn't use it.Retry-After(seconds or HTTP-date) if a proxy sent one, otherwise untilX-Sentry-Rate-Limit-Reset. The reset is measured against the responseDatewhen the local clock is off by more than 2s, plus a small jitter. Concurrency 429s and gateway errors use equal-jitter exponential backoff starting at 250ms.ApiRateLimitErrorwith a realretryAfterMs, and its message ends with "Retry in N seconds." When there is no header, it says to wait a few seconds and make fewer parallel requests.signalends the wait immediately.ApiRateLimitError.retryAfter(seconds, from the body) is replaced byretryAfterMs, taken from the headers. Nothing read the old field.The CLI currently has its own interim retry fix in a separate CLI-only PR. It moves onto this shared module in a follow-up after that PR lands. Once the MCP is on the shared transport (#1669 task 6.1),
SentryApiServicedrops this loop so retries don't stack.via @BYK.
--
View Junior Session [Sentry]