Skip to content

fix(mcp): Retry rate-limited GETs using Sentry rate-limit reset headers - #1673

Merged
BYK merged 5 commits into
mainfrom
fix/mcp-429-retry
Oct 27, 2026
Merged

BYK merged 5 commits into
mainfrom
fix/mcp-429-retry

Conversation

@sentry-junior

@sentry-junior sentry-junior Bot commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

The hosted MCP gets rate limited by the Sentry API a lot. In the last 7 days sentry/mcp-server logged about 32.6K http.client 429s, nearly all on GETs, usually "Limit is 40 requests in 1 seconds", plus about 72 concurrency-limit hits. SentryApiService.request() only retried 502/503/504, so every 429 failed the tool call. ApiRateLimitError.retryAfter was read from body fields Sentry never sends, so nothing could say when to try again.

Sentry's 429 body is just {"detail": ...}. The timing comes in headers set by src/sentry/middleware/ratelimit.py: X-Sentry-Rate-Limit-Reset is the Unix second when the next window starts, and Remaining is 0 only when the fixed window is used up. Concurrency 429s carry a reset that doesn't apply to them.

This adds @sentry/toolkit-core/api-retry, a pure retry policy with no Node APIs, so it is safe on Workers. It is task 5.0 in #1669. The MCP adopts it here:

  • What gets retried: GET and HEAD requests that get a 429 or a 502/503/504. Writes are never retried. The policy's idempotent opt-in exists for the CLI's content-addressed uploads, but the MCP doesn't use it.
  • How long a 429 waits: until Retry-After (seconds or HTTP-date) if a proxy sent one, otherwise until X-Sentry-Rate-Limit-Reset. The reset is measured against the response Date when the local clock is off by more than 2s, plus a small jitter. Concurrency 429s and gateway errors use equal-jitter exponential backoff starting at 250ms.
  • Caps: at most 2 retries, 2s per wait and 3s in total. Sentry's windows are mostly 1s, so the reset is usually under a second away and both retries fit in about 2s. A reset further out than 2s means a longer window. Then the call fails at once instead of holding the Worker request open. The error is an ApiRateLimitError with a real retryAfterMs, and its message ends with "Retry in N seconds." When there is no header, it says to wait a few seconds and make fewer parallel requests.
  • Abort: aborting the request's signal ends the wait immediately.

ApiRateLimitError.retryAfter (seconds, from the body) is replaced by retryAfterMs, taken from the headers. Nothing read the old field.

The CLI currently has its own interim retry fix in a separate CLI-only PR. It moves onto this shared module in a follow-up after that PR lands. Once the MCP is on the shared transport (#1669 task 6.1), SentryApiService drops this loop so retries don't stack.

via @BYK.

--

View Junior Session [Sentry]

The hosted MCP saw about 32,600 Sentry API 429s in 7 days, nearly all on
GETs ("Limit is 40 requests in 1 seconds"). request() only retried
502/503/504, so every one failed the tool call, and ApiRateLimitError's
retryAfter read body fields Sentry never sends.

Add a pure retry policy in toolkit-core/api-retry (OpenSpec task 5.0 in
#1669). It retries only GET/HEAD/OPTIONS unless a request opts in as
idempotent, waits for Retry-After or X-Sentry-Rate-Limit-Reset with a
Date-header clock-skew guard and per-wait/total caps, falls back to
jittered backoff for concurrency 429s and gateway errors, and sleeps
abortably.

SentryApiService.request() now retries 429 and 502/503/504 on reads, at most
2 times, 2s per wait, and 3s in total. A reset further out fails at once with
an ApiRateLimitError whose retryAfterMs and message say when to retry. Writes
are still never retried.

Co-Authored-By: Burak Yigit Kaya <126780+BYK@users.noreply.github.com>
@vercel

vercel Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
sentry-local Ready Ready Preview Oct 27, 2026 3:30pm UTC

Request Review

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit a4f9a98. Configure here.

Comment thread packages/toolkit-core/src/api-retry.ts
A local clock up to 2s behind the response Date was treated as in sync, so a
1s Sentry window could look 2s+ away and exceed the MCP's 2s per-wait cap,
dropping a retry that would have succeeded. Clamp the local estimate into the
range the Date header allows instead of switching clocks at a threshold.
Comment thread packages/mcp-core/src/api-client/client.test.ts Outdated
Comment thread packages/mcp-core/src/api-client/client.ts Outdated
Comment thread packages/mcp-core/src/api-client/errors.ts Outdated
…back off

Sentry's rate-limit middleware rejects a 429 in process_request, before the
endpoint runs, so replaying it is safe for any method. Retry 429s for writes
too; 5xx responses still retry only for GET/HEAD since the endpoint may have
run. Add 500 to the MCP's retryable statuses; 501, 505, and other 5xx codes
stay out because they are deterministic.

Format rate-limit tool errors as "Rate Limited" instead of "Input Error",
with the retry time and advice to stop calling tools in parallel.
sentry-junior Bot added a commit that referenced this pull request Oct 27, 2026
…uses

Sentry's RatelimitMiddleware rejects a 429 in process_request, before the view
runs, so replaying one cannot repeat a write. The sentry-api-transport spec
now retries 429s for any method, and retries 5xx, timeouts, and network errors
only for GET/HEAD/OPTIONS or an explicit idempotent opt-in.

The retried statuses are 429, 500, 502, 503, and 504. Other 5xx codes are
deterministic. The clock-skew wording now matches the Date-bounded delay from
#1673. The long requirements are split so `openspec validate --strict` passes.

@BYK BYK left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should then abstract this logic out either into toolkit-core or @sentry/api (ideally) and make it shared across CLI and MCP

Comment thread docs/contributing/api-patterns.md Outdated
`SentryApiService.request()` transparently retries rate limits (`429`) and
transient upstream gateway failures (`502`/`503`/`504`), using the shared
policy in `@sentry/toolkit-core/api-retry`. Retries are:
transient server failures (`500`/`502`/`503`/`504`), using the shared policy in

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should link this information to the code or from the code. Ideally the code should have something like:

const RETRIABLES = {
  429: true,
  500: ["GET", "HEAD", "OPTIONS"],
};

or other way around:

const RETRIABLES = {
  "GET": [429, 500, 502, 503, 504],
  "POST": [429],
}

etc. Or some rule. Also ideally these will be Map and Set instances yadda yadda.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in abc41b8. packages/toolkit-core/src/api-retry.ts now has RETRYABLE_STATUSES: ReadonlyMap<number, "any" | "idempotent">, which maps 429 to any and 500/502/503/504 to idempotent. IDEMPOTENT_METHODS is a ReadonlySet. The any/idempotent rationale lives on the map. RetryPolicy lost retryableStatuses, so a product only sets caps and can't drift from the rule. This doc section now shows the same table and links to the file, and the code comment points back here.

Replace the per-product retryableStatuses list and the 429 special case
with RETRYABLE_STATUSES, a Map from status to the requests it may replay
("any" for 429, "idempotent" for 500/502/503/504). Products now set only
caps, so the CLI and MCP share one rule. The contributor docs show the
table and link to the code.
@sentry-junior

sentry-junior Bot commented Oct 27, 2026

Copy link
Copy Markdown
Contributor Author

On sharing this between CLI and MCP: the rule now lives only in @sentry/toolkit-core/api-retry, as the RETRYABLE_STATUSES map, the header parsing, and nextRetryDelayMs/sleep. The MCP supplies just its caps (abc41b8).

I haven't switched the CLI in this PR. #1677 (fix/cli-retry-idempotent-429) is open and rewrites the same retry code in packages/cli/src/lib/sentry-client.ts, so touching it here would conflict. The CLI moves onto this module in a follow-up once #1677 lands. That's task 5.0's "CLI follow-up" in #1669, and 5.1 then moves the whole transport.

@sentry/api is the better long-term home, and #1669 already lists "move pagination, then the transport, into @sentry/api" as a follow-up. I kept it in toolkit-core for now so this fix doesn't wait on an @sentry/api release.

Comment thread docs/contributing/api-patterns.md
The contributor table and caps copied RETRYABLE_STATUSES and
REQUEST_RETRY_POLICY by hand, so they could drift. Point to the code
instead, and add a short user-facing note on rate limits to the README.
@BYK
BYK merged commit 9ef6ab5 into main Oct 27, 2026
38 checks passed
@BYK
BYK deleted the fix/mcp-429-retry branch October 27, 2026 17:20
BYK added a commit that referenced this pull request Oct 27, 2026
…reset on 429 (#1677)

## Problem
`packages/cli/src/lib/sentry-client.ts` retried **every** HTTP method,
including POST/PUT/DELETE, on 408/429/500/502/503/504 and on network
errors and timeouts. That can replay writes. A 429 also only got the
generic 1s/2s backoff: the client never read Sentry's
`X-Sentry-Rate-Limit-Reset` header, and Sentry doesn't send
`Retry-After` on API rate limits.

## Behavior changes
- **5xx, timeouts and network errors** now retry only for GET, HEAD and
OPTIONS, or when a request sets the new `idempotent: true` opt-in on
`getSdkConfig` / `ApiRequestOptions`.
- **429 retries for any method.** Sentry's rate-limit middleware rejects
the request in `process_view`, before the endpoint runs, so nothing was
applied.
- **Retried statuses stay 408/429/500/502/503/504.** Deterministic 5xx
like 501 and 505 are not retried.
- **PUT and DELETE are deliberately not retried.** Sentry PUTs have side
effects (merge, activity), and a replayed DELETE returns 404.
- **The 401-refresh retry still covers every method**, and no longer
waits 1s first.
- **A timed-out write** now gets a "may still have been applied" hint.
- **The 429 wait** is the longer of `X-Sentry-Rate-Limit-Reset` (read
only when Remaining is 0) and `Retry-After` (seconds or HTTP date), both
measured against the response `Date` to guard against clock skew.
Concurrency-limit 429s, or 429s with no hint, use jittered backoff: the
upper half of min(1s·2^n, 10s).
- **Caps are unchanged:** `MAX_RETRIES` is 2 and each wait is at most
10s.
- **New `RateLimitError`** (extends `ApiError`, status 429). It's thrown
when the wait would exceed the cap or retries run out. The message is
agent-friendly: server detail, then "Retry in N seconds; do not repeat
the request before then. Make fewer requests in parallel."
- **The policy is pure**, with no CLI imports, in
`packages/cli/src/lib/retry-policy.ts`. That way the CLI can switch to
the shared `@sentry/toolkit-core/api-retry` from #1673 without a
rewrite.

## POST callers given the `idempotent` opt-in
Each was checked against its getsentry/sentry handler:
- **Chunk uploads** (`chunk-upload.ts`): blobs are stored by SHA-1
(`FileBlob.from_files`).
- **Artifact-bundle assemble and `pollAssembly`** (`sourcemaps.ts`).
- **DIF assemble** in `debug-files.ts`, `proguard.ts` and
`dart-symbols.ts`: the server checks the cached assemble status by
checksum.

**Not opted in:** preprod assemble (each call creates a new
`PreprodArtifact`), Seer `/autofix/`, snapshot create and archive
trigger, and SDK and `sentry api` writes.

## Tests
- The POST-retry body-cloning tests now go through the 429 and opt-in
paths.
- New cases:
  - a rate-limited POST is resent with the same body
  - a 501 is not retried
  - a 503 on a non-idempotent method is not retried
- the reset-header and `Retry-After` wait, clock skew, and the cap
leading to `RateLimitError`
  - the exact rate-limit message text
- `api-client.coverage` used 429 as a sample error and now uses 400.

## Checks
- `tsc --noEmit` and `pnpm run lint` pass.
- The sentry-client, retry-policy, api-client and api-client.coverage
tests pass: 204/204.
- Full CLI suite on `bf3b710`: 11,482 passed and 6 failed. The same 6
fail on `main` (`37c7193`) in this sandbox. They're chmod-based
permission tests (completions ×2, code scanner, init file-changes,
telemetry readonly, upgrade lock), and the sandbox runs with full
capabilities, so they can't fail the way they expect.
- After the 429-any-method change (`7099e2b`), all other CLI test files
that mention 429 pass (1,195), apart from the 2 known sandbox permission
failures.

## Relation to #1669
This matches the retry policy in #1669 (task 5.0 and the
`sentry-api-transport` spec). Task 5.1 will move this behavior into
toolkit-core's shared transport unchanged.

<!-- junior-request-attribution:start -->
via **BYK**.
<!-- junior-request-attribution:end -->

<!-- junior-session-footer:start -->
<!-- junior-conversation-id:slack%3AC08J1NSPU6S%3A1792772794.175999 -->

--

[View Junior
Session](https://junior-prod.sentry.dev/conversations/slack%3AC08J1NSPU6S%3A1792772794.175999)
[[Sentry]](https://sentry.sentry.io/explore/conversations/slack%3AC08J1NSPU6S%3A1792772794.175999/?project=4510944073809921)

<!-- junior-session-footer:end -->

---------

Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com>
Co-authored-by: Burak Yigit Kaya <126780+BYK@users.noreply.github.com>

This branch was successfully deployed

1 active deployment
Preview — 6c293837 Deployed Oct 27, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk: medium PR risk score: medium

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant