From 880d25230f04994741b421306ae4bd9575251be4 Mon Sep 17 00:00:00 2001 From: "sentry-junior[bot]" <264270552+sentry-junior[bot]@users.noreply.github.com> Date: Mon, 26 Oct 2026 20:16:57 +0000 Subject: [PATCH 1/8] docs(openspec): propose a shared Sentry API layer for the CLI and MCP Share pagination, response caching, and request transport between the CLI and MCP through toolkit-core on top of @sentry/api, let sentry mcp supply the CLI's authenticated transport, and add a tiered (memory + R2), zstd-compressed cache for the hosted MCP. Co-Authored-By: BYK --- .../unify-sentry-api-layer/.openspec.yaml | 2 + .../changes/unify-sentry-api-layer/design.md | 278 ++++++++++++++++++ .../unify-sentry-api-layer/proposal.md | 89 ++++++ .../specs/mcp-host-transport/spec.md | 64 ++++ .../specs/sentry-api-pagination/spec.md | 72 +++++ .../specs/sentry-api-response-cache/spec.md | 129 ++++++++ .../changes/unify-sentry-api-layer/tasks.md | 63 ++++ 7 files changed, 697 insertions(+) create mode 100644 openspec/changes/unify-sentry-api-layer/.openspec.yaml create mode 100644 openspec/changes/unify-sentry-api-layer/design.md create mode 100644 openspec/changes/unify-sentry-api-layer/proposal.md create mode 100644 openspec/changes/unify-sentry-api-layer/specs/mcp-host-transport/spec.md create mode 100644 openspec/changes/unify-sentry-api-layer/specs/sentry-api-pagination/spec.md create mode 100644 openspec/changes/unify-sentry-api-layer/specs/sentry-api-response-cache/spec.md create mode 100644 openspec/changes/unify-sentry-api-layer/tasks.md diff --git a/openspec/changes/unify-sentry-api-layer/.openspec.yaml b/openspec/changes/unify-sentry-api-layer/.openspec.yaml new file mode 100644 index 000000000..d58ba0232 --- /dev/null +++ b/openspec/changes/unify-sentry-api-layer/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-10-26 diff --git a/openspec/changes/unify-sentry-api-layer/design.md b/openspec/changes/unify-sentry-api-layer/design.md new file mode 100644 index 000000000..b9d188260 --- /dev/null +++ b/openspec/changes/unify-sentry-api-layer/design.md @@ -0,0 +1,278 @@ +# Design + +## Context + +See proposal.md for motivation. Current state that shapes the approach: + +- **Dependency direction.** `packages/cli` depends on `@sentry/mcp-server` + (`workspace:*`), and its build compiles `mcp-core` and `mcp-server` first. + `cli/src/lib/mcp.ts` already passes credentials into `runMcpServer()` + through `resolveAccessToken` and `onUpstreamUnauthorized`. Shared code + must therefore live below both products, in `toolkit-core`. +- **CLI request stack.** + - `sentry-client.ts` handles auth, refresh on 401, timeouts, GET retries, + redirect validation, and custom headers. + - `response-cache.ts` caches responses as JSON files under + `~/.sentry/cache/responses/`, using `http-cache-semantics`, an HMAC + identity key, TTL tiers, and a 500-entry cap. + - `api/infrastructure.ts` has `paginate()` and `fetchAllPages()`, with + mid-page skip counts encoded in the cursor's `is_prev` field. +- **MCP request stack.** `mcp-core/src/api-client/client.ts` has about 105 + methods and hits 91 distinct endpoint paths. It owns its own + `request`/`requestOnce`, does GET-only gateway retries, validates with Zod, + and has no response cache. Against `@sentry/api@0.404.0`, about 67 of + those paths have generated operations; the rest are missing, mainly + legacy alert rules, uptime, monitor stats, issue activities/notes/user + reports, the search agent, and event committers. +- **Workers runtime.** + - 128 MB memory per isolate, and one isolate serves many users at once. + - No persistent filesystem. + - `nodejs_compat` is enabled: `AsyncLocalStorage`, `node:crypto`, and + `node:zlib` are available. + - `node:zlib` zstd works under the current compatibility date + (`2025-03-21`) in local workerd, verified with `wrangler dev` 4.151.3. + Cloudflare's docs list only gzip, deflate, and brotli, so this must be + confirmed on a deployed preview. + - `CompressionStream` has no zstd. +- **`@sentry/api` already has** `fetchPage_*`, `paginateAll_*`, and + `paginateUpTo_*` helpers, plus `./zod`, `./valibot`, and `./browser` + entry points. Its `paginateUpTo` drops `nextCursor` when trimming a page + would make continuing unsafe. The CLI's skip-count cursor closes exactly + that gap. + +## Goals / Non-Goals + +**Goals:** + +- One implementation of pagination, cache policy, and transport, used by + the CLI, `sentry mcp`, and the hosted MCP. +- Shared modules run unchanged in Node and Workers, and avoid + Node-only APIs where a web standard exists, so they can later move into + `@sentry/api` and run in the browser. +- No CLI behavior change while its internals move. + +**Non-Goals:** + +- Client-side caching in the Sentry frontend. Browser HTTP caching driven + by server headers covers that (getsentry/sentry#128818). +- Using `createSentrySDK()` from the MCP. +- Moving code into `@sentry/api` in this change. +- Replacing MCP Zod response validation with generated schemas everywhere. + That happens per domain where `@sentry/api/zod` covers the endpoint. + +## Decisions + +### Shared code lives in `toolkit-core`, behind injected dependencies + +New `toolkit-core` modules: + +- `pagination`: `paginate`, `fetchAllPages`, cursor encode/decode, and page + sizing. Pure logic. Errors come back as typed results, and the host maps + them to its own error classes. +- `response-cache`: the cache policy. Covers tier classification, key + derivation, invalidation prefixes (from `cli/src/lib/cache-keys.ts`), and + freshness through `http-cache-semantics`. Storage sits behind a + `CacheStore` interface. +- `api-transport`: a fetch wrapper that takes a `CredentialProvider` (get the + token, refresh on 401), an optional `CacheStore`, a logger, a span hook, + and retry/timeout options. Redirect validation and region routing move + with it. + +Product code keeps credential storage, error classes, user-facing +messages, and telemetry setup. + +_Alternative: a new package._ Rejected for now. `toolkit-core` is already +the bundled-private shared package, and a split can come later when moving +code into `@sentry/api`. + +### `CacheStore` is three byte-based operations + +```ts +interface CacheStore { + get(key: string): Promise; + set(key: string, entry: CacheEntry, ttlMs: number): Promise; + deleteMatching(identity: string, urlPrefix: string): Promise; +} +``` + +`CacheEntry` stores the compressed body bytes plus metadata: URL, identity, +tier, encoding, serialized cache policy, and size. Every backend enforces a +byte budget and a per-entry cap, not an entry count. + +Backends: + +- **Filesystem (CLI).** Keeps today's on-disk layout and adds compression. + It moves from a 500-entry cap to a byte budget, with the default set so + current users see a similar footprint. +- **Memory (hosted MCP, tests).** + - LRU limited by bytes, counted over compressed bytes. + - Default budget 24 MB per isolate and a 1 MB per-entry cap, configurable + by env. + - Module-scoped, so it lives as long as the isolate. `deleteMatching` + scans the in-memory index. +- **R2 (hosted MCP, immutable tier only).** + - Object key is `/`, with the expiry in + custom metadata and checked on read. + - A bucket lifecycle rule deletes objects after 1 day. + - Writes happen in `ctx.waitUntil()`, so they never delay a response. + - The immutable tier is never invalidated, so R2 never needs + `deleteMatching`. + +_Alternatives:_ + +- **Workers KV.** Rejected because it is eventually consistent, which + breaks invalidation after mutations. +- **D1.** A good fit for prefix deletes, but it has a single primary region, + is single-threaded, and limits rows to 2 MB. Deferred until hit-rate data + says stable-tier sharing across isolates is worth it. +- **Durable Objects per user.** Deferred for the same reason. + +### Composite store for hosted + +`TieredStore(memory, r2)`: + +- `get` checks memory first, then R2 for immutable keys, and copies R2 hits + into memory. +- `set` writes to memory, and also to R2 when the tier is immutable. +- `deleteMatching` only touches memory, since only volatile and stable + entries are ever invalidated. + +### Identity + +The identity string is an HMAC input. It is never a raw token, and it stays +stable across token refresh. + +- **CLI:** keeps `getIdentityFingerprint()`, which hashes the refresh token + for OAuth logins. +- **Hosted:** hashes the user ID and grant ID taken from the MCP wrapper + bearer token (`userId:grantId:secret`, parsed in + `mcp-cloudflare/src/server/lib/mcp-handler.ts`). +- **Standalone MCP with a static token:** hashes the token. + +Key derivation uses WebCrypto (`crypto.subtle`) HMAC-SHA256 instead of +`node:crypto`, so the module also runs in browsers. This makes key +derivation async. The cache API is already async, so no call sites change +shape. + +### Compression: zstd level 3, with fallbacks + +Measured on Node 22 against MCP mock fixtures: + +- **5.7 MB trace payload.** zstd level 3 compresses about 52x in about 5 ms + and decompresses in about 4 ms. Level 9 reaches 72x but takes about + 24 ms to compress. +- **Small payloads (median 1.2 KB).** zstd, brotli at quality 4, and gzip + all land between 2.9x and 3.2x. + +Decisions: + +- **Memory tier: zstd level 3.** It runs on the response path and is + charged as Worker CPU time. +- **R2 tier: zstd level 9.** Compression runs in `waitUntil()` and the entry + is read many times. +- **Small bodies: stored uncompressed** below a 1 KB threshold, where the + savings do not pay for the CPU. +- **Fallback: brotli (quality 4) via `node:zlib`, then uncompressed.** The + CLI supports Node.js 20+, which lacks zstd, so the codec is detected at + runtime the way `api/infrastructure.ts` already detects + `zstdCompress`. +- The `encoding` is recorded per entry, so mixed entries stay readable + after upgrades. + +Compression saves memory and storage. It does not save CPU: a cache hit +costs decompression instead of JSON from the network, and on Workers the +network wait is not billed CPU. Its job is to fit more entries into the +budget. + +_Future:_ a zstd dictionary trained on Sentry API responses would mostly +help small payloads. Deferred until there is real hit data. + +### `runMcpServer()` takes a transport + +`McpServerOptions` gains `fetch?: typeof fetch`. + +- `sentry mcp` passes the CLI's authenticated fetch, so local sessions get + CLI auth, refresh, retries, and the on-disk cache. +- `mcp-cloudflare` builds an `api-transport` around the OAuth token and the + tiered store. + +Inside `mcp-core`, `SentryApiService` sends all requests through the +injected fetch, and `@sentry/api` operations receive the same fetch through +their client options. + +### `fresh` is a shared read-tool input + +Read-only tools gain an optional `fresh: boolean`, defined once in the +shared tool schema helpers. The tool handler sets a request-scoped bypass +(`AsyncLocalStorage`), and the transport honors it. The tool count does not +change. Token cost is checked with `pnpm run measure-tokens`, and the +description stays short. + +### Move MCP methods onto `@sentry/api` by domain + +The order is: + +1. organizations, projects, teams (already partial) +2. issues and events +3. explore, traces, logs +4. releases and replays +5. monitors, alerts, and uptime + +Endpoints that `@sentry/api` lacks call the shared transport directly and +keep their Zod schemas. Each domain lands as its own PR. Endpoint behavior +is checked against `~/src/sentry`, as `AGENTS.md` requires. + +### Version alignment first + +Add `@sentry/api` to the pnpm catalog at `0.400.0`, which is already in +`minimumReleaseAgeExclude`, and point both packages at `catalog:`. Later +bumps move both products together. + +## Risks / Trade-offs + +- **Cross-user leakage on the shared Worker.** Every key includes an + identity HMAC. There is a test that two identities never share entries + for the same URL, and R2 keys are prefixed by identity. +- **Stale data in agent answers.** Volatile entries last 60 s. There is + invalidation after mutations, and `fresh` for explicit bypass. The tool + description tells agents when to use it. +- **Worker OOM.** The memory store has a byte budget and a per-entry cap. + Compressed size is what counts. A load test fills the store with many + identities and checks its peak memory. +- **zstd unavailable in production Workers.** Confirm on a preview deploy + before relying on it. The brotli fallback keeps caching working either + way. +- **Transport move regresses the CLI.** The CLI's existing tests run against + the shared modules unchanged. The move happens one module at a time, and + each step has no behavior change. +- **Generated `@sentry/api` types disagree with responses.** Keep Zod + validation at the MCP boundary. Pin and bump `@sentry/api` through the + catalog. +- **Snapshot and eval churn.** Each domain migration lands with its own + snapshot updates and an eval run. +- **CLI-issued tokens lack MCP scopes.** The MCP and CLI use different OAuth + client IDs and scope lists today. Compare them before deprecating the + standalone login. If they differ, widen the CLI's requested scopes. + +## Migration Plan + +1. Land the catalog alignment and the pure `toolkit-core` modules (pagination, + cache policy, codecs), with the CLI switched over and no behavior change. +2. Land `api-transport` and the CLI switch-over. +3. Land the MCP transport injection and `sentry mcp` passing the CLI fetch, + then the hosted memory store behind a flag (`MCP_RESPONSE_CACHE=memory`). +4. Enable the hosted memory cache. Add the R2 binding and the immutable tier + behind `MCP_RESPONSE_CACHE=tiered`. +5. Run the domain-by-domain `@sentry/api` migration PRs. +6. Deprecate the standalone MCP login. + +**Rollback:** the hosted cache is gated by an env flag, and setting it to +`off` restores today's uncached behavior. Each CLI step is a refactor that +can be reverted on its own. + +## Open Questions + +- The exact byte budgets (CLI disk, Worker memory) and the per-entry cap. + To be tuned with hit-rate and memory metrics after launch. +- Whether R2 reads beat API latency for immutable lookups in practice. + Metrics will decide whether the R2 tier stays. diff --git a/openspec/changes/unify-sentry-api-layer/proposal.md b/openspec/changes/unify-sentry-api-layer/proposal.md new file mode 100644 index 000000000..0420a101b --- /dev/null +++ b/openspec/changes/unify-sentry-api-layer/proposal.md @@ -0,0 +1,89 @@ +# Proposal + +## Why + +The CLI and the MCP server talk to the same Sentry API through two separate +request layers. The CLI has resume-safe pagination, an identity-scoped +response cache, 401 refresh, timeouts, and validated redirects. The MCP has +its own transport in `mcp-core/src/api-client/client.ts` (about 6,000 lines), +uses `@sentry/api` for only three operations, and caches nothing. Fixes and +features land in one product and not the other, and the two pin different +`@sentry/api` versions (`0.337.0` and `^0.400.0`). + +`createSentrySDK()` cannot be the shared layer: + +- The CLI depends on `@sentry/mcp-server` for `sentry mcp`. +- The SDK needs SQLite and a filesystem, which Cloudflare Workers lack. +- SDK errors flatten HTTP status into exit codes. + +The shared layer must therefore sit below both products, on `@sentry/api`. + +## What Changes + +- Move the CLI's pagination into `@sentry/toolkit-core` and use it from both + products: limit-bounded walks, even page sizing, and cursors that resume + mid-page. +- Move the CLI's response-cache policy into `@sentry/toolkit-core`. That + covers TTL tiers, identity-scoped keys, invalidation after mutations, and + bypass. Storage becomes a pluggable backend with a byte budget and + compression. +- Move the CLI's request transport into `@sentry/toolkit-core`, with + credentials, storage, and telemetry injected by the host. It covers retries, + timeouts, redirect validation, and region routing. +- `runMcpServer()` accepts a host-supplied transport. `sentry mcp` passes the + CLI's authenticated transport, so local MCP sessions reuse the CLI login, + refresh, and on-disk cache. +- The hosted MCP on Cloudflare uses a tiered cache: + - an in-memory LRU, limited by bytes, for every tier + - R2 for immutable responses (events, traces) + - zstd compression on both +- MCP read tools accept a `fresh` parameter that bypasses the cache, matching + the CLI's `--fresh` flag. +- MCP API methods move onto `@sentry/api` operations domain by domain. The + endpoints `@sentry/api` lacks go through the same shared transport. +- Both products use one `@sentry/api` version from the pnpm catalog. +- **Deprecation:** the standalone `@sentry/mcp-server` device-code login is + deprecated in favor of `sentry mcp` and `sentry auth login`. + +Out of scope, tracked separately: + +- Server-side `Cache-Control` and `ETag` for immutable API responses + (getsentry/sentry#128818). +- Moving the shared infrastructure into `@sentry/api` once it has proven + itself in `toolkit-core`. + +## Capabilities + +### New Capabilities + +- `sentry-api-pagination`: Limit-bounded, resume-safe pagination over Sentry + list endpoints, shared by the CLI and MCP. +- `sentry-api-response-cache`: Identity-scoped caching of Sentry API GET + responses. Covers TTL tiers, bypass, invalidation after mutations, and + bounded, compressed storage. +- `mcp-host-transport`: How an MCP host supplies credentials and the Sentry + API transport, including `sentry mcp` reusing the CLI's login and cache. + +### Modified Capabilities + +None. + +## Impact + +- `packages/toolkit-core`: new `pagination`, `response-cache`, and transport + modules, with tests. +- `packages/cli/src/lib/api/infrastructure.ts`, + `packages/cli/src/lib/response-cache.ts`, + `packages/cli/src/lib/sentry-client.ts`: switch to the shared modules. + CLI behavior does not change. +- `packages/mcp-core/src/api-client/client.ts`: the transport is replaced and + methods move to `@sentry/api`. +- `packages/mcp-core` read tools gain a `fresh` input, and generated tool + definitions change. +- `packages/mcp-server/src/index.ts`: `McpServerOptions` gains a transport + option, and the standalone auth flow is deprecated. +- `packages/cli/src/lib/mcp.ts`: passes the CLI transport. +- `packages/mcp-cloudflare`: in-memory cache, an R2 bucket binding for + immutable responses, and a wrangler config change. +- `pnpm-workspace.yaml`: a catalog entry for `@sentry/api`. +- Snapshot and eval churn in `mcp-core` and `mcp-server-evals`. diff --git a/openspec/changes/unify-sentry-api-layer/specs/mcp-host-transport/spec.md b/openspec/changes/unify-sentry-api-layer/specs/mcp-host-transport/spec.md new file mode 100644 index 000000000..c59fca7d6 --- /dev/null +++ b/openspec/changes/unify-sentry-api-layer/specs/mcp-host-transport/spec.md @@ -0,0 +1,64 @@ +# Spec Delta + +## Purpose + +Defines how a host process gives the MCP server its Sentry credentials and +API transport, so one login and one request stack serve both the CLI and +the local MCP server. + +## ADDED Requirements + +### Requirement: Host-supplied transport + +The MCP server SHALL accept a Sentry API transport from its host. When one +is supplied, every Sentry API request made by MCP tools SHALL go through it. + +#### Scenario: Transport supplied + +- **WHEN** a host starts the MCP server with a transport +- **THEN** tool calls send their Sentry API requests through that transport + +#### Scenario: No transport supplied + +- **WHEN** the MCP server starts without a host transport +- **THEN** it uses the shared default transport with the configured access token + +### Requirement: sentry mcp reuses the CLI session + +`sentry mcp` SHALL start the MCP server with the CLI's stored login, token +refresh, and response cache. It SHALL not start a separate login flow. + +#### Scenario: Logged-in CLI + +- **WHEN** a user who has run `sentry auth login` starts `sentry mcp` +- **THEN** MCP tools authenticate with that login without prompting again + +#### Scenario: Cache shared with CLI commands + +- **WHEN** a CLI command and an MCP tool, for the same identity, request the same immutable resource +- **THEN** the second request is served from the shared on-disk cache + +#### Scenario: Token rejected mid-session + +- **WHEN** the API rejects the token during a `sentry mcp` session +- **THEN** the CLI refreshes the credential and later tool calls succeed without restarting the server + +### Requirement: CLI login covers MCP tool scopes + +The OAuth scopes granted by `sentry auth login` SHALL include every scope +that MCP tools require. + +#### Scenario: Write tool after CLI login + +- **WHEN** a user who logged in with `sentry auth login` calls an MCP tool that needs write scopes +- **THEN** the call is not rejected for missing scopes + +### Requirement: Standalone login is deprecated + +The standalone `@sentry/mcp-server` device-code login SHALL keep working and +SHALL tell users that `sentry mcp` is the supported local entry point. + +#### Scenario: Standalone auth + +- **WHEN** a user runs the standalone `@sentry/mcp-server` auth command +- **THEN** login succeeds and the output points the user to `sentry mcp` diff --git a/openspec/changes/unify-sentry-api-layer/specs/sentry-api-pagination/spec.md b/openspec/changes/unify-sentry-api-layer/specs/sentry-api-pagination/spec.md new file mode 100644 index 000000000..cbcb7e147 --- /dev/null +++ b/openspec/changes/unify-sentry-api-layer/specs/sentry-api-pagination/spec.md @@ -0,0 +1,72 @@ +# Spec Delta + +## Purpose + +Lets the CLI and MCP fetch a bounded number of rows from any Sentry list +endpoint and resume exactly where the previous call stopped. + +## ADDED Requirements + +### Requirement: Limit-bounded walks + +A list operation SHALL return at most the requested number of rows. It SHALL +use the fewest requests that can hold them, with page sizes spread as evenly +as possible and no page larger than the API maximum of 100. + +#### Scenario: Limit above one page + +- **WHEN** a caller requests 201 rows from an endpoint with more than 201 rows +- **THEN** the walk makes three requests of 67 rows each and returns 201 rows + +#### Scenario: Limit within one page + +- **WHEN** a caller requests 25 rows +- **THEN** the walk makes one request with a page size of 25 + +### Requirement: Resume-safe cursors + +A walk that stops before the end of the results SHALL return a cursor that +resumes at the first row the caller has not received, including when it +stopped inside a page. + +#### Scenario: Stop inside a page + +- **WHEN** a walk returns 30 rows of a 67-row page and the caller passes the returned cursor back +- **THEN** the next walk starts at row 31 of that page without repeating or skipping rows + +#### Scenario: Cursor accepted by the API and older clients + +- **WHEN** a caller sends a returned cursor directly to the Sentry API, or to an older CLI +- **THEN** the request succeeds and returns rows at or before the resume point, never after it + +### Requirement: Page-number cursors keep their page size + +Resuming a page-number cursor SHALL reuse the page size encoded in that +cursor, whatever the new limit. + +#### Scenario: Resume with a different limit + +- **WHEN** a page-number cursor produced with a page size of 50 is resumed with a limit of 10 +- **THEN** the walk requests pages of 50 and returns the first 10 unseen rows + +### Requirement: Bounded page count + +A walk SHALL stop after a fixed maximum number of pages. It SHALL return +the rows collected so far, a resume cursor, and an indication that the page +cap stopped it. + +#### Scenario: Page cap reached + +- **WHEN** an endpoint keeps returning next pages beyond the page cap +- **THEN** the walk stops at the cap and returns a cursor that resumes after the last returned row + +### Requirement: Complete walks fail loudly + +A walk that must see every row, such as one whose result a mutation depends +on, SHALL fail rather than return partial results. That applies when a page +is invalid, a cursor repeats, or the page cap is reached. + +#### Scenario: Repeated cursor + +- **WHEN** an endpoint returns a next cursor it already returned during the same walk +- **THEN** the walk fails with an error naming the operation diff --git a/openspec/changes/unify-sentry-api-layer/specs/sentry-api-response-cache/spec.md b/openspec/changes/unify-sentry-api-layer/specs/sentry-api-response-cache/spec.md new file mode 100644 index 000000000..ff8b5495e --- /dev/null +++ b/openspec/changes/unify-sentry-api-layer/specs/sentry-api-response-cache/spec.md @@ -0,0 +1,129 @@ +# Spec Delta + +## Purpose + +Lets the CLI and MCP reuse Sentry API GET responses without one user or +server ever seeing another's data, and without holding more memory or +storage than the host allows. + +## ADDED Requirements + +### Requirement: Identity-scoped entries + +Every cache entry SHALL be keyed by the requesting identity, the request +method, and the full request URL including host and normalized query. The +identity SHALL stay stable across access-token rotation of the same login. + +#### Scenario: Different users, same URL + +- **WHEN** two identities request the same URL +- **THEN** neither receives the other's cached response + +#### Scenario: Different servers, same path + +- **WHEN** one identity requests the same path on two Sentry hosts +- **THEN** the responses are cached separately + +#### Scenario: Token refresh + +- **WHEN** an OAuth access token is refreshed for the same login +- **THEN** entries cached under the previous access token remain usable + +### Requirement: Server cache headers take precedence + +When a response carries HTTP caching headers, the cache SHALL follow them +(RFC 9111). Only responses without such headers SHALL use the TTL tiers. + +#### Scenario: Server sends no-store + +- **WHEN** a response carries `Cache-Control: no-store` +- **THEN** the response is not cached + +#### Scenario: Server sends a validator + +- **WHEN** a stale entry has an `ETag` and the server answers the revalidation with 304 +- **THEN** the cached body is served and its freshness is renewed + +### Requirement: Fallback TTL tiers + +Responses without caching headers SHALL use a TTL by URL class: + +- immutable: 24 hours (event, trace, and trace-item lookups by ID) +- volatile: 60 seconds (issue endpoints, log and span queries) +- no-cache: never cached (autofix and root-cause polling) +- stable: 5 minutes (everything else) + +#### Scenario: Event lookup + +- **WHEN** an event is fetched by ID without caching headers +- **THEN** the response is reused for up to 24 hours + +#### Scenario: Polling endpoint + +- **WHEN** autofix state is fetched +- **THEN** the response is never served from cache + +### Requirement: Mutations invalidate related entries + +A successful non-GET request SHALL invalidate the requesting identity's +cached entries whose URLs fall under the mutated resource's prefixes. Other +identities' entries SHALL be unaffected. + +#### Scenario: Resolve then list + +- **WHEN** an identity updates an issue and then lists issues within the volatile TTL +- **THEN** the list is fetched from the API, not from cache + +### Requirement: Fresh reads bypass the cache + +A caller SHALL be able to request a fresh read. A fresh read SHALL fetch +from the API and replace the cached entry. The CLI exposes this as `--fresh` +/ `-f`, and MCP read tools as a `fresh` input. + +#### Scenario: MCP fresh read + +- **WHEN** an MCP read tool is called with `fresh: true` +- **THEN** every API request made by that call bypasses cached entries + +### Requirement: Bounded storage + +Each storage backend SHALL enforce a byte budget and a per-entry size cap. +Entries over the cap SHALL not be stored. When the budget is exceeded, the +least recently used entries SHALL be evicted first. + +#### Scenario: Oversized response + +- **WHEN** a response is larger than the per-entry cap +- **THEN** it is returned to the caller and not stored + +#### Scenario: Shared Worker isolate + +- **WHEN** many identities fill the in-memory cache of one Cloudflare Worker isolate +- **THEN** total cache memory stays within the configured budget and the isolate does not run out of memory + +### Requirement: Compressed entries + +Stored entries SHALL be compressed with zstd when the runtime supports it, +and otherwise stored with a supported fallback. Reading an entry SHALL +return a body identical to the original response body. + +#### Scenario: Runtime without zstd + +- **WHEN** the CLI runs on a Node.js version without zstd support +- **THEN** caching still works using the fallback encoding + +### Requirement: Hosted tiers + +The hosted MCP SHALL cache every tier in memory, and SHALL also persist +immutable-tier entries to durable object storage shared across isolates. +Volatile and stable entries SHALL never be persisted outside the isolate. + +#### Scenario: Immutable hit in another isolate + +- **WHEN** an identity fetches an event in one isolate and later fetches it in another +- **THEN** the second fetch is served from durable storage without calling the API + +#### Scenario: Volatile entry + +- **WHEN** an issue list is cached on the hosted MCP +- **THEN** it exists only in that isolate's memory diff --git a/openspec/changes/unify-sentry-api-layer/tasks.md b/openspec/changes/unify-sentry-api-layer/tasks.md new file mode 100644 index 000000000..9621dd9fb --- /dev/null +++ b/openspec/changes/unify-sentry-api-layer/tasks.md @@ -0,0 +1,63 @@ +# Tasks + +## 1. Version alignment + +- [ ] 1.1 Add `@sentry/api: 0.400.0` to the pnpm catalog and point `packages/cli` and `packages/mcp-core` at `catalog:`. Verify `pnpm install --frozen-lockfile` succeeds and the lockfile has one `@sentry/api` version. +- [ ] 1.2 Fix any `mcp-core` breakage from the `0.337.0` → `0.400.0` bump. Verify `pnpm --filter @sentry/mcp-core run tsc` and `pnpm --filter @sentry/mcp-core run test` pass. + +## 2. Shared pagination + +- [ ] 2.1 Move `paginate`, `fetchAllPages`, page sizing, and cursor encode/decode from `cli/src/lib/api/infrastructure.ts` into `toolkit-core/src/pagination.ts`. Failures are returned as typed results, and the host supplies the logger. Verify with toolkit-core unit tests that cover each `sentry-api-pagination` scenario. +- [ ] 2.2 Switch the CLI to the shared module, mapping errors to `ApiError` and keeping the `sentry-cli/no-manual-pagination` lint rule. Verify the CLI unit tests pass with no snapshot changes. +- [ ] 2.3 Document the module in `packages/toolkit-core/README.md`, and verify `pnpm run docs:check` passes. + +## 3. Shared cache policy and codecs + +- [ ] 3.1 Move tier classification, key derivation (moved to WebCrypto HMAC), URL normalization, and invalidation prefixes into `toolkit-core/src/response-cache/`. Verify with unit tests for the tier, identity-scoping, and invalidation scenarios. +- [ ] 3.2 Add a codec module (zstd level N, brotli fallback, raw below 1 KB) with runtime detection. Verify round-trip tests in Node 22, and a test with zstd stubbed out. +- [ ] 3.3 Define the `CacheStore` interface and add the byte-budgeted memory store. Verify tests for LRU eviction, the per-entry cap, and the budget staying within its bound under many identities. +- [ ] 3.4 Re-implement the CLI filesystem store on `CacheStore`, with compressed entries and a byte budget, while reading existing uncompressed entries. Verify the CLI cache tests pass and an old-format entry is still served. +- [ ] 3.5 Add conditional revalidation: send `If-None-Match` for stale entries that have an `ETag`, and treat a 304 as a hit. Verify with a test against a mocked `ETag` endpoint. + +## 4. Shared transport + +- [ ] 4.1 Move auth headers, 401 refresh, timeouts, GET retry/backoff, redirect validation, and region routing from `cli/src/lib/sentry-client.ts` into `toolkit-core/src/api-transport.ts`, with an injected `CredentialProvider`, `CacheStore`, logger, and span hook. Verify with transport unit tests. +- [ ] 4.2 Switch the CLI to the shared transport, wiring in the database credentials, the filesystem store, and telemetry spans. Verify the CLI unit and e2e tests pass. +- [ ] 4.3 Add a request-scoped fresh-read bypass to the transport, and verify a test that a bypassed request skips cache reads and replaces the entry. + +## 5. MCP transport injection + +- [ ] 5.1 Add `fetch` to `McpServerOptions`, and route all `SentryApiService` requests and `@sentry/api` calls through it. Verify an `mcp-server` test where a supplied fetch receives every tool request. +- [ ] 5.2 Pass the CLI's authenticated fetch from `cli/src/lib/mcp.ts`. Verify a CLI test where `sentry mcp` tool calls hit the shared cache, and `/mcp-qa` stdio QA passes. +- [ ] 5.3 Add the `fresh` input to read tools through the shared schema helpers. Regenerate definitions, then verify `pnpm run measure-tokens` and the tool tests pass. + +## 6. Hosted cache + +- [ ] 6.1 Wire the memory store into `mcp-cloudflare` behind `MCP_RESPONSE_CACHE` (`off`/`memory`/`tiered`), with identity taken from the wrapper token's user and grant IDs. Verify the Workers tests pass, and a two-identity isolation test. +- [ ] 6.2 Confirm `node:zlib` zstd on a deployed preview Worker. If it is unavailable, keep brotli as the hosted codec and record that in `docs/cloudflare/architecture.md`. +- [ ] 6.3 Add the R2 binding, the immutable-tier store, and `TieredStore`, with writes in `waitUntil` and a 1-day lifecycle rule. Verify tests for cross-isolate immutable hits and memory-only volatile entries. +- [ ] 6.4 Add cache hit, miss, and eviction metrics and memory-budget logging. Document them in `docs/operations/monitoring.md`, and verify they appear in a preview deploy. + +## 7. Move MCP endpoints onto `@sentry/api` + +- [ ] 7.1 Organizations, projects, and teams. Verify the MCP tests and snapshots for the affected tools. +- [ ] 7.2 Issues and events. Verify the MCP tests and snapshots for the affected tools. +- [ ] 7.3 Explore, traces, and logs. Verify the MCP tests, snapshots, and a search-events eval run. +- [ ] 7.4 Releases and replays. Verify the MCP tests and snapshots for the affected tools. +- [ ] 7.5 Monitors, alerts, and uptime, keeping the shared-transport fallback for the endpoints `@sentry/api` lacks. Verify the MCP tests and snapshots. +- [ ] 7.6 Update `docs/contributing/api-patterns.md` for the new client layering, and verify `pnpm run docs:check` passes. + +## 8. Standalone login deprecation + +- [ ] 8.1 Compare the OAuth scopes the CLI requests with the MCP `SCOPES`, and widen the CLI's requested scopes if needed. Verify a test that the CLI's scope list covers every MCP tool's required scopes. +- [ ] 8.2 Add a deprecation notice to the standalone `@sentry/mcp-server` auth flow and docs pointing to `sentry mcp`. Verify with an output test and an update to `docs/operations/stdio-auth.md`. + +## 9. Integration + +- [ ] 9.1 Run `pnpm run tsc && pnpm run lint && pnpm run test` and the MCP eval suite on the final state, and record the results in the PR. + +## Workflow follow-up + +- Archive the change after review once all tasks are complete. +- Follow up on getsentry/sentry#128818 and revisit the TTL tiers when server cache headers ship. +- Propose moving pagination, then the transport, into `@sentry/api`. From 9ba30f434db6cb7bbb9b959d62489aeaae8d8b81 Mon Sep 17 00:00:00 2001 From: "sentry-junior[bot]" <264270552+sentry-junior[bot]@users.noreply.github.com> Date: Mon, 26 Oct 2026 20:27:46 +0000 Subject: [PATCH 2/8] docs(openspec): pick codecs from real-response benchmarks Benchmark zstd, brotli, gzip and trained zstd dictionaries on real API responses. Keep plain zstd for now, cap entries on compressed size, and move dictionary support to a follow-up gated on safe training data and Workers support (workerd ignores the zstd dictionary option). Co-Authored-By: BYK --- .../changes/unify-sentry-api-layer/design.md | 110 +++++++++++++----- .../specs/sentry-api-response-cache/spec.md | 19 ++- .../changes/unify-sentry-api-layer/tasks.md | 12 +- 3 files changed, 102 insertions(+), 39 deletions(-) diff --git a/openspec/changes/unify-sentry-api-layer/design.md b/openspec/changes/unify-sentry-api-layer/design.md index b9d188260..babbccf2f 100644 --- a/openspec/changes/unify-sentry-api-layer/design.md +++ b/openspec/changes/unify-sentry-api-layer/design.md @@ -106,8 +106,8 @@ Backends: current users see a similar footprint. - **Memory (hosted MCP, tests).** - LRU limited by bytes, counted over compressed bytes. - - Default budget 24 MB per isolate and a 1 MB per-entry cap, configurable - by env. + - Default budget 24 MB per isolate and a 1 MB per-entry cap on compressed + size, configurable by env. - Module-scoped, so it lives as long as the isolate. `deleteMatching` scans the in-memory index. - **R2 (hosted MCP, immutable tier only).** @@ -154,38 +154,88 @@ Key derivation uses WebCrypto (`crypto.subtle`) HMAC-SHA256 instead of derivation async. The cache API is already async, so no call sites change shape. -### Compression: zstd level 3, with fallbacks - -Measured on Node 22 against MCP mock fixtures: - -- **5.7 MB trace payload.** zstd level 3 compresses about 52x in about 5 ms - and decompresses in about 4 ms. Level 9 reaches 72x but takes about - 24 ms to compress. -- **Small payloads (median 1.2 KB).** zstd, brotli at quality 4, and gzip - all land between 2.9x and 3.2x. +### Compression: plain zstd now, trained dictionary as a follow-up + +Benchmarked on Node 22 with real API responses from Sentry's own +organization. The samples were kept in the sandbox and not committed: + +- 255 responses from one project (events, issue details, issue lists, + traces, logs, spans, projects, teams), split 70/30 into training and test + sets. +- 90 responses from five other projects (Android, Node, Next.js, Rust, + Python) as a cross-project check. + +Ratios are total raw bytes divided by total compressed bytes, with each +response compressed on its own, the way the cache stores entries: + +| Codec | Same project, ≤1 MB | Other projects | >256 KB (traces) | Compress, ms/MB | Decompress, ms/MB | +| ---------------------------------- | ------------------- | -------------- | ---------------- | --------------- | ----------------- | +| zstd-3 | 8.7x | 6.3x | 37x | 1.6 | 1.2 | +| zstd-9 | — | — | 45x | 5.6 | 0.9 | +| zstd-3 + trained 110 KB dictionary | 17.0x | 7.6x | 37x | 2.0 | 0.8 | +| brotli-4 | 9.3x | 6.6x | 42x | 3.0 | 0.6 | +| brotli-6 | — | — | 45x | 5.7 | 0.8 | +| brotli-11 | — | — | 49x | 775 | 2.3 | +| gzip-6 | — | — | 30x | 3.9 | 0.8 | + +Findings: + +- **Brotli is not the better target.** + - At the same CPU, zstd-9 and brotli-6 tie. + - brotli-4 beats zstd-3 by 5–12% but costs about 2x the compression CPU. + - Brotli's best ratio, quality 11, costs about 500x the CPU of zstd-3. + - `node:zlib` brotli has no custom dictionary support. +- **A trained dictionary is the big lever for small and medium responses, + which are most entries by count.** On the same project it doubled the + overall ratio (8.7x → 17.0x), and below 16 KB it went from 3.4x to 9.8x. + On other projects the gain shrank to about 20% (6.3x → 7.6x). The + dictionary had learned project-specific values (paths, function names, + slugs), not just JSON keys. It did nothing for large traces. +- **Dictionaries built from public data did not help.** One trained on the + repo's mock fixtures, and one trained on response skeletons with all values + blanked, stayed at about 3.4–3.8x for small responses. The gain comes from + real values. +- **Workers ignores `dictionary`.** Local workerd (`wrangler dev` 4.151.3) + accepts the `dictionary` option on `zstdCompressSync` but silently ignores + it: output is byte-for-byte the same size as without a dictionary. Node 22 + honors it (5.3 KB issue: 1,819 → 632 bytes). +- **The per-entry cap must apply to compressed size.** Traces in this + sample had a median of 1.5 MB raw (max 14 MB), but compress 37–45x. A cap on + raw size would make most traces impossible to cache. Decisions: -- **Memory tier: zstd level 3.** It runs on the response path and is - charged as Worker CPU time. -- **R2 tier: zstd level 9.** Compression runs in `waitUntil()` and the entry +- **Memory tier: zstd level 3, no dictionary.** It runs on the response path + and is charged as Worker CPU time. +- **R2 tier: zstd level 9.** Compression runs in `waitUntil()`, and the entry is read many times. -- **Small bodies: stored uncompressed** below a 1 KB threshold, where the - savings do not pay for the CPU. -- **Fallback: brotli (quality 4) via `node:zlib`, then uncompressed.** The - CLI supports Node.js 20+, which lacks zstd, so the codec is detected at - runtime the way `api/infrastructure.ts` already detects - `zstdCompress`. -- The `encoding` is recorded per entry, so mixed entries stay readable - after upgrades. - -Compression saves memory and storage. It does not save CPU: a cache hit -costs decompression instead of JSON from the network, and on Workers the -network wait is not billed CPU. Its job is to fit more entries into the -budget. - -_Future:_ a zstd dictionary trained on Sentry API responses would mostly -help small payloads. Deferred until there is real hit data. +- **Fallback: brotli quality 4 via `node:zlib`, then uncompressed.** The CLI + supports Node.js 20+, which lacks zstd, so the codec is detected at + runtime, the way `api/infrastructure.ts` already detects `zstdCompress`. +- **Small bodies:** stored uncompressed below 256 bytes. +- **Per-entry cap on compressed size**, plus a raw-size ceiling (default + 32 MB) that bounds memory spikes during decompression. +- **Encoding recorded per entry.** `encoding` and `dictId` are recorded per + entry, so mixed entries stay readable after upgrades and dictionary + rotations. +- **Dictionary support is a follow-up (task group 9), gated on three + checks:** + 1. **Training data safety.** A dictionary embeds verbatim strings from its + training data, so one trained on customer or internal data must never + ship in a public package (`sentry`, `@sentry/mcp-server`). Options: + - train on a public demo organization + - train per deployment, for example the hosted MCP from its own cache + contents within an organization + 2. **Workers support.** Use a WASM zstd build for dictionary compression, + or wait for workerd to honor the option. A startup self-test compresses + a known sample and checks that the dictionary took effect, so a silently + ignored option is caught. + 3. **A broader benchmark** across many organizations and projects before + committing to dictionary size and rotation. + +Compression saves memory and storage, not CPU. A cache hit costs +decompression instead of JSON from the network, and on Workers the network +wait is not billed CPU. Its job is to fit more entries into the budget. ### `runMcpServer()` takes a transport diff --git a/openspec/changes/unify-sentry-api-layer/specs/sentry-api-response-cache/spec.md b/openspec/changes/unify-sentry-api-layer/specs/sentry-api-response-cache/spec.md index ff8b5495e..f67f9f381 100644 --- a/openspec/changes/unify-sentry-api-layer/specs/sentry-api-response-cache/spec.md +++ b/openspec/changes/unify-sentry-api-layer/specs/sentry-api-response-cache/spec.md @@ -87,15 +87,21 @@ from the API and replace the cached entry. The CLI exposes this as `--fresh` ### Requirement: Bounded storage -Each storage backend SHALL enforce a byte budget and a per-entry size cap. -Entries over the cap SHALL not be stored. When the budget is exceeded, the -least recently used entries SHALL be evicted first. +Each storage backend SHALL enforce a byte budget over stored bytes. It SHALL +enforce a per-entry cap on stored (compressed) size and a ceiling on raw +size. Entries over either limit SHALL not be stored. When the budget is +exceeded, the least recently used entries SHALL be evicted first. #### Scenario: Oversized response -- **WHEN** a response is larger than the per-entry cap +- **WHEN** a response's compressed size is larger than the per-entry cap - **THEN** it is returned to the caller and not stored +#### Scenario: Large but compressible response + +- **WHEN** a 5 MB trace compresses below the per-entry cap and its raw size is under the ceiling +- **THEN** it is stored + #### Scenario: Shared Worker isolate - **WHEN** many identities fill the in-memory cache of one Cloudflare Worker isolate @@ -104,8 +110,9 @@ least recently used entries SHALL be evicted first. ### Requirement: Compressed entries Stored entries SHALL be compressed with zstd when the runtime supports it, -and otherwise stored with a supported fallback. Reading an entry SHALL -return a body identical to the original response body. +and otherwise stored with a supported fallback. Each entry SHALL record its +encoding. Reading an entry SHALL return a body identical to the original +response body. #### Scenario: Runtime without zstd diff --git a/openspec/changes/unify-sentry-api-layer/tasks.md b/openspec/changes/unify-sentry-api-layer/tasks.md index 9621dd9fb..2007562b4 100644 --- a/openspec/changes/unify-sentry-api-layer/tasks.md +++ b/openspec/changes/unify-sentry-api-layer/tasks.md @@ -14,7 +14,7 @@ ## 3. Shared cache policy and codecs - [ ] 3.1 Move tier classification, key derivation (moved to WebCrypto HMAC), URL normalization, and invalidation prefixes into `toolkit-core/src/response-cache/`. Verify with unit tests for the tier, identity-scoping, and invalidation scenarios. -- [ ] 3.2 Add a codec module (zstd level N, brotli fallback, raw below 1 KB) with runtime detection. Verify round-trip tests in Node 22, and a test with zstd stubbed out. +- [ ] 3.2 Add a codec module (zstd level 3 or 9 by tier, brotli-4 fallback, raw below 256 bytes, `encoding` and `dictId` recorded per entry) with runtime detection. Verify round-trip tests in Node 22, and a test with zstd stubbed out. - [ ] 3.3 Define the `CacheStore` interface and add the byte-budgeted memory store. Verify tests for LRU eviction, the per-entry cap, and the budget staying within its bound under many identities. - [ ] 3.4 Re-implement the CLI filesystem store on `CacheStore`, with compressed entries and a byte budget, while reading existing uncompressed entries. Verify the CLI cache tests pass and an old-format entry is still served. - [ ] 3.5 Add conditional revalidation: send `If-None-Match` for stale entries that have an `ETag`, and treat a 304 as a hit. Verify with a test against a mocked `ETag` endpoint. @@ -52,9 +52,15 @@ - [ ] 8.1 Compare the OAuth scopes the CLI requests with the MCP `SCOPES`, and widen the CLI's requested scopes if needed. Verify a test that the CLI's scope list covers every MCP tool's required scopes. - [ ] 8.2 Add a deprecation notice to the standalone `@sentry/mcp-server` auth flow and docs pointing to `sentry mcp`. Verify with an output test and an update to `docs/operations/stdio-auth.md`. -## 9. Integration +## 9. Trained compression dictionary (follow-up) -- [ ] 9.1 Run `pnpm run tsc && pnpm run lint && pnpm run test` and the MCP eval suite on the final state, and record the results in the PR. +- [ ] 9.1 Build a training corpus that is safe to ship: a public demo organization, or a per-deployment corpus that never leaves the hosted MCP. Verify that no customer or internal strings appear in a shipped dictionary (`strings` audit in CI). +- [ ] 9.2 Add dictionary compression with a versioned `dictId` and a startup self-test that detects a silently ignored `dictionary` option (as in workerd today). Verify a test where an ignored dictionary falls back to plain zstd. +- [ ] 9.3 On Workers, evaluate WASM zstd for dictionary compression: bundle size, CPU in ms/MB, and ratio against native zstd-3. Record the results in design.md and adopt it only if the memory-tier gain outweighs the CPU cost. + +## 10. Integration + +- [ ] 10.1 Run `pnpm run tsc && pnpm run lint && pnpm run test` and the MCP eval suite on the final state, and record the results in the PR. ## Workflow follow-up From c3ccf46b59e05d4429eee782f5178a76130bf94e Mon Sep 17 00:00:00 2001 From: "sentry-junior[bot]" <264270552+sentry-junior[bot]@users.noreply.github.com> Date: Mon, 26 Oct 2026 23:13:41 +0000 Subject: [PATCH 3/8] docs(openspec): narrow cache tiers to what the server spike allows The getsentry/sentry#128818 spike found that no event or trace endpoint is safe to cache as immutable. Issue and project event details embed next/previous IDs and user reports, `latest` resolves to different events over time, and full traces gain spans. The CLI's 24-hour tier still matches all of them, including `/issues//events/latest/`. Replace the immutable tier with a 5-minute detail tier limited to organization event details and project trace-item details, and move issue events, project events, and full traces to 60 seconds. Add the CLI tier correction as its own task group ahead of the toolkit-core move. R2 now persists only detail-tier entries that the server marks `private, max-age`, for at most 5 minutes, and needs a security sign-off before production. Use zstd level 3 for every tier instead of level 9 for R2. Co-Authored-By: BYK --- .../changes/unify-sentry-api-layer/design.md | 139 ++++++++++++++---- .../unify-sentry-api-layer/proposal.md | 19 ++- .../specs/mcp-host-transport/spec.md | 2 +- .../specs/sentry-api-response-cache/spec.md | 64 ++++++-- .../changes/unify-sentry-api-layer/tasks.md | 86 ++++++----- 5 files changed, 224 insertions(+), 86 deletions(-) diff --git a/openspec/changes/unify-sentry-api-layer/design.md b/openspec/changes/unify-sentry-api-layer/design.md index babbccf2f..be9e4b78a 100644 --- a/openspec/changes/unify-sentry-api-layer/design.md +++ b/openspec/changes/unify-sentry-api-layer/design.md @@ -14,7 +14,10 @@ See proposal.md for motivation. Current state that shapes the approach: redirect validation, and custom headers. - `response-cache.ts` caches responses as JSON files under `~/.sentry/cache/responses/`, using `http-cache-semantics`, an HMAC - identity key, TTL tiers, and a 500-entry cap. + identity key, TTL tiers, and a 500-entry cap. Its 24-hour "immutable" + tier matches any `/events//`, `/trace/<32 hex>/`, or + `/trace-items//` URL, and is checked before the `/issues/` volatile + pattern, so `/issues//events/latest/` is cached for 24 hours too. - `api/infrastructure.ts` has `paginate()` and `fetchAllPages()`, with mid-page skip counts encoded in the cursor's `is_prev` field. - **MCP request stack.** `mcp-core/src/api-client/client.ts` has about 105 @@ -49,7 +52,8 @@ See proposal.md for motivation. Current state that shapes the approach: - Shared modules run unchanged in Node and Workers, and avoid Node-only APIs where a web standard exists, so they can later move into `@sentry/api` and run in the browser. -- No CLI behavior change while its internals move. +- No CLI behavior change while its internals move, apart from the cache + tier correction, which lands first as its own PR. **Non-Goals:** @@ -62,6 +66,61 @@ See proposal.md for motivation. Current state that shapes the approach: ## Decisions +### Narrow the CLI's cache tiers before moving them + +The getsentry/sentry#128818 spike (findings in +[this comment](https://github.com/getsentry/sentry/issues/128818#issuecomment-6426889162), +checked against sentry `06a3bc41dda`) found that none of the "immutable" +endpoints are safe to mark `immutable`: + +- **Project and issue event details** embed `nextEventID`, + `previousEventID`, user reports, and release data, all of which change + later. Reprocessing and issue merges can change the response for the same + event ID. `latest`, `oldest`, and `recommended` resolve to different events + over time. +- **Full traces** gain spans as they arrive. +- **Organization event details** (excluding `?averageColumn`, which reads + live aggregates) and **project trace-item details** (excluding + superuser/staff requests and `?debug`) are stable enough for + `Cache-Control: private, max-age=300`. The prototype in + getsentry/sentry#128828 sends exactly that, off by default, with + `Vary: Origin, Cookie, Authorization`. +- Every endpoint can still change visibility: deletion and access + revocation are not reflected until the entry expires. That bounds the TTL, + not just the tier. + +So the shared policy replaces `immutable` with `detail`: + +| Tier | TTL | Matches | +| -------- | ---- | ------------------------------------------------------------------------------------------------------------------------------------------ | +| no-cache | 0 | autofix and root-cause polling | +| detail | 5 m | `/organizations/{org}/events/{project}:{event_id}/` without `averageColumn`; `/projects/{org}/{project}/trace-items/{id}/` without `debug` | +| volatile | 60 s | `/issues/` (including issue events), `/projects/{org}/{project}/events/{id}/`, `/organizations/{org}/trace/{id}/`, log and span queries | +| stable | 5 m | everything else | + +- Patterns are exact path shapes, anchored on the API prefix, not + substring matches on `/events/`. Anything that does not match an + allowlisted shape falls through to volatile or stable. +- `detail` has the same TTL as `stable` in the CLI. It exists as its own + tier because it is the only tier eligible for R2 on the hosted MCP. +- Today the CLI reads events through project and issue endpoints + (`getEvent`, `getLatestEvent` in `cli/src/lib/api/events.ts`), and the MCP + through `/organizations/{org}/issues/{id}/events/{event_id}/`. So in + practice the detail tier covers trace items until a caller moves to the + organization event endpoint. That is worth doing per call site only where + the response has every field the caller uses. +- When server headers are present they still win. The CLI keeps honoring + `max-age`, so if the server later extends it, the CLI follows without a + release. +- This lands as its own CLI PR before the move into `toolkit-core`, so the + move itself stays a no-behavior-change refactor. The existing + `response-cache.property.test.ts` cases that assert 24-hour event and + trace caching are replaced with scenarios from the spec delta. + +The cost is more API calls for repeated event and trace reads. A 24-hour +window was a correctness bug, not a tunable: an agent re-reading an issue's +latest event an hour later got the wrong event. + ### Shared code lives in `toolkit-core`, behind injected dependencies New `toolkit-core` modules: @@ -110,13 +169,20 @@ Backends: size, configurable by env. - Module-scoped, so it lives as long as the isolate. `deleteMatching` scans the in-memory index. -- **R2 (hosted MCP, immutable tier only).** - - Object key is `/`, with the expiry in - custom metadata and checked on read. - - A bucket lifecycle rule deletes objects after 1 day. +- **R2 (hosted MCP, detail tier only).** + - Persists an entry only when it is in the detail tier _and_ the response + carries `Cache-Control: private` with a positive `max-age`. The server + header is the opt-in: until getsentry/sentry#128828 ships and is + enabled, R2 stores nothing. + - Object key is `/`, with the expiry, + the lesser of the server `max-age` and 5 minutes, in custom metadata and + checked on read. + - A bucket lifecycle rule deletes objects after 1 day, the shortest rule + R2 supports. It is garbage collection only; reads enforce the expiry. - Writes happen in `ctx.waitUntil()`, so they never delay a response. - - The immutable tier is never invalidated, so R2 never needs - `deleteMatching`. + - Detail entries are never targeted by mutation invalidation, so R2 never + needs `deleteMatching`. The short expiry bounds how long a revoked or + deleted item can still be served. _Alternatives:_ @@ -131,9 +197,10 @@ _Alternatives:_ `TieredStore(memory, r2)`: -- `get` checks memory first, then R2 for immutable keys, and copies R2 hits - into memory. -- `set` writes to memory, and also to R2 when the tier is immutable. +- `get` checks memory first, then R2 for detail keys, and copies R2 hits + into memory with the remaining expiry. +- `set` writes to memory, and also to R2 when the entry is R2-eligible (see + above). - `deleteMatching` only touches memory, since only volatile and stable entries are ever invalidated. @@ -205,10 +272,14 @@ Findings: Decisions: -- **Memory tier: zstd level 3, no dictionary.** It runs on the response path - and is charged as Worker CPU time. -- **R2 tier: zstd level 9.** Compression runs in `waitUntil()`, and the entry - is read many times. +- **zstd level 3 everywhere, no dictionary.** One codec setting for the + CLI disk, the hosted memory tier, and R2. + - It runs on the response path and is charged as Worker CPU time. + - zstd-9 only helps large traces (37x → 45x) at 3.5x the compression + CPU. With R2 entries now living at most 5 minutes, an entry is read + only a few times, so the extra CPU does not pay back. + - A single level keeps entries portable between tiers: an R2 hit is + copied into memory without recompressing. - **Fallback: brotli quality 4 via `node:zlib`, then uncompressed.** The CLI supports Node.js 20+, which lacks zstd, so the codec is detected at runtime, the way `api/infrastructure.ts` already detects `zstdCompress`. @@ -218,7 +289,7 @@ Decisions: - **Encoding recorded per entry.** `encoding` and `dictId` are recorded per entry, so mixed entries stay readable after upgrades and dictionary rotations. -- **Dictionary support is a follow-up (task group 9), gated on three +- **Dictionary support is a follow-up (task group 10), gated on three checks:** 1. **Training data safety.** A dictionary embeds verbatim strings from its training data, so one trained on customer or internal data must never @@ -283,9 +354,16 @@ bumps move both products together. - **Cross-user leakage on the shared Worker.** Every key includes an identity HMAC. There is a test that two identities never share entries for the same URL, and R2 keys are prefixed by identity. -- **Stale data in agent answers.** Volatile entries last 60 s. There is +- **Stale data in agent answers.** Volatile entries last 60 s, and no entry + lasts longer than 5 minutes without a server header. There is invalidation after mutations, and `fresh` for explicit bypass. The tool description tells agents when to use it. +- **Revoked access or deleted data served from cache.** A user who loses + access to a project, or whose event is deleted, can still get a cached + copy until it expires: up to 5 minutes for detail entries, including + from R2 on the hosted MCP. Enabling R2 in production needs a privacy and + security sign-off on that window, alongside the server-side headers in + getsentry/sentry#128828. - **Worker OOM.** The memory store has a byte budget and a per-entry cap. Compressed size is what counts. A load test fills the store with many identities and checks its peak memory. @@ -306,15 +384,18 @@ bumps move both products together. ## Migration Plan -1. Land the catalog alignment and the pure `toolkit-core` modules (pagination, - cache policy, codecs), with the CLI switched over and no behavior change. -2. Land `api-transport` and the CLI switch-over. -3. Land the MCP transport injection and `sentry mcp` passing the CLI fetch, +1. Land the catalog alignment (#1672) and the CLI cache tier correction, + each as its own PR. +2. Land the pure `toolkit-core` modules (pagination, cache policy, codecs), + with the CLI switched over and no behavior change. +3. Land `api-transport` and the CLI switch-over. +4. Land the MCP transport injection and `sentry mcp` passing the CLI fetch, then the hosted memory store behind a flag (`MCP_RESPONSE_CACHE=memory`). -4. Enable the hosted memory cache. Add the R2 binding and the immutable tier - behind `MCP_RESPONSE_CACHE=tiered`. -5. Run the domain-by-domain `@sentry/api` migration PRs. -6. Deprecate the standalone MCP login. +5. Enable the hosted memory cache. Add the R2 binding and the detail tier + behind `MCP_RESPONSE_CACHE=tiered`, and enable it only after the server + headers ship and the security sign-off. +6. Run the domain-by-domain `@sentry/api` migration PRs. +7. Deprecate the standalone MCP login. **Rollback:** the hosted cache is gated by an env flag, and setting it to `off` restores today's uncached behavior. Each CLI step is a refactor that @@ -324,5 +405,9 @@ can be reverted on its own. - The exact byte budgets (CLI disk, Worker memory) and the per-entry cap. To be tuned with hit-rate and memory metrics after launch. -- Whether R2 reads beat API latency for immutable lookups in practice. - Metrics will decide whether the R2 tier stays. +- Whether R2 pays off with a 5-minute window. Cross-isolate hits only help + when the same identity re-reads the same event or span within minutes, + which agents often do, but metrics will decide whether the R2 tier stays. +- Whether the server should send `private, max-age` on more endpoints + later, such as stable organization and project metadata. The CLI follows + server headers automatically; R2 eligibility stays an explicit allowlist. diff --git a/openspec/changes/unify-sentry-api-layer/proposal.md b/openspec/changes/unify-sentry-api-layer/proposal.md index 0420a101b..c56632dcb 100644 --- a/openspec/changes/unify-sentry-api-layer/proposal.md +++ b/openspec/changes/unify-sentry-api-layer/proposal.md @@ -23,6 +23,12 @@ The shared layer must therefore sit below both products, on `@sentry/api`. - Move the CLI's pagination into `@sentry/toolkit-core` and use it from both products: limit-bounded walks, even page sizing, and cursors that resume mid-page. +- **Behavior change:** narrow the CLI's 24-hour "immutable" cache tier. It + matches every `/events//` and `/trace//` URL, but issue events, + project events, and full traces change after they are first read + (getsentry/sentry#128818). Only organization event details and project + trace-item details stay in a separate tier, at 5 minutes. Issue events, + project events, and full traces drop to 60 seconds. - Move the CLI's response-cache policy into `@sentry/toolkit-core`. That covers TTL tiers, identity-scoped keys, invalidation after mutations, and bypass. Storage becomes a pluggable backend with a byte budget and @@ -35,8 +41,9 @@ The shared layer must therefore sit below both products, on `@sentry/api`. refresh, and on-disk cache. - The hosted MCP on Cloudflare uses a tiered cache: - an in-memory LRU, limited by bytes, for every tier - - R2 for immutable responses (events, traces) - - zstd compression on both + - R2 for organization event details and project trace-item details, only + when the server marks them `private, max-age`, for at most 5 minutes + - zstd level 3 compression on both - MCP read tools accept a `fresh` parameter that bypasses the cache, matching the CLI's `--fresh` flag. - MCP API methods move onto `@sentry/api` operations domain by domain. The @@ -47,8 +54,8 @@ The shared layer must therefore sit below both products, on `@sentry/api`. Out of scope, tracked separately: -- Server-side `Cache-Control` and `ETag` for immutable API responses - (getsentry/sentry#128818). +- Server-side `Cache-Control` for event and trace-item detail responses + (getsentry/sentry#128818, prototype in getsentry/sentry#128828). - Moving the shared infrastructure into `@sentry/api` once it has proven itself in `toolkit-core`. @@ -75,7 +82,7 @@ None. - `packages/cli/src/lib/api/infrastructure.ts`, `packages/cli/src/lib/response-cache.ts`, `packages/cli/src/lib/sentry-client.ts`: switch to the shared modules. - CLI behavior does not change. + Apart from the narrower cache tiers, CLI behavior does not change. - `packages/mcp-core/src/api-client/client.ts`: the transport is replaced and methods move to `@sentry/api`. - `packages/mcp-core` read tools gain a `fresh` input, and generated tool @@ -84,6 +91,6 @@ None. option, and the standalone auth flow is deprecated. - `packages/cli/src/lib/mcp.ts`: passes the CLI transport. - `packages/mcp-cloudflare`: in-memory cache, an R2 bucket binding for - immutable responses, and a wrangler config change. + detail responses, and a wrangler config change. - `pnpm-workspace.yaml`: a catalog entry for `@sentry/api`. - Snapshot and eval churn in `mcp-core` and `mcp-server-evals`. diff --git a/openspec/changes/unify-sentry-api-layer/specs/mcp-host-transport/spec.md b/openspec/changes/unify-sentry-api-layer/specs/mcp-host-transport/spec.md index c59fca7d6..16ad37513 100644 --- a/openspec/changes/unify-sentry-api-layer/specs/mcp-host-transport/spec.md +++ b/openspec/changes/unify-sentry-api-layer/specs/mcp-host-transport/spec.md @@ -35,7 +35,7 @@ refresh, and response cache. It SHALL not start a separate login flow. #### Scenario: Cache shared with CLI commands -- **WHEN** a CLI command and an MCP tool, for the same identity, request the same immutable resource +- **WHEN** a CLI command and an MCP tool, for the same identity, request the same cacheable resource within its TTL - **THEN** the second request is served from the shared on-disk cache #### Scenario: Token rejected mid-session diff --git a/openspec/changes/unify-sentry-api-layer/specs/sentry-api-response-cache/spec.md b/openspec/changes/unify-sentry-api-layer/specs/sentry-api-response-cache/spec.md index f67f9f381..a2331f7c4 100644 --- a/openspec/changes/unify-sentry-api-layer/specs/sentry-api-response-cache/spec.md +++ b/openspec/changes/unify-sentry-api-layer/specs/sentry-api-response-cache/spec.md @@ -48,21 +48,53 @@ When a response carries HTTP caching headers, the cache SHALL follow them Responses without caching headers SHALL use a TTL by URL class: -- immutable: 24 hours (event, trace, and trace-item lookups by ID) -- volatile: 60 seconds (issue endpoints, log and span queries) +- detail: 5 minutes (organization event details and project trace-item + details, by exact path) +- volatile: 60 seconds (issue endpoints, project event details, full + traces, log and span queries) - no-cache: never cached (autofix and root-cause polling) - stable: 5 minutes (everything else) -#### Scenario: Event lookup +#### Scenario: Organization event lookup -- **WHEN** an event is fetched by ID without caching headers -- **THEN** the response is reused for up to 24 hours +- **WHEN** `/organizations/{org}/events/{project}:{event_id}/` is fetched without caching headers +- **THEN** the response is reused for up to 5 minutes + +#### Scenario: Live aggregate on an event + +- **WHEN** an organization event detail is fetched with an `averageColumn` query +- **THEN** it is classified as stable, not detail + +#### Scenario: Trace-item debug lookup + +- **WHEN** a project trace-item detail is fetched with a `debug` query +- **THEN** it is classified as stable, not detail + +#### Scenario: Issue event lookup + +- **WHEN** an issue's latest, oldest, recommended, or specific event is fetched without caching headers +- **THEN** the response is reused for at most 60 seconds + +#### Scenario: Full trace lookup + +- **WHEN** a full trace is fetched by trace ID without caching headers +- **THEN** the response is reused for at most 60 seconds, since spans can still arrive #### Scenario: Polling endpoint - **WHEN** autofix state is fetched - **THEN** the response is never served from cache +### Requirement: No immutable fallback + +No URL class SHALL treat a response as immutable. Without server caching +headers, no response SHALL be reused for longer than 5 minutes. + +#### Scenario: Event re-read after an hour + +- **WHEN** any event or trace URL was cached without server headers more than 5 minutes ago +- **THEN** the next read fetches from the API + ### Requirement: Mutations invalidate related entries A successful non-GET request SHALL invalidate the requesting identity's @@ -109,8 +141,8 @@ exceeded, the least recently used entries SHALL be evicted first. ### Requirement: Compressed entries -Stored entries SHALL be compressed with zstd when the runtime supports it, -and otherwise stored with a supported fallback. Each entry SHALL record its +Stored entries SHALL be compressed with zstd at level 3 when the runtime +supports it, and otherwise stored with a supported fallback. Each entry SHALL record its encoding. Reading an entry SHALL return a body identical to the original response body. @@ -121,15 +153,23 @@ response body. ### Requirement: Hosted tiers -The hosted MCP SHALL cache every tier in memory, and SHALL also persist -immutable-tier entries to durable object storage shared across isolates. -Volatile and stable entries SHALL never be persisted outside the isolate. +The hosted MCP SHALL cache every tier in memory. It SHALL persist an entry +to durable object storage shared across isolates only when the entry is in +the detail tier and the response carries a server `Cache-Control` with +`private` and a positive `max-age`. A persisted entry SHALL expire at the +lesser of the server `max-age` and 5 minutes. Volatile and stable entries +SHALL never be persisted outside the isolate. -#### Scenario: Immutable hit in another isolate +#### Scenario: Detail hit in another isolate -- **WHEN** an identity fetches an event in one isolate and later fetches it in another +- **WHEN** an identity fetches an organization event detail that carries `Cache-Control: private, max-age=300` in one isolate, and fetches it again in another isolate within 5 minutes - **THEN** the second fetch is served from durable storage without calling the API +#### Scenario: Detail response without server headers + +- **WHEN** a detail-tier response carries no server caching headers +- **THEN** it is cached in that isolate's memory only + #### Scenario: Volatile entry - **WHEN** an issue list is cached on the hosted MCP diff --git a/openspec/changes/unify-sentry-api-layer/tasks.md b/openspec/changes/unify-sentry-api-layer/tasks.md index 2007562b4..9be576118 100644 --- a/openspec/changes/unify-sentry-api-layer/tasks.md +++ b/openspec/changes/unify-sentry-api-layer/tasks.md @@ -5,65 +5,71 @@ - [ ] 1.1 Add `@sentry/api: 0.400.0` to the pnpm catalog and point `packages/cli` and `packages/mcp-core` at `catalog:`. Verify `pnpm install --frozen-lockfile` succeeds and the lockfile has one `@sentry/api` version. - [ ] 1.2 Fix any `mcp-core` breakage from the `0.337.0` → `0.400.0` bump. Verify `pnpm --filter @sentry/mcp-core run tsc` and `pnpm --filter @sentry/mcp-core run test` pass. -## 2. Shared pagination +## 2. CLI cache tier correction -- [ ] 2.1 Move `paginate`, `fetchAllPages`, page sizing, and cursor encode/decode from `cli/src/lib/api/infrastructure.ts` into `toolkit-core/src/pagination.ts`. Failures are returned as typed results, and the host supplies the logger. Verify with toolkit-core unit tests that cover each `sentry-api-pagination` scenario. -- [ ] 2.2 Switch the CLI to the shared module, mapping errors to `ApiError` and keeping the `sentry-cli/no-manual-pagination` lint rule. Verify the CLI unit tests pass with no snapshot changes. -- [ ] 2.3 Document the module in `packages/toolkit-core/README.md`, and verify `pnpm run docs:check` passes. +- [ ] 2.1 Replace the CLI's `immutable` tier in `packages/cli/src/lib/response-cache.ts` with `detail` (5 minutes): exact path shapes for organization event details without `averageColumn` and project trace-item details without `debug`. Move issue events (including `latest`, `oldest`, `recommended`), project event details, and full traces to `volatile`, and check `/issues/` before any event pattern. Verify with `response-cache.property.test.ts` cases for each `Fallback TTL tiers` scenario, replacing the 24-hour assertions. +- [ ] 2.2 Ship it as its own PR with a changelog note, since repeated event and trace reads will now call the API more often. Verify the CLI unit tests pass. -## 3. Shared cache policy and codecs +## 3. Shared pagination -- [ ] 3.1 Move tier classification, key derivation (moved to WebCrypto HMAC), URL normalization, and invalidation prefixes into `toolkit-core/src/response-cache/`. Verify with unit tests for the tier, identity-scoping, and invalidation scenarios. -- [ ] 3.2 Add a codec module (zstd level 3 or 9 by tier, brotli-4 fallback, raw below 256 bytes, `encoding` and `dictId` recorded per entry) with runtime detection. Verify round-trip tests in Node 22, and a test with zstd stubbed out. -- [ ] 3.3 Define the `CacheStore` interface and add the byte-budgeted memory store. Verify tests for LRU eviction, the per-entry cap, and the budget staying within its bound under many identities. -- [ ] 3.4 Re-implement the CLI filesystem store on `CacheStore`, with compressed entries and a byte budget, while reading existing uncompressed entries. Verify the CLI cache tests pass and an old-format entry is still served. -- [ ] 3.5 Add conditional revalidation: send `If-None-Match` for stale entries that have an `ETag`, and treat a 304 as a hit. Verify with a test against a mocked `ETag` endpoint. +- [ ] 3.1 Move `paginate`, `fetchAllPages`, page sizing, and cursor encode/decode from `cli/src/lib/api/infrastructure.ts` into `toolkit-core/src/pagination.ts`. Failures are returned as typed results, and the host supplies the logger. Verify with toolkit-core unit tests that cover each `sentry-api-pagination` scenario. +- [ ] 3.2 Switch the CLI to the shared module, mapping errors to `ApiError` and keeping the `sentry-cli/no-manual-pagination` lint rule. Verify the CLI unit tests pass with no snapshot changes. +- [ ] 3.3 Document the module in `packages/toolkit-core/README.md`, and verify `pnpm run docs:check` passes. -## 4. Shared transport +## 4. Shared cache policy and codecs -- [ ] 4.1 Move auth headers, 401 refresh, timeouts, GET retry/backoff, redirect validation, and region routing from `cli/src/lib/sentry-client.ts` into `toolkit-core/src/api-transport.ts`, with an injected `CredentialProvider`, `CacheStore`, logger, and span hook. Verify with transport unit tests. -- [ ] 4.2 Switch the CLI to the shared transport, wiring in the database credentials, the filesystem store, and telemetry spans. Verify the CLI unit and e2e tests pass. -- [ ] 4.3 Add a request-scoped fresh-read bypass to the transport, and verify a test that a bypassed request skips cache reads and replaces the entry. +- [ ] 4.1 Move tier classification, key derivation (moved to WebCrypto HMAC), URL normalization, and invalidation prefixes into `toolkit-core/src/response-cache/`. Verify with unit tests for the tier, identity-scoping, and invalidation scenarios. +- [ ] 4.2 Add a codec module (zstd level 3 for every tier, brotli-4 fallback, raw below 256 bytes, `encoding` and `dictId` recorded per entry) with runtime detection. Verify round-trip tests in Node 22, and a test with zstd stubbed out. +- [ ] 4.3 Define the `CacheStore` interface and add the byte-budgeted memory store. Verify tests for LRU eviction, the per-entry cap, and the budget staying within its bound under many identities. +- [ ] 4.4 Re-implement the CLI filesystem store on `CacheStore`, with compressed entries and a byte budget, while reading existing uncompressed entries. Verify the CLI cache tests pass and an old-format entry is still served. +- [ ] 4.5 Add conditional revalidation: send `If-None-Match` for stale entries that have an `ETag`, and treat a 304 as a hit. Verify with a test against a mocked `ETag` endpoint. -## 5. MCP transport injection +## 5. Shared transport -- [ ] 5.1 Add `fetch` to `McpServerOptions`, and route all `SentryApiService` requests and `@sentry/api` calls through it. Verify an `mcp-server` test where a supplied fetch receives every tool request. -- [ ] 5.2 Pass the CLI's authenticated fetch from `cli/src/lib/mcp.ts`. Verify a CLI test where `sentry mcp` tool calls hit the shared cache, and `/mcp-qa` stdio QA passes. -- [ ] 5.3 Add the `fresh` input to read tools through the shared schema helpers. Regenerate definitions, then verify `pnpm run measure-tokens` and the tool tests pass. +- [ ] 5.1 Move auth headers, 401 refresh, timeouts, GET retry/backoff, redirect validation, and region routing from `cli/src/lib/sentry-client.ts` into `toolkit-core/src/api-transport.ts`, with an injected `CredentialProvider`, `CacheStore`, logger, and span hook. Verify with transport unit tests. +- [ ] 5.2 Switch the CLI to the shared transport, wiring in the database credentials, the filesystem store, and telemetry spans. Verify the CLI unit and e2e tests pass. +- [ ] 5.3 Add a request-scoped fresh-read bypass to the transport, and verify a test that a bypassed request skips cache reads and replaces the entry. -## 6. Hosted cache +## 6. MCP transport injection -- [ ] 6.1 Wire the memory store into `mcp-cloudflare` behind `MCP_RESPONSE_CACHE` (`off`/`memory`/`tiered`), with identity taken from the wrapper token's user and grant IDs. Verify the Workers tests pass, and a two-identity isolation test. -- [ ] 6.2 Confirm `node:zlib` zstd on a deployed preview Worker. If it is unavailable, keep brotli as the hosted codec and record that in `docs/cloudflare/architecture.md`. -- [ ] 6.3 Add the R2 binding, the immutable-tier store, and `TieredStore`, with writes in `waitUntil` and a 1-day lifecycle rule. Verify tests for cross-isolate immutable hits and memory-only volatile entries. -- [ ] 6.4 Add cache hit, miss, and eviction metrics and memory-budget logging. Document them in `docs/operations/monitoring.md`, and verify they appear in a preview deploy. +- [ ] 6.1 Add `fetch` to `McpServerOptions`, and route all `SentryApiService` requests and `@sentry/api` calls through it. Verify an `mcp-server` test where a supplied fetch receives every tool request. +- [ ] 6.2 Pass the CLI's authenticated fetch from `cli/src/lib/mcp.ts`. Verify a CLI test where `sentry mcp` tool calls hit the shared cache, and `/mcp-qa` stdio QA passes. +- [ ] 6.3 Add the `fresh` input to read tools through the shared schema helpers. Regenerate definitions, then verify `pnpm run measure-tokens` and the tool tests pass. -## 7. Move MCP endpoints onto `@sentry/api` +## 7. Hosted cache -- [ ] 7.1 Organizations, projects, and teams. Verify the MCP tests and snapshots for the affected tools. -- [ ] 7.2 Issues and events. Verify the MCP tests and snapshots for the affected tools. -- [ ] 7.3 Explore, traces, and logs. Verify the MCP tests, snapshots, and a search-events eval run. -- [ ] 7.4 Releases and replays. Verify the MCP tests and snapshots for the affected tools. -- [ ] 7.5 Monitors, alerts, and uptime, keeping the shared-transport fallback for the endpoints `@sentry/api` lacks. Verify the MCP tests and snapshots. -- [ ] 7.6 Update `docs/contributing/api-patterns.md` for the new client layering, and verify `pnpm run docs:check` passes. +- [ ] 7.1 Wire the memory store into `mcp-cloudflare` behind `MCP_RESPONSE_CACHE` (`off`/`memory`/`tiered`), with identity taken from the wrapper token's user and grant IDs. Verify the Workers tests pass, and a two-identity isolation test. +- [ ] 7.2 Confirm `node:zlib` zstd on a deployed preview Worker. If it is unavailable, keep brotli as the hosted codec and record that in `docs/cloudflare/architecture.md`. +- [ ] 7.3 Add the R2 binding, the detail-tier store, and `TieredStore`. Persist only detail-tier entries whose response carries `Cache-Control: private` with a positive `max-age`, expiring at the lesser of that `max-age` and 5 minutes. Write in `waitUntil` and add a 1-day lifecycle rule for garbage collection. Verify tests for a cross-isolate detail hit, no R2 write without server headers, expiry enforced on read, and memory-only volatile entries. +- [ ] 7.4 Get privacy and security sign-off on the 5-minute stale window after access revocation or deletion before setting `MCP_RESPONSE_CACHE=tiered` in production. Record the decision in `docs/cloudflare/architecture.md`. +- [ ] 7.5 Add cache hit, miss, and eviction metrics and memory-budget logging. Document them in `docs/operations/monitoring.md`, and verify they appear in a preview deploy. -## 8. Standalone login deprecation +## 8. Move MCP endpoints onto `@sentry/api` -- [ ] 8.1 Compare the OAuth scopes the CLI requests with the MCP `SCOPES`, and widen the CLI's requested scopes if needed. Verify a test that the CLI's scope list covers every MCP tool's required scopes. -- [ ] 8.2 Add a deprecation notice to the standalone `@sentry/mcp-server` auth flow and docs pointing to `sentry mcp`. Verify with an output test and an update to `docs/operations/stdio-auth.md`. +- [ ] 8.1 Organizations, projects, and teams. Verify the MCP tests and snapshots for the affected tools. +- [ ] 8.2 Issues and events. Verify the MCP tests and snapshots for the affected tools. +- [ ] 8.3 Explore, traces, and logs. Verify the MCP tests, snapshots, and a search-events eval run. +- [ ] 8.4 Releases and replays. Verify the MCP tests and snapshots for the affected tools. +- [ ] 8.5 Monitors, alerts, and uptime, keeping the shared-transport fallback for the endpoints `@sentry/api` lacks. Verify the MCP tests and snapshots. +- [ ] 8.6 Update `docs/contributing/api-patterns.md` for the new client layering, and verify `pnpm run docs:check` passes. -## 9. Trained compression dictionary (follow-up) +## 9. Standalone login deprecation -- [ ] 9.1 Build a training corpus that is safe to ship: a public demo organization, or a per-deployment corpus that never leaves the hosted MCP. Verify that no customer or internal strings appear in a shipped dictionary (`strings` audit in CI). -- [ ] 9.2 Add dictionary compression with a versioned `dictId` and a startup self-test that detects a silently ignored `dictionary` option (as in workerd today). Verify a test where an ignored dictionary falls back to plain zstd. -- [ ] 9.3 On Workers, evaluate WASM zstd for dictionary compression: bundle size, CPU in ms/MB, and ratio against native zstd-3. Record the results in design.md and adopt it only if the memory-tier gain outweighs the CPU cost. +- [ ] 9.1 Compare the OAuth scopes the CLI requests with the MCP `SCOPES`, and widen the CLI's requested scopes if needed. Verify a test that the CLI's scope list covers every MCP tool's required scopes. +- [ ] 9.2 Add a deprecation notice to the standalone `@sentry/mcp-server` auth flow and docs pointing to `sentry mcp`. Verify with an output test and an update to `docs/operations/stdio-auth.md`. -## 10. Integration +## 10. Trained compression dictionary (follow-up) -- [ ] 10.1 Run `pnpm run tsc && pnpm run lint && pnpm run test` and the MCP eval suite on the final state, and record the results in the PR. +- [ ] 10.1 Build a training corpus that is safe to ship: a public demo organization, or a per-deployment corpus that never leaves the hosted MCP. Verify that no customer or internal strings appear in a shipped dictionary (`strings` audit in CI). +- [ ] 10.2 Add dictionary compression with a versioned `dictId` and a startup self-test that detects a silently ignored `dictionary` option (as in workerd today). Verify a test where an ignored dictionary falls back to plain zstd. +- [ ] 10.3 On Workers, evaluate WASM zstd for dictionary compression: bundle size, CPU in ms/MB, and ratio against native zstd-3. Record the results in design.md and adopt it only if the memory-tier gain outweighs the CPU cost. + +## 11. Integration + +- [ ] 11.1 Run `pnpm run tsc && pnpm run lint && pnpm run test` and the MCP eval suite on the final state, and record the results in the PR. ## Workflow follow-up - Archive the change after review once all tasks are complete. -- Follow up on getsentry/sentry#128818 and revisit the TTL tiers when server cache headers ship. +- Follow up on getsentry/sentry#128818 and getsentry/sentry#128828. When the server headers ship, confirm the CLI honors them and enable the R2 tier. - Propose moving pagination, then the transport, into `@sentry/api`. From a81439e7224be8834c7917786807bebfc43c9d61 Mon Sep 17 00:00:00 2001 From: "sentry-junior[bot]" <264270552+sentry-junior[bot]@users.noreply.github.com> Date: Mon, 26 Oct 2026 23:27:43 +0000 Subject: [PATCH 4/8] docs(openspec): plan a shared retry policy that honors Sentry 429 headers Neither product handles 429s well. The CLI retries every method, POSTs included, and ignores rate-limit headers. The MCP retries only GET 502/503/504, so the ~32.6K rate-limited GETs it sees a week fail immediately. Add task 5.0 for a pure `toolkit-core/src/api-retry.ts` policy: retry only GET/HEAD/OPTIONS unless a request opts in as idempotent, wait for Retry-After or X-Sentry-Rate-Limit-Reset with a clock-skew guard and caps, fall back to jittered backoff, and sleep abortably. The MCP adopts it first. Task 5.1 now moves idempotent-only retry through it and calls the CLI's all-methods retry a bug fixed separately; 6.1 drops the MCP's own retry once it is on the transport so retries don't stack. Add a sentry-api-transport spec delta for these rules. Co-Authored-By: Burak Yigit Kaya <126780+BYK@users.noreply.github.com> --- .../changes/unify-sentry-api-layer/design.md | 42 ++++++++- .../unify-sentry-api-layer/proposal.md | 15 +++- .../specs/sentry-api-transport/spec.md | 88 +++++++++++++++++++ .../changes/unify-sentry-api-layer/tasks.md | 5 +- 4 files changed, 140 insertions(+), 10 deletions(-) create mode 100644 openspec/changes/unify-sentry-api-layer/specs/sentry-api-transport/spec.md diff --git a/openspec/changes/unify-sentry-api-layer/design.md b/openspec/changes/unify-sentry-api-layer/design.md index be9e4b78a..0796ce72c 100644 --- a/openspec/changes/unify-sentry-api-layer/design.md +++ b/openspec/changes/unify-sentry-api-layer/design.md @@ -132,10 +132,13 @@ New `toolkit-core` modules: derivation, invalidation prefixes (from `cli/src/lib/cache-keys.ts`), and freshness through `http-cache-semantics`. Storage sits behind a `CacheStore` interface. +- `api-retry`: the retry policy. Pure logic with no Node APIs, so Workers + can use it. It lands first, ahead of the transport, so the MCP can fix + its 429 handling now. - `api-transport`: a fetch wrapper that takes a `CredentialProvider` (get the token, refresh on 401), an optional `CacheStore`, a logger, a span hook, - and retry/timeout options. Redirect validation and region routing move - with it. + and retry/timeout options. Retries go through `api-retry`. Redirect + validation and region routing move with it. Product code keeps credential storage, error classes, user-facing messages, and telemetry setup. @@ -144,6 +147,37 @@ _Alternative: a new package._ Rejected for now. `toolkit-core` is already the bundled-private shared package, and a split can come later when moving code into `@sentry/api`. +### Retries are for idempotent requests and honor rate-limit headers + +Neither product handles 429s well today. The CLI retries every method on +408, 429, and 5xx, POSTs included, and ignores rate-limit headers. The MCP +retries only GETs on 502, 503, and 504, so a 429 fails at once. In 7 days +the hosted MCP saw about 32,600 429s from the Sentry API, nearly all on GETs +("Limit is 40 requests in 1 seconds"). + +Sentry's 429 body is only `{"detail": ...}`. The timing is in headers: +`X-Sentry-Rate-Limit-Reset` is the Unix second the next window starts, with +`-Remaining`, `-Limit`, `-ConcurrentRemaining`, and `-ConcurrentLimit` +alongside (`src/sentry/middleware/ratelimit.py`). Sentry sends no +`Retry-After`, but a proxy in front of it can. + +`api-retry` decides whether and how long to wait: + +- Only GET, HEAD, and OPTIONS are retried, unless a request opts in with + `idempotent: true`. A replayed write can duplicate a comment or an alert. +- `Retry-After` wins when present. Otherwise a 429 waits until + `X-Sentry-Rate-Limit-Reset`, measured against the response `Date` when the + local clock disagrees with it by more than a couple of seconds. +- A concurrency-limit 429 (`ConcurrentRemaining: 0`) and a response with no + usable header fall back to jittered exponential backoff. +- Each product sets a per-wait and a total-wait cap. When the header asks for + longer, the request fails at once with a rate-limit error that says when + to retry, rather than holding a Worker request open. +- Sleeps take an `AbortSignal`. + +The 401 refresh is not a retry in this sense. It replays any method once, +because the request never reached the endpoint. + ### `CacheStore` is three byte-based operations ```ts @@ -384,8 +418,8 @@ bumps move both products together. ## Migration Plan -1. Land the catalog alignment (#1672) and the CLI cache tier correction, - each as its own PR. +1. Land the catalog alignment (#1672), the CLI cache tier correction, and + `api-retry` with the MCP 429 fix, each as its own PR. 2. Land the pure `toolkit-core` modules (pagination, cache policy, codecs), with the CLI switched over and no behavior change. 3. Land `api-transport` and the CLI switch-over. diff --git a/openspec/changes/unify-sentry-api-layer/proposal.md b/openspec/changes/unify-sentry-api-layer/proposal.md index c56632dcb..0ace61610 100644 --- a/openspec/changes/unify-sentry-api-layer/proposal.md +++ b/openspec/changes/unify-sentry-api-layer/proposal.md @@ -33,9 +33,14 @@ The shared layer must therefore sit below both products, on `@sentry/api`. covers TTL tiers, identity-scoped keys, invalidation after mutations, and bypass. Storage becomes a pluggable backend with a byte budget and compression. +- Add a shared retry policy to `@sentry/toolkit-core`: only idempotent + requests are retried, and 429s wait for Sentry's rate-limit reset header + within a cap. The MCP adopts it first, so rate-limited GETs retry instead of + failing. - Move the CLI's request transport into `@sentry/toolkit-core`, with - credentials, storage, and telemetry injected by the host. It covers retries, - timeouts, redirect validation, and region routing. + credentials, storage, and telemetry injected by the host. It covers retries + through the shared policy, timeouts, redirect validation, and region + routing. - `runMcpServer()` accepts a host-supplied transport. `sentry mcp` passes the CLI's authenticated transport, so local MCP sessions reuse the CLI login, refresh, and on-disk cache. @@ -68,6 +73,8 @@ Out of scope, tracked separately: - `sentry-api-response-cache`: Identity-scoped caching of Sentry API GET responses. Covers TTL tiers, bypass, invalidation after mutations, and bounded, compressed storage. +- `sentry-api-transport`: When a Sentry API request is retried, how long it + waits, and what a caller sees when it gives up. - `mcp-host-transport`: How an MCP host supplies credentials and the Sentry API transport, including `sentry mcp` reusing the CLI's login and cache. @@ -77,8 +84,8 @@ None. ## Impact -- `packages/toolkit-core`: new `pagination`, `response-cache`, and transport - modules, with tests. +- `packages/toolkit-core`: new `pagination`, `response-cache`, retry, and + transport modules, with tests. - `packages/cli/src/lib/api/infrastructure.ts`, `packages/cli/src/lib/response-cache.ts`, `packages/cli/src/lib/sentry-client.ts`: switch to the shared modules. diff --git a/openspec/changes/unify-sentry-api-layer/specs/sentry-api-transport/spec.md b/openspec/changes/unify-sentry-api-layer/specs/sentry-api-transport/spec.md new file mode 100644 index 000000000..7c6ffff62 --- /dev/null +++ b/openspec/changes/unify-sentry-api-layer/specs/sentry-api-transport/spec.md @@ -0,0 +1,88 @@ +# Spec Delta + +## Purpose + +Defines when a Sentry API request from the CLI or MCP is retried, how long +it waits first, and what the caller sees when it gives up. + +## ADDED Requirements + +### Requirement: Only idempotent requests are retried + +The transport SHALL automatically retry a failed request only when its +method is GET, HEAD, or OPTIONS, or when the caller marks that request +`idempotent`. A 401 that leads to a successful credential refresh SHALL be +replayed once for any method. + +#### Scenario: POST without the opt-in + +- **WHEN** a POST without the `idempotent` opt-in gets a 429, a 5xx, or a timeout +- **THEN** the transport returns that failure without sending the request again + +#### Scenario: POST with the opt-in + +- **WHEN** a POST marked `idempotent`, such as a content-addressed chunk upload, gets a 503 +- **THEN** the transport retries it like a GET + +#### Scenario: Token refresh + +- **WHEN** any request, of any method, gets a 401 and the credential provider refreshes the token +- **THEN** the transport sends the request once more with the new token + +### Requirement: Rate-limited requests wait for the reset + +On a retryable 429, the transport SHALL wait for the delay the response +gives before retrying: `Retry-After` (seconds or an HTTP-date) when +present, otherwise `X-Sentry-Rate-Limit-Reset`. It SHALL measure the reset +against the response `Date` when the local clock disagrees with it by more +than a few seconds. When the response gives no usable delay, or the 429 is +for concurrency (`X-Sentry-Rate-Limit-ConcurrentRemaining: 0`), it SHALL use +jittered exponential backoff. + +#### Scenario: Reset header within the cap + +- **WHEN** a GET gets a 429 whose `X-Sentry-Rate-Limit-Reset` is 1 second away and the per-wait cap is 2 seconds +- **THEN** the transport waits about 1 second and retries the GET + +#### Scenario: Skewed local clock + +- **WHEN** the local clock is 10 minutes behind the response `Date` and the reset is 1 second after that `Date` +- **THEN** the transport waits about 1 second, not 10 minutes + +#### Scenario: Concurrency limit + +- **WHEN** a GET gets a 429 with `X-Sentry-Rate-Limit-ConcurrentRemaining: 0` +- **THEN** the transport retries after a jittered backoff delay, not at the window reset + +### Requirement: Retry waits are capped + +Each product SHALL set a maximum number of retries, a per-wait cap, and a +total-wait cap. When the next wait would exceed either cap, the transport +SHALL stop retrying and return a rate-limit error that says how many seconds +to wait before trying again. An aborted request SHALL stop waiting at once. + +#### Scenario: Reset beyond the cap + +- **WHEN** a GET gets a 429 whose reset is 30 seconds away and the per-wait cap is 2 seconds +- **THEN** the transport does not wait and returns a rate-limit error saying to retry in 30 seconds + +#### Scenario: Retries exhausted + +- **WHEN** a GET is still rate limited after the last allowed retry +- **THEN** the caller gets a rate-limit error with the delay from the last response + +#### Scenario: Aborted during a wait + +- **WHEN** the caller aborts a request while it waits to retry +- **THEN** the wait ends and the request fails with the abort reason + +### Requirement: Retries do not stack + +A request SHALL be retried by one layer only. When the MCP sends its +requests through a host-supplied or shared transport, `SentryApiService` +SHALL not retry them again. + +#### Scenario: MCP on the shared transport + +- **WHEN** an MCP tool's GET is rate limited on every attempt through the shared transport +- **THEN** the Sentry API receives at most `maxRetries + 1` requests for that call diff --git a/openspec/changes/unify-sentry-api-layer/tasks.md b/openspec/changes/unify-sentry-api-layer/tasks.md index 9be576118..bc28196b9 100644 --- a/openspec/changes/unify-sentry-api-layer/tasks.md +++ b/openspec/changes/unify-sentry-api-layer/tasks.md @@ -26,13 +26,14 @@ ## 5. Shared transport -- [ ] 5.1 Move auth headers, 401 refresh, timeouts, GET retry/backoff, redirect validation, and region routing from `cli/src/lib/sentry-client.ts` into `toolkit-core/src/api-transport.ts`, with an injected `CredentialProvider`, `CacheStore`, logger, and span hook. Verify with transport unit tests. +- [ ] 5.0 Add a pure, Workers-safe retry policy in `toolkit-core/src/api-retry.ts`: retryable statuses; retryable methods (GET, HEAD, and OPTIONS by default, plus an explicit per-request `idempotent` opt-in, such as content-addressed chunk uploads); the delay from `Retry-After` (seconds or HTTP-date) or, on a 429, `X-Sentry-Rate-Limit-Reset`, with a clock-skew guard against the response `Date` and a per-wait and total-wait cap; a jittered exponential fallback, used for concurrency-limit 429s too; and an abortable sleep. The 401 refresh is not part of this policy and stays allowed for any method. MCP `request()` adopts it in the same PR, retrying rate-limited GETs and filling `ApiRateLimitError.retryAfter` from the headers. Verify with toolkit-core unit tests for each `sentry-api-transport` retry scenario and MCP client tests. The CLI's interim local fix switches to this module in a follow-up PR. +- [ ] 5.1 Move auth headers, 401 refresh (any method), timeouts, idempotent-method retry and backoff through `api-retry`, redirect validation, and region routing from `cli/src/lib/sentry-client.ts` into `toolkit-core/src/api-transport.ts`, with an injected `CredentialProvider`, `CacheStore`, logger, and span hook. The CLI used to retry every method, POSTs included; that was a bug, fixed separately before this move. Verify with transport unit tests. - [ ] 5.2 Switch the CLI to the shared transport, wiring in the database credentials, the filesystem store, and telemetry spans. Verify the CLI unit and e2e tests pass. - [ ] 5.3 Add a request-scoped fresh-read bypass to the transport, and verify a test that a bypassed request skips cache reads and replaces the entry. ## 6. MCP transport injection -- [ ] 6.1 Add `fetch` to `McpServerOptions`, and route all `SentryApiService` requests and `@sentry/api` calls through it. Verify an `mcp-server` test where a supplied fetch receives every tool request. +- [ ] 6.1 Add `fetch` to `McpServerOptions`, and route all `SentryApiService` requests and `@sentry/api` calls through it. `SentryApiService` drops its own retry when it moves onto the transport, so retries do not stack. Verify an `mcp-server` test where a supplied fetch receives every tool request, and a test where a rate-limited GET reaches the API at most `maxRetries + 1` times. - [ ] 6.2 Pass the CLI's authenticated fetch from `cli/src/lib/mcp.ts`. Verify a CLI test where `sentry mcp` tool calls hit the shared cache, and `/mcp-qa` stdio QA passes. - [ ] 6.3 Add the `fresh` input to read tools through the shared schema helpers. Regenerate definitions, then verify `pnpm run measure-tokens` and the tool tests pass. From b5523f5e76f39ed55ce600725056d90997feaed7 Mon Sep 17 00:00:00 2001 From: "sentry-junior[bot]" <264270552+sentry-junior[bot]@users.noreply.github.com> Date: Tue, 27 Oct 2026 09:30:07 +0000 Subject: [PATCH 5/8] docs(openspec): Retry 429s for any method and add 500 to retried statuses Sentry's RatelimitMiddleware rejects a 429 in process_request, before the view runs, so replaying one cannot repeat a write. The sentry-api-transport spec now retries 429s for any method, and retries 5xx, timeouts, and network errors only for GET/HEAD/OPTIONS or an explicit idempotent opt-in. The retried statuses are 429, 500, 502, 503, and 504. Other 5xx codes are deterministic. The clock-skew wording now matches the Date-bounded delay from #1673. The long requirements are split so `openspec validate --strict` passes. --- .../changes/unify-sentry-api-layer/design.md | 13 +++-- .../unify-sentry-api-layer/proposal.md | 6 +-- .../specs/sentry-api-transport/spec.md | 52 ++++++++++++++----- .../changes/unify-sentry-api-layer/tasks.md | 4 +- 4 files changed, 53 insertions(+), 22 deletions(-) diff --git a/openspec/changes/unify-sentry-api-layer/design.md b/openspec/changes/unify-sentry-api-layer/design.md index 0796ce72c..b9845750f 100644 --- a/openspec/changes/unify-sentry-api-layer/design.md +++ b/openspec/changes/unify-sentry-api-layer/design.md @@ -147,7 +147,7 @@ _Alternative: a new package._ Rejected for now. `toolkit-core` is already the bundled-private shared package, and a split can come later when moving code into `@sentry/api`. -### Retries are for idempotent requests and honor rate-limit headers +### Retries replay only requests that cannot repeat a write, and honor rate-limit headers Neither product handles 429s well today. The CLI retries every method on 408, 429, and 5xx, POSTs included, and ignores rate-limit headers. The MCP @@ -163,11 +163,16 @@ alongside (`src/sentry/middleware/ratelimit.py`). Sentry sends no `api-retry` decides whether and how long to wait: -- Only GET, HEAD, and OPTIONS are retried, unless a request opts in with +- The retried statuses are 429, 500, 502, 503, and 504. The other 5xx codes, + such as 501 and 505, are deterministic, so they are not retried. +- A 429 is retried for any method. `RatelimitMiddleware.process_request` + rejects it before the view runs, so replaying it cannot repeat a write. +- A 5xx, a timeout, or a network error may come after the endpoint ran. Those + are retried only for GET, HEAD, and OPTIONS, unless a request opts in with `idempotent: true`. A replayed write can duplicate a comment or an alert. - `Retry-After` wins when present. Otherwise a 429 waits until - `X-Sentry-Rate-Limit-Reset`, measured against the response `Date` when the - local clock disagrees with it by more than a couple of seconds. + `X-Sentry-Rate-Limit-Reset`. The delay is kept within the range the response + `Date` allows, so a skewed local clock moves it by at most a second. - A concurrency-limit 429 (`ConcurrentRemaining: 0`) and a response with no usable header fall back to jittered exponential backoff. - Each product sets a per-wait and a total-wait cap. When the header asks for diff --git a/openspec/changes/unify-sentry-api-layer/proposal.md b/openspec/changes/unify-sentry-api-layer/proposal.md index 0ace61610..826ac06a1 100644 --- a/openspec/changes/unify-sentry-api-layer/proposal.md +++ b/openspec/changes/unify-sentry-api-layer/proposal.md @@ -33,9 +33,9 @@ The shared layer must therefore sit below both products, on `@sentry/api`. covers TTL tiers, identity-scoped keys, invalidation after mutations, and bypass. Storage becomes a pluggable backend with a byte budget and compression. -- Add a shared retry policy to `@sentry/toolkit-core`: only idempotent - requests are retried, and 429s wait for Sentry's rate-limit reset header - within a cap. The MCP adopts it first, so rate-limited GETs retry instead of +- Add a shared retry policy to `@sentry/toolkit-core`: 429s are retried for + any method and wait for Sentry's rate-limit reset header within a cap, while + 5xx responses and timeouts are retried only for idempotent requests. The MCP adopts it first, so rate-limited GETs retry instead of failing. - Move the CLI's request transport into `@sentry/toolkit-core`, with credentials, storage, and telemetry injected by the host. It covers retries diff --git a/openspec/changes/unify-sentry-api-layer/specs/sentry-api-transport/spec.md b/openspec/changes/unify-sentry-api-layer/specs/sentry-api-transport/spec.md index 7c6ffff62..b39a57559 100644 --- a/openspec/changes/unify-sentry-api-layer/specs/sentry-api-transport/spec.md +++ b/openspec/changes/unify-sentry-api-layer/specs/sentry-api-transport/spec.md @@ -7,16 +7,39 @@ it waits first, and what the caller sees when it gives up. ## ADDED Requirements -### Requirement: Only idempotent requests are retried +### Requirement: Retried statuses -The transport SHALL automatically retry a failed request only when its -method is GET, HEAD, or OPTIONS, or when the caller marks that request -`idempotent`. A 401 that leads to a successful credential refresh SHALL be -replayed once for any method. +The transport SHALL retry only 429, 500, 502, 503, and 504 responses, plus +timeouts and network errors where the product enables them. Other 5xx codes, +such as 501 and 505, are deterministic, so retrying them only adds latency. + +#### Scenario: Deterministic 5xx + +- **WHEN** a GET gets a 501 +- **THEN** the transport returns the failure without retrying + +### Requirement: Rate-limited requests are retried for any method + +The transport SHALL retry a 429 for any method. Sentry's +`RatelimitMiddleware` rejects the request in `process_request`, before the +view runs (`src/sentry/middleware/ratelimit.py`), so replaying it cannot +repeat a write. + +#### Scenario: Rate-limited POST + +- **WHEN** a POST without the `idempotent` opt-in gets a 429 whose reset is within the cap +- **THEN** the transport waits for the reset and sends the POST again + +### Requirement: Other failures are retried only for idempotent requests + +A 5xx, a timeout, or a network error may come after the endpoint ran. The +transport SHALL NOT retry one unless the method is GET, HEAD, or OPTIONS, or +the caller marks the request `idempotent`. A 401 that leads to a successful +credential refresh SHALL be replayed once for any method. #### Scenario: POST without the opt-in -- **WHEN** a POST without the `idempotent` opt-in gets a 429, a 5xx, or a timeout +- **WHEN** a POST without the `idempotent` opt-in gets a 5xx, a timeout, or a network error - **THEN** the transport returns that failure without sending the request again #### Scenario: POST with the opt-in @@ -31,13 +54,10 @@ replayed once for any method. ### Requirement: Rate-limited requests wait for the reset -On a retryable 429, the transport SHALL wait for the delay the response -gives before retrying: `Retry-After` (seconds or an HTTP-date) when -present, otherwise `X-Sentry-Rate-Limit-Reset`. It SHALL measure the reset -against the response `Date` when the local clock disagrees with it by more -than a few seconds. When the response gives no usable delay, or the 429 is -for concurrency (`X-Sentry-Rate-Limit-ConcurrentRemaining: 0`), it SHALL use -jittered exponential backoff. +On a 429, the transport SHALL wait for `Retry-After` (seconds or an +HTTP-date) when present, otherwise for `X-Sentry-Rate-Limit-Reset`. It SHALL +keep that delay within the range the response `Date` allows, so a skewed +local clock moves it by at most one second. #### Scenario: Reset header within the cap @@ -49,6 +69,12 @@ jittered exponential backoff. - **WHEN** the local clock is 10 minutes behind the response `Date` and the reset is 1 second after that `Date` - **THEN** the transport waits about 1 second, not 10 minutes +### Requirement: Backoff without a usable delay + +When a retryable response gives no usable delay, or a 429 is for +concurrency (`X-Sentry-Rate-Limit-ConcurrentRemaining: 0`), the transport +SHALL wait a jittered exponential backoff delay. + #### Scenario: Concurrency limit - **WHEN** a GET gets a 429 with `X-Sentry-Rate-Limit-ConcurrentRemaining: 0` diff --git a/openspec/changes/unify-sentry-api-layer/tasks.md b/openspec/changes/unify-sentry-api-layer/tasks.md index bc28196b9..6fb96f250 100644 --- a/openspec/changes/unify-sentry-api-layer/tasks.md +++ b/openspec/changes/unify-sentry-api-layer/tasks.md @@ -26,8 +26,8 @@ ## 5. Shared transport -- [ ] 5.0 Add a pure, Workers-safe retry policy in `toolkit-core/src/api-retry.ts`: retryable statuses; retryable methods (GET, HEAD, and OPTIONS by default, plus an explicit per-request `idempotent` opt-in, such as content-addressed chunk uploads); the delay from `Retry-After` (seconds or HTTP-date) or, on a 429, `X-Sentry-Rate-Limit-Reset`, with a clock-skew guard against the response `Date` and a per-wait and total-wait cap; a jittered exponential fallback, used for concurrency-limit 429s too; and an abortable sleep. The 401 refresh is not part of this policy and stays allowed for any method. MCP `request()` adopts it in the same PR, retrying rate-limited GETs and filling `ApiRateLimitError.retryAfter` from the headers. Verify with toolkit-core unit tests for each `sentry-api-transport` retry scenario and MCP client tests. The CLI's interim local fix switches to this module in a follow-up PR. -- [ ] 5.1 Move auth headers, 401 refresh (any method), timeouts, idempotent-method retry and backoff through `api-retry`, redirect validation, and region routing from `cli/src/lib/sentry-client.ts` into `toolkit-core/src/api-transport.ts`, with an injected `CredentialProvider`, `CacheStore`, logger, and span hook. The CLI used to retry every method, POSTs included; that was a bug, fixed separately before this move. Verify with transport unit tests. +- [ ] 5.0 Add a pure, Workers-safe retry policy in `toolkit-core/src/api-retry.ts`: retryable statuses (429, 500, 502, 503, and 504, not every 5xx, since 501, 505, and the rest are deterministic); a 429 retried for any method, because Sentry's `RatelimitMiddleware` rejects it before the view runs; 5xx, timeouts, and network errors retried only for GET, HEAD, and OPTIONS, or for an explicit per-request `idempotent` opt-in, such as content-addressed chunk uploads; the delay from `Retry-After` (seconds or HTTP-date) or, on a 429, `X-Sentry-Rate-Limit-Reset`, kept within the range the response `Date` allows, and a per-wait and total-wait cap; a jittered exponential fallback, used for concurrency-limit 429s too; and an abortable sleep. The 401 refresh is not part of this policy and stays allowed for any method. MCP `request()` adopts it in the same PR, retrying rate-limited requests and filling `ApiRateLimitError.retryAfter` from the headers. Verify with toolkit-core unit tests for each `sentry-api-transport` retry scenario and MCP client tests. The CLI's interim local fix switches to this module in a follow-up PR. +- [ ] 5.1 Move auth headers, 401 refresh (any method), timeouts, retry and backoff through `api-retry` (429 for any method; 5xx, timeouts, and network errors for idempotent methods only), redirect validation, and region routing from `cli/src/lib/sentry-client.ts` into `toolkit-core/src/api-transport.ts`, with an injected `CredentialProvider`, `CacheStore`, logger, and span hook. The CLI used to retry every method on 5xx and timeouts, POSTs included; that was a bug, fixed separately before this move. Verify with transport unit tests. - [ ] 5.2 Switch the CLI to the shared transport, wiring in the database credentials, the filesystem store, and telemetry spans. Verify the CLI unit and e2e tests pass. - [ ] 5.3 Add a request-scoped fresh-read bypass to the transport, and verify a test that a bypassed request skips cache reads and replaces the entry. From a2a032a8c673e8efcefb54a12c6123a5a8100c11 Mon Sep 17 00:00:00 2001 From: "sentry-junior[bot]" <264270552+sentry-junior[bot]@users.noreply.github.com> Date: Tue, 27 Oct 2026 17:41:15 +0000 Subject: [PATCH 6/8] docs(openspec): mark version alignment done (#1672) Co-Authored-By: BYK --- openspec/changes/unify-sentry-api-layer/tasks.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/openspec/changes/unify-sentry-api-layer/tasks.md b/openspec/changes/unify-sentry-api-layer/tasks.md index 6fb96f250..32d73b664 100644 --- a/openspec/changes/unify-sentry-api-layer/tasks.md +++ b/openspec/changes/unify-sentry-api-layer/tasks.md @@ -2,8 +2,8 @@ ## 1. Version alignment -- [ ] 1.1 Add `@sentry/api: 0.400.0` to the pnpm catalog and point `packages/cli` and `packages/mcp-core` at `catalog:`. Verify `pnpm install --frozen-lockfile` succeeds and the lockfile has one `@sentry/api` version. -- [ ] 1.2 Fix any `mcp-core` breakage from the `0.337.0` → `0.400.0` bump. Verify `pnpm --filter @sentry/mcp-core run tsc` and `pnpm --filter @sentry/mcp-core run test` pass. +- [x] 1.1 Add `@sentry/api: 0.400.0` to the pnpm catalog and point `packages/cli` and `packages/mcp-core` at `catalog:`. Verify `pnpm install --frozen-lockfile` succeeds and the lockfile has one `@sentry/api` version. +- [x] 1.2 Fix any `mcp-core` breakage from the `0.337.0` → `0.400.0` bump. Verify `pnpm --filter @sentry/mcp-core run tsc` and `pnpm --filter @sentry/mcp-core run test` pass. ## 2. CLI cache tier correction From 317a80dd87edc1cab08792269ba04d3c248860ea Mon Sep 17 00:00:00 2001 From: "sentry-junior[bot]" <264270552+sentry-junior[bot]@users.noreply.github.com> Date: Tue, 27 Oct 2026 19:12:03 +0000 Subject: [PATCH 7/8] docs(openspec): Fold the detail tier into stable and allowlist R2 instead Matches #1681 as merged: the CLI has no separate detail tier, since it would share the stable TTL. R2 eligibility on the hosted MCP becomes an explicit URL allowlist owned by that store. Mark task group 2 done. Co-Authored-By: BYK --- .../changes/unify-sentry-api-layer/design.md | 71 ++++++++++--------- .../unify-sentry-api-layer/proposal.md | 19 ++--- .../specs/sentry-api-response-cache/spec.md | 31 +++----- .../changes/unify-sentry-api-layer/tasks.md | 6 +- 4 files changed, 59 insertions(+), 68 deletions(-) diff --git a/openspec/changes/unify-sentry-api-layer/design.md b/openspec/changes/unify-sentry-api-layer/design.md index b9845750f..3810f2f95 100644 --- a/openspec/changes/unify-sentry-api-layer/design.md +++ b/openspec/changes/unify-sentry-api-layer/design.md @@ -89,33 +89,26 @@ endpoints are safe to mark `immutable`: revocation are not reflected until the entry expires. That bounds the TTL, not just the tier. -So the shared policy replaces `immutable` with `detail`: - -| Tier | TTL | Matches | -| -------- | ---- | ------------------------------------------------------------------------------------------------------------------------------------------ | -| no-cache | 0 | autofix and root-cause polling | -| detail | 5 m | `/organizations/{org}/events/{project}:{event_id}/` without `averageColumn`; `/projects/{org}/{project}/trace-items/{id}/` without `debug` | -| volatile | 60 s | `/issues/` (including issue events), `/projects/{org}/{project}/events/{id}/`, `/organizations/{org}/trace/{id}/`, log and span queries | -| stable | 5 m | everything else | - -- Patterns are exact path shapes, anchored on the API prefix, not - substring matches on `/events/`. Anything that does not match an - allowlisted shape falls through to volatile or stable. -- `detail` has the same TTL as `stable` in the CLI. It exists as its own - tier because it is the only tier eligible for R2 on the hosted MCP. -- Today the CLI reads events through project and issue endpoints - (`getEvent`, `getLatestEvent` in `cli/src/lib/api/events.ts`), and the MCP - through `/organizations/{org}/issues/{id}/events/{event_id}/`. So in - practice the detail tier covers trace items until a caller moves to the - organization event endpoint. That is worth doing per call site only where - the response has every field the caller uses. +So the CLI drops `immutable` (getsentry/toolkit#1681): + +| Tier | TTL | Matches | +| -------- | ---- | --------------------------------------------------------------------------------------------------------------------------------------- | +| no-cache | 0 | autofix and root-cause polling | +| volatile | 60 s | `/issues/` (including issue events), `/projects/{org}/{project}/events/{id}/`, `/organizations/{org}/trace/{id}/`, log and span queries | +| stable | 5 m | everything else, including organization event details and trace-item details | + +- No separate tier for the safe detail endpoints. In the CLI it would have + the same TTL as `stable`, so it was folded in during review. Which + responses the hosted MCP may persist to R2 is a decision for that store, + made with its own allowlist (see the R2 backend below), not a TTL tier. - When server headers are present they still win. The CLI keeps honoring `max-age`, so if the server later extends it, the CLI follows without a release. -- This lands as its own CLI PR before the move into `toolkit-core`, so the - move itself stays a no-behavior-change refactor. The existing - `response-cache.property.test.ts` cases that assert 24-hour event and - trace caching are replaced with scenarios from the spec delta. +- Classification stays URL-regex based for now. A TODO in + `response-cache.ts` tracks moving it to generated `@sentry/api` operation + metadata (getsentry/sentry-api-schema#81, #102). +- This landed as its own CLI PR before the move into `toolkit-core`, so the + move itself stays a no-behavior-change refactor. The cost is more API calls for repeated event and trace reads. A 24-hour window was a correctness bug, not a tunable: an agent re-reading an issue's @@ -208,19 +201,27 @@ Backends: size, configurable by env. - Module-scoped, so it lives as long as the isolate. `deleteMatching` scans the in-memory index. -- **R2 (hosted MCP, detail tier only).** - - Persists an entry only when it is in the detail tier _and_ the response - carries `Cache-Control: private` with a positive `max-age`. The server - header is the opt-in: until getsentry/sentry#128828 ships and is - enabled, R2 stores nothing. +- **R2 (hosted MCP, allowlisted detail endpoints only).** + - Persists an entry only when its URL matches the store's explicit + allowlist _and_ the response carries `Cache-Control: private` with a + positive `max-age`. The allowlist holds exact path shapes: + `/organizations/{org}/events/{project}:{event_id}/` without + `averageColumn`, and `/projects/{org}/{project}/trace-items/{id}/` + without `debug`. The server header is the opt-in: until + getsentry/sentry#128828 ships and is enabled, R2 stores nothing. + - Today the CLI and MCP read events through project and issue endpoints, + which are not on the allowlist. So in practice R2 covers trace items + until a caller moves to the organization event endpoint, which is worth + doing per call site only where the response has every field the caller + uses. - Object key is `/`, with the expiry, the lesser of the server `max-age` and 5 minutes, in custom metadata and checked on read. - A bucket lifecycle rule deletes objects after 1 day, the shortest rule R2 supports. It is garbage collection only; reads enforce the expiry. - Writes happen in `ctx.waitUntil()`, so they never delay a response. - - Detail entries are never targeted by mutation invalidation, so R2 never - needs `deleteMatching`. The short expiry bounds how long a revoked or + - Allowlisted entries are never targeted by mutation invalidation, so R2 + never needs `deleteMatching`. The short expiry bounds how long a revoked or deleted item can still be served. _Alternatives:_ @@ -236,8 +237,8 @@ _Alternatives:_ `TieredStore(memory, r2)`: -- `get` checks memory first, then R2 for detail keys, and copies R2 hits - into memory with the remaining expiry. +- `get` checks memory first, then R2 for allowlisted URLs, and copies R2 + hits into memory with the remaining expiry. - `set` writes to memory, and also to R2 when the entry is R2-eligible (see above). - `deleteMatching` only touches memory, since only volatile and stable @@ -399,7 +400,7 @@ bumps move both products together. description tells agents when to use it. - **Revoked access or deleted data served from cache.** A user who loses access to a project, or whose event is deleted, can still get a cached - copy until it expires: up to 5 minutes for detail entries, including + copy until it expires: up to 5 minutes for stable entries, including from R2 on the hosted MCP. Enabling R2 in production needs a privacy and security sign-off on that window, alongside the server-side headers in getsentry/sentry#128828. @@ -430,7 +431,7 @@ bumps move both products together. 3. Land `api-transport` and the CLI switch-over. 4. Land the MCP transport injection and `sentry mcp` passing the CLI fetch, then the hosted memory store behind a flag (`MCP_RESPONSE_CACHE=memory`). -5. Enable the hosted memory cache. Add the R2 binding and the detail tier +5. Enable the hosted memory cache. Add the R2 binding and its allowlist behind `MCP_RESPONSE_CACHE=tiered`, and enable it only after the server headers ship and the security sign-off. 6. Run the domain-by-domain `@sentry/api` migration PRs. diff --git a/openspec/changes/unify-sentry-api-layer/proposal.md b/openspec/changes/unify-sentry-api-layer/proposal.md index 826ac06a1..6aa3c0af5 100644 --- a/openspec/changes/unify-sentry-api-layer/proposal.md +++ b/openspec/changes/unify-sentry-api-layer/proposal.md @@ -23,12 +23,12 @@ The shared layer must therefore sit below both products, on `@sentry/api`. - Move the CLI's pagination into `@sentry/toolkit-core` and use it from both products: limit-bounded walks, even page sizing, and cursors that resume mid-page. -- **Behavior change:** narrow the CLI's 24-hour "immutable" cache tier. It - matches every `/events//` and `/trace//` URL, but issue events, - project events, and full traces change after they are first read - (getsentry/sentry#128818). Only organization event details and project - trace-item details stay in a separate tier, at 5 minutes. Issue events, - project events, and full traces drop to 60 seconds. +- **Behavior change (landed in #1681):** drop the CLI's 24-hour + "immutable" cache tier. It matched every `/events//` and + `/trace//` URL, but issue events, project events, and full traces + change after they are first read (getsentry/sentry#128818). Issue events, + project events, and full traces drop to 60 seconds; organization event + details and trace-item details fall to the default 5 minutes. - Move the CLI's response-cache policy into `@sentry/toolkit-core`. That covers TTL tiers, identity-scoped keys, invalidation after mutations, and bypass. Storage becomes a pluggable backend with a byte budget and @@ -46,8 +46,9 @@ The shared layer must therefore sit below both products, on `@sentry/api`. refresh, and on-disk cache. - The hosted MCP on Cloudflare uses a tiered cache: - an in-memory LRU, limited by bytes, for every tier - - R2 for organization event details and project trace-item details, only - when the server marks them `private, max-age`, for at most 5 minutes + - R2 for an allowlist of organization event details and project + trace-item details, only when the server marks them `private, max-age`, + for at most 5 minutes - zstd level 3 compression on both - MCP read tools accept a `fresh` parameter that bypasses the cache, matching the CLI's `--fresh` flag. @@ -98,6 +99,6 @@ None. option, and the standalone auth flow is deprecated. - `packages/cli/src/lib/mcp.ts`: passes the CLI transport. - `packages/mcp-cloudflare`: in-memory cache, an R2 bucket binding for - detail responses, and a wrangler config change. + allowlisted detail responses, and a wrangler config change. - `pnpm-workspace.yaml`: a catalog entry for `@sentry/api`. - Snapshot and eval churn in `mcp-core` and `mcp-server-evals`. diff --git a/openspec/changes/unify-sentry-api-layer/specs/sentry-api-response-cache/spec.md b/openspec/changes/unify-sentry-api-layer/specs/sentry-api-response-cache/spec.md index a2331f7c4..bac7fea02 100644 --- a/openspec/changes/unify-sentry-api-layer/specs/sentry-api-response-cache/spec.md +++ b/openspec/changes/unify-sentry-api-layer/specs/sentry-api-response-cache/spec.md @@ -48,28 +48,17 @@ When a response carries HTTP caching headers, the cache SHALL follow them Responses without caching headers SHALL use a TTL by URL class: -- detail: 5 minutes (organization event details and project trace-item - details, by exact path) - volatile: 60 seconds (issue endpoints, project event details, full traces, log and span queries) - no-cache: never cached (autofix and root-cause polling) -- stable: 5 minutes (everything else) +- stable: 5 minutes (everything else, including organization event details + and trace-item details) #### Scenario: Organization event lookup - **WHEN** `/organizations/{org}/events/{project}:{event_id}/` is fetched without caching headers - **THEN** the response is reused for up to 5 minutes -#### Scenario: Live aggregate on an event - -- **WHEN** an organization event detail is fetched with an `averageColumn` query -- **THEN** it is classified as stable, not detail - -#### Scenario: Trace-item debug lookup - -- **WHEN** a project trace-item detail is fetched with a `debug` query -- **THEN** it is classified as stable, not detail - #### Scenario: Issue event lookup - **WHEN** an issue's latest, oldest, recommended, or specific event is fetched without caching headers @@ -154,20 +143,20 @@ response body. ### Requirement: Hosted tiers The hosted MCP SHALL cache every tier in memory. It SHALL persist an entry -to durable object storage shared across isolates only when the entry is in -the detail tier and the response carries a server `Cache-Control` with -`private` and a positive `max-age`. A persisted entry SHALL expire at the -lesser of the server `max-age` and 5 minutes. Volatile and stable entries -SHALL never be persisted outside the isolate. +to durable object storage shared across isolates only when its URL matches +an explicit allowlist of detail endpoints and the response carries a server +`Cache-Control` with `private` and a positive `max-age`. A persisted entry SHALL expire at the +lesser of the server `max-age` and 5 minutes. Entries outside the +allowlist SHALL never be persisted outside the isolate. -#### Scenario: Detail hit in another isolate +#### Scenario: Allowlisted hit in another isolate - **WHEN** an identity fetches an organization event detail that carries `Cache-Control: private, max-age=300` in one isolate, and fetches it again in another isolate within 5 minutes - **THEN** the second fetch is served from durable storage without calling the API -#### Scenario: Detail response without server headers +#### Scenario: Allowlisted response without server headers -- **WHEN** a detail-tier response carries no server caching headers +- **WHEN** an allowlisted response carries no server caching headers - **THEN** it is cached in that isolate's memory only #### Scenario: Volatile entry diff --git a/openspec/changes/unify-sentry-api-layer/tasks.md b/openspec/changes/unify-sentry-api-layer/tasks.md index 32d73b664..6f9a822ce 100644 --- a/openspec/changes/unify-sentry-api-layer/tasks.md +++ b/openspec/changes/unify-sentry-api-layer/tasks.md @@ -7,8 +7,8 @@ ## 2. CLI cache tier correction -- [ ] 2.1 Replace the CLI's `immutable` tier in `packages/cli/src/lib/response-cache.ts` with `detail` (5 minutes): exact path shapes for organization event details without `averageColumn` and project trace-item details without `debug`. Move issue events (including `latest`, `oldest`, `recommended`), project event details, and full traces to `volatile`, and check `/issues/` before any event pattern. Verify with `response-cache.property.test.ts` cases for each `Fallback TTL tiers` scenario, replacing the 24-hour assertions. -- [ ] 2.2 Ship it as its own PR with a changelog note, since repeated event and trace reads will now call the API more often. Verify the CLI unit tests pass. +- [x] 2.1 Remove the CLI's `immutable` tier in `packages/cli/src/lib/response-cache.ts`. Move issue events (including `latest`, `oldest`, `recommended`), project event details, and full traces to `volatile`; organization event and trace-item details fall to `stable`. Verify with classifier and fake-clock expiry tests for each `Fallback TTL tiers` scenario. +- [x] 2.2 Ship it as its own PR (#1681). Verify the CLI unit tests pass. ## 3. Shared pagination @@ -41,7 +41,7 @@ - [ ] 7.1 Wire the memory store into `mcp-cloudflare` behind `MCP_RESPONSE_CACHE` (`off`/`memory`/`tiered`), with identity taken from the wrapper token's user and grant IDs. Verify the Workers tests pass, and a two-identity isolation test. - [ ] 7.2 Confirm `node:zlib` zstd on a deployed preview Worker. If it is unavailable, keep brotli as the hosted codec and record that in `docs/cloudflare/architecture.md`. -- [ ] 7.3 Add the R2 binding, the detail-tier store, and `TieredStore`. Persist only detail-tier entries whose response carries `Cache-Control: private` with a positive `max-age`, expiring at the lesser of that `max-age` and 5 minutes. Write in `waitUntil` and add a 1-day lifecycle rule for garbage collection. Verify tests for a cross-isolate detail hit, no R2 write without server headers, expiry enforced on read, and memory-only volatile entries. +- [ ] 7.3 Add the R2 binding, its URL allowlist (organization event details without `averageColumn`, project trace-item details without `debug`), and `TieredStore`. Persist only allowlisted entries whose response carries `Cache-Control: private` with a positive `max-age`, expiring at the lesser of that `max-age` and 5 minutes. Write in `waitUntil` and add a 1-day lifecycle rule for garbage collection. Verify tests for a cross-isolate allowlisted hit, no R2 write without server headers, expiry enforced on read, and memory-only volatile entries. - [ ] 7.4 Get privacy and security sign-off on the 5-minute stale window after access revocation or deletion before setting `MCP_RESPONSE_CACHE=tiered` in production. Record the decision in `docs/cloudflare/architecture.md`. - [ ] 7.5 Add cache hit, miss, and eviction metrics and memory-budget logging. Document them in `docs/operations/monitoring.md`, and verify they appear in a preview deploy. From 5f877f28ecd098ba25c4780ac7f65fdf8d790f9e Mon Sep 17 00:00:00 2001 From: "sentry-junior[bot]" <264270552+sentry-junior[bot]@users.noreply.github.com> Date: Tue, 27 Oct 2026 19:35:56 +0000 Subject: [PATCH 8/8] docs(openspec): Mark shared pagination done (#1683) Task group 3 moved paginate, fetchAllPages, page sizing, and cursor encoding into @sentry/toolkit-core/pagination behind CLI wrappers. Co-Authored-By: BYK --- openspec/changes/unify-sentry-api-layer/tasks.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/openspec/changes/unify-sentry-api-layer/tasks.md b/openspec/changes/unify-sentry-api-layer/tasks.md index 6f9a822ce..fa750bf82 100644 --- a/openspec/changes/unify-sentry-api-layer/tasks.md +++ b/openspec/changes/unify-sentry-api-layer/tasks.md @@ -12,9 +12,9 @@ ## 3. Shared pagination -- [ ] 3.1 Move `paginate`, `fetchAllPages`, page sizing, and cursor encode/decode from `cli/src/lib/api/infrastructure.ts` into `toolkit-core/src/pagination.ts`. Failures are returned as typed results, and the host supplies the logger. Verify with toolkit-core unit tests that cover each `sentry-api-pagination` scenario. -- [ ] 3.2 Switch the CLI to the shared module, mapping errors to `ApiError` and keeping the `sentry-cli/no-manual-pagination` lint rule. Verify the CLI unit tests pass with no snapshot changes. -- [ ] 3.3 Document the module in `packages/toolkit-core/README.md`, and verify `pnpm run docs:check` passes. +- [x] 3.1 Move `paginate`, `fetchAllPages`, page sizing, and cursor encode/decode from `cli/src/lib/api/infrastructure.ts` into `toolkit-core/src/pagination.ts`. Failures are returned as typed results, and the host supplies the logger. Verify with toolkit-core unit tests that cover each `sentry-api-pagination` scenario. +- [x] 3.2 Switch the CLI to the shared module, mapping errors to `ApiError` and keeping the `sentry-cli/no-manual-pagination` lint rule. Verify the CLI unit tests pass with no snapshot changes. +- [x] 3.3 Document the module in `packages/toolkit-core/README.md`, and verify `pnpm run docs:check` passes. Shipped in #1683. ## 4. Shared cache policy and codecs