Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Added

- Add `auto-instrumentation` knowledge to the `observability` domain: which Sentry spans the browser SDK creates without a `trace()` call, and why an `http.client` count is a floor on requests. `sentry-quota` now reviews requests a diff adds or re-triggers, and `sentry-mcp-queries` checks the active-span condition before using an endpoint as a volume anchor.
- Add `navigation` skill with a repo-agnostic base and a MetaMask Mobile overlay for `Routes` and `NavigationService`. Marked `base: true` so it installs even when its domain is filtered out.
- Add `feature-flags` skill with a repo-agnostic base and a MetaMask Mobile overlay for version-gated remote flags. Marked `base: true` so it installs even when its domain is filtered out. ([#147](https://github.com/MetaMask/skills/pull/147))
- Add `analytics` skill (`platform/analytics`, moved from `coding`) with a repo-agnostic base and a MetaMask Mobile overlay for the canonical tracking API. Marked `base: true` so it installs even when its domain is filtered out. ([#140](https://github.com/MetaMask/skills/pull/140))
Expand Down
62 changes: 62 additions & 0 deletions domains/observability/knowledge/auto-instrumentation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
---
name: auto-instrumentation
domain: observability
description: Which Sentry spans the browser SDK creates with no trace() call, when it creates them, and how a diff with no trace() site still moves span volume
---

# Auto-Instrumentation Coverage

`browserTracingIntegration()` creates most of the spans a browser client sends, and none of them has a `trace(` site. A review that counts only `trace(` sites and `TraceName` entries cannot see a change that makes these spans more frequent.

## What the SDK creates

| Span op | Option (default) | Created when |
|---|---|---|
| `pageload`, `navigation` | `instrumentPageLoad`, `instrumentNavigation` (on) | each page load and each route change. Both are idle spans, active until activity stops |
| `http.client` | `traceFetch`, `traceXHR` (on) | each `fetch` or XHR that starts **while a span is active** and passes `shouldCreateSpanForRequest` |
| `browser.*`, `resource.*`, `paint`, `mark`, `measure` | none; added from Performance API entries when a `pageload` or `navigation` span ends | one span per entry, so per page load or route change |
| `ui.long-animation-frame`, falling back to `ui.long-task` | `enableLongAnimationFrame`, `enableLongTask` (on) | each long frame or task under an active span |
| `ui.interaction.*` | `enableInp` (on) | slow interactions |

Sources, `@sentry/*` [10.38.0](https://github.com/getsentry/sentry-javascript/tree/ed7956a01f3f6962d3e76ebf91dc3683027e71d8):

- defaults: [browserTracingIntegration.ts L316-L326](https://github.com/getsentry/sentry-javascript/blob/ed7956a01f3f6962d3e76ebf91dc3683027e71d8/packages/browser/src/tracing/browserTracingIntegration.ts#L316-L326)
- entries added at span end: [L429-L438](https://github.com/getsentry/sentry-javascript/blob/ed7956a01f3f6962d3e76ebf91dc3683027e71d8/packages/browser/src/tracing/browserTracingIntegration.ts#L429-L438)
- request defaults: [request.ts L126-L127](https://github.com/getsentry/sentry-javascript/blob/ed7956a01f3f6962d3e76ebf91dc3683027e71d8/packages/browser/src/tracing/request.ts#L126-L127)

## The parent condition

The SDK creates an `http.client` span only if a span is active when the request starts ([fetch.ts L106-L109](https://github.com/getsentry/sentry-javascript/blob/ed7956a01f3f6962d3e76ebf91dc3683027e71d8/packages/core/src/fetch.ts#L106-L109), [request.ts L361-L364](https://github.com/getsentry/sentry-javascript/blob/ed7956a01f3f6962d3e76ebf91dc3683027e71d8/packages/browser/src/tracing/request.ts#L361-L364) for XHR). With no active span, the request carries a non-recording span and nothing is sent. A recorded span is kept or dropped with the trace it joins.

Two consequences follow.

- **An `http.client` count is a floor on requests.** Requests that start with no active span leave no span. The count matches the request count only for a call site that always runs inside an active span.
- **A `trace()` callback that awaits requests fans out.** The callback form keeps its span active until the callback settles, so every request inside becomes a child. Spans per trigger is one plus the requests made, and the children are kept at the rate the transaction was drawn at.

## A new timing trace may already be recorded

Before reviewing a new custom span for volume, ask whether an automatic span already records what it times. A time-to-content or load-duration trace usually waits on a request, and that request is often already an `http.client` span with a duration.

1. List the requests the traced interval waits on, in the realm where each one runs. A UI surface can wait on a request the background makes.
2. Query `span.op:http.client` with `span.description` matching each endpoint, grouped by `transaction`, with `p50(span.duration)`. Rows mean the request is already recorded, and the `transaction` column says which spans it is recorded under.
3. For a background request, check whether any row sits under the RPC or controller path the new trace depends on. A background request gets a span only when a span is active there, so a UI trigger can be recorded under a sampled `Background RPC: <method>` wrapper and nowhere else.
4. Put the difference to the author: what the new span measures that those spans do not, such as time in the UI after the response, Redux propagation, or render. A new span that only restates a request duration duplicates volume without adding signal.

Extension instance, 30 days to 2026-09-17: the carousel's Contentful `entries` request (`span.description:*promotionalBanner*`) was recorded under every UI page transaction, with a median of 58 to 78 ms by page. The notification list request (`POST .../api/v4/notifications`) was recorded under background transactions, among them `Background RPC: fetchAndUpdateMetamaskNotifications`. Both were already recorded before a PR added time-to-content spans that wait on them.

## What moves span volume with no `trace(` site

A diff can add spans when it:

- adds a request, or makes one fire more often: a new effect dependency, a refetch on a new trigger, a shorter poll;
- moves a request into a `trace()` callback, or wraps a request-making call in one;
- adds routes, page loads, or images, iframes and scripts to a page, since each Performance API entry becomes a span on sampled page loads;
- changes `browserTracingIntegration()` options or the `shouldCreateSpanForRequest` filter.

Estimate each as requests or entries per trigger × triggers × the kept rate of the trace the span joins. Under a UI `pageload` or `navigation` span that is the default `tracesSampleRate`. Under a context continued as sampled it is close to 1 (`span-sub-sampling`).

## Measured share, extension

In project `metamask` (273505), over the 30 days to 2026-09-17, `http.client` was about nine in ten billed-equivalent child spans. Billed-equivalent means `count()` × client rate, or `count()` where no client rate is recorded. Most of those `http.client` spans had no client rate and sat under background transactions such as `Simulate` and `Smart Transactions: Fetch Liveness`, which make requests inside their callbacks.

Queries: [child spans by op and client rate](https://metamask.sentry.io/explore/traces/?query=environment%3Aproduction+project.id%3A273505+is_transaction%3Afalse&aggregateField=%7B%22groupBy%22%3A%22span.op%22%7D&aggregateField=%7B%22groupBy%22%3A%22client_sample_rate%22%7D&aggregateField=%7B%22yAxes%22%3A%5B%22count%28%29%22%2C%22count_sample%28%29%22%5D%7D&mode=aggregate&sort=-count_sample%28%29&statsPeriod=30d&table=span), [`http.client` with no client rate, by transaction](https://metamask.sentry.io/explore/traces/?query=environment%3Aproduction+project.id%3A273505+span.op%3Ahttp.client+%21has%3Aclient_sample_rate&aggregateField=%7B%22groupBy%22%3A%22transaction%22%7D&aggregateField=%7B%22yAxes%22%3A%5B%22count%28%29%22%2C%22count_sample%28%29%22%5D%7D&mode=aggregate&sort=-count%28%29&statsPeriod=30d&table=span). The window rolls, so re-run them before citing the share.
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,30 @@ The serialized context carries no sampled flag, so the background continues ever

The UI and background timestamps do not share a clock. In 17 of 96 measured traces they disagreed by up to 67 minutes, so a duration computed across the boundary is not reliable.

## Span Tags, Data and Measurements

`trace()` routes `tags` by the type of each value, and the field name does not show the split. Non-numeric values reach `scope.setTag` in [`initScope`](https://github.com/MetaMask/metamask-extension/blob/e24e5a017af7a84eeaac7f4f6053eb83cd237f5c/shared/lib/trace.ts#L618-L626). Numeric values never become tags: [`initSpan`](https://github.com/MetaMask/metamask-extension/blob/e24e5a017af7a84eeaac7f4f6053eb83cd237f5c/shared/lib/trace.ts#L635-L643) passes them to `sentrySetMeasurement(key, value, 'none')` so they can be queried numerically, which also means a numeric tag will not match a tag filter.

`trace()` takes `data` as well, and it becomes the span's `attributes` at creation in [`startSpan`](https://github.com/MetaMask/metamask-extension/blob/e24e5a017af7a84eeaac7f4f6053eb83cd237f5c/shared/lib/trace.ts#L536-L560).

`endTrace()` takes its own `data` and applies it with `span.setAttribute` immediately before ending the span, at [trace.ts L303-L308](https://github.com/MetaMask/metamask-extension/blob/e24e5a017af7a84eeaac7f4f6053eb83cd237f5c/shared/lib/trace.ts#L303-L308).

`scope.setTag` is called in one place in the file and `initScope` runs only from `startSpan`, so no path adds a tag after start. A value that must be a tag has to be known when `trace()` is called.

## `endTrace()` No-Ops

The pending trace is keyed `<name>:<id>`, with `id` defaulting to `'default'` ([trace.ts L125](https://github.com/MetaMask/metamask-extension/blob/e24e5a017af7a84eeaac7f4f6053eb83cd237f5c/shared/lib/trace.ts#L125), [L596-L605](https://github.com/MetaMask/metamask-extension/blob/e24e5a017af7a84eeaac7f4f6053eb83cd237f5c/shared/lib/trace.ts#L596-L605)). [`endTrace`](https://github.com/MetaMask/metamask-extension/blob/e24e5a017af7a84eeaac7f4f6053eb83cd237f5c/shared/lib/trace.ts#L292-L301) looks that key up in `tracesByKey`, and on a miss it logs `No pending trace found` and returns without ending anything. A mismatched `id` between `trace()` and `endTrace()` produces no span and no error, so nothing reaches Sentry and the case is indistinguishable there from a span that was never started.

[`startTrace`](https://github.com/MetaMask/metamask-extension/blob/e24e5a017af7a84eeaac7f4f6053eb83cd237f5c/shared/lib/trace.ts#L442-L468) writes `tracesByKey` unconditionally, so a second `trace()` under the same name and `id` overwrites the first entry. The first span becomes unreachable by `endTrace` and never ends. Concurrent traces sharing a name need distinct `id` values.

The `data` block in `endTrace` is additionally guarded on `pendingTrace.span`, which is `null` when `globalThis.sentry` is absent ([trace.ts L693-L704](https://github.com/MetaMask/metamask-extension/blob/e24e5a017af7a84eeaac7f4f6053eb83cd237f5c/shared/lib/trace.ts#L693-L704)). That is the Sentry-not-loaded case, not the sampled-out case.

## Debounced Background State Update

`sendUpdate` is a debounced wrapper, `MILLISECOND * 200` with `{ maxWait: SECOND }`, assigned in the controller constructor at [metamask-controller.js L465-L469](https://github.com/MetaMask/metamask-extension/blob/e24e5a017af7a84eeaac7f4f6053eb83cd237f5c/app/scripts/metamask-controller.js#L465-L469). The underlying [`privateSendUpdate`](https://github.com/MetaMask/metamask-extension/blob/e24e5a017af7a84eeaac7f4f6053eb83cd237f5c/app/scripts/metamask-controller.js#L6218-L6220) is what emits `update` with `this.getState()`.

A UI loading flag flips when the RPC response arrives, but the data reaches the Redux store only on that debounced update. A span ending on a loading flag therefore excludes 200 to 1000 ms plus render, and measures time to response under a time to content name.

## Sentry Sample Rate

```bash
Expand Down
28 changes: 25 additions & 3 deletions domains/observability/skills/instrumentation/skill.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,17 +27,34 @@ description: Create and update Sentry spans, MetaMetrics events, and Segment eve

### Creating a Span

1. **Register a named trace entry** in the repo's trace name enum before writing any span code. Unnamed spans are invisible in Sentry filters.
2. **Use the repo's `trace()` wrapper**, not raw `Sentry.startSpan()`. Wrappers handle cross-process context propagation, active-span inheritance, and consistent tag injection.
3. **Inherit parent automatically** — when no `parentContext` is provided, the wrapper inherits from `Sentry.getActiveSpan()`, making the new span a child of the active parent (e.g., a `pageload` span). That parent is whichever span is active at the call, not necessarily the one that caused the work, which metamask-extension#45527 (stop spans silently attaching to whatever trace happens to be active) proposes to fix. A span started by `trace()` without a callback is active only inside that call, so spans created before its `endTrace()` do not nest under it.
1. **Check what already records this interval.** Most spans a browser client sends have no `trace()` site: `pageload`, `navigation`, `http.client` and the Performance API entries all come from `browserTracingIntegration()`. A timing span that waits on a request is usually waiting on one that already carries an `http.client` span with a duration. Query `span.op:http.client` with `span.description` matching each endpoint the interval waits on, grouped by `transaction`. Rows mean the timing exists; the new span then has to measure something those do not, and the PR should say what. See `auto-instrumentation`.
2. **Register a named trace entry** in the repo's trace name enum before writing any span code. Unnamed spans are invisible in Sentry filters.
3. **Use the repo's `trace()` wrapper**, not raw `Sentry.startSpan()`. Wrappers handle cross-process context propagation, active-span inheritance, and consistent tag injection.
4. **Inherit parent automatically** — when no `parentContext` is provided, the wrapper inherits from `Sentry.getActiveSpan()`, making the new span a child of the active parent (e.g., a `pageload` span). That parent is whichever span is active at the call, not necessarily the one that caused the work, which metamask-extension#45527 (stop spans silently attaching to whatever trace happens to be active) proposes to fix. A span started by `trace()` without a callback is active only inside that call, so spans created before its `endTrace()` do not nest under it.

### Updating a Span

- `trace()` takes `tags`, `endTrace()` takes `data`. There is no post-start tag path, so a value that must be a tag has to be known when `trace()` is called
- A `tags` entry is routed by the type of its value. A non-numeric value becomes a tag; a numeric one is skipped and set as a Sentry measurement instead, so it never becomes filterable as a tag. Put a number in `tags` only when a measurement is what you want
- Adding a tag: no governance required
- Renaming a trace name enum entry: grep all callsites; update enum and references atomically
- Changing an `op` value: breaks saved queries and dashboards — coordinate with whoever owns them
- Moving a span's start (`trace()`) or end (`endTrace()`): changes what its duration measures, so a release-over-release delta mixes a performance change with a definition change

### Timing To Content

**End the span on the state the UI renders from, not on the request settling.** A loading flag flips when the response arrives; the data reaches the component later. In metamask-extension the background's `sendUpdate` is debounced 200 ms with a 1 s `maxWait`, so a span ending on a loading flag excludes 200 to 1000 ms plus render. That excluded window is the only part an `http.client` span does not already cover, so the span measures time-to-response under a time-to-content name.

**Falsifier, before the span ships:** on a cold load, assert the content is in the store at the moment `endTrace` runs. If it is not, the end condition is wrong. A test that only asserts the span ended cannot see this.

### Cross-Platform Parity

**Parity is a property of the definition, not of the name.** Two platforms sharing a trace name and an `op` produce one queryable series, so a dashboard puts them side by side whatever the code does. They are comparable only if the start point, the end condition and every tag derivation match.

- Read the other platform's implementation before choosing the name, not after.
- A tag derived differently means one value selects different populations. Extension and mobile both emit `source: cold|warm` under `notification.performance`: mobile takes `cold` from the render immediately before the span ends, the extension from any loading render since the span started, and the extension span also waits on `isPending`, so it ends at least a render later.
- Where the thing timed differs, no naming makes the numbers comparable. Mobile's banner trace times a Braze banner with SDK targeting in its path; the extension carousel times a Contentful fetch.

---

## MetaMetrics / Segment Events
Expand Down Expand Up @@ -85,3 +102,8 @@ Caveats: sample population is MetaMetrics opted-in users only. The extension's S
| New span with no trace name enum entry | Register enum entry first; unnamed spans are invisible in Sentry filters |
| Multiply a span `count()` by `1 / tracesSampleRate` | `count()` is already extrapolated, so read it as the estimate |
| Treat Sentry estimates as exact counts | Probabilistic sample — state sample size and confidence |
| New timing span for a request that already has an `http.client` span | Query the endpoint first, and say what the new span measures that the automatic one does not |
| Span ends when the fetch settles, under a "time to content" name | End on the state the UI renders from; a debounced store update sits between the response and the render |
| Same trace name as another platform, different end condition or tag derivation | Parity is the definition. Match start, end and every tag, or use a different name |
| A value that must be a tag, passed to `endTrace` | `tags` are start-only via `scope.setTag`; `endTrace` data becomes attributes |
| Effect that owns a span listing dependencies it never reads | React Compiler's effect-dependency validation errors and skips the whole function, so the hook ships unmemoized behind a green build |
Loading
Loading