Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Add `navigation` skill with a repo-agnostic base and a MetaMask Mobile overlay for `Routes` and `NavigationService`. Marked `base: true` so it installs even when its domain is filtered out.
- Add `feature-flags` skill with a repo-agnostic base and a MetaMask Mobile overlay for version-gated remote flags. Marked `base: true` so it installs even when its domain is filtered out. ([#147](https://github.com/MetaMask/skills/pull/147))
- Add `analytics` skill (`platform/analytics`, moved from `coding`) with a repo-agnostic base and a MetaMask Mobile overlay for the canonical tracking API. Marked `base: true` so it installs even when its domain is filtered out. ([#140](https://github.com/MetaMask/skills/pull/140))
- Add a MetaMask Extension overlay to `analytics` for MetaMetrics and Segment events, and the `metametrics-identity` and `segment-governance` knowledge files to the `platform` domain, which install beside each `platform` skill an install selects: `analytics` in MetaMask Extension, and `analytics`, `feature-flags` and `navigation` in MetaMask Mobile. ([#143](https://github.com/MetaMask/skills/pull/143))

### Changed

- Move MetaMetrics and Segment event guidance from `observability/instrumentation` to `platform/analytics`. `instrumentation` now covers Sentry spans and volume estimation from Sentry span data. ([#143](https://github.com/MetaMask/skills/pull/143))

## [0.3.1]

Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -391,7 +391,7 @@ homepage) are preserved through install — only `name`, `description`,
`--exclude` / `SKILLS_EXCLUDE` still wins. The maturity filter runs before the
base bypass, so `--maturity stable` drops a `base: true` experimental skill.
A skill with a `repos/` directory and no overlay for `--repo` is skipped
(this `analytics` skill installs for Mobile and is skipped for Extension).
(the `navigation` skill installs for Mobile and is skipped for Extension).

The 1,536-character ceiling is a repo budget rather than an operator limit — the
description is always-on context for every installed skill, so it is capped
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,9 @@ parent: instrumentation
|---------|------|
| Sentry trace wrapper | `shared/lib/trace.ts` |
| Trace name enum | `shared/lib/trace.ts` → `TraceName` |
| MetaMetrics controller | `app/scripts/controllers/metametrics-controller.ts` |
| Anonymous-event marking (`excludeMetaMetricsId`) | `app/scripts/controllers/analytics/analytics.ts` → `applyAnonymousEventOptions()` |
| Event enum | `shared/constants/metametrics.ts` → `MetaMetricsEventName` |
| Traces buffered until MetaMetrics opt-in | `app/scripts/controllers/metametrics-controller.ts` → `bufferedTrace()` |
| Sentry setup + sample rate | `app/scripts/lib/setupSentry.js` → `getTracesSampleRate()` |
| Sentry `user.id` (set to the MetaMetrics `analyticsId`) | `app/scripts/lib/sentry-metametrics.ts` → `metaMetricsIntegration()` |
| Segment tracking plan | `Consensys/segment-schema` → `tracking-plans/metamask-extension.yaml`, which lists event libraries. Event definitions live in `libraries/events/<library>/` |

## Cross-Process Context (UI → Background)

Expand Down Expand Up @@ -53,16 +50,3 @@ Query: span.op:http.client span.description:*{endpoint}*
Group by: span.description, transaction
Sort: -count(span.duration)
```

## Detect `excludeMetaMetricsId` Misuse

```bash
grep -rn "excludeMetaMetricsId: true" app/ ui/ shared/ --include="*.ts" --include="*.tsx" --include="*.js"
# Each hit sends its event under the shared anonymous id. Confirm the event must not carry identity.
# Event names matching /^send|^confirm/iu are anonymous by default: check new event names too.
```

## Data Council Contact

- Slack: `#metamask-metametrics`
- Team: `@consensys/data-council`
29 changes: 4 additions & 25 deletions domains/observability/skills/instrumentation/skill.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,22 @@
---
maturity: experimental
name: instrumentation
description: Create and update Sentry spans, MetaMetrics events, and Segment events — methodology, policies, common pitfalls
description: Create and update Sentry performance spans, and estimate span or event volume from Sentry span data — methodology, policies, common pitfalls
---

# Analytics Instrumentation
# Sentry Span Instrumentation

## When To Use

- Adding or modifying a MetaMetrics (Segment) event
- Adding or modifying a Sentry performance span
- Estimating event or span volume from production data
- Auditing existing instrumentation for correctness
- Auditing existing span instrumentation for correctness

---

## Do Not Use When

- Adding or modifying a MetaMetrics / Segment event (use the `analytics` skill, `platform/analytics`)
- Adding local debug logging with no telemetry destination
- Investigating an existing Sentry error report (use `sentry-mcp-queries`)
- Internal feature flag evaluation not surfaced as an analytics event
Expand All @@ -40,25 +40,6 @@ description: Create and update Sentry spans, MetaMetrics events, and Segment eve

---

## MetaMetrics / Segment Events

### Creating an Event

1. **Check the event name enum** — event may already exist under a different phrasing.
2. **Check the segment tracking plan** — event may be registered under a different name than the enum key.
3. **Add to the enum**, then implement the `trackEvent` call.
4. **Pass `excludeMetaMetricsId: true` only for an event that must not carry the user's identity.** It sends the event under the shared anonymous id and drops the profile ids, for every user, not only those who have not opted in. Event names matching `/^send|^confirm/iu` get it by default unless the caller passes `excludeMetaMetricsId: false` (see data domain `knowledge/metrametrics-identity.md`).
5. **Open a data governance review** before merging. There is usually no CI enforcement on schema registration — this step is easy to skip (see data domain `knowledge/segment-governance.md`).
6. **Register in the team's segment tracking plan** before shipping.

### Updating an Event

- Adding a property: requires governance review and schema update
- Renaming an event: deprecate old + add new in tracking plan; coordinate on migration window
- Removing an event: confirm no active dashboards depend on it before removing

---

## Volume Estimation via Sentry

When direct Segment access is unavailable, estimate from Sentry production span data:
Expand All @@ -79,8 +60,6 @@ Caveats: sample population is MetaMetrics opted-in users only. The extension's S

| Mistake | Correct Approach |
|---------|-----------------|
| `excludeMetaMetricsId: true` on an event that needs user identity | It sends the event under the shared anonymous id for every user. Reserve it for events that must be anonymous |
| Ship event without tracking-plan registration | No CI gate — add governance review explicitly to PR checklist |
| Raw `Sentry.startSpan()` instead of the repo's `trace()` wrapper | Use the wrapper — handles cross-process context and active-span inheritance |
| New span with no trace name enum entry | Register enum entry first; unnamed spans are invisible in Sentry filters |
| Multiply a span `count()` by `1 / tracesSampleRate` | `count()` is already extrapolated, so read it as the estimate |
Expand Down
6 changes: 3 additions & 3 deletions domains/observability/skills/sentry-quota/skill.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Find and fix custom Sentry span instrumentation that blows the project span budg
## Do Not Use When

- Reading the live span counts themselves — that's `sentry-mcp-queries` (Volume Estimation).
- Product-analytics events (Segment / `trackEvent`) — that's `instrumentation`, with data domain `knowledge/segment-governance.md` for Segment governance.
- Product-analytics events (Segment / `trackEvent`) — that's the `analytics` skill (`platform/analytics`), with platform domain `knowledge/segment-governance.md` for Segment governance.
- The span is already behind a per-trace sample gate **and** a kill-switch — already mitigated.
- Error volume. Errors are metered separately from spans and transactions, so no change here moves the error quota.

Expand Down Expand Up @@ -67,7 +67,7 @@ Pick the lowest tier that stops the bleed.
| **0 — Immediate** | a span fans out and is actively breaching on the live release | disable the `trace()` call at source (or env-guard it) + **cherry-pick to the release branch** + file a sev-1 release blocker on the in-flight release milestone |
| **1 — Release containment** | spike concentrated in an old, already-patched release with lingering users. A sampler fix in a newer build does not change that release's rates unless it reads its rate remotely | Sentry **inbound filter** dropping `release:<bad>` spans + force-update. The only dashboard action. Filters target a whole release, not one span — don't filter a release you still want data from. There is no inbound filter by transaction name, only a fixed health-check one. Filtered events do not consume quota, so confirm the drop in the filtered outcomes (`stats_v2` grouped by `reason`), not in Explore. It is not instant: one recorded release filter took 3.8 days from filing to taking effect |
| **2 — Durable** | the span is justified long-term but ungated | deterministic `traceId`-hash sub-sample gate before the span (`span-sub-sampling`) |
| **3 — Wrong tool** | the metric needs full fidelity; sampling loses the signal | move the metric off trace spans — they are the wrong substrate for always-on high-cardinality metrics. Segment is the usual target, but its events can ship unregistered, with no CI check and no billing review (data domain `knowledge/segment-governance.md`), so it is not a free lunch |
| **3 — Wrong tool** | the metric needs full fidelity; sampling loses the signal | move the metric off trace spans — they are the wrong substrate for always-on high-cardinality metrics. Segment is the usual target, but its events can ship unregistered, with no CI check and no billing review (platform domain `knowledge/segment-governance.md`), so it is not a free lunch |

Tier 0 + 1 stop the bleed; Tier 2 is the follow-up so the metric returns.

Expand All @@ -82,7 +82,7 @@ Tier 0 + 1 stop the bleed; Tier 2 is the follow-up so the metric returns.
| Inbound-filter a release you still need data from | Filters drop the whole release — fix in code (Tier 0/2) instead |
| "No grep hits, so it's safe" | The culprit may be on a release ref not checked out — verify the version/ref |
| Disable the span on `main` only | Cherry-pick to the active release branch — `main` alone leaves the live release breaching |
| Treat "move to Segment" as free | Segment events ship without CI governance or billing review (data domain `knowledge/segment-governance.md`) |
| Treat "move to Segment" as free | Segment events ship without CI governance or billing review (platform domain `knowledge/segment-governance.md`) |
| Ship new always-on instrumentation with no kill-switch | Add an env disable flag on day one — turns a future cut into a config flip, not a cherry-pick |
| An optional `trace?` param passes review because it emits nothing | It is a dormant fan-out — it detonates when any caller supplies the argument. Remove the *param*, not just the argument, so one line can't re-arm it. |
| Disable one entry point of a multi-path change | One change can reach the backend by more than one path (a controller callback *and* a selector param). Audit every entry point it added, not just the one that fired. |
Expand Down
40 changes: 40 additions & 0 deletions domains/platform/knowledge/metametrics-identity.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
---
name: metametrics-identity
domain: platform
description: Which extension MetaMetrics events are sent without the user's analytics ID, set by the excludeMetaMetricsId option or by an event name starting with Send or Confirm
---

# MetaMetrics Identity on Anonymous Events

Read at `metamask-extension` [`c31416a`](https://github.com/MetaMask/metamask-extension/commit/c31416a47811bc4355a904925021a30f4c5564bb).

## The Mechanism

Each event is sent under the user's analytics ID or under one shared anonymous ID, decided per event, except that an event with sensitive properties is sent twice (see Sensitive Properties):

1. The background `trackEvent` reads `excludeMetaMetricsId` from the event's build options. When it is true, [`applyAnonymousEventOptions`](https://github.com/MetaMask/metamask-extension/blob/c31416a47811bc4355a904925021a30f4c5564bb/app/scripts/controllers/analytics/analytics.ts#L303-L333) sets an `anonymous: true` marker on the event's properties.
2. The platform adapter reads the marker. A marked event is sent with `anonymousId` set to `METAMETRICS_ANONYMOUS_ID` ([`0x0000000000000000`](https://github.com/MetaMask/metamask-extension/blob/c31416a47811bc4355a904925021a30f4c5564bb/shared/constants/metametrics.ts#L777)), and an unmarked one with `userId` set to the analytics ID ([`platform-adapter.ts:345-355`](https://github.com/MetaMask/metamask-extension/blob/c31416a47811bc4355a904925021a30f4c5564bb/app/scripts/controllers/analytics/platform-adapter.ts#L345-L355)). The analytics ID is a hex string rather than a UUIDv4, so the adapter sets `skipUUIDv4Check: true` ([`platform-adapter.ts:308-321`](https://github.com/MetaMask/metamask-extension/blob/c31416a47811bc4355a904925021a30f4c5564bb/app/scripts/controllers/analytics/platform-adapter.ts#L308-L321)). Sentry attaches the same value to its events as `user.id` ([`sentry-metametrics.ts:47-51`](https://github.com/MetaMask/metamask-extension/blob/c31416a47811bc4355a904925021a30f4c5564bb/app/scripts/lib/sentry-metametrics.ts#L47-L51)), so an identified Segment row and a Sentry event from one install carry one ID.
3. [`enrichEventProperties`](https://github.com/MetaMask/metamask-extension/blob/c31416a47811bc4355a904925021a30f4c5564bb/app/scripts/controllers/analytics/platform-adapter.ts#L137-L172) adds the profile identity properties to unmarked events only, and deletes `profile_id`, `canonical_profile_id` and the marker from marked ones.
4. Some events are renamed on the anonymous path through [`anonymousEventNameOverrides`](https://github.com/MetaMask/metamask-extension/blob/c31416a47811bc4355a904925021a30f4c5564bb/app/scripts/controllers/analytics/platform-adapter.ts#L246), for example the transaction lifecycle events and `SignatureRequested`.

No event is sent while basic functionality is off ([`analytics.ts:351`](https://github.com/MetaMask/metamask-extension/blob/c31416a47811bc4355a904925021a30f4c5564bb/app/scripts/controllers/analytics/analytics.ts#L351)).

## The Name Default

`excludeMetaMetricsId` is not only the caller's choice. An event whose name matches `/^send|^confirm/iu` is marked anonymous unless the caller passes `excludeMetaMetricsId: false` ([`analytics.ts:321-324`](https://github.com/MetaMask/metamask-extension/blob/c31416a47811bc4355a904925021a30f4c5564bb/app/scripts/controllers/analytics/analytics.ts#L321-L324)). The comment above it calls the match a carry-over from the previous implementation.

So a new event whose name begins with `Send` or `Confirm` loses user-level attribution by default. Every such event shares one ID in Segment, and user-level dimensions such as account type or feature flags cannot be joined to it.

## Sensitive Properties

An event carrying `sensitiveProperties` cannot also set `excludeMetaMetricsId: true`. `applyAnonymousEventOptions` throws ([`analytics.ts:307-315`](https://github.com/MetaMask/metamask-extension/blob/c31416a47811bc4355a904925021a30f4c5564bb/app/scripts/controllers/analytics/analytics.ts#L307-L315)), and `trackEvent` catches the error and reports it to Sentry ([`analytics.ts:391-392`](https://github.com/MetaMask/metamask-extension/blob/c31416a47811bc4355a904925021a30f4c5564bb/app/scripts/controllers/analytics/analytics.ts#L391-L392)), so the event is not sent.

Otherwise an event with sensitive properties is sent twice: once without them, under whichever ID the event would otherwise get (the anonymous ID when the name default has marked it), and once with them and the anonymous marker, under the anonymous name where `anonymousEventNameOverrides` has one ([`AnalyticsController.ts:1452-1483`](https://github.com/MetaMask/core/blob/d0487f6f68af1bb9be22f5aa4eb72e036b0f4452/packages/analytics-controller/src/AnalyticsController.ts#L1452-L1483) in `@metamask/analytics-controller` 2.1.0, with the feature enabled at [`analytics-controller-init.ts:59`](https://github.com/MetaMask/metamask-extension/blob/c31416a47811bc4355a904925021a30f4c5564bb/app/scripts/messenger-client-init/analytics-controller-init.ts#L59)). A count over both rows counts the event twice.

## Detection

```bash
git grep -n 'excludeMetaMetricsId: true' -- app shared ui ':!*.test.*'
```

At `c31416a` this returns the phishing-detection, `eth_requestAccounts` and MetaMetrics data-deletion call sites, among others. A new hit, or a new event name beginning with `Send` or `Confirm`, drops the analytics ID from that event. Check that this is intended.
40 changes: 40 additions & 0 deletions domains/platform/knowledge/segment-governance.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
---
name: segment-governance
domain: platform
description: Segment event governance via segment-schema is advisory — no CI enforcement prevents unregistered events from shipping
---

# Segment Event Governance

## Architecture

| Component | Location |
|-----------|----------|
| Tracking plan | `Consensys/segment-schema` → `tracking-plans/metamask-extension.yaml` |
| Event registry | `shared/constants/metametrics.ts` → `MetaMetricsEventName` enum (300+ entries) |
| Review process | `CONTRIBUTING.md` in segment-schema; Data Council review |
| Governance contact | `#metamask-metametrics` in Slack, and a Data Council delegate, who is a required reviewer on each `Consensys/segment-schema` pull request |

## The Gap

There is **no CI enforcement** in the extension repo. A developer can:

1. Add entry to `MetaMetricsEventName` enum
2. Call `trackEvent` with it
3. Merge and ship to production

...without registering in segment-schema or going through Data Council review.

## Implications

- Schema drift between tracking plan and production events
- No property schema validation for unregistered events
- Billing impact goes unreviewed
- Data Council review is bypassable by omission

## Recommended Fix

CI check that:
1. Parses `MetaMetricsEventName` entries
2. Validates each against `tracking-plans/metamask-extension.yaml`
3. Fails build if event is missing from the plan
Loading