Skip to content

Stable Sentry error grouping across deploys - #1781

Merged
RhysSullivan merged 3 commits into
mainfrom
fix/stable-sentry-fingerprints
Aug 28, 2026
Merged

Stable Sentry error grouping across deploys#1781
RhysSullivan merged 3 commits into
mainfrom
fix/stable-sentry-fingerprints

Conversation

@RhysSullivan

@RhysSullivan RhysSullivan commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator

Problem

The same crash kept opening a brand-new issue after every deploy, so a single bug had no count, no history and no regression signal — one problem spread across many issues. Our bundles ship content-hashed chunks and those frames are not resolved back to source, so error grouping falls back to minified chunk names like execution-rate-limit-<hash>, and the hash rotates on every build. On desktop the same effect splits one renderer bug across several issues, and Chromium soft-assert minidumps split by load address.

Fix

A shared @executor-js/sdk/sentry-grouping helper computes an explicit Sentry fingerprint with the volatile hash segment removed, wired into the cloud worker/DO beforeSend, the desktop main init and the desktop renderer init. Each process installs the same withStableGroupingFingerprint helper rather than its own copy, and the options objects handed to Sentry.init/withSentry are assembled in the modules their tests can reach.

The event itself is sent untouched — filenames keep their hashes, so server-side sourcemap resolution is unaffected. Only the grouping key is normalized, and only for events that actually carry a hash; everything else keeps the default grouping algorithm. Desktop additionally collapses DumpWithoutCrashing minidumps onto one fingerprint. Nothing is dropped, downgraded or filtered.

Mechanism choice: the SDKs offer no sourcemap-based grouping switch. The alternative is uploading sourcemaps per release so grouping happens on original frames — worth doing separately, since it also fixes the frames engineers read — but it is a release-pipeline change and does nothing for native minidumps, so beforeSend fingerprints are the minimal reliable mechanism.

Testing

  • e2e: vitest run --project cloud cloud/sentry-otel-correlation.test.ts — black-box against the running worker. Its correlation payload exists only if the worker actually ran the beforeSend from cloudSentryOptions, so this is the proof that the hook is installed and still captures. Three consecutive green runs, ~20s each.

  • Grouping keys are decided inside the Sentry SDK and never reach a product surface, and a locally served worker/renderer is not content-hashed, so the e2e harness has nothing to observe. Coverage is unit-level by exception, split so both halves are checked:

    • Normalizer (packages/core/sdk): the same frame across two chunk hashes fingerprints identically; different modules and different callers stay distinct; hash-free events pass through untouched. The "is this a hash?" threshold is pinned from both sides, so relaxing it — the tempting fix for the word-shaped hashes it deliberately misses — fails instead of silently merging real chunk names like connections-Settings. Two chunks sharing a name are separated by their caller chain, and shrinking that chain to the crashing frame fails.
    • Wiring (apps/cloud/src/observability, apps/desktop/src/main): the exact options object handed to withSentry, instrumentDurableObjectWithSentry and the main-process Sentry.init is asserted to carry the fingerprinting hook, so unhooking the normalizer fails a test rather than passing quietly.
    • Known gap: the desktop renderer's Sentry.init lives in packages/app, which has no test runner. Its beforeSend is a one-symbol reference to the covered helper, but deleting that line would not fail a test.
  • Checked against a real desktop build: 146 of 147 chunk names normalize; the one holdout is a word-shaped hash left alone on purpose.

    • Composition with capture ownership (rebased onto main): the cloud beforeSend now runs two independent stages. The capture-owner pass decides whether an event is reported at all — a cause the session Durable Object already claimed is dropped — and the grouping pass then decides how whatever survives is grouped. A dropped event is never fingerprinted, and a report the DO owns still gets a stable fingerprint across chunk hashes. Both directions are asserted.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 27, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
executor-cloud ccc859f Aug 28 2026, 12:04 AM

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Aug 27, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
executor-marketing ccc859f Commit Preview URL

Branch Preview URL
Aug 28 2026, 12:03 AM

@github-actions

github-actions Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Cloudflare preview

Torn down — the PR is closed.

@pkg-pr-new

pkg-pr-new Bot commented Aug 27, 2026

Copy link
Copy Markdown

Open in StackBlitz

@executor-js/codemode-core

npm i https://pkg.pr.new/@executor-js/codemode-core@1781

@executor-js/runtime-quickjs

npm i https://pkg.pr.new/@executor-js/runtime-quickjs@1781

@executor-js/cli

npm i https://pkg.pr.new/@executor-js/cli@1781

@executor-js/config

npm i https://pkg.pr.new/@executor-js/config@1781

@executor-js/execution

npm i https://pkg.pr.new/@executor-js/execution@1781

@executor-js/sdk

npm i https://pkg.pr.new/@executor-js/sdk@1781

@executor-js/plugin-file-secrets

npm i https://pkg.pr.new/@executor-js/plugin-file-secrets@1781

@executor-js/plugin-graphql

npm i https://pkg.pr.new/@executor-js/plugin-graphql@1781

@executor-js/plugin-keychain

npm i https://pkg.pr.new/@executor-js/plugin-keychain@1781

@executor-js/plugin-mcp

npm i https://pkg.pr.new/@executor-js/plugin-mcp@1781

@executor-js/plugin-onepassword

npm i https://pkg.pr.new/@executor-js/plugin-onepassword@1781

@executor-js/plugin-openapi

npm i https://pkg.pr.new/@executor-js/plugin-openapi@1781

executor

npm i https://pkg.pr.new/executor@1781

commit: ccc859f

@RhysSullivan
RhysSullivan force-pushed the fix/stable-sentry-fingerprints branch from 545ee73 to 7b82895 Compare August 27, 2026 20:25
@RhysSullivan RhysSullivan changed the title Stable Sentry grouping across deploys Stable Sentry error grouping across deploys Aug 27, 2026
Normalize build content hashes out of the grouping key in the cloud, desktop main and desktop renderer Sentry inits, and merge address-keyed Chromium soft-assert minidumps into one fingerprint.
@RhysSullivan
RhysSullivan force-pushed the fix/stable-sentry-fingerprints branch from 5038374 to ccc859f Compare August 28, 2026 00:00
@RhysSullivan
RhysSullivan marked this pull request as ready for review August 28, 2026 01:28
@RhysSullivan
RhysSullivan merged commit 36c901b into main Aug 28, 2026
44 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant