feat(extension): m2 extension observation v1 and quarantine ingest

Refs #1175

This is milestone M2 of Docs/Proposals/2026-10-28-extension-data-source.md (option B, chosen by JOY on 2026-11-04, with the recommendations). M1 shipped in #1602 and #1605.

Outcome

An opted-in extension can send a minimal ExtensionObservation v1: price, list price, stock, SKU, listing, time and marketplace. It goes to a new quarantine table with a pseudonymous per-install submitter id. Nothing reaches public prices. Verification is M3, in DOS/Rate-Services.

Work (DOS/Rate)

Schema. ExtensionObservation v1 in packages/shared-data: zod schema, types and limits. Following the recommendations, ratings are not part of it.
Extractor. The extension reads stock state, stock quantity and the SKU (variant id) where a store shows them. It reads them only; nothing new is sent in this PR.
Ingest code behind a server flag that stays off until the table exists:
- issue a submitter id;
- accept an observation;
- delete a submitter's data.
The routes use only narrow RPCs; there is no table access from the web. The raw IP is never stored: only a salted, short-lived hash of the network range goes to the RPC.
Migration proposal for DOS-Me: the quarantine table, the submitters table and the RPCs, as a Docs/Proposals PR with sql, verify, rollback and a Docker replay. Nothing is applied. JOY approves the apply through DOS-Me.

Later, not in M2's first PRs

The extension switches its opt-in payload to v1.
The disclosures (opt-in text, consent page, privacy policy, store doc) gain the submitter id and the network hash.
Both happen only after the table exists and the flag is on.

Not in scope

Publishing to users or stores, verification and promotion (M3, M4), and any write to public prices.

Size: L

docs(proposals): extension observations revision 2, ingest key and li…

fix(web): observed_at bounds match postgres, brake retry-after

fix(proposals): extension observations purge lock order and key checks

fix(shared-data,web): host check without a built regex, test key at r…

feat(extension,web): extension observation v1, stock and sku, flagged…

docs(proposals): dos-me extension observation quarantine (m2) (#1608)

feat(extension): opted-in installs send observation v1, with disclosu…

chore(db): follow the ingest key rotation to version 20261109090000

chore(db): mirror the m2 extension observation migrations from dos-me (…

chore(db): mirror the four older drift migrations from dos-me (#1612)

M2 status after the DOS.Me promotes, 2026-11-06. JOY approved M2 on 2026-11-04 ("Duyệt M2"), the Vercel secrets on 2026-11-05 ("Cho đặt lên Vercel"), and the labels for the promotes. Nothing is published to users or a store, and the web flag is off.

Done and merged

What	DOS/Rate	DOS.Me (dev, then main)
v1 validator, extractor (stock, SKU), flagged ingest routes	#1609
Quarantine migration proposal	#1608
Quarantine tables, RPCs and the first key row		#1560, then #1561 (main `8bc7fdae`)
Daily purge, pg_cron 03:30 UTC		#1564, then #1568 (main `11555895`)
Key rotation row (version moved to `20261109090000` after a version collision)		#1569 and #1578, then #1577 (main `2380a859`)
Opted-in installs send v1, with every disclosure	#1610
Mirrors of the four M2 migrations, and of four older drift files	#1611, #1612

Evidence

Production database: the Supabase production apply succeeded on the three main merges above, which carry the tables, the purge job and the key row. The key row holds the sha256 aff995fa98271f0b1bb45f92567da3d425287726ccf759aa44e7cd3234aa5979, from a key that was generated with a CSPRNG and is never in git.
Vercel: RATE_EXTENSION_INGEST_KEY and RATE_NETWORK_HASH_SALT are set on Production and Preview of metados/rate; vercel env ls lists all four. RATE_EXTENSION_OBSERVATIONS is not set.
Not yet deployed: the web code of feat(extension,web): extension observation v1, stock and sku, flagged ingest #1609 and feat(extension): opted-in installs send observation v1, with disclosures #1610 is on dev, not yet on main. On rate.vn, GET /api/v1/rate/extension/submitter and /observation answer 404, and /api/health reports the main commit f598f849 with the database ok.
Tests: the extension, shared-data and write-path tests pass. The DOS/Rate proposal replay and the DOS.Me rate schema replay (apply, verify, rollback, limits, ingest key, purge races) pass in CI.
Reviews: two fresh-context Claude reviews per PR. Codex was not run.

Left before the flag can go on

Verify on production, read-only (through the Rate-Services audit route; no DB access here): V1 to V12 of scripts/ops/rate-extension-observations-verify.sql return 0, and SELECT jobname, schedule FROM cron.job WHERE jobname = 'rate_extension_observations_purge' returns one row. V12 proves the key row is set.
Promote the web code (feat(extension,web): extension observation v1, stock and sku, flagged ingest #1609, feat(extension): opted-in installs send observation v1, with disclosures #1610 and the mirrors) from dev to main, so the routes exist and Vercel builds with the secrets that are already set.
A real install test of the unpacked extension in Chrome (RELEASE.md, steps 4): the consent page opens once with "No" focused, opt-in, a product page, and opt-out erases the contributions.
JOY's decision to set RATE_EXTENSION_OBSERVATIONS=1 on Production. Rollback is removing that variable and redeploying.
After the flag is on: one real submit, and the check that the observation row exists with status = 'quarantined'.

Later milestones, not part of M2

M3: Rate-Services verification in shadow mode, with the thresholds kept private.
M4: promotion of corroborated observations, a legal opinion, Limited Use certification and the store submissions.

Handoff to agent PO. The Web agent's work on this issue (M2 of the extension data source) is done: no PR of mine is open, and the PO takes the claim over with board.mjs --take-over.

What is left, none of it code:

Read-only production check by RS: V1 to V12 of scripts/ops/rate-extension-observations-verify.sql return 0 (V12 proves the key row from DOS.Me feat(web): store review form v2 through submit_community_review_v2 #1577 is set), and cron.job has one row named rate_extension_observations_purge.
Promote chore(release): promote dev to main #1613 (dev to main) puts the web routes into production. They answer 404 on rate.vn until then. The ingest key and salt are already on Vercel (Production and Preview), and take effect with that build.
JOY's decision to set RATE_EXTENSION_OBSERVATIONS=1 on Production. Then one real submit, and a check that the row exists with status = 'quarantined'. Rollback is removing the variable and redeploying.
Not done, by decision: the Chrome install test of the unpacked extension (RELEASE.md, step 4).

Evidence and the list of merged PRs are in the comment above.

Navigation Menu

Repository navigation

feat(extension): m2 extension observation v1 and quarantine ingest #1606

Outcome

Work (DOS/Rate)

Later, not in M2's first PRs

Not in scope

Done and merged

Evidence

Left before the flag can go on

Later milestones, not part of M2

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

feat(extension): m2 extension observation v1 and quarantine ingest #1606

Description

Outcome

Work (DOS/Rate)

Later, not in M2's first PRs

Not in scope

Activity

JOY commented on Nov 6, 2026

Done and merged

Evidence

Left before the flag can go on

Later milestones, not part of M2

JOY commented on Nov 6, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions