Skip to content

chore(release): promote dev to main - #1617

Merged
JOY (JOY) merged 2 commits into
mainfrom
dev
Nov 6, 2026
Merged

JOY (JOY) merged 2 commits into
mainfrom
dev

Conversation

@JOY

@JOY JOY (JOY) commented Nov 6, 2026 •

Copy link
Copy Markdown
Contributor

Promote dev to main.

Merge with a merge commit, never squash (Docs/Resources/RELEASE.md).


📌 TL;DR

This PR introduces a per-review author_key mechanism to allow the review analyzer to distinguish between authors after Rate-Services masks author_name for privacy. It adds a new rate.review_author_keys table with strict RLS, updates the ingestion pipeline to store and load these keys, and modifies the duplicate-detection logic to use the key as the primary author identifier.

🎯 Type of Change

  • 🚀 New feature
  • 🐛 Bugfix
  • 🧹 Refactor
  • ⚡ Performance
  • 📚 Documentation
  • ⚙️ CI / Configuration

🔍 Changes Walkthrough

File Summary of Changes
.gitignore Un-ignores specific SQL proposal files to allow them to be tracked in the repository.
Docs/Proposals/2026-11-06-dos-me-review-author-keys.md Documents the proposal, ingest contract, storage strategy, and analyzer logic for the new author key feature.
Docs/Proposals/2026-11-06-dos-me-review-author-keys.sql Proposal DDL for creating the rate.review_author_keys table with RLS policies.
Docs/Proposals/2026-11-06-dos-me-review-author-keys-verify.sql Read-only verification queries to ensure the table exists, RLS is enabled, and data integrity is maintained.
Docs/Proposals/2026-11-06-dos-me-review-author-keys-rollback.sql Rollback script to drop the rate.review_author_keys table.
supabase/migrations/20261109130000_rate_review_author_keys.sql The actual migration file applying the DDL from the proposal to the database.
packages/shared-data/src/database.types.ts Adds TypeScript type definitions for the new review_author_keys table.
packages/shared-data/src/schemas/specs.schema.ts Adds authorKey to ScrapedReviewSchema with validation (16-128 chars, alphanumeric/underscore/hyphen) and a .catch(undefined) to drop invalid keys without rejecting the listing.
packages/shared-data/src/__tests__/shared-data.test.ts Adds tests to verify that valid authorKey values are kept and invalid ones are dropped during schema parsing.
apps/importer/src/shopee/types.ts Adds optional authorKey field to the ShopeeReview interface.
apps/importer/src/shopee/ai-review-analyzer.ts Adds author_key to ReviewInputItem. Introduces reviewAuthorIdentity() helper to generate a unique identity string (key:, name:, or id: prefix) for duplicate detection. Updates the analyzer to use this new identity function.
apps/importer/src/shopee/shopee-to-supabase.ts Adds storeAuthorKeys() to upsert keys into the DB after review ingestion. Adds loadAuthorKeys() to fetch keys for existing reviews before analysis. Integrates these into the main pipeline.
apps/importer/src/shopee/test-review-analyzer.ts Adds tests verifying that the analyzer correctly distinguishes authors by author_key even when author_name is masked or identical.
apps/importer/src/shopee/__tests__/shopee-to-supabase.test.ts Updates pipeline stubs to mock review_author_keys table interactions and verifies that keys are correctly upserted during ingestion.

📊 Architectural Flow

sequenceDiagram
    participant RS as Rate-Services
    participant Importer as Shopee Ingestion Pipeline
    participant DB as Supabase (rate.review_author_keys)
    participant Analyzer as AI Review Analyzer

    RS->>Importer: Sends reviews with masked author_name & authorKey (HMAC)
    
    Note over Importer: Ingestion Phase
    Importer->>DB: Upsert review_author_keys (review_id, author_key)
    DB-->>Importer: Success
    
    Note over Importer: Analysis Phase
    Importer->>DB: Load author_keys for stored reviews
    DB-->>Importer: Map of review_id -> author_key
    
    Importer->>Importer: Build ReviewInputItem[] with author_key
    Importer->>Analyzer: Pass reviews to analyzeWithHeuristics
    
    Note over Analyzer: Duplicate Detection
    Analyzer->>Analyzer: Use reviewAuthorIdentity() to group authors
    alt author_key exists
        Analyzer->>Analyzer: Identity = "key:<author_key>"
    else author_name exists
        Analyzer->>Analyzer: Identity = "name:<lowercased_name>"
    else
        Analyzer->>Analyzer: Identity = "id:<review_id>"
    end
    Analyzer-->>Importer: Returns analysis results (seeding flags, etc.)
Loading

JOY (JOY) and others added 2 commits November 6, 2026 17:59
* feat(shared-data): per-review author key for duplicate-text analysis

Rate-Services will mask stored author names (RS#1073), so the analyzer can no
longer tell authors apart by author_name. Reviews may now carry an optional
authorKey (keyed HMAC computed by RS, 16 to 128 chars of [A-Za-z0-9_-]).

- ScrapedReviewSchema and ShopeeReview accept authorKey.
- The importer stores it in rate.review_author_keys (a table anon cannot read,
  unlike rate.reviews) and loads it for the analysis; failures log a warning.
- Duplicate-text and near-duplicate seeding checks group by author_key when
  present, else by author_name, else by review id.
- Proposal SQL, verify, rollback and the ingest contract in Docs/Proposals.

Closes #1614

* fix(shared-data): track proposal SQL, all-or-nothing key load, drop bad keys

- Force-add the three proposal SQL files (git-ignored by *.sql).
- loadAuthorKeys returns an empty map on any failure so one analysis never
  mixes key and name identities.
- A malformed authorKey is dropped instead of rejecting the listing.
- Document the mixed-identity limits.

* chore: allow the author-keys proposal SQL in .gitignore

Refs #1614

* docs(proposals): author-keys migration after the second review

Worker policy limited to crawled reviews, lock_timeout, no unused index,
privilege-matrix verify, and the secret-rotation limit in the contract.

Refs #1614

* fix(shared-data): leave database.types.ts to the migration mirror

gen-db-types --check compares the file with supabase/migrations, and the
review_author_keys migration is still a DOS-Me proposal. The generated type
arrives with the mirror once DOS-Me applies it.

Refs #1614

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
* chore(db): mirror the review author keys migration from dos-me

Mirrors 20261109130000_rate_review_author_keys.sql byte for byte (DOS.Me #1584,
promote DOS.Me #1585) and regenerates database.types.ts.

Refs #1614

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Code review skipped — your organization has no extra usage available to pay for this review.

If your organization's extra usage balance is empty, an organization admin can add extra usage credits at claude.ai/admin-settings/usage. If its monthly spend limit was reached, an admin can raise it on the same page. If neither applies, contact Anthropic support.

Once extra usage is available, someone with write access to this repository can comment @claude review on this pull request to trigger a review.

@vercel

vercel Bot commented Nov 6, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
rate Ready Ready Preview Nov 6, 2026 6:45pm UTC

Request Review

@coderabbitai

coderabbitai Bot commented Nov 6, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 1c340dc1-75ae-43e7-a63a-1aa9a97c17f1

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@github-actions github-actions Bot added docs Documentation changes area:shared Shared packages / contracts / configs area:crawlers area:database labels Nov 6, 2026
dos[bot]
dos Bot approved these changes Nov 6, 2026

@dos dos Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⏱️ Code Review completed (14 eligible files · 30,837 chars · 1 model call(s) · 2 PR unit(s))

Whole diff with bounded context at base 1ba7325 / head ef40467.

✅ CLEAN_PASS: Complete diff reviewed

@JOY
JOY (JOY) merged commit 6b52974 into main Nov 6, 2026
23 checks passed

This branch was successfully deployed

1 active deployment
Preview — ef404671 Deployed Nov 6, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:crawlers area:database area:shared Shared packages / contracts / configs docs Documentation changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant