diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d298b5358..0b4f6967c 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -85,6 +85,9 @@ jobs: - name: Replay the DOS-Me review value proposal (Docker) run: pnpm --filter @repo/shared-data build && node scripts/test-review-value-proposal.mjs + - name: Replay the DOS-Me review AI points proposal (Docker) + run: node scripts/test-review-ai-points-proposal.mjs + - name: Check generated rate schema types match the migrations run: pnpm check:db-types diff --git a/.gitignore b/.gitignore index 1e2109638..be909f563 100644 --- a/.gitignore +++ b/.gitignore @@ -161,6 +161,9 @@ playwright-report-item-model/ !Docs/Proposals/2026-11-02-dos-me-review-value-score.sql !Docs/Proposals/2026-11-02-dos-me-review-value-score-rollback.sql !Docs/Proposals/2026-11-02-dos-me-review-value-score-verify.sql +!Docs/Proposals/2026-11-03-dos-me-review-ai-points.sql +!Docs/Proposals/2026-11-03-dos-me-review-ai-points-rollback.sql +!Docs/Proposals/2026-11-03-dos-me-review-ai-points-verify.sql * - Copy* * - Bản sao* *.dump diff --git a/Docs/Proposals/2026-11-03-ai-review-points.html b/Docs/Proposals/2026-11-03-ai-review-points.html new file mode 100644 index 000000000..cca943450 --- /dev/null +++ b/Docs/Proposals/2026-11-03-ai-review-points.html @@ -0,0 +1,289 @@ + + + + + + + AI trích ưu nhược từ review sàn + + + +
+ Đề xuất thiết kế + Cần JOY quyết +

AI trích ưu nhược từ review trên sàn

+

Ngày 03/11/2026. Người soạn: Core (Rate). Thuộc #1531 (sao theo tiêu chí, ưu nhược, điểm giá trị review), mục 4.3 của đề xuất ngày 31/10. Đây chỉ là thiết kế: chưa có code gọi AI, chưa tốn tiền, chưa đụng database.

+ +

0. Tóm tắt trong một phút

+ +

Từ ngữ: "token" là đơn vị AI dùng để đếm chữ; tiền tính theo số token đọc vào và viết ra. "Batch API" là cách gửi nhiều yêu cầu một lúc, trả kết quả trong vòng 24 giờ, rẻ hơn 50%. "Prompt caching" là cách để AI nhớ phần hướng dẫn giống nhau giữa các lần gọi; phần nhớ được tính 10% giá.

+ +

1. Review nào được xử lý

+

Chỉ xử lý review có đủ chữ để rút ra một ý thật. Review nào không qua các điều kiện dưới đây thì bỏ qua, không gọi AI và không tốn tiền.

+
+ + + + + + + +
Điều kiệnGiá trị đề xuấtLý do
Nguồnorigin = 'crawl', reviewer_type = 'user'Review viết trên Rate đã có ưu nhược do người viết ghi. Review chuyên gia đi theo đường riêng (mục 4.3 của đề xuất gốc).
Chưa có dòng ưu nhượcKhông có dòng method = 'writer' nàoKhông ghi đè lời người viết.
Qua cổng chặnCó review_flags, include_in_rating không phải false, trọng số lớn hơn 0 (cổng của job product-points, jobs/product-points.ts), và độ sâu không phải "viết cho có" (kiểm tra của job review-values, jobs/review-values.ts)Review giả, seeding hoặc viết cho có không được tạo ưu nhược.
Độ dàiTừ 40 ký tự và 8 từ trở lên, sau khi bỏ emoji và ký tự lặp"Hàng đẹp, giao nhanh" không có ý gì về sản phẩm. Ngưỡng này loại phần lớn review một câu.
Ngôn ngữTiếng Việt hoặc tiếng Anh (đoán bằng dấu tiếng Việt và từ thông dụng, không gọi AI)Các dòng ưu nhược hiện bằng tiếng Việt. Review ngôn ngữ khác rất hiếm, bỏ qua để giữ prompt đơn giản.
Ngành hàngListing đã gắn với một sản phẩm gốc, và ngành hàng có danh sách tiêu chíPhải biết danh sách tiêu chí để gắn dòng. Ngành "dịch vụ" chưa mở.
+

Các ý chỉ nói về giao hàng, đóng gói, thái độ shop thì không lấy: đó là chuyện của người bán, đã có mô hình tin cậy người bán lo.

+ +

2. Prompt và dạng kết quả

+

2.1 Gửi gì cho AI

+ +

2.2 Luật trích (tóm tắt prompt)

+
    +
  1. Chỉ lấy điều review nói ra. Không suy ra từ số sao, không đoán.
    2. +
  2. Mỗi review tối đa 3 ưu và 3 nhược. Không có gì rõ ràng thì trả danh sách rỗng; danh sách rỗng là kết quả bình thường.
    3. +
  3. Mỗi dòng dài 5 đến 150 ký tự, viết lại gọn bằng tiếng Việt, giữ đúng ý và đúng mức độ ("pin hơi yếu" không thành "pin rất tệ").
    4. +
  4. Mỗi dòng có criterion_key thuộc danh sách của ngành hàng, hoặc null nếu là ý chung.
    5. +
  5. Mỗi dòng có evidence: một đoạn trích nguyên văn ngắn từ review chứa ý đó.
    6. +
  6. Bỏ qua chuyện giao hàng, đóng gói, người bán, khuyến mãi.
    7. +
+

2.3 Dạng kết quả (structured output, JSON có schema)

+

Dùng tính năng structured outputs của Claude API: kết quả luôn đúng schema, không cần đoán cách đọc chữ.

+
{
+  "reviews": [
+    {
+      "ref": 3,
+      "points": [
+        { "kind": "pro", "criterion_key": "battery",
+          "body": "Pin dùng hết ngày với nhu cầu vừa phải",
+          "evidence": "pin xài cả ngày vẫn còn" },
+        { "kind": "con", "criterion_key": "screen",
+          "body": "Màn hình khó nhìn ngoài nắng",
+          "evidence": "ra nắng hơi khó nhìn" }
+      ]
+    }
+  ]
+}
+

criterion_key trong schema là một danh sách liệt kê đúng các khoá của ngành hàng, nên AI không thể trả một khoá lạ.

+ +

3. Chọn model

+
+ + + + +
ModelGiá đọc vào / viết ra (USD mỗi 1 triệu token)Nhận xét
Claude Haiku 5.5 (claude-haiku-5-5)0,10 / 0,50Rẻ nhất. Việc này đơn giản: đọc một đoạn ngắn, rút ý có trích dẫn. Đề xuất dùng mặc định.
Claude Sonnet 5.5 (claude-sonnet-5-5)2 / 10Dự phòng nếu Haiku không đạt ngưỡng ở mục 7 (ví dụ hiểu sai tiếng lóng, viết tắt).
Claude Opus 5.5 (claude-opus-5-5)4 / 20Không cần cho việc này.
+

Giá theo bảng giá API của Anthropic. Batch API giảm 50% cả hai chiều. Phần hướng dẫn cache được tính khoảng 10% giá đọc vào; lần ghi cache đầu tính 125%. Haiku 5.5 mặc định có suy nghĩ ở mức medium. Đề xuất đặt mức low: việc này không cần suy luận sâu, và token suy nghĩ cũng tính tiền như token viết ra.

+

Gọi qua đâu. Importer hiện gọi AI qua chuẩn OpenAI (biến AI_PROVIDER, cổng dos-ai). Đề xuất bước này gọi thẳng Claude API bằng SDK chính thức của Anthropic, vì cần structured outputs, Batch API và prompt caching. Khoá API chỉ nằm trong secret của Rate-Services. Câu hỏi này để JOY quyết ở D2.

+ +

4. Gửi theo lô và chỉ xử lý một lần

+ + +

5. Ước tính chi phí

+

Giả định:

+ +
+ + + +
Model1.000 review, giá thường1.000 review, Batch APIKho hôm nay (~250 review)5.000 review mới mỗi tháng
Haiku 5.5~0,10 đến 0,30 USD~0,05 đến 0,15 USDdưới 0,05 USD~0,25 đến 0,75 USD/tháng
Sonnet 5.5~2 đến 5 USD~1 đến 2,5 USD~0,25 đến 0,6 USD~5 đến 12,5 USD/tháng
+

Cách tính, với Haiku 5.5, cho 1.000 review (50 yêu cầu):

+ +

Mức tăng hằng tháng phụ thuộc tốc độ crawl. 5.000 review mỗi tháng là giả định rộng tay; phần lớn review sàn bị loại ở mục 1 nên số thật gửi đi sẽ ít hơn.

+

Chi phí thật nhỏ hơn rủi ro chất lượng, nên giới hạn tiền ở mục 8 chủ yếu để chặn lỗi vòng lặp, không phải để tiết kiệm.

+ +

6. Nơi chạy và ranh giới open-core

+ + +

7. Kiểm tra chất lượng

+

7.1 Trước khi bật (một lần, trên Beta)

+ +

7.2 Mỗi lần chạy (tự động, trong code)

+ +

7.3 Sau khi chạy

+ + +

8. Công tắc tắt và giới hạn

+ + +

9. Dòng AI trong ngưỡng D9

+

D9 hiện tại (trong packages/shared-data/src/rules/product-points.ts): một ưu hoặc nhược của sản phẩm hiện khi có ít nhất 2 người khác nhau nhắc và tổng trọng số từ 1,5 trở lên, hoặc 1 chuyên gia; luôn giữ ít nhất 1 nhược điểm nếu có. Câu hỏi là dòng AI được tính bao nhiêu.

+
+ + + + +
Phương ánCách tínhHệ quả
(a) Như lời người viếtDòng AI tính đủ trọng số của reviewHiện nhiều ý nhất. Nhưng một lỗi trích của AI có sức nặng bằng lời người thật viết.
(b) Nửa trọng sốMỗi review vẫn tính là 1 người viết, nhưng dòng AI chỉ góp 0,5 × min(trọng số sau khi giảm theo tuổi, 1) vào tổng. Nhân 0,5 sau mức trần 1 mỗi người.Một ý chỉ từ review sàn cần ít nhất 3 review mới đạt 1,5. Nếu nhân trước mức trần, 2 review trọng số cao đã đủ. Lỗi lẻ của AI khó tự tạo thành một ưu nhược của sản phẩm.
(c) Chỉ để phụ hoạDòng AI chỉ cộng vào ý đã có ít nhất 1 dòng của người viếtAn toàn nhất. Nhưng hôm nay gần như không có review Rate nào, nên trang sản phẩm sẽ vẫn trống.
+

Phương án (a) hoặc (b) cần sửa job product-points: bỏ bộ lọc chỉ đọc dòng 'writer', và thêm method vào mỗi dòng đem gom. Câu hiển thị của một cụm hiện lấy từ dòng được nhắc nhiều nhất, nên có thể là câu AI viết. Khi đó khối ưu nhược phải ghi nguồn như dưới đây.

+

Các điều giữ nguyên dù chọn phương án nào:

+ + +

10. Các quyết định cần JOY

+
D1. Model. Haiku 5.5 làm mặc định, chuyển lên Sonnet 5.5 nếu không đạt ngưỡng ở 7.1. +

Đề xuất: duyệt. Chi phí cả hai đều nhỏ; chất lượng quyết định bằng bộ mẫu, không bằng đoán.

+
D2. Gọi AI qua đâu. (a) Thẳng Claude API bằng SDK Anthropic, khoá trong Rate-Services; (b) qua cổng dos-ai kiểu OpenAI như importer đang làm. +

Đề xuất: (a). Cần structured outputs, Batch API và prompt caching. Qua cổng kiểu OpenAI thì mất hoặc phải tự làm lại các tính năng này.

+
D3. Điều kiện chọn review. Các ngưỡng ở mục 1: từ 40 ký tự và 8 từ, tiếng Việt hoặc tiếng Anh, qua cổng chặn, chưa có dòng của người viết. +

Đề xuất: duyệt. Chỉnh lại sau lần chạy thử đầu nếu bỏ sót quá nhiều.

+
D4. Trọng số dòng AI trong D9 (mục 9). +

Đề xuất: (b) nửa trọng số, chốt sau một lần gom thử trên Beta với kho ~250 review: xem (b) và (c) mỗi cách hiện bao nhiêu ưu nhược. Trang sản phẩm có ưu nhược thật từ dữ liệu sàn, nhưng một lỗi lẻ của AI không tự thành ưu nhược của sản phẩm.

+
D5. Giới hạn. 5 USD mỗi tháng với Haiku 5.5 (15 USD nếu phải dùng Sonnet 5.5), 2.000 review mỗi đêm, hai công tắc (dừng gọi AI, ẩn dòng đã ghi) và lệnh xoá sạch. +

Đề xuất: duyệt.

+
D6. Migration DOS-Me. Bảng rate.review_ai_extractions và hàm rate.store_review_ai_points cho rate_worker. Core soạn SQL đề xuất kèm file kiểm tra, file hoàn tác và replay. +

Đề xuất: duyệt để Core soạn. Không áp dụng gì khi chưa có lệnh của JOY.

+ +

11. Thứ tự làm

+
    +
  1. Sau khi JOY duyệt: Core soạn đề xuất migration ở D6 (chưa áp dụng).
    2. +
  2. Viết job trong apps/importer, test bằng client AI giả. Chưa tốn tiền.
    3. +
  3. Chạy thử bộ mẫu 100 review trên Beta (dưới 1 USD), báo kết quả theo ngưỡng ở 7.1. Bước này cần JOY cho phép tiêu tiền và cấp khoá API.
    4. +
  4. Áp dụng migration lên Beta, chạy job một lần ở chế độ chạy thử, rồi --apply trên kho ~250 review.
    5. +
  5. Production và lịch chạy hằng đêm trong Rate-Services.
    6. +
+ +

12. Không nằm trong đề xuất này

+ +
+ + diff --git a/Docs/Proposals/2026-11-03-dos-me-review-ai-points-rollback.sql b/Docs/Proposals/2026-11-03-dos-me-review-ai-points-rollback.sql new file mode 100644 index 000000000..7ef3fd4f0 --- /dev/null +++ b/Docs/Proposals/2026-11-03-dos-me-review-ai-points-rollback.sql @@ -0,0 +1,20 @@ +-- ROLLBACK the PROPOSAL 2026-11-03-dos-me-review-ai-points.sql (DOS/Rate#1531, AI design D6). +-- Safe to run twice. Stop the review AI points job first (REVIEW_AI_POINTS_ENABLED=false in the +-- Rate-Services schedule), and cancel any open batch, so nothing writes while this runs. +-- +-- It deletes every 'ai' line of rate.review_points (the job can extract them again after a +-- re-apply, at the cost of the model calls), then the two functions and the two tables. Writer +-- lines are not touched. The next product-points run aggregates without the 'ai' lines. +-- The batches table holds the month's spend: copy it first if the record is wanted. + +BEGIN; + +DELETE FROM rate.review_points WHERE method = 'ai'; + +DROP FUNCTION IF EXISTS rate.store_review_ai_points(jsonb); +DROP FUNCTION IF EXISTS rate.purge_review_ai_points(text); +DROP FUNCTION IF EXISTS rate.review_ai_text_hash(text); +DROP TABLE IF EXISTS rate.review_ai_extractions; +DROP TABLE IF EXISTS rate.review_ai_batches; + +COMMIT; diff --git a/Docs/Proposals/2026-11-03-dos-me-review-ai-points-verify.sql b/Docs/Proposals/2026-11-03-dos-me-review-ai-points-verify.sql new file mode 100644 index 000000000..d8c51f800 --- /dev/null +++ b/Docs/Proposals/2026-11-03-dos-me-review-ai-points-verify.sql @@ -0,0 +1,108 @@ +-- VERIFY the PROPOSAL 2026-11-03-dos-me-review-ai-points.sql (DOS/Rate#1531, AI design D6). +-- Read-only. Run after the migration on preview, beta and production, and after each run of +-- the review AI points job. V1 to V7 return 0; V8 is information. Run as postgres or +-- service_role (V1 calls a function only service_role and rate_worker may execute). + +-- V1 the text hash is sha256 of the NFC text: a decomposed and a composed "Pin tốt" hash the +-- same, and the empty text is sha256 of nothing; expect 0 +SELECT count(*) FROM (VALUES + (rate.review_ai_text_hash(NULL), 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'), + (rate.review_ai_text_hash(E'Pin tốt'), rate.review_ai_text_hash(E'Pin tốt')) +) AS t(got, expected) +WHERE t.got IS DISTINCT FROM t.expected; + +-- V2 the objects and grants: anon and authenticated cannot read the two tables or call the +-- functions; rate_worker reads both tables, writes batches but not markers, and still has no +-- write on rate.review_points; the functions are security definer with an empty search_path; +-- expect 0 +SELECT count(*) FROM ( + SELECT c.relname::text AS problem FROM pg_class c + WHERE c.oid IN ('rate.review_ai_batches'::regclass, 'rate.review_ai_extractions'::regclass) + AND NOT c.relrowsecurity + UNION ALL + SELECT p.proname::text FROM pg_proc p JOIN pg_namespace n ON n.oid = p.pronamespace + WHERE n.nspname = 'rate' AND p.proname IN ('store_review_ai_points', 'purge_review_ai_points') + AND NOT (p.prosecdef AND p.proconfig @> ARRAY['search_path=""']) + UNION ALL + SELECT r.role || ' ' || t.tab FROM unnest(ARRAY['anon', 'authenticated']) AS r(role) + CROSS JOIN unnest(ARRAY['rate.review_ai_batches', 'rate.review_ai_extractions']) AS t(tab) + WHERE has_table_privilege(r.role, t.tab, 'SELECT') + OR has_table_privilege(r.role, t.tab, 'INSERT') + OR has_table_privilege(r.role, t.tab, 'UPDATE') + OR has_table_privilege(r.role, t.tab, 'DELETE') + UNION ALL + SELECT r.role || ' ' || f.fn FROM unnest(ARRAY['anon', 'authenticated']) AS r(role) + CROSS JOIN unnest(ARRAY['rate.store_review_ai_points(jsonb)', 'rate.purge_review_ai_points(text)', + 'rate.review_ai_text_hash(text)']) AS f(fn) + WHERE has_function_privilege(r.role, f.fn, 'EXECUTE') + UNION ALL + SELECT 'rate_worker grants' WHERE NOT ( + has_table_privilege('rate_worker', 'rate.review_ai_batches', 'SELECT') + AND has_table_privilege('rate_worker', 'rate.review_ai_batches', 'INSERT') + AND has_table_privilege('rate_worker', 'rate.review_ai_batches', 'UPDATE') + AND NOT has_table_privilege('rate_worker', 'rate.review_ai_batches', 'DELETE') + AND has_table_privilege('rate_worker', 'rate.review_ai_extractions', 'SELECT') + AND NOT has_table_privilege('rate_worker', 'rate.review_ai_extractions', 'INSERT') + AND NOT has_table_privilege('rate_worker', 'rate.review_ai_extractions', 'UPDATE') + AND NOT has_table_privilege('rate_worker', 'rate.review_ai_extractions', 'DELETE') + AND NOT has_table_privilege('rate_worker', 'rate.review_points', 'INSERT') + AND NOT has_table_privilege('rate_worker', 'rate.review_points', 'UPDATE') + AND NOT has_table_privilege('rate_worker', 'rate.review_points', 'DELETE') + AND has_function_privilege('rate_worker', 'rate.store_review_ai_points(jsonb)', 'EXECUTE') + AND has_function_privilege('rate_worker', 'rate.purge_review_ai_points(text)', 'EXECUTE')) +) s; + +-- V3 every 'ai' line is on a crawled user review with a marker, and no review has both writer +-- and 'ai' lines; expect 0 +SELECT count(*) FROM rate.review_points p +JOIN rate.reviews r ON r.id = p.review_id +LEFT JOIN rate.review_ai_extractions x ON x.review_id = p.review_id +WHERE p.method = 'ai' + AND (x.review_id IS NULL + OR r.origin IS DISTINCT FROM 'crawl' OR r.reviewer_type IS DISTINCT FROM 'user' + OR EXISTS (SELECT 1 FROM rate.review_points w + WHERE w.review_id = p.review_id AND w.method = 'writer')); + +-- V4 each marker counts the review's 'ai' lines and holds one evidence quote per line; expect 0 +SELECT count(*) FROM rate.review_ai_extractions x +LEFT JOIN (SELECT review_id, count(*) AS n FROM rate.review_points WHERE method = 'ai' + GROUP BY review_id) l ON l.review_id = x.review_id +WHERE x.point_count <> COALESCE(l.n, 0) + OR jsonb_array_length(x.evidence) <> x.point_count; + +-- V5 at most 3 pros and 3 cons per review, and every criterion key in the criteria list the +-- marker was processed with (its version and category; a review whose master moved keeps its +-- keys until the job processes it again); expect 0 +SELECT count(*) FROM ( + SELECT review_id FROM rate.review_points WHERE method = 'ai' + GROUP BY review_id, kind HAVING count(*) > 3 + UNION ALL + SELECT p.review_id FROM rate.review_points p + JOIN rate.review_ai_extractions x ON x.review_id = p.review_id + WHERE p.method = 'ai' AND p.criterion_key IS NOT NULL + AND NOT EXISTS (SELECT 1 FROM rate.rating_criteria k + WHERE k.criteria_version = x.criteria_version + AND k.category_slug = x.category_slug AND k.key = p.criterion_key) +) s; + +-- V6 no stored batch without its reported cost, and no batch left open for more than 48 hours +-- (the Batch API ends every batch within 24); expect 0 +SELECT count(*) FROM rate.review_ai_batches +WHERE (status = 'stored' AND cost_usd IS NULL) + OR (status IN ('submitted', 'ended') AND submitted_at < now() - interval '48 hours'); + +-- V7 no marker points at a batch of another model or prompt version; expect 0 +SELECT count(*) FROM rate.review_ai_extractions x +JOIN rate.review_ai_batches b ON b.id = x.batch_id +WHERE b.model <> x.model OR b.prompt_version <> x.prompt_version; + +-- V8 information: markers by prompt version and outcome, and this month's spend per model +SELECT 'marker' AS what, prompt_version AS key, outcome AS detail, count(*)::numeric AS n +FROM rate.review_ai_extractions GROUP BY prompt_version, outcome +UNION ALL +SELECT 'spend', model, to_char(date_trunc('month', now()), 'YYYY-MM'), + sum(COALESCE(cost_usd, reserved_usd)) +FROM rate.review_ai_batches +WHERE submitted_at >= date_trunc('month', now()) +GROUP BY model +ORDER BY 1, 2, 3; diff --git a/Docs/Proposals/2026-11-03-dos-me-review-ai-points.md b/Docs/Proposals/2026-11-03-dos-me-review-ai-points.md new file mode 100644 index 000000000..e0d1b77aa --- /dev/null +++ b/Docs/Proposals/2026-11-03-dos-me-review-ai-points.md @@ -0,0 +1,172 @@ +# DOS-Me proposal: review AI points, the marker and the writer (D6) + +DOS/Rate#1531, D6 of `Docs/Proposals/2026-11-03-ai-review-points.html` (all six decisions +approved by JOY on 2026-11-03: "Duyệt theo khuyến nghị"). Status: **proposal, nothing +applied.** It goes through DOS-Me like P2, P3 and P6. + +| File | What | +| ------------------------------------------------- | ------------------------------------------------------- | +| `2026-11-03-dos-me-review-ai-points.sql` | the migration, one transaction | +| `2026-11-03-dos-me-review-ai-points-verify.sql` | read-only checks, V1 to V7 return 0 (V8 is information) | +| `2026-11-03-dos-me-review-ai-points-rollback.sql` | removes the 'ai' lines, the functions and tables | +| `scripts/test-review-ai-points-proposal.mjs` | replays it all on PostgreSQL in Docker | + +## What it does + +- **`rate.review_ai_batches`** (new): one row per Batch API batch the job sends. + - It holds the batch id, written the moment the batch is created. A run that crashes + collects that batch next time instead of sending the reviews again. + - The manifest maps each request to its reviews: review ids, refs and text hashes. It + never holds review text. + - The cost: the ceiling the job reserved before sending, then the cost from the usage the + API reported. The month's spend per model is read from here (design section 8). + - `rate_worker` reads and inserts it, and updates a batch until it is stored. A stored + batch's cost, which the monthly cap reads, is closed to it. No role but `service_role` + deletes. +- **`rate.review_ai_extractions`** (new): the process-once marker, one row per review. + - The key: the NFC text hash, prompt version, criteria version and category it was + processed with, plus the model. A review is due again when any part of the key changes, + including a move of its master to another category. + - The outcome: `stored`, `empty` (the model found nothing), `rejected` (every line it + returned failed the job's checks) or `failed` (cut off, refused or errored). + - A failed result is never recorded as `empty`: the writer requires the `failed` flag on + every row. It counts attempts; the writer refuses a fourth attempt with the same key, and + the table holds at most 3. + - The prompt version that wrote the review's lines. A failed retry under a new version keeps + the old lines, and the purge of the old version still finds them. + - The evidence quote of each line, for the weekly read-back (design 7.2). + - `rate_worker` reads it. Only the two functions write it. +- **`rate.review_ai_text_hash(text)`**: sha256 hex of the text in NFC. The job computes the + same hash; the replay checks they agree, decomposed diacritics included. +- **`rate.store_review_ai_points(p_rows)`**, for `rate_worker` and `service_role`. + - It writes a review's 'ai' lines and its marker in one transaction. + - Up to 500 rows and 2 MB per call. One malformed row refuses the whole call (22023, with + the row's index), before anything is written. + - Each line is checked: + - at most 3 pros and 3 cons; + - 5 to 150 characters with words in it, stored as P3 stores a line; + - a con that only dodges ("không có gì để chê") is refused; + - no line twice, including as both a pro and a con; + - the criterion key is null or in the category's criteria list of that version; + - an evidence quote of 2 to 300 characters and at least two words. + - A batch id must be a row of `rate.review_ai_batches`; an unknown one fails with 23503. + - Some reviews are skipped and counted, not refused, so one odd review never stops a night: + - `missing`: the review no longer exists; + - `not_crawl`: not a crawled user review (a community or expert review); + - `writer`: it has a writer line; 'ai' and writer lines never mix; + - `stale`: its comment changed since it was sent (the hash differs), or its master is in + another category now. The job sends it again; + - `unquoted`: a quote marked `exact` is not in the comment, compared as P3 compares lines. + Nothing is written and the job reports it. A quote marked `unaccented` (found only + without diacritics) is checked by the job alone; + - `done`: a failed result for a review that already has a result with the same key. A late + or duplicate failure never undoes a result; + - `exhausted`: a fourth failed attempt with the same key. + - The reviews are locked in id order first, so two calls never interleave on one review. +- **`rate.purge_review_ai_points(p_prompt_version)`**: the job's `--purge`. + - `'all'` removes every 'ai' line and marker. A version removes the markers of that version + or holding lines it wrote, with all the 'ai' lines of those reviews; they are processed + again on the next run. + - There is no default: a bare call does not exist, so nothing is purged by accident. No + prompt version may be called `all`. + - It locks its reviews in the writer's order, so a purge and a store never leave a line + without its marker. + - Batches stay, since they hold the month's spend. Writer lines are never touched. + +`rate_worker` still has only SELECT on `rate.review_points`. 'ai' lines come only through the +writer, as writer lines come only through the review RPC. + +**Rating impact at apply time: none.** Nothing reads 'ai' lines yet: + +- the P2 rating never reads `review_points`; +- P6 counts only 'writer' lines; +- product-points reads only 'writer' lines until its D4 change ships. + +## Order and dependencies + +- **Needs:** P2 (`20261101120000`) and P3 (`20261102090000`, for `rate.rating_criteria` and + the line helpers). Both are in production. +- **After the apply:** + - The job (`apps/importer`, a follow-up PR) runs dry by default. Only with `--apply` does + it send batches and call the writer, as `rate_worker`. + - Rate-Services schedules it after a dry run and holds the API key. Nothing calls a model + before that. + - Run one job at a time. The batches table is how a second run would see the first one's + open batch. + +## Decisions + +The design's D6 named one table and one function. This proposal adds three things to it: + +1. **A second table for batches**, rather than keeping the spend in the marker. One request + carries up to 20 reviews, so its cost belongs to the batch, not to each review. A batch also + exists before any review has a result, and that is when its id must be kept. +2. **A purge function**, rather than a purge mode in the writer. Each function then does one + thing, and its grant says so. +3. **`rate_worker` writes the batches table directly.** It is the job's own bookkeeping: it + holds no review text, and no page or rating reads it. Its CHECKs keep it well-formed, and a + stored batch is closed to it. Until a batch is stored, the worker could lower its reserved + cost and so loosen the cap; the cap guards against a looping job, not against the worker's + own credentials. The lines and the markers, which pages and the aggregation read, stay + behind the writer. + +Known limits: + +- If the job dies between creating a batch and inserting its row, that batch is paid for but + not collected, and its reviews are sent again. The window is one HTTP call, and the cost is + at most one night's batch (under $0.20 with Haiku). +- The database checks an `exact` quote against the comment but not an `unaccented` one: it + has no diacritic folding. The job checks both, and counts `unaccented` lines apart. +- The replay runs one session, so it does not exercise two calls blocking on each other's + locks; the lock order is the same as P6's writer. + +## Apply + +1. Copy the `.sql` into DOS-Me `supabase/migrations/` with a new timestamp, and the verify and + rollback files into `scripts/ops/`, as for P6. +2. After the beta and production applies, run the verify file read-only as `postgres` or + `service_role`. V1 to V7 must return 0. Run it again after each job run. +3. **Rollback:** the rollback file, safe to run twice. Stop the job first + (`REVIEW_AI_POINTS_ENABLED=false`) and cancel any open batch. + - It deletes every 'ai' line, then the functions and both tables. Writer lines stay. + - The batch records, and so the month's spend, are lost: copy them first if wanted. + - The job can extract the lines again after a re-apply, at the cost of the model calls. + +## Tests (local, Docker) + +`node scripts/test-review-ai-points-proposal.mjs`: + +- the proposal applied twice; no rating moves; +- the hash equal to the job's, for decomposed and composed text, emoji and the empty string; +- a batch row inserted and updated as `rate_worker`; +- lines and markers stored as `rate_worker`: + - positions per kind, the evidence and its match kind; + - a review listed twice (the last row wins); + - a re-store that replaces the lines, and an empty result that clears them; +- the skips: a missing review, a community and an expert review, a review with a writer line, + a changed comment, a made-up exact quote, a master moved to another category (V5 still 0); +- failed results: + - a late failure over a result of the same key changes nothing (`done`); + - under a new key, attempts counted and the old lines kept with their version; + - a fourth attempt refused (`exhausted`); + - the purge of the old version finding the kept lines; +- the refusals, where nothing is written: + - not an array, too many rows; + - a bad id, hash, version (`all` included), model, batch id, flag, count or key; + - a row without its `failed` flag, or with a null one; + - an unknown criteria list, a key of another category; + - a failed row with points, seven lines, four pros; + - a short, long, letterless or evasive line, a line both pro and con; + - one word of evidence, or too much; + - an unknown batch id (23503); + - `anon` and `authenticated` calling the writer; +- direct writes stopped: + - `rate_worker` on `rate.review_points` and on the markers; + - deleting batches, and updating a stored one; + - a stored batch without its cost; +- the public reading neither table, and reading 'ai' lines like writer lines; +- the purge of one prompt version, then of everything (`'all'`); a bare call and a bad version + refused; writer lines stay; +- V1 to V7 after each step, and V8; +- the rollback twice: every object and every 'ai' line gone, writer lines and the rating kept; +- the proposal again, with a store on it. diff --git a/Docs/Proposals/2026-11-03-dos-me-review-ai-points.sql b/Docs/Proposals/2026-11-03-dos-me-review-ai-points.sql new file mode 100644 index 000000000..5c9290237 --- /dev/null +++ b/Docs/Proposals/2026-11-03-dos-me-review-ai-points.sql @@ -0,0 +1,496 @@ +-- ============================================================================= +-- PROPOSAL (DOS/Rate#1531, D6 of Docs/Proposals/2026-11-03-ai-review-points.html; all six +-- decisions approved by JOY on 2026-11-03 as recommended): the database side of the AI +-- extraction of pros and cons from marketplace reviews (rate.review_points method 'ai'). A +-- DOS-Me migration to review and apply; nothing here is applied by DOS/Rate. +-- Verify: 2026-11-03-dos-me-review-ai-points-verify.sql (V1 to V7 return 0). +-- Rollback: 2026-11-03-dos-me-review-ai-points-rollback.sql. +-- +-- What it adds: +-- rate.review_ai_batches one row per Batch API batch the job sends: its id (kept the +-- moment the batch is created, so a crashed run collects it +-- instead of paying twice), the manifest (review ids, refs and +-- text hashes; never review text), the cost reserved before +-- sending and the cost the API reported. The month's spend per +-- model is read from here (design section 8). rate_worker reads, +-- inserts and updates it until it is stored; nobody deletes +-- through the API. +-- rate.review_ai_extractions the process-once marker, one row per review: the NFC text hash, +-- prompt version, criteria version, category and model it was +-- processed with, the outcome (stored, empty, rejected, failed), +-- the attempts of a failed result (at most 3), the version that +-- wrote its lines, and the evidence quote of each line. +-- rate_worker reads it; only the functions write. +-- rate.review_ai_text_hash(text) sha256 hex of the text in Unicode NFC; the job hashes the +-- same way (the replay compares them). +-- rate.store_review_ai_points(p_rows) the writer for the job (rate_worker) and service_role: +-- the 'ai' lines of a review and its marker in one transaction. +-- Only crawled user reviews with no writer lines; at most 3 pros +-- and 3 cons, lines 5 to 150 characters with words, no evasive +-- con, no line twice, criterion keys of the review's category +-- list, an evidence quote per line found in the comment. +-- rate.purge_review_ai_points(p_prompt_version) removes every 'ai' line and marker ('all'), +-- or those of one prompt version (the job's --purge). +-- +-- rate_worker still has only SELECT on rate.review_points: 'ai' lines come only through the +-- writer, as writer lines come only through the review RPC. +-- +-- Depends on P2 (20261101120000) and P3 (20261102090000, rate.rating_criteria and the line +-- helpers); both are in production. +-- +-- Rating impact at apply time: none. Nothing reads 'ai' lines yet: the P2 rating never read +-- review_points, P6 counts only 'writer' lines, and product-points reads 'writer' lines until +-- its 'ai' change ships (D4). +-- +-- Idempotent: IF NOT EXISTS for tables and indexes, CREATE OR REPLACE for functions, DROP ... +-- IF EXISTS before each policy. One transaction. +-- ============================================================================= + +BEGIN; + +-- 1. rate.review_ai_batches -------------------------------------------------------------------- + +CREATE TABLE IF NOT EXISTS rate.review_ai_batches ( + -- The Batch API's id (msgbatch_...). + id text PRIMARY KEY CHECK (id ~ '^[A-Za-z0-9_-]{8,128}$'), + model text NOT NULL CHECK (model ~ '^[a-z0-9][a-z0-9.-]{1,63}$'), + prompt_version text NOT NULL CHECK (prompt_version ~ '^[a-z0-9][a-z0-9._-]{0,31}$'), + status text NOT NULL DEFAULT 'submitted' + CHECK (status IN ('submitted', 'ended', 'stored', 'canceled', 'expired')), + request_count integer NOT NULL CHECK (request_count BETWEEN 1 AND 10000), + review_count integer NOT NULL CHECK (review_count BETWEEN 1 AND 100000), + -- [{custom_id, category, criteria_version, reviews: [{ref, id, text_hash}]}]: what each + -- request carried, so results map back to reviews after a crash. No review text. + manifest jsonb NOT NULL + CHECK (jsonb_typeof(manifest) = 'array' AND pg_column_size(manifest) <= 1048576), + -- The ceiling the job reserved before sending (requests x max output, at list price), then + -- the cost from the usage the API reported. The month's spend counts cost, else the ceiling. + reserved_usd numeric(10, 4) NOT NULL CHECK (reserved_usd >= 0), + cost_usd numeric(10, 4) CHECK (cost_usd >= 0), + input_tokens bigint NOT NULL DEFAULT 0 CHECK (input_tokens >= 0), + output_tokens bigint NOT NULL DEFAULT 0 CHECK (output_tokens >= 0), + cache_read_tokens bigint NOT NULL DEFAULT 0 CHECK (cache_read_tokens >= 0), + cache_write_tokens bigint NOT NULL DEFAULT 0 CHECK (cache_write_tokens >= 0), + submitted_at timestamptz NOT NULL DEFAULT now(), + ended_at timestamptz, + stored_at timestamptz, + CONSTRAINT review_ai_batches_stored_cost CHECK (status <> 'stored' OR cost_usd IS NOT NULL) +); +CREATE INDEX IF NOT EXISTS review_ai_batches_month_idx + ON rate.review_ai_batches (model, submitted_at); +CREATE INDEX IF NOT EXISTS review_ai_batches_open_idx + ON rate.review_ai_batches (submitted_at) WHERE status IN ('submitted', 'ended'); + +COMMENT ON TABLE rate.review_ai_batches IS + 'Batches the review AI points job sent (DOS/Rate#1531, AI design D6): id, manifest, reserved and reported cost. No review text.'; + +ALTER TABLE rate.review_ai_batches ENABLE ROW LEVEL SECURITY; +REVOKE ALL ON rate.review_ai_batches FROM PUBLIC, anon, authenticated, rate_worker; +GRANT ALL ON rate.review_ai_batches TO service_role; +GRANT SELECT, INSERT, UPDATE ON rate.review_ai_batches TO rate_worker; +DROP POLICY IF EXISTS "Service role access review_ai_batches" ON rate.review_ai_batches; +CREATE POLICY "Service role access review_ai_batches" ON rate.review_ai_batches + FOR ALL TO service_role USING (true) WITH CHECK (true); +DROP POLICY IF EXISTS "Rate worker read review_ai_batches" ON rate.review_ai_batches; +CREATE POLICY "Rate worker read review_ai_batches" ON rate.review_ai_batches + FOR SELECT TO rate_worker USING (true); +DROP POLICY IF EXISTS "Rate worker insert review_ai_batches" ON rate.review_ai_batches; +CREATE POLICY "Rate worker insert review_ai_batches" ON rate.review_ai_batches + FOR INSERT TO rate_worker WITH CHECK (true); +DROP POLICY IF EXISTS "Rate worker update review_ai_batches" ON rate.review_ai_batches; +-- A stored batch is closed: its cost, which the month's cap reads, cannot be rewritten. +CREATE POLICY "Rate worker update review_ai_batches" ON rate.review_ai_batches + FOR UPDATE TO rate_worker USING (status <> 'stored') WITH CHECK (true); + +-- 2. rate.review_ai_extractions ---------------------------------------------------------------- +-- A review is done while its marker has the same text hash, prompt version, criteria version +-- and category, unless the outcome is 'failed' with fewer than 3 attempts (a cut-off or refused +-- result is never recorded as 'empty'; it is retried at most twice, and the writer refuses a +-- fourth attempt). + +CREATE TABLE IF NOT EXISTS rate.review_ai_extractions ( + review_id uuid PRIMARY KEY REFERENCES rate.reviews(id) ON DELETE CASCADE, + text_hash text NOT NULL CHECK (text_hash ~ '^[0-9a-f]{64}$'), + -- 'all' is the purge's word for every version, so no version is called that. + prompt_version text NOT NULL CHECK (prompt_version ~ '^[a-z0-9][a-z0-9._-]{0,31}$' + AND prompt_version <> 'all'), + criteria_version integer NOT NULL CHECK (criteria_version >= 1), + -- The category of the review's master when it was processed: a move to another category + -- makes the review due again. + category_slug text NOT NULL CHECK (category_slug ~ '^[a-z_-]{2,64}$'), + model text NOT NULL CHECK (model ~ '^[a-z0-9][a-z0-9.-]{1,63}$'), + -- stored: lines written; empty: the model found nothing; rejected: every line it returned + -- failed the job's checks; failed: cut off (max_tokens), refused or errored, to retry. + outcome text NOT NULL CHECK (outcome IN ('stored', 'empty', 'rejected', 'failed')), + -- The 'ai' lines of the review now, and the prompt version that wrote them (a failed retry + -- under a new version keeps the lines of the old one; the purge reads this). + point_count smallint NOT NULL DEFAULT 0 CHECK (point_count BETWEEN 0 AND 6), + lines_prompt_version text, + -- Lines the job dropped (evidence not found, too short, a key of another category...). + rejected_count smallint NOT NULL DEFAULT 0 CHECK (rejected_count BETWEEN 0 AND 20), + attempts smallint NOT NULL DEFAULT 1 CHECK (attempts BETWEEN 1 AND 3), + -- [{kind, criterion_key, position, evidence, match}] of the review's lines; match is + -- 'exact' or 'unaccented' (found only without diacritics, counted apart by the job). + evidence jsonb NOT NULL DEFAULT '[]'::jsonb CHECK (jsonb_typeof(evidence) = 'array'), + batch_id text REFERENCES rate.review_ai_batches(id) ON DELETE SET NULL, + processed_at timestamptz NOT NULL DEFAULT now(), + CONSTRAINT review_ai_extractions_stored CHECK (outcome <> 'stored' OR point_count > 0), + CONSTRAINT review_ai_extractions_failed CHECK (outcome = 'failed' OR attempts = 1), + CONSTRAINT review_ai_extractions_lines CHECK ((point_count > 0) = (lines_prompt_version IS NOT NULL)) +); +CREATE INDEX IF NOT EXISTS review_ai_extractions_version_idx + ON rate.review_ai_extractions (prompt_version); +CREATE INDEX IF NOT EXISTS review_ai_extractions_lines_version_idx + ON rate.review_ai_extractions (lines_prompt_version) WHERE lines_prompt_version IS NOT NULL; +CREATE INDEX IF NOT EXISTS review_ai_extractions_batch_idx + ON rate.review_ai_extractions (batch_id) WHERE batch_id IS NOT NULL; + +COMMENT ON TABLE rate.review_ai_extractions IS + 'Process-once marker of the review AI points job (DOS/Rate#1531, AI design D6). Written only by rate.store_review_ai_points and rate.purge_review_ai_points.'; + +ALTER TABLE rate.review_ai_extractions ENABLE ROW LEVEL SECURITY; +REVOKE ALL ON rate.review_ai_extractions FROM PUBLIC, anon, authenticated, rate_worker; +GRANT ALL ON rate.review_ai_extractions TO service_role; +GRANT SELECT ON rate.review_ai_extractions TO rate_worker; +DROP POLICY IF EXISTS "Service role access review_ai_extractions" ON rate.review_ai_extractions; +CREATE POLICY "Service role access review_ai_extractions" ON rate.review_ai_extractions + FOR ALL TO service_role USING (true) WITH CHECK (true); +DROP POLICY IF EXISTS "Rate worker read review_ai_extractions" ON rate.review_ai_extractions; +CREATE POLICY "Rate worker read review_ai_extractions" ON rate.review_ai_extractions + FOR SELECT TO rate_worker USING (true); + +-- 3. The text hash ----------------------------------------------------------------------------- +-- The review's comment as the job sent it: Unicode NFC, UTF-8, sha256, lower-case hex. The job +-- computes createHash('sha256').update(comment.normalize('NFC'), 'utf8'). + +CREATE OR REPLACE FUNCTION rate.review_ai_text_hash(p_text text) +RETURNS text +LANGUAGE sql +IMMUTABLE +PARALLEL SAFE +SET search_path = '' +AS $$ + SELECT pg_catalog.encode(pg_catalog.sha256( + pg_catalog.convert_to(normalize(COALESCE(p_text, ''), NFC), 'UTF8')), 'hex') +$$; + +REVOKE ALL ON FUNCTION rate.review_ai_text_hash(text) FROM PUBLIC, anon, authenticated; +GRANT EXECUTE ON FUNCTION rate.review_ai_text_hash(text) TO service_role, rate_worker; + +-- 4. The writer -------------------------------------------------------------------------------- +-- p_rows: a JSON array of at most 500 rows and 2 MB, each +-- {"id": review uuid, "text_hash": hex, "prompt_version": "...", "criteria_version": n, +-- "category": slug, "model": "...", "batch_id": "..." | null, "failed": true | false, +-- "rejected": 0-20 (optional), +-- "points": [{"kind": "pro" | "con", "criterion_key": key | null, "body": "...", +-- "evidence": "...", "match": "exact" | "unaccented"}]} +-- "failed" is required, so a cut-off or refused result can never pass as an empty one. +-- Checked for every row before anything is written; one malformed row fails the call (22023, +-- with its index): +-- the shape and types above; a failed row carries no points; +-- (category, criteria_version) a list in rate.rating_criteria, every criterion key in it; +-- at most 3 pros and 3 cons; each body 5 to 150 characters as stored (review_text_stored) +-- with words in it, a con not evasive, no line twice across pros and cons (compared form); +-- each evidence 2 to 300 characters as stored, at least two words. +-- A batch_id must be a row of rate.review_ai_batches (else 23503, from its foreign key). +-- Then, per review (a review listed twice: the last row wins), in id order, after the reviews +-- are locked FOR NO KEY UPDATE in id order (two calls never interleave on one review), each +-- row is one of: +-- 'missing' the review no longer exists; +-- 'not_crawl' it is not a crawled user review (origin 'crawl', reviewer_type 'user'); +-- 'writer' it has a writer line: writer lines are never mixed with 'ai'; +-- 'stale' its comment no longer hashes to text_hash, or its master is in another +-- category now: the job sends it again with what it is now; +-- 'unquoted' an 'exact' evidence quote is not in the comment (compared form, as P3 +-- compares lines): nothing is written, and the job reports it; +-- 'done' a failed row for a review whose marker already holds a result of the same +-- text, prompt version, criteria version and category: the result stays; +-- 'exhausted' a failed row for a review already failed 3 times with the same key: refused +-- as a fourth attempt; +-- 'failed' the marker says failed and counts the attempt (1 under a new key); the +-- review's lines and their evidence stay, with the version that wrote them; +-- 'stored', 'rejected', 'empty' the review's 'ai' lines are replaced by its points +-- (position 0 to 2 per kind), and the marker says which. +-- Returns the count of each. +CREATE OR REPLACE FUNCTION rate.store_review_ai_points(p_rows jsonb) +RETURNS jsonb +LANGUAGE plpgsql +SECURITY DEFINER +SET search_path = '' +AS $$ +DECLARE + v_row jsonb; + v_point jsonb; + v_index integer := 0; + v_pros integer; + v_cons integer; + v_seen text[]; + v_compared text; + v_body text; + v_len integer; + v_evidence text; + v_review record; + v_marker rate.review_ai_extractions%ROWTYPE; + v_same_key boolean; + v_outcome text; + v_count integer; + v_result jsonb := '{"stored": 0, "empty": 0, "rejected": 0, "failed": 0, "missing": 0, "not_crawl": 0, "writer": 0, "stale": 0, "unquoted": 0, "done": 0, "exhausted": 0}'::jsonb; +BEGIN + IF jsonb_typeof(p_rows) IS DISTINCT FROM 'array' THEN + RAISE EXCEPTION 'p_rows must be a JSON array' USING ERRCODE = '22023'; + END IF; + IF jsonb_array_length(p_rows) > 500 OR octet_length(p_rows::text) > 2097152 THEN + RAISE EXCEPTION 'at most 500 rows and 2 MB per call' USING ERRCODE = '22023'; + END IF; + + FOR v_row IN SELECT value FROM jsonb_array_elements(p_rows) LOOP + IF jsonb_typeof(v_row) IS DISTINCT FROM 'object' + OR v_row - ARRAY['id', 'text_hash', 'prompt_version', 'criteria_version', 'category', + 'model', 'batch_id', 'failed', 'rejected', 'points'] <> '{}'::jsonb + OR NOT v_row ?& ARRAY['id', 'text_hash', 'prompt_version', 'criteria_version', 'category', + 'model', 'failed', 'points'] + OR jsonb_typeof(v_row -> 'id') IS DISTINCT FROM 'string' + OR (v_row ->> 'id') !~ '^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$' + OR jsonb_typeof(v_row -> 'text_hash') IS DISTINCT FROM 'string' + OR (v_row ->> 'text_hash') !~ '^[0-9a-f]{64}$' + OR jsonb_typeof(v_row -> 'prompt_version') IS DISTINCT FROM 'string' + OR (v_row ->> 'prompt_version') !~ '^[a-z0-9][a-z0-9._-]{0,31}$' + OR v_row ->> 'prompt_version' = 'all' + OR jsonb_typeof(v_row -> 'criteria_version') IS DISTINCT FROM 'number' + OR (v_row ->> 'criteria_version') !~ '^[1-9][0-9]{0,5}$' + OR jsonb_typeof(v_row -> 'category') IS DISTINCT FROM 'string' + OR (v_row ->> 'category') !~ '^[a-z_-]{2,64}$' + OR jsonb_typeof(v_row -> 'model') IS DISTINCT FROM 'string' + OR (v_row ->> 'model') !~ '^[a-z0-9][a-z0-9.-]{1,63}$' + OR jsonb_typeof(v_row -> 'batch_id') NOT IN ('string', 'null') + OR (jsonb_typeof(v_row -> 'batch_id') = 'string' + AND (v_row ->> 'batch_id') !~ '^[A-Za-z0-9_-]{8,128}$') + OR jsonb_typeof(v_row -> 'failed') IS DISTINCT FROM 'boolean' + OR jsonb_typeof(v_row -> 'rejected') NOT IN ('number', 'null') + OR (jsonb_typeof(v_row -> 'rejected') = 'number' + AND (v_row ->> 'rejected') !~ '^([0-9]|1[0-9]|20)$') + OR jsonb_typeof(v_row -> 'points') IS DISTINCT FROM 'array' + OR jsonb_array_length(v_row -> 'points') > 6 + OR ((v_row ->> 'failed')::boolean AND jsonb_array_length(v_row -> 'points') > 0) THEN + RAISE EXCEPTION 'row % is not a review AI result', v_index USING ERRCODE = '22023'; + END IF; + IF NOT EXISTS (SELECT 1 FROM rate.rating_criteria + WHERE criteria_version = (v_row ->> 'criteria_version')::integer + AND category_slug = v_row ->> 'category') THEN + RAISE EXCEPTION 'row % has no criteria list %/%', v_index, v_row ->> 'category', + v_row ->> 'criteria_version' USING ERRCODE = '22023'; + END IF; + + v_pros := 0; + v_cons := 0; + v_seen := '{}'; + FOR v_point IN SELECT value FROM jsonb_array_elements(v_row -> 'points') LOOP + IF jsonb_typeof(v_point) IS DISTINCT FROM 'object' + OR v_point - ARRAY['kind', 'criterion_key', 'body', 'evidence', 'match'] <> '{}'::jsonb + OR NOT v_point ?& ARRAY['kind', 'criterion_key', 'body', 'evidence', 'match'] + OR jsonb_typeof(v_point -> 'kind') IS DISTINCT FROM 'string' + OR (v_point ->> 'kind') NOT IN ('pro', 'con') + OR jsonb_typeof(v_point -> 'criterion_key') NOT IN ('string', 'null') + OR jsonb_typeof(v_point -> 'body') IS DISTINCT FROM 'string' + OR jsonb_typeof(v_point -> 'evidence') IS DISTINCT FROM 'string' + OR jsonb_typeof(v_point -> 'match') IS DISTINCT FROM 'string' + OR (v_point ->> 'match') NOT IN ('exact', 'unaccented') THEN + RAISE EXCEPTION 'row % has a line that is not a review AI line', v_index + USING ERRCODE = '22023'; + END IF; + IF v_point ->> 'kind' = 'pro' THEN + v_pros := v_pros + 1; + ELSE + v_cons := v_cons + 1; + END IF; + v_body := rate.review_text_stored(v_point ->> 'body'); + v_len := rate.review_text_length(v_body); + v_evidence := rate.review_text_stored(v_point ->> 'evidence'); + v_compared := rate.review_point_compare(v_body); + IF v_pros > 3 OR v_cons > 3 + OR v_len < 5 OR v_len > 150 + OR NOT rate.review_point_has_content(v_body) + OR (v_point ->> 'kind' = 'con' AND rate.review_point_is_evasive_con(v_body)) + OR v_compared = ANY (v_seen) + OR rate.review_text_length(v_evidence) NOT BETWEEN 2 AND 300 + OR rate.review_point_compare(v_evidence) !~ ' ' + OR (jsonb_typeof(v_point -> 'criterion_key') = 'string' + AND NOT EXISTS (SELECT 1 FROM rate.rating_criteria + WHERE criteria_version = (v_row ->> 'criteria_version')::integer + AND category_slug = v_row ->> 'category' + AND key = v_point ->> 'criterion_key')) THEN + RAISE EXCEPTION 'row % has a line out of range', v_index USING ERRCODE = '22023'; + END IF; + v_seen := v_seen || v_compared; + END LOOP; + v_index := v_index + 1; + END LOOP; + + PERFORM 1 FROM rate.reviews r + WHERE r.id IN (SELECT (e ->> 'id')::uuid FROM jsonb_array_elements(p_rows) e) + ORDER BY r.id + FOR NO KEY UPDATE; + + FOR v_row IN + SELECT DISTINCT ON ((e ->> 'id')::uuid) e + FROM jsonb_array_elements(p_rows) WITH ORDINALITY AS a(e, n) + ORDER BY (e ->> 'id')::uuid, n DESC + LOOP + SELECT r.id, r.comment, r.origin, r.reviewer_type, c.slug AS category, + EXISTS (SELECT 1 FROM rate.review_points rp + WHERE rp.review_id = r.id AND rp.method = 'writer') AS has_writer + INTO v_review + FROM rate.reviews r + LEFT JOIN rate.products p ON p.id = r.product_id AND p.origin = 'crawl' + LEFT JOIN rate.master_products m ON m.id = p.cluster_id + LEFT JOIN rate.categories c ON c.id = m.category_id + WHERE r.id = (v_row ->> 'id')::uuid; + v_outcome := CASE + WHEN NOT FOUND THEN 'missing' + WHEN v_review.origin IS DISTINCT FROM 'crawl' + OR v_review.reviewer_type IS DISTINCT FROM 'user' THEN 'not_crawl' + WHEN v_review.has_writer THEN 'writer' + WHEN rate.review_ai_text_hash(v_review.comment) <> v_row ->> 'text_hash' + OR v_review.category IS DISTINCT FROM v_row ->> 'category' THEN 'stale' + WHEN EXISTS (SELECT 1 FROM jsonb_array_elements(v_row -> 'points') p + WHERE p ->> 'match' = 'exact' + AND strpos(rate.review_point_compare(v_review.comment), + rate.review_point_compare(p ->> 'evidence')) = 0) THEN 'unquoted' + END; + IF v_outcome IS NOT NULL THEN + v_result := jsonb_set(v_result, ARRAY[v_outcome], to_jsonb((v_result ->> v_outcome)::integer + 1)); + CONTINUE; + END IF; + + SELECT * INTO v_marker FROM rate.review_ai_extractions WHERE review_id = v_review.id; + v_same_key := FOUND + AND v_marker.text_hash = v_row ->> 'text_hash' + AND v_marker.prompt_version = v_row ->> 'prompt_version' + AND v_marker.criteria_version = (v_row ->> 'criteria_version')::integer + AND v_marker.category_slug = v_row ->> 'category'; + + IF (v_row ->> 'failed')::boolean THEN + v_outcome := CASE WHEN v_same_key AND v_marker.outcome <> 'failed' THEN 'done' + WHEN v_same_key AND v_marker.attempts >= 3 THEN 'exhausted' + ELSE 'failed' END; + IF v_outcome = 'failed' THEN + INSERT INTO rate.review_ai_extractions AS x ( + review_id, text_hash, prompt_version, criteria_version, category_slug, model, outcome, + point_count, lines_prompt_version, rejected_count, attempts, evidence, batch_id, + processed_at) + VALUES (v_review.id, v_row ->> 'text_hash', v_row ->> 'prompt_version', + (v_row ->> 'criteria_version')::integer, v_row ->> 'category', v_row ->> 'model', + 'failed', 0, NULL, 0, 1, '[]'::jsonb, v_row ->> 'batch_id', now()) + ON CONFLICT (review_id) DO UPDATE + SET attempts = CASE WHEN v_same_key THEN x.attempts + 1 ELSE 1 END, + text_hash = EXCLUDED.text_hash, + prompt_version = EXCLUDED.prompt_version, + criteria_version = EXCLUDED.criteria_version, + category_slug = EXCLUDED.category_slug, + model = EXCLUDED.model, + outcome = 'failed', + rejected_count = 0, + batch_id = EXCLUDED.batch_id, + processed_at = EXCLUDED.processed_at; + END IF; + v_result := jsonb_set(v_result, ARRAY[v_outcome], to_jsonb((v_result ->> v_outcome)::integer + 1)); + CONTINUE; + END IF; + + DELETE FROM rate.review_points WHERE review_id = v_review.id AND method = 'ai'; + INSERT INTO rate.review_points (review_id, criterion_key, kind, body, position, method) + SELECT v_review.id, p ->> 'criterion_key', p ->> 'kind', rate.review_text_stored(p ->> 'body'), + (row_number() OVER (PARTITION BY p ->> 'kind' ORDER BY n) - 1)::smallint, 'ai' + FROM jsonb_array_elements(v_row -> 'points') WITH ORDINALITY AS t(p, n); + GET DIAGNOSTICS v_count = ROW_COUNT; + v_outcome := CASE WHEN v_count > 0 THEN 'stored' + WHEN COALESCE((v_row ->> 'rejected')::integer, 0) > 0 THEN 'rejected' + ELSE 'empty' END; + INSERT INTO rate.review_ai_extractions AS x ( + review_id, text_hash, prompt_version, criteria_version, category_slug, model, outcome, + point_count, lines_prompt_version, rejected_count, attempts, evidence, batch_id, + processed_at) + VALUES (v_review.id, v_row ->> 'text_hash', v_row ->> 'prompt_version', + (v_row ->> 'criteria_version')::integer, v_row ->> 'category', v_row ->> 'model', + v_outcome, v_count, CASE WHEN v_count > 0 THEN v_row ->> 'prompt_version' END, + COALESCE((v_row ->> 'rejected')::integer, 0), 1, + COALESCE((SELECT jsonb_agg(s.line ORDER BY s.n) + FROM (SELECT n, jsonb_build_object( + 'kind', p ->> 'kind', 'criterion_key', p -> 'criterion_key', + 'position', row_number() OVER (PARTITION BY p ->> 'kind' ORDER BY n) - 1, + 'evidence', rate.review_text_stored(p ->> 'evidence'), + 'match', p ->> 'match') AS line + FROM jsonb_array_elements(v_row -> 'points') WITH ORDINALITY AS t(p, n)) s), + '[]'::jsonb), + v_row ->> 'batch_id', now()) + ON CONFLICT (review_id) DO UPDATE + SET text_hash = EXCLUDED.text_hash, + prompt_version = EXCLUDED.prompt_version, + criteria_version = EXCLUDED.criteria_version, + category_slug = EXCLUDED.category_slug, + model = EXCLUDED.model, + outcome = EXCLUDED.outcome, + point_count = EXCLUDED.point_count, + lines_prompt_version = EXCLUDED.lines_prompt_version, + rejected_count = EXCLUDED.rejected_count, + attempts = 1, + evidence = EXCLUDED.evidence, + batch_id = EXCLUDED.batch_id, + processed_at = EXCLUDED.processed_at; + v_result := jsonb_set(v_result, ARRAY[v_outcome], to_jsonb((v_result ->> v_outcome)::integer + 1)); + END LOOP; + + RETURN v_result; +END $$; + +REVOKE EXECUTE ON FUNCTION rate.store_review_ai_points(jsonb) FROM PUBLIC, anon, authenticated; +GRANT EXECUTE ON FUNCTION rate.store_review_ai_points(jsonb) TO service_role, rate_worker; + +-- 5. The purge --------------------------------------------------------------------------------- +-- 'all': every 'ai' line and every marker. A version: every marker of that version or holding +-- lines it wrote (lines_prompt_version, so lines kept through a failed retry under a later +-- version go too), with all the 'ai' lines of those reviews; they are processed again on the +-- next run. There is no default: a bare call is refused. The reviews concerned are locked in +-- id order first, as the writer locks them, so a purge and a store never leave a line without +-- its marker. Batches stay (they hold the month's spend). The next product-points run +-- aggregates without the lines; no writer line is touched. Returns {"lines": n, "markers": n}. +CREATE OR REPLACE FUNCTION rate.purge_review_ai_points(p_prompt_version text) +RETURNS jsonb +LANGUAGE plpgsql +SECURITY DEFINER +SET search_path = '' +AS $$ +DECLARE + v_all boolean := p_prompt_version = 'all'; + v_lines integer; + v_markers integer; +BEGIN + IF p_prompt_version IS NULL + OR (NOT v_all AND p_prompt_version !~ '^[a-z0-9][a-z0-9._-]{0,31}$') THEN + RAISE EXCEPTION 'p_prompt_version must be ''all'' or a prompt version' USING ERRCODE = '22023'; + END IF; + PERFORM 1 FROM rate.reviews r + WHERE r.id IN (SELECT x.review_id FROM rate.review_ai_extractions x + WHERE v_all OR x.prompt_version = p_prompt_version + OR x.lines_prompt_version = p_prompt_version + UNION + SELECT rp.review_id FROM rate.review_points rp WHERE v_all AND rp.method = 'ai') + ORDER BY r.id + FOR NO KEY UPDATE; + + DELETE FROM rate.review_points rp + WHERE rp.method = 'ai' + AND (v_all OR rp.review_id IN (SELECT x.review_id FROM rate.review_ai_extractions x + WHERE x.prompt_version = p_prompt_version + OR x.lines_prompt_version = p_prompt_version)); + GET DIAGNOSTICS v_lines = ROW_COUNT; + DELETE FROM rate.review_ai_extractions x + WHERE v_all OR x.prompt_version = p_prompt_version OR x.lines_prompt_version = p_prompt_version; + GET DIAGNOSTICS v_markers = ROW_COUNT; + RETURN jsonb_build_object('lines', v_lines, 'markers', v_markers); +END $$; + +REVOKE EXECUTE ON FUNCTION rate.purge_review_ai_points(text) FROM PUBLIC, anon, authenticated; +GRANT EXECUTE ON FUNCTION rate.purge_review_ai_points(text) TO service_role, rate_worker; + +COMMIT; diff --git a/apps/extension/CHROMEWEBSTORE.md b/apps/extension/CHROMEWEBSTORE.md index 6824333bb..5f79ca19b 100644 --- a/apps/extension/CHROMEWEBSTORE.md +++ b/apps/extension/CHROMEWEBSTORE.md @@ -1,6 +1,6 @@ # Chrome Web Store Listing & Compliance - Rate.vn Browser Extension (MV3) -- **Last Updated:** 2026-10-14 +- **Last Updated:** 2026-11-04 - **Current Version:** 2.0.0 - **Category:** Shopping @@ -18,7 +18,7 @@ Xem lịch sử giá 90 ngày, phát hiện giảm giá ảo, tìm nơi bán r ### Detailed Description -Rate.vn giúp người tiêu dùng mua sắm đồ công nghệ minh bạch và tiết kiệm hơn ngay khi đang xem sản phẩm trên Shopee, Lazada, Tiki, CellphoneS, Thế Giới Di Động và FPT Shop: +Rate.vn giúp người tiêu dùng mua sắm đồ công nghệ minh bạch và tiết kiệm hơn ngay khi đang xem sản phẩm trên Shopee, Lazada, Tiki, TikTok Shop, CellphoneS, Thế Giới Di Động, FPT Shop, Hoàng Hà Mobile, Di Động Việt và Phong Vũ: - Phát hiện Giảm Giá Ảo: Tự động đối chiếu mức giá hiện tại và mức giá gạch ngang với lịch sử giá đã xác minh để nhận diện khuyến mãi thật hay nâng giá gốc rồi giảm ảo. - So Sánh Giá Đa Sàn Tức Thì: Hiển thị ngay các gian hàng chính hãng khác đang bán cùng mẫu điện thoại, máy tính bảng, laptop với mức giá thấp hơn. @@ -32,23 +32,24 @@ Rate.vn giúp người tiêu dùng mua sắm đồ công nghệ minh bạch và This table lists exactly what `manifest.json` requests; `test-extension-pipeline.ts` fails when the two disagree (DOS/Rate#1231). -| Permission / Host Permission | Plain-English Justification | -| :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `storage` | Saves the user's preferred Rate.vn server endpoint, a temporary product lookup cache for the active session, and the user's explicit opt-in preference for contributing product price signals. | -| `sidePanel` | Displays the price history chart, cross-merchant price comparison table and AI review authenticity breakdown beside the shopping page when the user clicks the extension badge or floating pill. | -| `https://shopee.vn/*`, `https://*.lazada.vn/*`, `https://tiki.vn/*`, `https://www.tiktok.com/*`, `https://shop.tiktok.com/*`, `https://cellphones.com.vn/*`, `https://*.thegioididong.com/*`, `https://fptshop.com.vn/*`, `https://hoanghamobile.com/*`, `https://didongviet.vn/*`, `https://phongvu.vn/*` | Runs the content script on the 10 supported Vietnamese stores. It reads the public product title, selected variant, listed price, original price and rating of a product page to show the floating price-check badge. The script loads on every page of these sites because they navigate in-page, but it reads and sends nothing on pages it does not recognise as a product page. TikTok is limited to the two hosts its product pages use (DOS/Rate#1232). | -| `https://rate.vn/*`, `https://www.rate.vn/*` | Fetches verified price history, cross-merchant offers and review authenticity scores from the Rate.vn API, and sends the opt-in crowd signal. | +| Permission / Host Permission | Plain-English Justification | +| :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `storage` | Saves the user's preferred Rate.vn server endpoint, a temporary product lookup cache for the active session, and the user's explicit opt-in preference for contributing product price signals. | +| `sidePanel` | Displays the price history chart, cross-merchant price comparison table and AI review authenticity breakdown beside the shopping page when the user clicks the extension badge or floating pill. | +| `https://shopee.vn/*`, `https://*.lazada.vn/*`, `https://tiki.vn/*`, `https://www.tiktok.com/*`, `https://shop.tiktok.com/*`, `https://cellphones.com.vn/*`, `https://*.thegioididong.com/*`, `https://fptshop.com.vn/*`, `https://hoanghamobile.com/*`, `https://didongviet.vn/*`, `https://phongvu.vn/*` | Runs the content script on the 10 supported Vietnamese stores. It reads the public product title, selected variant, listed price, original price and rating of a product page to show the floating price-check badge. The script loads on every page of these sites because they navigate in-page, but it reads and sends nothing on pages it does not recognise as a product page. On `www.tiktok.com` it loads only on product page paths (`/view/product/`, `/@shop/product/`, `/shop/.../pdp/`), not on the video feed or video pages (DOS/Rate#1601). A product page reached by in-page navigation from the feed gets the badge after a reload; `shop.tiktok.com` serves only shop pages (DOS/Rate#1232). | +| `https://rate.vn/*`, `https://www.rate.vn/*` | Fetches verified price history, cross-merchant offers and review authenticity scores from the Rate.vn API, and sends the opt-in crowd signal. | --- ## 3. Privacy & User Data Disclosure -1. **Where it runs:** the content script loads on the 10 supported stores listed above (`shopee.vn`, `lazada.vn`, `tiki.vn`, `www.tiktok.com`, `shop.tiktok.com`, `cellphones.com.vn`, `thegioididong.com`, `fptshop.com.vn`, `hoanghamobile.com`, `didongviet.vn`, `phongvu.vn`). It acts only on pages it recognises as a product page; it never opens or fetches pages the user did not open. -2. **Lookup (on by default, no opt-in):** on a recognised product page the extension sends `GET /api/v1/rate/extension/lookup` with the store, the product URL with query parameters and tracking tokens (`utm_*`, `sp_atk`, `xptdk`, `fbclid`, `gclid`) stripped locally, the product title, the store's item id and shop id, the selected variant, the listed price and the original price. The request is read-only and carries no account or personal data. -3. **Opt-in crowd signal (off by default):** only when the user turns on "Đóng góp phát hiện sản phẩm mới (chỉ khi bạn bật)" in the Side Panel, the extension also sends `POST /api/v1/rate/extension/ingest` with the store, item id, shop id, cleaned URL, title, selected variant, price, original price, rating and rating count. The toggle's label lists these same fields (DOS/Rate#1230). Submissions go to a moderation queue and never change public prices directly. -4. **Stored in the browser only:** the server address and the opt-in choice (`chrome.storage.local`), and a session-only cache of the products viewed in each tab, the latest side panel search and their lookup results (`chrome.storage.session`). -5. **Side panel requests:** text the user types into the side panel search is sent to the same lookup endpoint; the side panel also reads public deals and merchant lists from the Rate.vn API. -6. **Privacy policy:** https://rate.vn/privacy-policy has a section on the extension. +1. **Where it runs:** the content script loads on the 10 supported stores listed above (`shopee.vn`, `lazada.vn`, `tiki.vn`, TikTok Shop product pages on `www.tiktok.com` and `shop.tiktok.com`, `cellphones.com.vn`, `thegioididong.com`, `fptshop.com.vn`, `hoanghamobile.com`, `didongviet.vn`, `phongvu.vn`). It acts only on pages it recognises as a product page. The extension itself never opens or fetches pages the user did not open, never works around a store's bot checks and never solves captchas. +2. **Lookup (on by default, no opt-in):** on a recognised product page the extension sends `GET /api/v1/rate/extension/lookup` with the store, the product URL with query parameters and tracking tokens (`utm_*`, `sp_atk`, `xptdk`, `fbclid`, `gclid`) stripped locally, the product title, the store's item id and shop id, the selected variant, the listed price and the original price. The request is read-only and carries no account, name, email or cookies. +3. **Opt-in crowd signal (off by default):** only when the user turns on "Đóng góp phát hiện sản phẩm mới (chỉ khi bạn bật)" (English: "Help find new products (only when you turn it on)") in the Side Panel, the extension also sends `POST /api/v1/rate/extension/ingest` with the store, item id, shop id, cleaned URL, title, selected variant, price, original price, rating and rating count. The toggle's label lists these same fields (DOS/Rate#1230). Submissions go to a moderation queue, stored with the time the server received them; Rate.vn's servers may re-check the listing on the store. They never change public prices directly. +4. **Stored in the browser only:** the server address and the opt-in choice (`chrome.storage.local`), and a session-only cache of the products viewed in each tab, the latest side panel search, their lookup results and a lookup wait time after a rate limit (`chrome.storage.session`). +5. **IP address:** like any web request, the Rate.vn server receives the user's IP address. The lookup and opt-in routes use it only for a per-minute request limit held in memory, and Rate.vn's code does not store it; the hosting provider keeps standard access logs for a short time. +6. **Side panel requests:** text the user types into the side panel search is sent to the same lookup endpoint; the side panel also reads public deals and merchant lists from the Rate.vn API. +7. **Privacy policy:** https://rate.vn/privacy-policy has a section on the extension. --- diff --git a/apps/extension/RELEASE.md b/apps/extension/RELEASE.md index 68468b0a6..3d41739bd 100644 --- a/apps/extension/RELEASE.md +++ b/apps/extension/RELEASE.md @@ -1,21 +1,21 @@ # Chrome Web Store release: checklist and changelog -Release-readiness check of the extension built from `dev` at `fc9b820d` on 2026-10-30. The check was read-only: nothing was published, and the live API was used with a few GET requests. +Release-readiness check of the extension built from `dev` at `fc9b820d` on 2026-10-30, with the TikTok content-script row updated on 2026-11-04 (#1601). The check was read-only: nothing was published, and the live API was used with a few GET requests. ## 1. Readiness check (2026-10-30) -| Check | Result | Evidence | -| :--------------------------------------------- | :------------------------ | :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| Build (`pnpm --filter @repo/extension build`) | Pass | `dist/` holds 11 files, 936 KB. The manifest is MV3, version `2.0.0`. | -| Extension tests, typecheck, lint | Pass | `test-extension-pipeline.ts` passes, including #1302, #1304 and #1326; `eslint --max-warnings 0` is clean. | -| Manifest permissions match `CHROMEWEBSTORE.md` | Pass | `storage` and `sidePanel`; 11 store host patterns (10 stores, TikTok on two hosts) plus `rate.vn` and `www.rate.vn`. The content-script matches equal the store hosts. The pipeline test enforces this (#1231). | -| `review_risk_score: null` shown as no data | Pass | The live lookup returns `null` for a seller with no analysed review, and the side panel shows `reviewRiskNoData` (#1266). | -| `dosafe_risk_score: null` | Fixed in #1465 (PR #1476) | A seller with no DOSafe score answers null from the API and reads "chưa đủ dữ liệu" in the side panel, never "0". | -| Verdict `messageKey` and the locale catalogue | Pass | All 10 server keys (`apps/web/src/server/rate/extension.ts`) map in `VERDICT_MESSAGES`. Every label and explanation exists in vi, en and cs with the right placeholder count. The live lookup returns `genuine_deal` and `fair_market` with their params. | -| Lookup ranking (#1379) | Pass | Live lookup of "Apple iPhone 16 128GB" matches `apple-iphone-16-128gb`, not the 16e. | -| 429 handling | Fixed in #1466 | After a 429 the extension sends no lookup until `Retry-After` (default 60 s, cap 600 s), the panel says how many seconds to wait, and the pill looks the product up again once the wait is over. | -| No locale sent | Pass | The lookup sends platform, URL, title, mode, ids, variant and prices, but no locale. The verdict is rendered from `messageKey` in the extension's language. | -| Version header | Fixed in #1466 | `X-Rate-Extension-Version` reads `chrome.runtime.getManifest().version`. | +| Check | Result | Evidence | +| :--------------------------------------------- | :------------------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| Build (`pnpm --filter @repo/extension build`) | Pass | `dist/` holds 11 files, 936 KB. The manifest is MV3, version `2.0.0`. | +| Extension tests, typecheck, lint | Pass | `test-extension-pipeline.ts` passes, including #1302, #1304 and #1326; `eslint --max-warnings 0` is clean. | +| Manifest permissions match `CHROMEWEBSTORE.md` | Pass | `storage` and `sidePanel`; 11 store host patterns (10 stores, TikTok on two hosts) plus `rate.vn` and `www.rate.vn`. The content-script matches cover the same origins; on `www.tiktok.com` they are limited to product page paths (#1601). The pipeline test enforces this (#1231). | +| `review_risk_score: null` shown as no data | Pass | The live lookup returns `null` for a seller with no analysed review, and the side panel shows `reviewRiskNoData` (#1266). | +| `dosafe_risk_score: null` | Fixed in #1465 (PR #1476) | A seller with no DOSafe score answers null from the API and reads "chưa đủ dữ liệu" in the side panel, never "0". | +| Verdict `messageKey` and the locale catalogue | Pass | All 10 server keys (`apps/web/src/server/rate/extension.ts`) map in `VERDICT_MESSAGES`. Every label and explanation exists in vi, en and cs with the right placeholder count. The live lookup returns `genuine_deal` and `fair_market` with their params. | +| Lookup ranking (#1379) | Pass | Live lookup of "Apple iPhone 16 128GB" matches `apple-iphone-16-128gb`, not the 16e. | +| 429 handling | Fixed in #1466 | After a 429 the extension sends no lookup until `Retry-After` (default 60 s, cap 600 s), the panel says how many seconds to wait, and the pill looks the product up again once the wait is over. | +| No locale sent | Pass | The lookup sends platform, URL, title, mode, ids, variant and prices, but no locale. The verdict is rendered from `messageKey` in the extension's language. | +| Version header | Fixed in #1466 | `X-Rate-Extension-Version` reads `chrome.runtime.getManifest().version`. | ## 2. Release checklist (JOY publishes) @@ -55,4 +55,4 @@ Release-readiness check of the extension built from `dev` at `fc9b820d` on 2026- - The toolbar badge shows at a glance whether the price you see is a good one. Sold-out listings no longer count towards the price comparison. - More accurate product matching: "iPhone 16" no longer matches the iPhone 16e. Category and brand pages are no longer mistaken for product pages. - Sellers with no analysed reviews show "No data yet" instead of a risk number. -- The "Help detect new products" option names exactly what it sends and is still off by default. On TikTok, the extension runs only on product pages. +- The "Help find new products" option names exactly what it sends and is still off by default. On TikTok, the extension runs only on product pages. diff --git a/apps/extension/manifest.json b/apps/extension/manifest.json index 47399927f..3b0c863dd 100644 --- a/apps/extension/manifest.json +++ b/apps/extension/manifest.json @@ -47,7 +47,9 @@ "https://shopee.vn/*", "https://*.lazada.vn/*", "https://tiki.vn/*", - "https://www.tiktok.com/*", + "https://www.tiktok.com/view/product/*", + "https://www.tiktok.com/@*/product/*", + "https://www.tiktok.com/shop/*pdp/*", "https://shop.tiktok.com/*", "https://cellphones.com.vn/*", "https://*.thegioididong.com/*", @@ -56,6 +58,7 @@ "https://didongviet.vn/*", "https://phongvu.vn/*" ], + "exclude_matches": ["https://www.tiktok.com/@*/video/*"], "js": ["content/content-script.js"], "run_at": "document_idle" } diff --git a/apps/extension/src/_locales/cs/messages.json b/apps/extension/src/_locales/cs/messages.json index fb8eb7a79..6d2564ce0 100644 --- a/apps/extension/src/_locales/cs/messages.json +++ b/apps/extension/src/_locales/cs/messages.json @@ -53,7 +53,7 @@ "message": "Pomoc s hledáním nových produktů (jen když ji zapnete):" }, "optInBody": { - "message": "Když je zapnuto, každá nabídka, kterou otevřete, se také odešle do nezávislé fronty ke kontrole Rate.vn s těmito údaji: obchod, ID produktu, ID obchodu, URL bez sledovacích parametrů, název produktu, zvolená varianta, cena, původní cena, počet hvězdiček a počet recenzí. Bez účtu a osobních údajů." + "message": "Když je zapnuto, každá nabídka, kterou otevřete, se také odešle do nezávislé fronty ke kontrole Rate.vn s těmito údaji: obchod, ID produktu, ID obchodu, URL bez sledovacích parametrů, název produktu, zvolená varianta, cena, původní cena, počet hvězdiček a počet recenzí. Bez účtu, jména, e-mailu a cookies." }, "crowdFieldPlatform": { "message": "obchod" diff --git a/apps/extension/src/_locales/en/messages.json b/apps/extension/src/_locales/en/messages.json index a524177e7..f69af4fcd 100644 --- a/apps/extension/src/_locales/en/messages.json +++ b/apps/extension/src/_locales/en/messages.json @@ -53,7 +53,7 @@ "message": "Help find new products (only when you turn it on):" }, "optInBody": { - "message": "When on, every listing you open is also sent to the independent Rate.vn review queue with: store, product ID, shop ID, URL with tracking parameters removed, product name, selected variant, price, original price, star rating and number of reviews. No account or personal data is included." + "message": "When on, every listing you open is also sent to the independent Rate.vn review queue with: store, product ID, shop ID, URL with tracking parameters removed, product name, selected variant, price, original price, star rating and number of reviews. No account, name, email or cookies are included." }, "crowdFieldPlatform": { "message": "store" diff --git a/apps/extension/src/_locales/vi/messages.json b/apps/extension/src/_locales/vi/messages.json index dcde804d6..188394535 100644 --- a/apps/extension/src/_locales/vi/messages.json +++ b/apps/extension/src/_locales/vi/messages.json @@ -53,7 +53,7 @@ "message": "Đóng góp phát hiện sản phẩm mới (chỉ khi bạn bật):" }, "optInBody": { - "message": "Khi bật, mỗi tin bán bạn mở còn được gửi vào hàng đợi kiểm duyệt độc lập của Rate.vn với: sàn, mã sản phẩm, mã gian hàng, URL đã cắt tham số theo dõi, tên sản phẩm, phiên bản đang chọn, giá bán, giá gốc, số sao và số lượt đánh giá. Không kèm tài khoản hay thông tin cá nhân." + "message": "Khi bật, mỗi tin bán bạn mở còn được gửi vào hàng đợi kiểm duyệt độc lập của Rate.vn với: sàn, mã sản phẩm, mã gian hàng, URL đã cắt tham số theo dõi, tên sản phẩm, phiên bản đang chọn, giá bán, giá gốc, số sao và số lượt đánh giá. Không kèm tài khoản, tên, email hay cookie." }, "crowdFieldPlatform": { "message": "sàn" diff --git a/apps/extension/src/test-extension-pipeline.ts b/apps/extension/src/test-extension-pipeline.ts index 1267c51bb..c641cfe40 100644 --- a/apps/extension/src/test-extension-pipeline.ts +++ b/apps/extension/src/test-extension-pipeline.ts @@ -485,7 +485,10 @@ async function runTests() { ) as { permissions?: string[]; host_permissions?: string[]; - content_scripts?: Array<{ matches?: string[] }>; + content_scripts?: Array<{ + matches?: string[]; + exclude_matches?: string[]; + }>; }; assert.deepEqual( manifestJson.permissions, @@ -499,22 +502,31 @@ async function runTests() { for (const pattern of [...hosts, ...matches]) { assert.ok(!pattern.includes('*.tiktok.com'), pattern); } - const matchesPattern = (pattern: string, url: string) => - new RegExp( + // Chrome matches a pattern's path against the URL's path and query, + // never its fragment. + const matchesPattern = (pattern: string, url: string) => { + const u = new URL(url); + return new RegExp( `^${pattern .replace(/[.+?^${}()|[\]\\]/g, '\\$&') .replace('://*\\.', '://(?:[^/]+\\.)?') .replace(/\*/g, '.*')}$` - ).test(url); + ).test(`${u.origin}${u.pathname}${u.search}`); + }; + const excludes = manifestJson.content_scripts?.[0]?.exclude_matches ?? []; + const injectedOn = (url: string) => + matches.some((p) => matchesPattern(p, url)) && + !excludes.some((p) => matchesPattern(p, url)); for (const url of [ 'https://www.tiktok.com/@appleflagship/product/1729384756', 'https://www.tiktok.com/view/product/1729384756', + 'https://www.tiktok.com/shop/pdp/iphone-16/1729384756', + 'https://www.tiktok.com/shop/vn/pdp/iphone-16/1729384756?src=x', 'https://shop.tiktok.com/view/product/1729401928401', ]) { - assert.ok( - matches.some((p) => matchesPattern(p, url)), - `content script must run on ${url}` - ); + assert.ok(injectedOn(url), `content script must run on ${url}`); + // ...and the extractor reads these as products, so the two agree. + assert.ok(extractIdsFromUrl('tiktok', url).externalItemId, url); assert.ok( hosts.some((p) => matchesPattern(p, url)), url @@ -539,8 +551,23 @@ async function runTests() { [...(manifestJson.permissions ?? []), ...hosts].sort(), 'CHROMEWEBSTORE.md permission table must match manifest.json' ); + // On www.tiktok.com the content script runs only on product pages, not + // on the video feed (DOS/Rate#1601); host permissions are per origin. + for (const url of [ + 'https://www.tiktok.com/', + 'https://www.tiktok.com/foryou', + 'https://www.tiktok.com/@appleflagship/video/7291234567890', + 'https://www.tiktok.com/search?q=iphone', + 'https://www.tiktok.com/@appleflagship', + 'https://www.tiktok.com/@appleflagship/video/7291234567890?ref=/product/1', + 'https://www.tiktok.com/foryou#/view/product/1', + ]) { + assert.ok(!injectedOn(url), `content script must not run on ${url}`); + } + const originOf = (pattern: string) => + pattern.replace(/^(https:\/\/[^/]+)\/.*$/, '$1/*'); assert.deepEqual( - [...matches].sort(), + [...new Set(matches.map(originOf))].sort(), hosts.filter((h) => !/rate\.vn/.test(h)).sort() ); diff --git a/apps/importer/package.json b/apps/importer/package.json index 4885316b0..fe8fa907a 100644 --- a/apps/importer/package.json +++ b/apps/importer/package.json @@ -37,11 +37,12 @@ "lint": "eslint . --max-warnings 0", "format": "prettier --write \"src/**/*.ts\"", "backfill:specs": "tsx src/shopee/jobs/backfill-specs.ts", - "test": "tsx src/shopee/__tests__/open-core-intelligence.test.ts && tsx src/shopee/__tests__/sourced-seeds.test.ts && tsx src/shopee/__tests__/entity-matcher.test.ts && tsx src/shopee/__tests__/item-matcher.test.ts && tsx src/shopee/__tests__/shopee-to-supabase.test.ts && tsx src/shopee/__tests__/paging.test.ts && tsx src/shopee/__tests__/backfill-review-risk.test.ts && tsx src/shopee/__tests__/item-inventory-report.test.ts && tsx src/shopee/__tests__/arena-audit.test.ts && tsx src/shopee/__tests__/item-aware-jobs.test.ts && tsx src/shopee/__tests__/product-points.test.ts && tsx src/shopee/__tests__/review-values.test.ts && tsx src/shopee/test-ai-enricher.ts && tsx src/shopee/test-review-analyzer.ts", + "test": "tsx src/shopee/__tests__/open-core-intelligence.test.ts && tsx src/shopee/__tests__/sourced-seeds.test.ts && tsx src/shopee/__tests__/entity-matcher.test.ts && tsx src/shopee/__tests__/item-matcher.test.ts && tsx src/shopee/__tests__/shopee-to-supabase.test.ts && tsx src/shopee/__tests__/paging.test.ts && tsx src/shopee/__tests__/backfill-review-risk.test.ts && tsx src/shopee/__tests__/item-inventory-report.test.ts && tsx src/shopee/__tests__/arena-audit.test.ts && tsx src/shopee/__tests__/item-aware-jobs.test.ts && tsx src/shopee/__tests__/product-points.test.ts && tsx src/shopee/__tests__/review-values.test.ts && tsx src/shopee/__tests__/review-ai-points.test.ts && tsx src/shopee/test-ai-enricher.ts && tsx src/shopee/test-review-analyzer.ts", "test:dist": "node src/shopee/__tests__/dist-exports.smoke.mjs", "backfill:review-risk": "tsx src/shopee/jobs/backfill-review-risk.ts", "product-points": "tsx src/shopee/jobs/product-points.ts", - "review-values": "tsx src/shopee/jobs/review-values.ts" + "review-values": "tsx src/shopee/jobs/review-values.ts", + "review-ai-points": "tsx src/shopee/jobs/review-ai-points.ts" }, "keywords": [ "product-normalizer", @@ -57,13 +58,14 @@ "node": ">=22.0.0" }, "dependencies": { + "@anthropic-ai/sdk": "1.3.0", "@repo/shared-data": "workspace:*", "@supabase/supabase-js": "^2.117.2", "dotenv": "^18.0.8" }, "devDependencies": { - "@types/node": "^26.6.5", "@repo/eslint-config": "workspace:*", + "@types/node": "^26.6.5", "eslint": "^9.39.5", "prettier": "^3.9.6", "tsx": "^4.23.15", diff --git a/apps/importer/src/shopee/__tests__/product-points.test.ts b/apps/importer/src/shopee/__tests__/product-points.test.ts index 300628437..e3e4f799d 100644 --- a/apps/importer/src/shopee/__tests__/product-points.test.ts +++ b/apps/importer/src/shopee/__tests__/product-points.test.ts @@ -1,6 +1,7 @@ import assert from 'node:assert/strict'; import { buildProductPoints, + hideAiLinesFrom, reviewPointWeight, runProductPointsJob, type PointReviewRow, @@ -164,7 +165,10 @@ async function run() { }); const dry = await runProductPointsJob(fake.client, { now: NOW }); assert.deepEqual(dry, { apply: false, masters: 3, points: 3, inserted: 0, deleted: 0 }); - assert.ok(fake.filters.includes('review_points:method=writer'), "only the writers' lines"); + assert.ok( + fake.filters.includes('review_points:method in writer,ai'), + "the writers' lines and AI's" + ); assert.equal(fake.writes.length, 0, 'a dry run writes nothing'); const applied = await runProductPointsJob(fake.client, { apply: true, now: NOW }); @@ -192,7 +196,86 @@ async function run() { ); await assert.rejects(runProductPointsJob(one.client, { masterId: 'not-a-uuid' }), /UUID/); + // Lines AI extracted from marketplace reviews (DOS/Rate#1531): each review is + // one writer at 0.5 x min(weight, 1), so three agreeing reviews show a point + // and two do not; --hide-ai leaves them all out. + const aiReviews = [ + review('m1', 'l-quiet', { helpfulness_weight: 3 }), + review('m2', 'l-quiet', { helpfulness_weight: 3 }), + review('m3', 'l-quiet'), + ]; + const aiPoint = (review_id: string, body: string, method = 'ai'): ReviewPointRow => ({ + ...point(review_id, 'pro', body), + method, + }); + const aiInput = (lines: ReviewPointRow[]) => ({ + points: lines, + reviews: aiReviews, + listings, + masters, + categories, + }); + const screen = 'Màn hình sáng rõ ngoài trời'; + const twoAi = buildProductPoints(aiInput([aiPoint('m1', screen), aiPoint('m2', screen)]), NOW); + assert.deepEqual(twoAi.get(QUIET), [], 'two AI lines (0.5 + 0.5) do not show'); + const threeAi = buildProductPoints( + aiInput([aiPoint('m1', screen), aiPoint('m2', screen), aiPoint('m3', screen)]), + NOW + ); + assert.deepEqual( + threeAi.get(QUIET)!.map(p => [p.label, p.support, p.reviewer_count, p.has_expert]), + [[screen, 1.5, 3, false]] + ); + assert.deepEqual( + buildProductPoints( + aiInput([aiPoint('m1', screen), aiPoint('m2', screen), aiPoint('m3', screen)]), + NOW, + { hideAiLines: true } + ).get(QUIET), + undefined, + 'hidden AI lines are not read at all' + ); + // A line of another method (a reworded label, say) is not a review line. + assert.deepEqual( + buildProductPoints( + aiInput([ + aiPoint('m1', screen, 'code'), + aiPoint('m2', screen, 'code'), + aiPoint('m3', screen, 'code'), + ]), + NOW + ).get(QUIET), + undefined + ); + // The switch: --hide-ai or PRODUCT_POINTS_HIDE_AI. + assert.equal(hideAiLinesFrom(['node', 'job', '--hide-ai'], {}), true); + assert.equal(hideAiLinesFrom(['node', 'job'], { PRODUCT_POINTS_HIDE_AI: 'true' }), true); + assert.equal(hideAiLinesFrom(['node', 'job'], { PRODUCT_POINTS_HIDE_AI: ' 1 ' }), true); + assert.equal(hideAiLinesFrom(['node', 'job'], { PRODUCT_POINTS_HIDE_AI: 'false' }), false); + assert.equal(hideAiLinesFrom(['node', 'job'], {}), false); + assert.equal(hideAiLinesFrom(['node', 'job', '--hide-ai=yes'], {}), true); + assert.equal(hideAiLinesFrom(['node', 'job', '--hide-ai=false'], {}), false); + assert.equal(hideAiLinesFrom(['node', 'job'], { PRODUCT_POINTS_HIDE_AI: 'on' }), true); + // A value it does not know fails, never shows lines meant to be hidden. + assert.throws( + () => hideAiLinesFrom(['node', 'job'], { PRODUCT_POINTS_HIDE_AI: 'ye' }), + /true or false/ + ); + assert.throws(() => hideAiLinesFrom(['node', 'job', '--hide-ai=maybe'], {}), /true or false/); + // The job reads only the writers' lines when AI lines are hidden. + const hiddenRun = fakeSupabase({ + review_points: points, + reviews, + products: listings, + master_products: masters, + categories, + product_points: [], + }); + await runProductPointsJob(hiddenRun.client, { now: NOW, hideAiLines: true }); + assert.ok(hiddenRun.filters.includes('review_points:method in writer')); + console.log(' ✅ weights, aggregation per master and category, dry run and apply verified'); + console.log(' ✅ AI lines at half weight, and the switch that hides them (#1531)'); } type Row = Record; @@ -216,7 +299,9 @@ function fakeSupabase(tables: Record) { return builder; }, in: (column: string, values: unknown[]) => { - rows = rows.filter(r => values.includes(r[column])); + filters.push(`${table}:${column} in ${values.join(',')}`); + // As eq: fixture rows without the column pass. + rows = rows.filter(r => r[column] === undefined || values.includes(r[column])); return builder; }, range: (from: number, to: number) => diff --git a/apps/importer/src/shopee/__tests__/review-ai-points.test.ts b/apps/importer/src/shopee/__tests__/review-ai-points.test.ts new file mode 100644 index 000000000..04c896aa7 --- /dev/null +++ b/apps/importer/src/shopee/__tests__/review-ai-points.test.ts @@ -0,0 +1,863 @@ +import assert from 'node:assert/strict'; +import { createHash } from 'node:crypto'; +import { comparablePoint, storedPoint } from '@repo/shared-data/rules'; +import { + batchCostUsd, + checkReviewPoints, + evidenceMatch, + reserveUsd, + reviewAiInstructions, + reviewAiOutputSchema, + reviewLanguage, + reviewTextHash, + REVIEW_AI_PROMPT_VERSION, + textQualifies, +} from '../review-ai-prompt.js'; +import { + aiCandidates, + buildRequests, + collectBatch, + qualityStop, + runReviewAiPointsJob, + type AiMarkerRow, + type AiReviewRow, + type AiStoreRow, + type BatchRequest, + type BatchResult, + type ReviewAiClient, +} from '../jobs/review-ai-points.js'; +import type { RateSupabaseClient } from '../types.js'; + +const PHONES = 'c0000000-0000-4000-8000-000000000001'; +const SERVICES = 'c0000000-0000-4000-8000-000000000002'; +const M1 = 'a0000000-0000-4000-8000-000000000001'; +const M2 = 'a0000000-0000-4000-8000-000000000002'; +const id = (n: number) => `d0000000-0000-4000-8000-${String(n).padStart(12, '0')}`; + +const LONG = + 'Pin xài cả ngày vẫn còn 30%, ra nắng hơi khó nhìn. Camera chụp đêm ổn, shop giao nhanh.'; +const review = (n: number, extra: Partial = {}): AiReviewRow => ({ + id: id(n), + product_id: 'l1', + origin: 'crawl', + reviewer_type: 'user', + helpfulness_weight: 1, + comment: LONG, + review_flags: { include_in_rating: true, verdict: 'authentic', flag_category: null }, + ...extra, +}); +const categories = [ + { id: PHONES, slug: 'smartphones' }, + { id: SERVICES, slug: 'services' }, +]; +const masters = [ + { id: M1, category_id: PHONES }, + { id: M2, category_id: SERVICES }, +]; +const listings = [ + { id: 'l1', cluster_id: M1, origin: 'crawl' }, + { id: 'l2', cluster_id: M2, origin: 'crawl' }, + { id: 'lseed', cluster_id: M1, origin: 'seed' }, +]; + +function testPrompt() { + // The hash is the database's: sha256 of the NFC text, decomposed or not. + assert.equal(reviewTextHash(LONG.normalize('NFD')), reviewTextHash(LONG)); + assert.equal( + reviewTextHash(''), + 'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855' + ); + assert.equal( + reviewTextHash(LONG), + createHash('sha256').update(LONG.normalize('NFC'), 'utf8').digest('hex') + ); + + // D3: length, words and language. + assert.equal(textQualifies(LONG), true); + assert.equal(textQualifies('Hàng đẹp, giao nhanh'), false, 'too short'); + assert.equal( + textQualifies('😍😍😍😍😍😍😍😍😍😍😍😍😍😍😍😍😍😍😍😍 tốt tốt tốt'), + false, + 'emoji do not count' + ); + assert.equal( + textQualifies('đẹppppppppppppppppppppppppppppppppppppppppppp lắm'), + false, + 'repeats do not count' + ); + assert.equal( + textQualifies('pin xai ca ngay van con, may chay muot khong lag, man hinh dep'), + true, + 'Vietnamese without diacritics' + ); + assert.equal( + textQualifies('good phone for the price, the battery works all day but the screen is dim'), + true + ); + assert.equal(reviewLanguage('这个手机很好用电池也很耐用屏幕很漂亮'), null); + assert.equal(reviewLanguage('Pin trâu'), 'vi'); + + // The schema limits keys to the category's list; services are not open. + const schema = JSON.stringify(reviewAiOutputSchema('smartphones')); + assert.ok(schema.includes('"battery"') && !schema.includes('"keyboard"')); + assert.ok(reviewAiInstructions('laptops').includes('keyboard: Bàn phím và touchpad')); + + // Evidence: in its own review, exact or only without diacritics; two words at least. + assert.equal(evidenceMatch(LONG, 'Pin XÀI cả ngày'), 'exact'); + assert.equal(evidenceMatch('pin xai ca ngay van con', 'pin xài cả ngày'), 'unaccented'); + assert.equal(evidenceMatch('Đẹp lắm, dùng ổn', 'dep lam'), 'unaccented'); + assert.equal(evidenceMatch(LONG, 'pin trâu lắm'), null); + assert.equal(evidenceMatch(LONG, 'pin'), null, 'one word'); + // Words as the writer counts them: punctuation is not a word, and is not a mismatch. + assert.equal(evidenceMatch('Máy đẹp :) dùng ổn', 'đẹp :)'), null); + assert.equal(evidenceMatch(LONG, 'ra nắng, hơi khó nhìn!'), 'exact'); + + const checked = checkReviewPoints(LONG, 'smartphones', [ + { + kind: 'pro', + criterion_key: 'battery', + body: 'Pin dùng hết ngày', + evidence: 'pin xài cả ngày vẫn còn', + }, + { + kind: 'pro', + criterion_key: 'battery', + body: 'Pin dùng hết ngày', + evidence: 'pin xài cả ngày', + }, + { + kind: 'con', + criterion_key: 'display', + body: 'Màn khó nhìn ngoài nắng', + evidence: 'ra nắng hơi khó nhìn', + }, + { + kind: 'con', + criterion_key: 'keyboard', + body: 'Bàn phím cứng', + evidence: 'ra nắng hơi khó nhìn', + }, + { kind: 'con', criterion_key: null, body: 'Không có gì để chê', evidence: 'ra nắng hơi' }, + { + kind: 'pro', + criterion_key: 'camera', + body: 'Chụp đêm rất đẹp', + evidence: 'chụp đêm cực đẹp', + }, + { kind: 'pro', criterion_key: 'camera', body: 'Tốt', evidence: 'chụp đêm ổn' }, + { kind: 'neutral', criterion_key: null, body: 'Giao nhanh', evidence: 'giao nhanh' }, + { kind: 'pro', criterion_key: 'camera', body: 'Chụp đêm ổn', evidence: 'Camera chụp đêm ổn' }, + { kind: 'pro', criterion_key: null, body: 'Dùng ổn định', evidence: 'chụp đêm ổn' }, + { kind: 'pro', criterion_key: null, body: 'Máy đẹp sang', evidence: 'Camera chụp' }, + ]); + assert.deepEqual( + checked.points.map(p => `${p.kind}:${p.criterion_key}:${p.body}`), + [ + 'pro:battery:Pin dùng hết ngày', + 'con:display:Màn khó nhìn ngoài nắng', + 'pro:camera:Chụp đêm ổn', + 'pro:null:Dùng ổn định', + ] + ); + assert.deepEqual(checked.rejected, { + shape: 1, + evidence: 1, + length: 1, + criterion: 1, + evasive: 1, + duplicate: 1, + too_many: 1, + }); + + // Prices: Haiku 5.5 at batch prices, cache reads and writes included. + assert.equal( + batchCostUsd('claude-haiku-5-5', { + input_tokens: 1_000_000, + output_tokens: 1_000_000, + cache_read_input_tokens: 1_000_000, + cache_creation_input_tokens: 1_000_000, + }).toFixed(4), + ((0.1 + 0.5 + 0.01 + 0.125) * 0.5).toFixed(4) + ); + assert.equal( + batchCostUsd('claude-sonnet-5-5', { output_tokens: 1_000_000 }), + 20 * batchCostUsd('claude-haiku-5-5', { output_tokens: 1_000_000 }) + ); + const ceiling = reserveUsd('claude-haiku-5-5', [ + { instructions: 'x'.repeat(3000), message: 'y'.repeat(8000) }, + ]); + assert.ok(ceiling > 0.0015 && ceiling < 0.003, `one request's ceiling ${ceiling}`); + console.log(' ✅ hash, qualifying text, schema, evidence, line checks and prices verified'); +} + +function testCandidates() { + const reviews = [ + review(1), + review(2, { origin: 'community' }), + review(3, { reviewer_type: 'expert' }), + review(4, { review_flags: null }), + review(5, { + review_flags: { include_in_rating: false, verdict: 'suspicious', flag_category: null }, + }), + review(6, { helpfulness_weight: 0 }), + review(7, { comment: 'Hàng đẹp, giao nhanh' }), + review(8, { product_id: 'l2' }), + review(9, { product_id: 'lseed' }), + review(10), + review(11), + review(12), + review(13), + review(14), + review(15), + review(16, { comment: LONG.normalize('NFD') }), + ]; + const hash = reviewTextHash(LONG); + const marker = (n: number, extra: Partial = {}): AiMarkerRow => ({ + review_id: id(n), + text_hash: hash, + prompt_version: REVIEW_AI_PROMPT_VERSION, + criteria_version: 1, + category_slug: 'smartphones', + outcome: 'stored', + attempts: 1, + batch_id: null, + ...extra, + }); + const due = aiCandidates({ + reviews, + writerReviewIds: new Set([id(10)]), + markers: [ + marker(11), + marker(12, { outcome: 'failed', attempts: 2 }), + marker(13, { outcome: 'failed', attempts: 3 }), + marker(14, { prompt_version: 'v0' }), + marker(15, { category_slug: 'tablets' }), + marker(16, { outcome: 'empty' }), + ], + listings, + masters, + categories, + pending: new Set([id(1)]), + }); + assert.deepEqual( + due.map(c => c.id), + [12, 14, 15].map(id), + 'failed with retries left, a new prompt version, a moved master' + ); + assert.equal(due[0]!.text, LONG); + + const many = Array.from({ length: 45 }, (_, i) => ({ + id: id(100 + i), + text: LONG, + textHash: hash, + category: i < 41 ? 'smartphones' : 'laptops', + })); + const built = buildRequests(many, 'claude-haiku-5-5'); + assert.deepEqual( + built.manifest.map(r => `${r.custom_id}:${r.reviews.length}`), + ['laptops-0:4', 'smartphones-1:20', 'smartphones-2:20', 'smartphones-3:1'] + ); + const request = built.requests[1]!; + assert.equal(request.params.model, 'claude-haiku-5-5'); + assert.equal(request.params.output_config.effort, 'low'); + assert.equal(request.params.system[0]!.cache_control.type, 'ephemeral'); + const sent = JSON.parse(request.params.messages[0]!.content.split('\n').slice(1).join('\n')) as { + ref: number; + text: string; + }[]; + assert.deepEqual( + sent.map(r => r.ref), + Array.from({ length: 20 }, (_, i) => i + 1) + ); + assert.ok(!request.params.messages[0]!.content.includes(id(100)), 'no review id is sent'); + console.log(' ✅ the gate, the process-once rule and the requests verified'); +} + +const answer = ( + reviews: unknown, + stop = 'end_turn', + usage = { input_tokens: 4000, output_tokens: 900 } +) => ({ + result: { + type: 'succeeded' as const, + message: { + stop_reason: stop, + content: [{ type: 'text', text: JSON.stringify({ reviews }) }], + usage, + }, + }, +}); +const GOOD_POINTS = [ + { + kind: 'pro', + criterion_key: 'battery', + body: 'Pin dùng hết ngày', + evidence: 'pin xài cả ngày vẫn còn', + }, + { + kind: 'con', + criterion_key: 'display', + body: 'Màn khó nhìn ngoài nắng', + evidence: 'ra nắng hơi khó nhìn', + }, +]; + +function testCollect() { + const manifest = [ + { + custom_id: 'smartphones-0', + category: 'smartphones', + criteria_version: 1, + reviews: [1, 2, 3].map(n => ({ ref: n, id: id(n), text_hash: 'h' + n })), + }, + { + custom_id: 'smartphones-1', + category: 'smartphones', + criteria_version: 1, + reviews: [{ ref: 1, id: id(4), text_hash: 'h4' }], + }, + { + custom_id: 'smartphones-2', + category: 'smartphones', + criteria_version: 1, + reviews: [{ ref: 1, id: id(5), text_hash: 'h5' }], + }, + { + custom_id: 'smartphones-3', + category: 'smartphones', + criteria_version: 1, + reviews: [{ ref: 1, id: id(6), text_hash: 'h6' }], + }, + { + custom_id: 'smartphones-4', + category: 'smartphones', + criteria_version: 1, + reviews: [{ ref: 1, id: id(7), text_hash: 'h7' }], + }, + ]; + const results: BatchResult[] = [ + // Review 3 left out by the model: failed, never empty. + { + custom_id: 'smartphones-0', + ...answer([ + { ref: 1, points: GOOD_POINTS }, + { ref: 2, points: [] }, + ]), + }, + { custom_id: 'smartphones-1', ...answer([{ ref: 1, points: [] }], 'max_tokens') }, + { custom_id: 'smartphones-2', ...answer([{ ref: 1, points: [] }], 'refusal') }, + { custom_id: 'smartphones-3', ...answer([{ ref: 1, points: 'none' }]) }, + // smartphones-4 has no result at all. + ]; + const texts = new Map([1, 2, 3, 4, 5, 6, 7].map(n => [id(n), LONG])); + const collected = collectBatch( + { id: 'msgbatch_test0001', model: 'claude-haiku-5-5', prompt_version: 'v1', manifest }, + results, + texts + ); + const rows = new Map(collected.rows.map(r => [r.id, r])); + assert.equal(rows.get(id(1))!.points.length, 2); + assert.equal(rows.get(id(1))!.failed, false); + assert.deepEqual( + [rows.get(id(2))!.failed, rows.get(id(2))!.points.length], + [false, 0], + 'an empty answer' + ); + for (const n of [3, 4, 5, 6, 7]) + assert.equal(rows.get(id(n))!.failed, true, `review ${n} failed`); + assert.equal(collected.failedReviews, 5); + assert.equal(rows.get(id(1))!.batch_id, 'msgbatch_test0001'); + assert.equal(collected.usage.input_tokens, 16000, 'cut-off and refused requests are paid for'); + assert.equal( + collected.costUsd, + batchCostUsd('claude-haiku-5-5', { input_tokens: 16000, output_tokens: 3600 }) + ); + const errored = collectBatch( + { + id: 'msgbatch_test0002', + model: 'claude-haiku-5-5', + prompt_version: 'v1', + manifest: manifest.slice(1, 2), + }, + [{ custom_id: 'smartphones-1', result: { type: 'errored' } }], + texts + ); + assert.deepEqual( + [errored.failedReviews, errored.costUsd], + [1, 0], + 'an errored request is failed and free' + ); + + // Quality stops. + const lines = (evidence: number, total: number) => ({ + ...collected, + lines: total, + rejects: { ...collected.rejects, evidence }, + }); + assert.equal(qualityStop(lines(2, 20), null), null); + assert.match(qualityStop(lines(3, 20), null) ?? '', /3 of 20 lines failed the evidence check/); + assert.equal(qualityStop(lines(3, 19), null), null, 'too few lines to judge'); + const share = (withLines: number, answered: number) => ({ + ...collected, + reviewsWithLines: withLines, + reviewsAnswered: answered, + }); + assert.match( + qualityStop(share(10, 100), { share: 0.6, reviews: 80 }) ?? '', + /moved from 0.60 to 0.10/ + ); + assert.equal(qualityStop(share(50, 100), { share: 0.6, reviews: 80 }), null); + assert.equal(qualityStop(share(10, 100), { share: 0.6, reviews: 20 }), null); + console.log(' ✅ results read, failures kept apart, cost and quality stops verified'); +} + +async function testJob() { + const reviews = [1, 2, 3].map(n => review(n)); + const seed = () => ({ + reviews: reviews.map(r => ({ ...r })), + review_points: [] as Row[], + review_ai_extractions: [] as Row[], + review_ai_batches: [] as Row[], + products: listings, + master_products: masters, + categories, + }); + const now = new Date('2026-11-20T00:00:00Z'); + + // Dry run: plans, never calls the model, writes nothing. + const dry = fakeSupabase(seed()); + const ai = fakeAi(requests => + requests.map(r => ({ + custom_id: r.custom_id, + ...answer(refsOf(r).map(ref => ({ ref, points: GOOD_POINTS }))), + })) + ); + const planned = await runReviewAiPointsJob(dry.client, { enabled: true, ai: ai.client, now }); + assert.equal(planned.candidates, 3); + assert.equal(planned.requests, 1); + assert.equal(ai.calls.length, 0, 'a dry run calls no model'); + assert.equal(dry.writes.length, 0); + const noKey = await runReviewAiPointsJob(dry.client, { enabled: true, now }); + assert.equal(noKey.candidates, 3, 'a dry run needs no key'); + + // Disabled: no model call, even with --apply. + const off = await runReviewAiPointsJob(dry.client, { + apply: true, + enabled: false, + ai: ai.client, + now, + }); + assert.equal(off.candidates, 0); + assert.equal(ai.calls.length, 0); + + // Apply: one batch, recorded before its results are read, then stored. + const db = fakeSupabase(seed()); + ai.onCreate = () => + assert.equal(db.tables.review_ai_batches!.length, 0, 'created before the row'); + ai.onResults = () => + assert.equal( + db.tables.review_ai_batches![0]!.status, + 'submitted', + 'the row exists before the results are read' + ); + const applied = await runReviewAiPointsJob(db.client, { + apply: true, + enabled: true, + ai: ai.client, + now, + wait: async () => {}, + }); + ai.onCreate = null; + ai.onResults = null; + assert.equal(applied.sent, 3); + assert.deepEqual(ai.calls, ['create', 'status', 'results']); + assert.equal(applied.stored.stored, 3); + const batch = db.tables.review_ai_batches![0]!; + assert.equal(batch.status, 'stored'); + assert.equal( + Number(batch.cost_usd), + Number(batchCostUsd('claude-haiku-5-5', { input_tokens: 4000, output_tokens: 900 }).toFixed(4)) + ); + assert.equal(db.stores.flat().length, 3); + assert.ok( + db.stores + .flat() + .every(r => r.batch_id === batch.id && r.prompt_version === REVIEW_AI_PROMPT_VERSION) + ); + + // The next run: nothing is due. + ai.calls.length = 0; + const again = await runReviewAiPointsJob(db.client, { + apply: true, + enabled: true, + ai: ai.client, + now, + }); + assert.equal(again.candidates, 0); + assert.equal(ai.calls.length, 0); + + // A crash after the batch was created: the next run collects it without sending again. + const crashed = fakeSupabase(seed()); + crashed.tables.review_ai_batches!.push({ + id: 'msgbatch_crashed01', + model: 'claude-haiku-5-5', + prompt_version: REVIEW_AI_PROMPT_VERSION, + status: 'submitted', + manifest: [ + { + custom_id: 'smartphones-0', + category: 'smartphones', + criteria_version: 1, + reviews: reviews.map((r, i) => ({ ref: i + 1, id: r.id, text_hash: reviewTextHash(LONG) })), + }, + ], + reserved_usd: 0.01, + cost_usd: null, + submitted_at: now.toISOString(), + stored_at: null, + }); + ai.calls.length = 0; + ai.batches.set('msgbatch_crashed01', [ + { custom_id: 'smartphones-0', ...answer([1, 2, 3].map(ref => ({ ref, points: GOOD_POINTS }))) }, + ]); + // A dry run while it is open does not count its reviews as due. + const dryOpen = await runReviewAiPointsJob(crashed.client, { enabled: true, now }); + assert.equal(dryOpen.candidates, 0); + // The crash came after review 1 was stored from this batch: it is not stored twice. + crashed.tables.review_ai_extractions!.push({ + review_id: id(1), + text_hash: reviewTextHash(LONG), + prompt_version: REVIEW_AI_PROMPT_VERSION, + criteria_version: 1, + category_slug: 'smartphones', + outcome: 'stored', + attempts: 1, + batch_id: 'msgbatch_crashed01', + }); + const resumed = await runReviewAiPointsJob(crashed.client, { + apply: true, + enabled: true, + ai: ai.client, + now, + }); + assert.deepEqual(resumed.collected, ['msgbatch_crashed01']); + assert.equal(resumed.sent, 0, 'nothing sent twice'); + assert.ok(!ai.calls.includes('create')); + assert.deepEqual( + crashed.stores.flat().map(r => r.id), + [id(2), id(3)], + 'rows already stored from this batch are skipped' + ); + + // A batch still running is left for the next run, and its reviews are not sent again. + const running = fakeSupabase(seed()); + running.tables.review_ai_batches!.push({ + ...crashed.tables.review_ai_batches![0]!, + id: 'msgbatch_running01', + status: 'submitted', + cost_usd: null, + }); + ai.calls.length = 0; + ai.status = 'in_progress'; + const waited = await runReviewAiPointsJob(running.client, { + apply: true, + enabled: true, + ai: ai.client, + now, + waitMs: 0, + wait: async () => {}, + }); + assert.deepEqual(waited.waiting, ['msgbatch_running01']); + assert.equal(waited.candidates, 0); + assert.ok(!ai.calls.includes('create')); + // ...and the kill switch cancels it. + const stopped = await runReviewAiPointsJob(running.client, { + apply: true, + enabled: false, + ai: ai.client, + now, + }); + assert.deepEqual(stopped.canceled, ['msgbatch_running01']); + assert.ok(ai.calls.includes('cancel')); + assert.equal( + running.tables.review_ai_batches![0]!.status, + 'submitted', + 'the row stays open: what finished before the cancel is paid for and collected later' + ); + // Switched on again: the canceled batch ends; its finished request is stored, the + // canceled one comes back as failed rows, and nothing is sent twice. + ai.status = 'ended'; + ai.batches.set('msgbatch_running01', [ + { custom_id: 'smartphones-0', result: { type: 'canceled' } }, + ]); + ai.calls.length = 0; + const back = await runReviewAiPointsJob(running.client, { + apply: true, + enabled: true, + ai: ai.client, + now, + }); + assert.equal(back.collected[0], 'msgbatch_running01', 'the old batch first, then the new one'); + assert.equal(back.stored.failed, 3); + // The fake writer has no attempts; the failed reviews are due again, in a new batch. + assert.equal(back.sent, 3); + + // The monthly cap: this month's spend plus the ceiling over $5 sends nothing. + const capped = fakeSupabase(seed()); + capped.tables.review_ai_batches!.push({ + id: 'msgbatch_spent0001', + model: 'claude-haiku-5-5', + prompt_version: 'v0', + status: 'stored', + manifest: [], + reserved_usd: 9, + cost_usd: 4.999, + submitted_at: '2026-11-02T00:00:00Z', + stored_at: '2026-11-02T01:00:00Z', + }); + capped.tables.review_ai_batches!.push({ + id: 'msgbatch_lastmonth', + model: 'claude-haiku-5-5', + prompt_version: 'v0', + status: 'stored', + manifest: [], + reserved_usd: 9, + cost_usd: 50, + submitted_at: '2026-10-31T23:00:00Z', + stored_at: '2026-11-01T01:00:00Z', + }); + ai.calls.length = 0; + const over = await runReviewAiPointsJob(capped.client, { + apply: true, + enabled: true, + ai: ai.client, + now, + }); + assert.equal(over.capped, true); + assert.equal(over.monthSpendUsd, 4.999, 'last month does not count'); + assert.equal(ai.calls.length, 0); + const sonnet = await runReviewAiPointsJob(capped.client, { + enabled: true, + model: 'claude-sonnet-5-5', + now, + }); + assert.equal(sonnet.capped, false, 'each model has its own cap'); + + // A quality stop: most quotes made up; nothing stored, the batch left ended. + const bad = fakeSupabase(seed()); + bad.tables.reviews = Array.from({ length: 10 }, (_, i) => ({ ...review(200 + i) }) as Row); + const madeUp = Array.from({ length: 3 }, (_, i) => ({ + kind: 'pro', + criterion_key: null, + body: `Ý bịa số ${i + 1}`, + evidence: `không có trong review ${i}`, + })); + const liar = fakeAi(requests => + requests.map(r => ({ + custom_id: r.custom_id, + ...answer(refsOf(r).map(ref => ({ ref, points: madeUp }))), + })) + ); + await assert.rejects( + runReviewAiPointsJob(bad.client, { apply: true, enabled: true, ai: liar.client, now }), + /not stored: 30 of 30 lines failed the evidence check/ + ); + assert.equal(bad.stores.length, 0); + assert.equal(bad.tables.review_ai_batches![0]!.status, 'ended'); + + // The batch row cannot be written: the batch is canceled at once, not left running unrecorded. + const lost = fakeSupabase(seed()); + lost.state.failInsert = true; + ai.calls.length = 0; + await assert.rejects( + runReviewAiPointsJob(lost.client, { apply: true, enabled: true, ai: ai.client, now }), + /insert refused\. The batch was canceled\./ + ); + assert.deepEqual(ai.calls, ['create', 'cancel']); + await assert.rejects( + runReviewAiPointsJob(dry.client, { enabled: true, now, waitMs: Number.NaN }), + /--wait-minutes/ + ); + + // The purge: a dry run says so; --apply calls the writer; a bad version is refused. + const purge = fakeSupabase(seed()); + const purgeDry = await runReviewAiPointsJob(purge.client, { purge: 'v1' }); + assert.equal(purgeDry.purged, null); + assert.equal(purge.rpcs.length, 0); + const purged = await runReviewAiPointsJob(purge.client, { apply: true, purge: 'all' }); + assert.deepEqual(purged.purged, { lines: 0, markers: 0 }); + assert.deepEqual(purge.rpcs, [['purge_review_ai_points', { p_prompt_version: 'all' }]]); + await assert.rejects( + runReviewAiPointsJob(purge.client, { apply: true, purge: 'V 1' }), + /'all' or a prompt version/ + ); + await assert.rejects(runReviewAiPointsJob(dry.client, { maxReviews: 0 }), /--max/); + console.log( + ' ✅ dry run, kill switch, apply, resume, waiting, cap, quality stop and purge verified' + ); +} + +function refsOf(request: BatchRequest): number[] { + const content = request.params.messages[0]!.content; + return (JSON.parse(content.slice(content.indexOf('\n') + 1)) as { ref: number }[]).map( + r => r.ref + ); +} + +function fakeAi(answerFor: (requests: BatchRequest[]) => BatchResult[]) { + const state = { + calls: [] as string[], + batches: new Map(), + status: 'ended' as 'in_progress' | 'ended', + onCreate: null as (() => void) | null, + onResults: null as (() => void) | null, + }; + let n = 0; + const client: ReviewAiClient = { + async createBatch(requests) { + state.calls.push('create'); + state.onCreate?.(); + const batchId = `msgbatch_fake${String((n += 1)).padStart(4, '0')}`; + state.batches.set(batchId, answerFor(requests)); + return { id: batchId }; + }, + async batchStatus() { + state.calls.push('status'); + return state.status; + }, + async *results(batchId) { + state.calls.push('results'); + state.onResults?.(); + for (const entry of state.batches.get(batchId) ?? []) yield entry; + }, + async cancelBatch() { + state.calls.push('cancel'); + }, + }; + return Object.assign(state, { client }); +} + +type Row = Record; + +/** The query builder calls the job makes, the batch writes, and the two RPCs. */ +function fakeSupabase(seed: Record) { + const tables: Record = Object.fromEntries( + Object.entries(seed).map(([name, rows]) => [name, rows.map(r => ({ ...(r as Row) }))]) + ); + const stores: AiStoreRow[][] = []; + const writes: string[] = []; + const rpcs: [string, unknown][] = []; + const state = { failInsert: false }; + const query = (table: string) => { + let rows = [...(tables[table] ?? [])]; + const builder = { + select: () => builder, + eq: (column: string, value: unknown) => { + rows = rows.filter(r => r[column] === value); + return builder; + }, + in: (column: string, values: unknown[]) => { + rows = rows.filter(r => values.includes(r[column])); + return builder; + }, + gt: (column: string, value: string) => { + rows = rows.filter(r => String(r[column]) > value); + return builder; + }, + order: (key: string) => { + rows.sort((a, b) => (String(a[key]) < String(b[key]) ? -1 : 1)); + return builder; + }, + limit: (n: number) => + Promise.resolve({ + data: rows.slice(0, Math.min(n, 1000)).map(r => ({ ...r })), + error: null, + }), + insert: (row: Row) => { + writes.push(`insert ${table}`); + if (state.failInsert) return Promise.resolve({ error: { message: 'insert refused' } }); + tables[table]!.push(JSON.parse(JSON.stringify(row)) as Row); + return Promise.resolve({ error: null }); + }, + // RLS: rate_worker updates a batch only while it is not stored (no error, no row). + update: (patch: Row) => ({ + eq: (column: string, value: unknown) => { + writes.push(`update ${table}`); + for (const r of tables[table]!) + if (r[column] === value && !(table === 'review_ai_batches' && r.status === 'stored')) + Object.assign(r, patch); + return Promise.resolve({ error: null }); + }, + }), + }; + return builder; + }; + const client = { + from: (table: string) => query(table), + rpc: (name: string, args: { p_rows?: AiStoreRow[]; p_prompt_version?: string }) => { + rpcs.push([name, args]); + if (name === 'purge_review_ai_points') + return Promise.resolve({ data: { lines: 0, markers: 0 }, error: null }); + assert.equal(name, 'store_review_ai_points'); + const rows = args.p_rows!; + // The writer's own checks (rate.store_review_ai_points): a boolean failed flag, and per + // line 5 to 150 characters and two words of evidence in the compared form. One bad row + // refuses the call. + const bad = rows.findIndex( + row => + typeof row.failed !== 'boolean' || + row.points.some(p => { + const body = [...storedPoint(p.body)].length; + return ( + body < 5 || body > 150 || !comparablePoint(storedPoint(p.evidence)).includes(' ') + ); + }) + ); + if (bad >= 0) + return Promise.resolve({ + data: null, + error: { code: '22023', message: `row ${bad} refused` }, + }); + stores.push(rows); + const counts: Record = { stored: 0, empty: 0, rejected: 0, failed: 0 }; + for (const row of rows) { + const outcome = row.failed + ? 'failed' + : row.points.length > 0 + ? 'stored' + : row.rejected > 0 + ? 'rejected' + : 'empty'; + counts[outcome] = (counts[outcome] ?? 0) + 1; + const markers = tables.review_ai_extractions!; + const old = markers.findIndex(m => m.review_id === row.id); + if (old >= 0) markers.splice(old, 1); + markers.push({ + review_id: row.id, + text_hash: row.text_hash, + prompt_version: row.prompt_version, + criteria_version: row.criteria_version, + category_slug: row.category, + outcome, + attempts: 1, + batch_id: row.batch_id, + }); + } + return Promise.resolve({ data: counts, error: null }); + }, + }; + return { client: client as unknown as RateSupabaseClient, tables, stores, writes, rpcs, state }; +} + +async function run() { + console.log('Review AI points job'); + testPrompt(); + testCandidates(); + testCollect(); + await testJob(); +} + +run().catch(err => { + console.error(err); + process.exit(1); +}); diff --git a/apps/importer/src/shopee/jobs/product-points.ts b/apps/importer/src/shopee/jobs/product-points.ts index b02a7f172..f1ff67427 100644 --- a/apps/importer/src/shopee/jobs/product-points.ts +++ b/apps/importer/src/shopee/jobs/product-points.ts @@ -2,7 +2,7 @@ * Product pros and cons writer (DOS/Rate#1546, design #1531 section 5, D9) * * Usage: - * pnpm --filter @repo/crawlers exec tsx src/shopee/jobs/product-points.ts [--apply] [--master ] + * pnpm --filter @repo/crawlers exec tsx src/shopee/jobs/product-points.ts [--apply] [--master ] [--hide-ai] * * Reads every review line (rate.review_points) with its review, flag, listing * and category, clusters the lines of each master with @@ -15,8 +15,12 @@ * product_points. * * Which lines count, as the P2 rating counts reviews: lines the writer wrote - * (method 'writer'), on crawled listings (origin 'crawl'), of crawled or - * community reviews; seed, synthetic, fixture and backfill rows never show. + * (method 'writer') and lines AI extracted from marketplace reviews (method + * 'ai', at half weight: POINT_AI_WEIGHT_SHARE, DOS/Rate#1531), on crawled + * listings (origin 'crawl'), of crawled or community reviews; seed, + * synthetic, fixture and backfill rows never show. `--hide-ai`, or + * PRODUCT_POINTS_HIDE_AI=true, leaves every 'ai' line out: the next run + * writes the points without them, and nothing is deleted from review_points. * A review is one writer: a Rate user writes one review per master. Two * reviews of one person on one master (after the matcher moves a listing) or * several marketplace reviews of one buyer count as several writers; the @@ -27,11 +31,13 @@ * * Per master, new rows are inserted before the old 'code' rows are deleted, so * a reader never sees a master with no points while the job runs. A master - * whose lines no longer show anything has its 'code' rows removed. Rows of - * another method ('ai', the rewording step of design 5.2) are left alone. + * whose lines no longer show anything has its 'code' rows removed. + * product_points rows of another method (product_points.method 'ai', the + * rewording step of design 5.2, not the review_points 'ai' lines above) are + * left alone. * * Not here yet: review_points.cluster_id (rate_worker has no UPDATE on - * review_points) and lines extracted by AI from marketplace reviews. + * review_points). */ import { pathToFileURL } from 'node:url'; import dotenv from 'dotenv'; @@ -51,6 +57,34 @@ export interface ReviewPointRow { criterion_key: string | null; kind: string; body: string; + /** 'writer', or 'ai' for a line extracted from a marketplace review. */ + method?: string | null; +} + +const SWITCH_ON = new Set(['true', '1', 'yes', 'on']); +const SWITCH_OFF = new Set(['', 'false', '0', 'no', 'off']); + +/** + * Whether `--hide-ai` (or `--hide-ai=`) or PRODUCT_POINTS_HIDE_AI asks + * to leave AI lines out. A value it does not know throws, so a typo never + * shows lines someone meant to hide. + */ +export function hideAiLinesFrom( + argv: readonly string[], + env: Record +): boolean { + const read = (raw: string, from: string): boolean => { + const value = raw.trim().toLowerCase(); + if (SWITCH_ON.has(value)) return true; + if (SWITCH_OFF.has(value)) return false; + throw new Error(`${from} must be true or false, not ${JSON.stringify(raw)}.`); + }; + let hide = read(env.PRODUCT_POINTS_HIDE_AI ?? '', 'PRODUCT_POINTS_HIDE_AI'); + for (const arg of argv) { + if (arg === '--hide-ai') hide = true; + else if (arg.startsWith('--hide-ai=')) hide = read(arg.slice('--hide-ai='.length), '--hide-ai'); + } + return hide; } export interface PointReviewRow { @@ -140,8 +174,10 @@ export function buildProductPoints( masters: readonly PointMasterRow[]; categories: readonly CategoryRow[]; }, - now: Date + now: Date, + options: { hideAiLines?: boolean } = {} ): Map { + const hideAiLines = options.hideAiLines === true; const reviewById = new Map(input.reviews.map(r => [r.id, r])); const masterByListing = new Map( input.listings @@ -163,6 +199,10 @@ export function buildProductPoints( const masterId = masterByListing.get(review.product_id); if (!masterId || !categoryByMaster.has(masterId)) continue; if (point.kind !== 'pro' && point.kind !== 'con') continue; + // Only the writer's lines and AI's; any other method is not a review line. + const method = point.method ?? 'writer'; + if (method !== 'writer' && method !== 'ai') continue; + if (hideAiLines && method === 'ai') continue; const lines = linesByMaster.get(masterId) ?? []; lines.push({ id: point.id, @@ -174,6 +214,7 @@ export function buildProductPoints( weight: reviewPointWeight(review), isExpert: review.reviewer_type === 'expert', writtenAt: review.platform_created_at ?? review.created_at ?? '', + method, }); linesByMaster.set(masterId, lines); } @@ -181,7 +222,9 @@ export function buildProductPoints( const out = new Map(); const stamp = now.toISOString(); for (const [masterId, lines] of linesByMaster) { - const result = aggregateProductPoints(categoryByMaster.get(masterId), lines, now); + const result = aggregateProductPoints(categoryByMaster.get(masterId), lines, now, { + hideAiLines, + }); // What a page shows: the top points of each criterion and the summary. const shown = new Map(); for (const sides of [...Object.values(result.byCriterion), result.summary]) @@ -232,7 +275,7 @@ const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i; export async function runProductPointsJob( supabase: RateSupabaseClient, - options: { apply?: boolean; masterId?: string | null; now?: Date } = {} + options: { apply?: boolean; masterId?: string | null; now?: Date; hideAiLines?: boolean } = {} ): Promise<{ apply: boolean; masters: number; @@ -248,8 +291,8 @@ export async function runProductPointsJob( const points = await fetchAllPages('review_points', (from, to) => supabase .from('review_points') - .select('id, review_id, criterion_key, kind, body') - .eq('method', 'writer') + .select('id, review_id, criterion_key, kind, body, method') + .in('method', options.hideAiLines === true ? ['writer'] : ['writer', 'ai']) .order('id') .range(from, to) ); @@ -295,7 +338,9 @@ export async function runProductPointsJob( supabase.from('categories').select('id, slug').order('id').range(from, to) ); - const built = buildProductPoints({ points, reviews, listings, masters, categories }, now); + const built = buildProductPoints({ points, reviews, listings, masters, categories }, now, { + hideAiLines: options.hideAiLines === true, + }); for (const id of masterIds) if (!built.has(id)) built.set(id, []); const oldIdsByMaster = new Map(); for (const row of existing) { @@ -382,11 +427,14 @@ if (isDirectExecution) { const apply = process.argv.includes('--apply'); const at = process.argv.indexOf('--master'); const masterId = at >= 0 ? (process.argv[at + 1] ?? '') : null; + const hideAiLines = hideAiLinesFrom(process.argv, process.env); // The host is not secret; it shows which project the job reads. - logger.info('product_points.target', `Target: ${host} (${apply ? 'apply' : 'dry run'})`, { - host, - }); - await runProductPointsJob(supabase, { apply, masterId }); + logger.info( + 'product_points.target', + `Target: ${host} (${apply ? 'apply' : 'dry run'}${hideAiLines ? ', AI lines hidden' : ''})`, + { host, hideAiLines } + ); + await runProductPointsJob(supabase, { apply, masterId, hideAiLines }); })().catch((err: unknown) => { logger.error( 'product_points.fatal', diff --git a/apps/importer/src/shopee/jobs/review-ai-points.ts b/apps/importer/src/shopee/jobs/review-ai-points.ts new file mode 100644 index 000000000..a04677923 --- /dev/null +++ b/apps/importer/src/shopee/jobs/review-ai-points.ts @@ -0,0 +1,1088 @@ +/** + * Review AI points job (DOS/Rate#1531, Docs/Proposals/2026-11-03-ai-review-points.html) + * + * Usage: + * pnpm --filter @repo/crawlers exec tsx src/shopee/jobs/review-ai-points.ts + * [--apply] [--purge ] [--max ] [--wait-minutes ] + * + * Reads crawled user reviews, picks those that qualify (design section 1, D3), + * sends them through the Message Batches API with a structured output schema + * per category (section 2), checks every line it gets back (section 7.2) and + * stores what passes as rate.review_points method 'ai' through + * rate.store_review_ai_points, which writes the lines and the process-once + * marker (rate.review_ai_extractions) in one transaction. + * + * Dry run by default: it reads, plans and logs counts and the cost ceiling, + * and never calls a model (no API key needed). `--apply` sends one batch, + * waits for it (at most --wait-minutes, default 60) and stores the results. + * It signs in as the rate_worker user (RATE_WORKER_EMAIL and + * RATE_WORKER_PASSWORD with the publishable key), which reads the review + * tables, writes the batch record and writes lines only through the writer. + * + * Switches and limits (section 8, D5): + * - REVIEW_AI_POINTS_ENABLED must be 'true', or the job exits before any + * model call; with --apply it also cancels a batch still running. + * - REVIEW_AI_MODEL: claude-haiku-5-5 (default) or claude-sonnet-5-5 (D1). + * - At most 2000 reviews a run (--max lowers it). + * - A monthly cap per model ($5 Haiku, $15 Sonnet). The month's spend is the + * reported cost of its batches, or their reserved ceiling while running, + * read from rate.review_ai_batches. A batch whose ceiling would pass the + * cap is not sent, and the CLI exits with code 3. + * - `--purge all` or `--purge ` removes 'ai' lines and markers + * (dry run: it only says so). The next product-points run aggregates + * without them. + * - product-points has its own switch to hide 'ai' lines without a purge. + * + * Process once (section 4): a review is due when it has no marker, when its + * text hash, the prompt version, the criteria version or its category differ + * from its marker's, or when its marker says failed with fewer than 3 + * attempts. A batch's id and manifest are written to rate.review_ai_batches + * right after it is created, so a run that dies collects it the next time + * instead of sending the reviews again. + * + * Quality stops (section 7.2): a batch whose lines fail the evidence check + * more than 10% of the time, or whose share of reviews with lines moves more + * than 30 points from the last stored batch, is not stored and the run fails. + * The batch stays 'ended': after a look, service_role marks it 'canceled' to + * drop it, or a fixed job stores it on the next run. + * + * Logs hold counts only: never review text, quotes or lines. Run it from one + * place at a time (the Rate-Services schedule), after the crawl and before + * product-points and review-values. + */ +import { pathToFileURL } from 'node:url'; +import dotenv from 'dotenv'; +import { reviewDepth } from '@repo/shared-data/rules'; +import { resolveBackfillSupabaseTarget } from '../paging.js'; +import { createImporterLogger } from '../logger.js'; +import { scopeSupabaseToRate, type RateSupabaseClient } from '../types.js'; +import { reviewPointWeight, type CategoryRow, type PointMasterRow } from './product-points.js'; +import { fetchByKey } from './review-values.js'; +import { + aiCategoryOpen, + batchCostUsd, + checkReviewPoints, + DEFAULT_REVIEW_AI_MODEL, + isReviewAiModel, + RATING_CRITERIA_VERSION, + REQUEST_MAX_TOKENS, + reserveUsd, + REVIEW_AI_MODELS, + REVIEW_AI_PROMPT_VERSION, + reviewAiInstructions, + reviewAiOutputSchema, + reviewAiUserMessage, + REVIEWS_PER_REQUEST, + reviewTextHash, + textQualifies, + type AiPoint, + type AiRejectReason, + type ReviewAiModel, +} from '../review-ai-prompt.js'; + +const logger = createImporterLogger('ReviewAiPointsJob'); + +/** Reviews a run sends at most (D5). */ +export const MAX_REVIEWS_PER_RUN = 2000; +/** Attempts a failed result gets in all (the writer refuses a fourth). */ +export const MAX_ATTEMPTS = 3; +/** Rows per writer call; the writer takes 500. */ +export const STORE_ROWS = 200; +/** Stop when more than this share of lines fail the evidence check. */ +export const EVIDENCE_STOP_SHARE = 0.1; +/** ...counted once a batch returns at least this many lines. */ +export const EVIDENCE_STOP_MIN_LINES = 20; +/** Stop when the share of reviews with lines moves more than this from the last batch. */ +export const SHARE_JUMP_STOP = 0.3; +/** ...when both batches have at least this many reviews. */ +export const SHARE_JUMP_MIN_REVIEWS = 50; +const DEADLOCK_ATTEMPTS = 4; +/** The Batch API keeps results 29 days; past 28 a batch is treated as gone. */ +const RESULTS_KEPT_MS = 28 * 24 * 60 * 60 * 1000; +const PROMPT_VERSION_RE = /^[a-z0-9][a-z0-9._-]{0,31}$/; + +// --- The model client: the Batch API as the job uses it, so tests pass a fake. ------------ + +export interface BatchRequest { + custom_id: string; + params: { + model: ReviewAiModel; + max_tokens: number; + system: { type: 'text'; text: string; cache_control: { type: 'ephemeral' } }[]; + messages: { role: 'user'; content: string }[]; + output_config: { + effort: 'low'; + format: { type: 'json_schema'; schema: Record }; + }; + }; +} + +export interface BatchUsage { + input_tokens?: number | null; + output_tokens?: number | null; + cache_read_input_tokens?: number | null; + cache_creation_input_tokens?: number | null; +} + +export type BatchResult = + | { + custom_id: string; + result: { + type: 'succeeded'; + message: { + stop_reason: string | null; + content: { type: string; text?: string }[]; + usage: BatchUsage; + }; + }; + } + | { custom_id: string; result: { type: 'errored' | 'expired' | 'canceled' } }; + +export interface ReviewAiClient { + createBatch(requests: BatchRequest[]): Promise<{ id: string }>; + batchStatus(id: string): Promise<'in_progress' | 'canceling' | 'ended'>; + results(id: string): AsyncIterable; + cancelBatch(id: string): Promise; +} + +/** The real client: the official Anthropic SDK, the key from ANTHROPIC_API_KEY. */ +export async function anthropicReviewAiClient(apiKey: string): Promise { + const { default: Anthropic } = await import('@anthropic-ai/sdk'); + const client = new Anthropic({ apiKey }); + return { + async createBatch(requests) { + const batch = await client.messages.batches.create({ + requests: requests as unknown as Parameters< + typeof client.messages.batches.create + >[0]['requests'], + }); + return { id: batch.id }; + }, + async batchStatus(id) { + return (await client.messages.batches.retrieve(id)).processing_status; + }, + async *results(id) { + for await (const entry of await client.messages.batches.results(id)) { + yield entry as unknown as BatchResult; + } + }, + async cancelBatch(id) { + await client.messages.batches.cancel(id); + }, + }; +} + +// --- Rows the job reads. ------------------------------------------------------------------ + +type FlagRow = { + include_in_rating: boolean | null; + verdict: string | null; + flag_category: string | null; +}; + +export interface AiReviewRow { + id: string; + product_id: string; + origin: string | null; + reviewer_type: string | null; + helpfulness_weight: number | string | null; + comment: string | null; + review_flags: FlagRow | FlagRow[] | null; +} + +export interface AiListingRow { + id: string; + cluster_id: string | null; + origin: string | null; +} + +export interface AiMarkerRow { + review_id: string; + text_hash: string; + prompt_version: string; + criteria_version: number; + category_slug: string; + outcome: string; + attempts: number; + batch_id: string | null; +} + +export interface ManifestReview { + ref: number; + id: string; + text_hash: string; +} + +export interface ManifestRequest { + custom_id: string; + category: string; + criteria_version: number; + reviews: ManifestReview[]; +} + +export interface AiBatchRow { + id: string; + model: string; + prompt_version: string; + status: string; + manifest: ManifestRequest[]; + reserved_usd: number | string; + cost_usd: number | string | null; + submitted_at: string; + stored_at: string | null; +} + +/** One row of rate.store_review_ai_points. */ +export interface AiStoreRow { + id: string; + text_hash: string; + prompt_version: string; + criteria_version: number; + category: string; + model: string; + batch_id: string | null; + failed: boolean; + rejected: number; + points: AiPoint[]; +} + +export type StoreCounts = Record< + | 'stored' + | 'empty' + | 'rejected' + | 'failed' + | 'missing' + | 'not_crawl' + | 'writer' + | 'stale' + | 'unquoted' + | 'done' + | 'exhausted', + number +>; + +const emptyCounts = (): StoreCounts => ({ + stored: 0, + empty: 0, + rejected: 0, + failed: 0, + missing: 0, + not_crawl: 0, + writer: 0, + stale: 0, + unquoted: 0, + done: 0, + exhausted: 0, +}); + +const flagOf = (review: AiReviewRow): FlagRow | null => + Array.isArray(review.review_flags) ? (review.review_flags[0] ?? null) : review.review_flags; + +// --- Which reviews are due (sections 1 and 4). -------------------------------------------- + +export interface AiCandidate { + id: string; + text: string; + textHash: string; + category: string; +} + +/** + * The reviews to send, in review id order: crawled user reviews past the + * gate (a flag that does not exclude them, a weight above 0, not filler), long + * enough and in Vietnamese or English, on a crawled listing of a master in an + * open category, with no writer line, not waiting in an open batch, and due + * by their marker. + */ +export function aiCandidates(input: { + reviews: readonly AiReviewRow[]; + writerReviewIds: ReadonlySet; + markers: readonly AiMarkerRow[]; + listings: readonly AiListingRow[]; + masters: readonly PointMasterRow[]; + categories: readonly CategoryRow[]; + pending: ReadonlySet; + promptVersion?: string; +}): AiCandidate[] { + const promptVersion = input.promptVersion ?? REVIEW_AI_PROMPT_VERSION; + const slugByCategory = new Map(input.categories.map(c => [c.id, c.slug])); + const slugByMaster = new Map( + input.masters.map(m => [ + m.id, + m.category_id ? (slugByCategory.get(m.category_id) ?? null) : null, + ]) + ); + const slugByListing = new Map( + input.listings + .filter(l => l.origin === 'crawl' && l.cluster_id) + .map(l => [l.id, slugByMaster.get(l.cluster_id as string) ?? null]) + ); + const markerById = new Map(input.markers.map(m => [m.review_id, m])); + const out: AiCandidate[] = []; + for (const review of input.reviews) { + if (review.origin !== 'crawl' || review.reviewer_type !== 'user') continue; + if (input.writerReviewIds.has(review.id) || input.pending.has(review.id)) continue; + const category = slugByListing.get(review.product_id) ?? null; + if (!aiCategoryOpen(category)) continue; + const flag = flagOf(review); + const weight = reviewPointWeight({ + id: review.id, + product_id: review.product_id, + origin: review.origin, + reviewer_type: review.reviewer_type, + helpfulness_weight: review.helpfulness_weight, + created_at: null, + platform_created_at: null, + review_flags: flag, + }); + if (weight === 0) continue; + const text = (review.comment ?? '').normalize('NFC'); + const depth = reviewDepth({ + comment: text, + verdict: null, + points: [], + reviewerType: review.reviewer_type, + flagCategory: flag?.flag_category ?? null, + }); + if (depth === 'filler' || !textQualifies(text)) continue; + const textHash = reviewTextHash(text); + const marker = markerById.get(review.id); + const sameKey = + marker !== undefined && + marker.text_hash === textHash && + marker.prompt_version === promptVersion && + Number(marker.criteria_version) === RATING_CRITERIA_VERSION && + marker.category_slug === category; + if (sameKey && !(marker.outcome === 'failed' && Number(marker.attempts) < MAX_ATTEMPTS)) { + continue; + } + out.push({ id: review.id, text, textHash, category }); + } + return out.sort((a, b) => (a.id < b.id ? -1 : a.id > b.id ? 1 : 0)); +} + +/** Requests of up to REVIEWS_PER_REQUEST reviews of one category, with their manifest. */ +export function buildRequests( + candidates: readonly AiCandidate[], + model: ReviewAiModel +): { requests: BatchRequest[]; manifest: ManifestRequest[]; instructions: string[] } { + const byCategory = new Map(); + for (const c of candidates) { + const list = byCategory.get(c.category) ?? []; + list.push(c); + byCategory.set(c.category, list); + } + const requests: BatchRequest[] = []; + const manifest: ManifestRequest[] = []; + const instructions: string[] = []; + for (const [category, list] of [...byCategory].sort(([a], [b]) => (a < b ? -1 : 1))) { + const system = reviewAiInstructions(category); + const schema = reviewAiOutputSchema(category); + for (let i = 0; i < list.length; i += REVIEWS_PER_REQUEST) { + const part = list.slice(i, i + REVIEWS_PER_REQUEST); + const custom_id = `${category}-${requests.length}`; + const refs = part.map((c, j) => ({ ref: j + 1, candidate: c })); + requests.push({ + custom_id, + params: { + model, + max_tokens: REQUEST_MAX_TOKENS, + system: [{ type: 'text', text: system, cache_control: { type: 'ephemeral' } }], + messages: [ + { + role: 'user', + content: reviewAiUserMessage(refs.map(r => ({ ref: r.ref, text: r.candidate.text }))), + }, + ], + output_config: { effort: 'low', format: { type: 'json_schema', schema } }, + }, + }); + instructions.push(system); + manifest.push({ + custom_id, + category, + criteria_version: RATING_CRITERIA_VERSION, + reviews: refs.map(r => ({ + ref: r.ref, + id: r.candidate.id, + text_hash: r.candidate.textHash, + })), + }); + } + } + return { requests, manifest, instructions }; +} + +// --- Reading a batch's results (section 7.2). --------------------------------------------- + +export interface CollectedBatch { + rows: AiStoreRow[]; + usage: Required<{ [K in keyof BatchUsage]: number }>; + costUsd: number; + lines: number; + rejects: Record; + unaccented: number; + failedReviews: number; + reviewsWithLines: number; + reviewsAnswered: number; +} + +/** The text the model returned, parsed, or null. */ +function parseOutput(message: { content: { type: string; text?: string }[] }): unknown { + const text = message.content + .filter(block => block.type === 'text' && typeof block.text === 'string') + .map(block => block.text) + .join(''); + try { + return JSON.parse(text); + } catch { + return null; + } +} + +/** + * The writer's rows for every review of a batch. A request that did not end + * normally (cut off at max_tokens, refused, errored, expired, canceled, or + * not JSON), or a review the model left out, gives failed rows: never an + * empty result. + */ +export function collectBatch( + batch: Pick, + results: readonly BatchResult[], + textById: ReadonlyMap +): CollectedBatch { + const model = batch.model as ReviewAiModel; + const byId = new Map(results.map(r => [r.custom_id, r])); + const usage = { + input_tokens: 0, + output_tokens: 0, + cache_read_input_tokens: 0, + cache_creation_input_tokens: 0, + }; + const out: CollectedBatch = { + rows: [], + usage, + costUsd: 0, + lines: 0, + rejects: { + shape: 0, + evidence: 0, + length: 0, + criterion: 0, + evasive: 0, + duplicate: 0, + too_many: 0, + }, + unaccented: 0, + failedReviews: 0, + reviewsWithLines: 0, + reviewsAnswered: 0, + }; + for (const request of batch.manifest) { + const result = byId.get(request.custom_id); + let entries: Map | null = null; + if (result?.result.type === 'succeeded') { + const message = result.result.message; + for (const key of Object.keys(usage) as (keyof typeof usage)[]) { + const n = Number(message.usage?.[key] ?? 0); + usage[key] += Number.isFinite(n) && n > 0 ? n : 0; + } + if (message.stop_reason === 'end_turn') { + const parsed = parseOutput(message) as { reviews?: unknown } | null; + if (parsed && Array.isArray(parsed.reviews)) { + entries = new Map(); + for (const entry of parsed.reviews as { ref?: unknown; points?: unknown }[]) { + if (entry && Number.isInteger(entry.ref) && !entries.has(entry.ref as number)) { + entries.set(entry.ref as number, entry.points); + } + } + } + } + } + for (const review of request.reviews) { + const base = { + id: review.id, + text_hash: review.text_hash, + prompt_version: batch.prompt_version, + criteria_version: request.criteria_version, + category: request.category, + model: batch.model, + batch_id: batch.id, + }; + const raw = entries?.get(review.ref); + if (entries === null || !Array.isArray(raw)) { + out.failedReviews += 1; + out.rows.push({ ...base, failed: true, rejected: 0, points: [] }); + continue; + } + const checked = checkReviewPoints(textById.get(review.id) ?? '', request.category, raw); + const rejected = Object.values(checked.rejected).reduce((s, n) => s + n, 0); + for (const [reason, n] of Object.entries(checked.rejected) as [AiRejectReason, number][]) { + out.rejects[reason] += n; + } + out.lines += checked.points.length + rejected; + out.unaccented += checked.points.filter(p => p.match === 'unaccented').length; + out.reviewsAnswered += 1; + if (checked.points.length > 0) out.reviewsWithLines += 1; + out.rows.push({ + ...base, + failed: false, + rejected: Math.min(rejected, 20), + points: checked.points, + }); + } + } + out.costUsd = isReviewAiModel(model) ? batchCostUsd(model, usage) : 0; + return out; +} + +/** Why a collected batch must not be stored, or null (section 7.2). */ +export function qualityStop( + collected: CollectedBatch, + previousShare: { share: number; reviews: number } | null +): string | null { + if ( + collected.lines >= EVIDENCE_STOP_MIN_LINES && + collected.rejects.evidence / collected.lines > EVIDENCE_STOP_SHARE + ) { + return `${collected.rejects.evidence} of ${collected.lines} lines failed the evidence check`; + } + if ( + previousShare !== null && + previousShare.reviews >= SHARE_JUMP_MIN_REVIEWS && + collected.reviewsAnswered >= SHARE_JUMP_MIN_REVIEWS + ) { + const share = collected.reviewsWithLines / collected.reviewsAnswered; + if (Math.abs(share - previousShare.share) > SHARE_JUMP_STOP) { + return `the share of reviews with lines moved from ${previousShare.share.toFixed(2)} to ${share.toFixed(2)}`; + } + } + return null; +} + +// --- Database calls. ---------------------------------------------------------------------- + +const sleep = (ms: number) => new Promise(resolve => setTimeout(resolve, ms)); + +/** Stores rows in calls of STORE_ROWS, retrying a deadlock; returns the summed counts. */ +export async function storeAiRows( + supabase: RateSupabaseClient, + rows: readonly AiStoreRow[], + wait: (ms: number) => Promise = sleep +): Promise { + const total = emptyCounts(); + for (let i = 0; i < rows.length; i += STORE_ROWS) { + const part = rows.slice(i, i + STORE_ROWS); + for (let attempt = 1; ; attempt++) { + const { data, error } = await supabase.rpc('store_review_ai_points', { p_rows: part }); + if (!error) { + for (const [key, n] of Object.entries((data ?? {}) as Record)) { + if (key in total) total[key as keyof StoreCounts] += Number(n) || 0; + } + break; + } + if (error.code === '40P01' && attempt < DEADLOCK_ATTEMPTS) { + await wait(250 * 2 ** (attempt - 1)); + continue; + } + throw new Error(`store_review_ai_points (${error.code ?? 'no code'}): ${error.message}`); + } + } + return total; +} + +const monthStart = (now: Date) => + new Date(Date.UTC(now.getUTCFullYear(), now.getUTCMonth(), 1)).toISOString(); + +/** This month's spend of a model: reported cost, or the reserved ceiling while open. */ +export function monthSpendUsd(batches: readonly AiBatchRow[], model: string, now: Date): number { + const start = monthStart(now); + return batches + .filter(b => b.model === model && b.submitted_at >= start) + .reduce((s, b) => s + Number(b.cost_usd ?? b.reserved_usd ?? 0), 0); +} + +const BATCH_COLUMNS = + 'id, model, prompt_version, status, manifest, reserved_usd, cost_usd, submitted_at, stored_at'; + +export interface ReviewAiPointsResult { + apply: boolean; + enabled: boolean; + model: ReviewAiModel; + candidates: number; + sent: number; + requests: number; + reservedUsd: number; + monthSpendUsd: number; + capped: boolean; + collected: string[]; + stored: StoreCounts; + canceled: string[]; + waiting: string[]; + purged: { lines: number; markers: number } | null; +} + +export interface ReviewAiPointsOptions { + apply?: boolean; + /** REVIEW_AI_POINTS_ENABLED === 'true'. */ + enabled?: boolean; + model?: ReviewAiModel; + purge?: string | null; + maxReviews?: number; + /** Needed only to send, collect or cancel (never in a dry run). */ + ai?: ReviewAiClient | null; + now?: Date; + waitMs?: number; + pollMs?: number; + wait?: (ms: number) => Promise; + promptVersion?: string; +} + +export async function runReviewAiPointsJob( + supabase: RateSupabaseClient, + options: ReviewAiPointsOptions = {} +): Promise { + const apply = options.apply === true; + const enabled = options.enabled === true; + const model = options.model ?? DEFAULT_REVIEW_AI_MODEL; + if (!isReviewAiModel(model)) throw new Error(`Unknown model '${String(model)}'.`); + const promptVersion = options.promptVersion ?? REVIEW_AI_PROMPT_VERSION; + const maxReviews = Math.min(options.maxReviews ?? MAX_REVIEWS_PER_RUN, MAX_REVIEWS_PER_RUN); + if (!Number.isInteger(maxReviews) || maxReviews < 1) throw new Error('--max must be 1 to 2000.'); + const now = options.now ?? new Date(); + const wait = options.wait ?? sleep; + const waitMs = options.waitMs ?? 60 * 60 * 1000; + if (!Number.isFinite(waitMs) || waitMs < 0 || waitMs > 24 * 60 * 60 * 1000) { + throw new Error('--wait-minutes must be 0 to 1440.'); + } + const pollMs = options.pollMs ?? 60 * 1000; + const result: ReviewAiPointsResult = { + apply, + enabled, + model, + candidates: 0, + sent: 0, + requests: 0, + reservedUsd: 0, + monthSpendUsd: 0, + capped: false, + collected: [], + stored: emptyCounts(), + canceled: [], + waiting: [], + purged: null, + }; + const needAi = (): ReviewAiClient => { + if (!options.ai) throw new Error('ANTHROPIC_API_KEY is needed to send or collect a batch.'); + return options.ai; + }; + + // The purge: one call, nothing else. + if (options.purge !== undefined && options.purge !== null) { + const version = options.purge; + if (version !== 'all' && !PROMPT_VERSION_RE.test(version)) { + throw new Error("--purge takes 'all' or a prompt version."); + } + if (!apply) { + logger.info('review_ai.purge', `Dry run: would purge ai lines of ${version}.`, { version }); + return result; + } + const { data, error } = await supabase.rpc('purge_review_ai_points', { + p_prompt_version: version, + }); + if (error) throw new Error(`purge_review_ai_points: ${error.message}`); + result.purged = data as { lines: number; markers: number }; + logger.info('review_ai.purge', `Purged ai lines of ${version}.`, { version, ...result.purged }); + return result; + } + + const batches = await fetchByKey(supabase, 'review_ai_batches', BATCH_COLUMNS, 'id'); + const open = batches.filter(b => b.status === 'submitted' || b.status === 'ended'); + + // The kill switch: no model call; --apply cancels what is still running at the API. The + // row stays open: requests that finished before the cancel are paid for, so the next + // enabled run collects them (the canceled ones come back as failed rows, to retry). + if (!enabled) { + if (apply) { + for (const batch of open.filter(b => b.status === 'submitted')) { + if ((await needAi().batchStatus(batch.id)) !== 'in_progress') continue; + await needAi().cancelBatch(batch.id); + result.canceled.push(batch.id); + } + } + logger.info( + 'review_ai.disabled', + `REVIEW_AI_POINTS_ENABLED is not 'true': no model call${result.canceled.length ? `, ${result.canceled.length} batches canceled` : ''}.`, + { canceled: result.canceled.length } + ); + return result; + } + + // Collect what an earlier run left open, then what this run sends. + const collectOpen = async (batch: AiBatchRow): Promise => { + const ai = needAi(); + const deadline = Date.now() + waitMs; + for (;;) { + const status = await ai.batchStatus(batch.id); + if (status === 'ended') break; + if (Date.now() >= deadline) { + result.waiting.push(batch.id); + logger.info('review_ai.waiting', `Batch ${batch.id} still running; collected next run.`, { + batch: batch.id, + }); + return false; + } + await wait(pollMs); + } + const results: BatchResult[] = []; + try { + for await (const entry of ai.results(batch.id)) results.push(entry); + } catch (err) { + // The API keeps results 29 days: an older batch is gone, so mark it and move on. + if (now.getTime() - new Date(batch.submitted_at).getTime() < RESULTS_KEPT_MS) throw err; + const { error } = await supabase + .from('review_ai_batches') + .update({ status: 'expired', ended_at: now.toISOString() }) + .eq('id', batch.id); + if (error) throw new Error(`review_ai_batches expire: ${error.message}`); + logger.warn('review_ai.expired', `Batch ${batch.id} results are gone; marked expired.`, { + batch: batch.id, + }); + return false; + } + const ids = batch.manifest.flatMap(r => r.reviews.map(v => v.id)); + const texts = await selectReviewsIn(supabase, ids); + const collected = collectBatch( + batch, + results, + new Map(texts.map(r => [r.id, (r.comment ?? '').normalize('NFC')])) + ); + const { error: endError } = await supabase + .from('review_ai_batches') + .update({ + status: 'ended', + ended_at: now.toISOString(), + input_tokens: collected.usage.input_tokens, + output_tokens: collected.usage.output_tokens, + cache_read_tokens: collected.usage.cache_read_input_tokens, + cache_write_tokens: collected.usage.cache_creation_input_tokens, + cost_usd: Number(collected.costUsd.toFixed(4)), + }) + .eq('id', batch.id); + if (endError) throw new Error(`review_ai_batches end: ${endError.message}`); + const latest = await fetchByKey(supabase, 'review_ai_batches', BATCH_COLUMNS, 'id'); + const stop = qualityStop(collected, await previousShare(supabase, latest, batch.id)); + if (stop !== null) { + throw new Error(`Batch ${batch.id} not stored: ${stop}. Look at it before the next run.`); + } + // A run that died after storing part of this batch: its rows already have markers of + // this batch. Storing them again would count a failed review's attempt twice. + const done = new Set( + ( + await fetchByKey<{ review_id: string }>( + supabase, + 'review_ai_extractions', + 'review_id', + 'review_id', + query => query.eq('batch_id', batch.id) + ) + ).map(m => m.review_id) + ); + const counts = await storeAiRows( + supabase, + collected.rows.filter(row => !done.has(row.id)), + wait + ); + for (const key of Object.keys(counts) as (keyof StoreCounts)[]) + result.stored[key] += counts[key]; + const { error: storeError } = await supabase + .from('review_ai_batches') + .update({ status: 'stored', stored_at: new Date().toISOString() }) + .eq('id', batch.id); + if (storeError) throw new Error(`review_ai_batches store: ${storeError.message}`); + result.collected.push(batch.id); + logger.info('review_ai.stored', `Stored batch ${batch.id}.`, { + batch: batch.id, + reviews: collected.rows.length, + lines: collected.lines, + rejects: collected.rejects, + unaccented: collected.unaccented, + failed_reviews: collected.failedReviews, + cost_usd: Number(collected.costUsd.toFixed(4)), + ...counts, + }); + return true; + }; + + if (apply) { + for (const batch of open) await collectOpen(batch); + } + + // Plan this run's batch. + const reviews = await fetchByKey( + supabase, + 'reviews', + 'id, product_id, origin, reviewer_type, helpfulness_weight, comment, review_flags(include_in_rating, verdict, flag_category)', + 'id', + query => query.eq('origin', 'crawl').eq('reviewer_type', 'user') + ); + const writerLines = await fetchByKey<{ id: string; review_id: string }>( + supabase, + 'review_points', + 'id, review_id', + 'id', + query => query.eq('method', 'writer') + ); + const markers = await fetchByKey( + supabase, + 'review_ai_extractions', + 'review_id, text_hash, prompt_version, criteria_version, category_slug, outcome, attempts, batch_id', + 'review_id' + ); + const listings = await fetchByKey( + supabase, + 'products', + 'id, cluster_id, origin', + 'id', + query => query.eq('origin', 'crawl') + ); + const masters = await fetchByKey( + supabase, + 'master_products', + 'id, category_id', + 'id' + ); + const categories = await fetchByKey(supabase, 'categories', 'id, slug', 'id'); + // Reviews in a batch still open are not due: after --apply, the ones still running; in a + // dry run, which collects nothing, every open batch. + const stillOpen = new Set(apply ? result.waiting : open.map(b => b.id)); + const pending = new Set( + batches + .filter(b => stillOpen.has(b.id)) + .flatMap(b => b.manifest.flatMap(r => r.reviews.map(v => v.id))) + ); + const candidates = aiCandidates({ + reviews, + writerReviewIds: new Set(writerLines.map(l => l.review_id)), + markers, + listings, + masters, + categories, + pending, + promptVersion, + }); + result.candidates = candidates.length; + const chosen = candidates.slice(0, maxReviews); + const { requests, manifest, instructions } = buildRequests(chosen, model); + result.requests = requests.length; + result.reservedUsd = reserveUsd( + model, + requests.map((r, i) => ({ + instructions: instructions[i]!, + message: r.params.messages[0]!.content, + })) + ); + const refreshed = await fetchByKey( + supabase, + 'review_ai_batches', + BATCH_COLUMNS, + 'id' + ); + result.monthSpendUsd = monthSpendUsd(refreshed, model, now); + const cap = REVIEW_AI_MODELS[model].monthlyCapUsd; + result.capped = requests.length > 0 && result.monthSpendUsd + result.reservedUsd > cap; + logger.info( + 'review_ai.plan', + `${apply ? 'Sending' : 'Would send'} ${chosen.length} of ${candidates.length} due reviews in ${requests.length} requests; ceiling $${result.reservedUsd.toFixed(4)}, month $${result.monthSpendUsd.toFixed(4)} of $${cap}.`, + { + apply, + model, + candidates: candidates.length, + reviews: chosen.length, + requests: requests.length, + reserved_usd: result.reservedUsd, + month_usd: result.monthSpendUsd, + cap_usd: cap, + capped: result.capped, + } + ); + if (result.capped) { + logger.warn( + 'review_ai.cap', + `The batch would pass the $${cap} monthly cap of ${model}; not sent.`, + { + model, + } + ); + return result; + } + if (!apply || requests.length === 0) return result; + + const created = await needAi().createBatch(requests); + // The id first, so a run that dies from here on collects this batch instead of re-sending. + const row = { + id: created.id, + model, + prompt_version: promptVersion, + status: 'submitted', + request_count: requests.length, + review_count: chosen.length, + manifest, + reserved_usd: result.reservedUsd, + submitted_at: now.toISOString(), + }; + const { error } = await supabase.from('review_ai_batches').insert(row); + if (error) { + // Unrecorded, its results would never be collected: stop it before more is spent. + const canceled = await needAi() + .cancelBatch(created.id) + .then( + () => 'canceled', + () => 'NOT canceled; cancel it by hand' + ); + throw new Error( + `review_ai_batches insert for ${created.id}: ${error.message}. The batch was ${canceled}.` + ); + } + result.sent = chosen.length; + await collectOpen({ ...row, cost_usd: null, stored_at: null }); + return result; +} + +async function selectReviewsIn( + supabase: RateSupabaseClient, + ids: readonly string[] +): Promise<{ id: string; comment: string | null }[]> { + const rows: { id: string; comment: string | null }[] = []; + const unique = [...new Set(ids)]; + for (let i = 0; i < unique.length; i += 150) { + const part = unique.slice(i, i + 150); + rows.push( + ...(await fetchByKey<{ id: string; comment: string | null }>( + supabase, + 'reviews', + 'id, comment', + 'id', + query => query.in('id', part) + )) + ); + } + return rows; +} + +/** The share of answered reviews with lines in the last stored batch, from its markers. */ +async function previousShare( + supabase: RateSupabaseClient, + batches: readonly AiBatchRow[], + current: string +): Promise<{ share: number; reviews: number } | null> { + const last = batches + .filter(b => b.status === 'stored' && b.id !== current && b.stored_at) + .sort((a, b) => ((a.stored_at ?? '') < (b.stored_at ?? '') ? 1 : -1))[0]; + if (!last) return null; + const markers = await fetchByKey<{ review_id: string; outcome: string }>( + supabase, + 'review_ai_extractions', + 'review_id, outcome', + 'review_id', + query => query.eq('batch_id', last.id) + ); + const answered = markers.filter(m => m.outcome !== 'failed'); + if (answered.length === 0) return null; + return { + share: answered.filter(m => m.outcome === 'stored').length / answered.length, + reviews: answered.length, + }; +} + +// --- CLI. --------------------------------------------------------------------------------- + +const argValue = (name: string): string | null => { + const at = process.argv.indexOf(name); + return at >= 0 ? (process.argv[at + 1] ?? '') : null; +}; + +const isDirectExecution = + typeof process !== 'undefined' && + Array.isArray(process.argv) && + Boolean(process.argv[1]) && + import.meta.url === pathToFileURL(process.argv[1]!).href; + +if (isDirectExecution) { + dotenv.config({ quiet: true }); + dotenv.config({ path: '../../.env', quiet: true }); + (async () => { + const { createClient } = await import('@supabase/supabase-js'); + const rawUrl = process.env.SUPABASE_URL || process.env.NEXT_PUBLIC_SUPABASE_URL || ''; + // The publishable key plus the worker user's sign-in: its access token + // carries the rate_worker role, which reads the review tables, writes the + // batch record and writes lines only through rate.store_review_ai_points. + const key = ( + process.env.SUPABASE_PUBLISHABLE_KEY || + process.env.NEXT_PUBLIC_SUPABASE_ANON_KEY || + '' + ).trim(); + const email = (process.env.RATE_WORKER_EMAIL || '').trim(); + const password = process.env.RATE_WORKER_PASSWORD || ''; + if (!rawUrl.trim() || !key || !email || !password) { + throw new Error( + 'Missing SUPABASE_URL, SUPABASE_PUBLISHABLE_KEY, RATE_WORKER_EMAIL or RATE_WORKER_PASSWORD in environment.' + ); + } + const modelName = (process.env.REVIEW_AI_MODEL || DEFAULT_REVIEW_AI_MODEL).trim(); + if (!isReviewAiModel(modelName)) { + throw new Error( + `REVIEW_AI_MODEL must be one of ${Object.keys(REVIEW_AI_MODELS).join(', ')}.` + ); + } + const apply = process.argv.includes('--apply'); + const enabled = (process.env.REVIEW_AI_POINTS_ENABLED || '').trim() === 'true'; + const max = argValue('--max'); + const waitMinutes = argValue('--wait-minutes'); + const { url, host } = resolveBackfillSupabaseTarget(rawUrl); + const client = createClient(url, key, { auth: { persistSession: false } }); + const { data: auth, error: authError } = await client.auth.signInWithPassword({ + email, + password, + }); + if (authError || !auth.session) { + throw new Error( + `Sign-in as the rate_worker user failed: ${authError?.message ?? 'no session'}` + ); + } + const role = (auth.user?.role ?? '').trim(); + if (role !== 'rate_worker') { + throw new Error(`The signed-in user has role '${role}', not rate_worker; refusing to run.`); + } + const supabase = scopeSupabaseToRate(client); + // The model key is read only for a run that may call the API. + const apiKey = (process.env.ANTHROPIC_API_KEY || '').trim(); + const ai = apply && apiKey ? await anthropicReviewAiClient(apiKey) : null; + logger.info( + 'review_ai.target', + `Target: ${host} (${apply ? 'apply' : 'dry run'}, ${enabled ? 'enabled' : 'disabled'}, ${modelName})`, + { host } + ); + try { + const result = await runReviewAiPointsJob(supabase, { + apply, + enabled, + model: modelName, + purge: argValue('--purge'), + maxReviews: max === null ? undefined : Number(max), + waitMs: waitMinutes === null ? undefined : Number(waitMinutes) * 60 * 1000, + ai, + }); + if (result.capped) process.exitCode = 3; + else if (result.stored.unquoted > 0 || result.stored.exhausted > 0) process.exitCode = 2; + } finally { + await client.auth.signOut().catch(() => undefined); + } + })().catch((err: unknown) => { + logger.error( + 'review_ai.fatal', + `Review AI points job failed: ${err instanceof Error ? err.message : String(err)}` + ); + process.exit(1); + }); +} diff --git a/apps/importer/src/shopee/review-ai-prompt.ts b/apps/importer/src/shopee/review-ai-prompt.ts new file mode 100644 index 000000000..48ab48551 --- /dev/null +++ b/apps/importer/src/shopee/review-ai-prompt.ts @@ -0,0 +1,388 @@ +/** + * The prompt, output schema and checks of the review AI points job + * (DOS/Rate#1531, Docs/Proposals/2026-11-03-ai-review-points.html). + * + * Plain functions only: no client, no network. The job + * (jobs/review-ai-points.ts) decides which reviews to send, sends them through + * the Batch API and stores what passes these checks through + * rate.store_review_ai_points, which checks the same limits again. + * + * What a model sees: the category's criteria (key, Vietnamese label, a line of + * explanation), the rules and a few made-up examples, then up to + * REVIEWS_PER_REQUEST review texts with a temporary ref each. Never a name, a + * user id, an order id or a link the job could leave out. + */ +import { createHash } from 'node:crypto'; +import { + comparablePoint, + getRatingCriteria, + hasRatingCriteria, + isEvasiveCon, + hasContent, + storedPoint, + RATING_CRITERIA_VERSION, + REVIEW_POINT_MAX, + REVIEW_POINT_MIN, +} from '@repo/shared-data/rules'; + +/** Bump on any change to the instructions, examples or schema (design 4). */ +export const REVIEW_AI_PROMPT_VERSION = 'v1'; +export { RATING_CRITERIA_VERSION }; + +/** Reviews per request: one category, one cached instruction block. */ +export const REVIEWS_PER_REQUEST = 20; +/** Lines per side per review (design 2.2). */ +export const AI_POINTS_PER_KIND = 3; +/** Output tokens per request: 20 reviews x ~120 tokens, with room for low-effort thinking. */ +export const REQUEST_MAX_TOKENS = 6000; +/** Evidence length, in characters as stored (the writer takes 2 to 300). */ +export const EVIDENCE_MAX = 300; + +/** Text length a review needs (D3): characters and words, without emoji and repeats. */ +export const REVIEW_AI_MIN_CHARS = 40; +export const REVIEW_AI_MIN_WORDS = 8; + +/** Models the job may use (D1) and their list prices in USD per million tokens. */ +export const REVIEW_AI_MODELS = { + 'claude-haiku-5-5': { input: 0.1, output: 0.5, monthlyCapUsd: 5 }, + 'claude-sonnet-5-5': { input: 2, output: 10, monthlyCapUsd: 15 }, +} as const; +export type ReviewAiModel = keyof typeof REVIEW_AI_MODELS; +export const DEFAULT_REVIEW_AI_MODEL: ReviewAiModel = 'claude-haiku-5-5'; +/** The Batch API halves every token price. */ +export const BATCH_DISCOUNT = 0.5; +/** Cache reads cost 0.1x the input price; cache writes 1.25x. */ +export const CACHE_READ_FACTOR = 0.1; +export const CACHE_WRITE_FACTOR = 1.25; + +export function isReviewAiModel(value: unknown): value is ReviewAiModel { + return typeof value === 'string' && Object.prototype.hasOwnProperty.call(REVIEW_AI_MODELS, value); +} + +/** Categories the job extracts for: a criteria list, and not services (design 1). */ +export function aiCategoryOpen(slug: string | null | undefined): slug is string { + return typeof slug === 'string' && slug !== 'services' && hasRatingCriteria(slug); +} + +/** The text hash the writer compares: sha256 hex of the comment in NFC. */ +export function reviewTextHash(text: string): string { + return createHash('sha256').update(text.normalize('NFC'), 'utf8').digest('hex'); +} + +// Pictographs and flag letters, then the joiners and selectors they leave behind. +const EMOJI = /\p{Extended_Pictographic}|[\u{1F1E6}-\u{1F1FF}]|\u{FE0F}|\u{200D}|\u{20E3}/gu; + +/** The text without emoji, a character repeated three times or more kept once. */ +export function substantiveText(text: string): string { + return text + .normalize('NFC') + .replace(EMOJI, ' ') + .replace(/(.)\1{2,}/gu, '$1') + .replace(/\s+/gu, ' ') + .trim(); +} + +// Vietnamese letters with diacritics, and common words written without them. +const VI_LETTERS = /[ăâđêôơưàáảãạằắẳẵặầấẩẫậèéẻẽẹềếểễệìíỉĩịòóỏõọồốổỗộờớởỡợùúủũụừứửữựỳýỷỹỵ]/iu; +const VI_WORDS = + /\b(?:khong|ko|rat|dung|may|pin|hang|tot|nhung|duoc|cung|minh|mua|giao|nhanh|dep|man hinh|sac|ok lam|chay|muot|nong|lag)\b/iu; +const EN_WORDS = + /\b(?:the|and|is|it|very|good|great|battery|phone|screen|not|but|works|quality|fast|slow)\b/iu; + +/** 'vi', 'en' or null, without a model call (D3). */ +export function reviewLanguage(text: string): 'vi' | 'en' | null { + if (VI_LETTERS.test(text)) return 'vi'; + // Letters outside Latin (Chinese, Thai...): another language. + if (/[^\p{Script=Latin}\p{N}\p{P}\p{S}\p{Z}\s]/u.test(text.replace(EMOJI, ''))) return null; + if (VI_WORDS.test(text)) return 'vi'; + if (EN_WORDS.test(text)) return 'en'; + return null; +} + +/** Whether a text is long enough and in a language the job reads (D3). */ +export function textQualifies(text: string | null | undefined): boolean { + if (typeof text !== 'string') return false; + const plain = substantiveText(text); + const words = plain.split(' ').filter(w => /[\p{L}\p{N}]/u.test(w)); + return ( + [...plain].length >= REVIEW_AI_MIN_CHARS && + words.length >= REVIEW_AI_MIN_WORDS && + reviewLanguage(plain) !== null + ); +} + +// One line of explanation per criterion, so the model knows what each key covers. +const CRITERION_HINTS: Readonly> = { + performance: 'tốc độ, độ mượt, lag, giật, chơi game, đa nhiệm', + camera: 'ảnh, video, chụp đêm, zoom, camera trước', + battery: 'thời lượng pin, tốc độ sạc, hao pin, nóng khi sạc', + display: 'độ sáng, màu, nhìn ngoài nắng, tần số quét, kích thước màn', + design: 'ngoại hình, chất liệu, cầm nắm, trọng lượng, độ bền, chống nước', + value: 'đáng tiền hay không so với giá', + sound: 'loa, âm lượng, chất âm khi xem phim nghe nhạc', + software: 'hệ điều hành, ứng dụng, bút, bàn phím rời, phụ kiện', + keyboard: 'bàn phím, hành trình phím, đèn phím, touchpad', + thermals: 'nóng, quạt, độ ồn khi tải nặng', + health: 'đo nhịp tim, giấc ngủ, SpO2, bước chân, độ chính xác', + apps: 'ứng dụng, thông báo, kết nối điện thoại, GPS, NFC', + anc: 'chống ồn chủ động, chế độ xuyên âm', + comfort: 'đeo lâu có đau không, vừa tai, chắc chắn', + calls: 'micro, gọi điện, người nghe có rõ không', +}; + +const EXAMPLES = `Ví dụ (minh hoạ, không phải review thật): +Review ref 1: "pin xài cả ngày vẫn còn 30%, mà ra nắng hơi khó nhìn. shop giao nhanh" +Kết quả: {"ref": 1, "points": [ + {"kind": "pro", "criterion_key": "battery", "body": "Pin dùng hết ngày vẫn còn dư", "evidence": "pin xài cả ngày vẫn còn 30%"}, + {"kind": "con", "criterion_key": "display", "body": "Màn hình hơi khó nhìn ngoài nắng", "evidence": "ra nắng hơi khó nhìn"}]} +(Không lấy "shop giao nhanh": đó là chuyện của người bán.) +Review ref 2: "hàng đẹp lắm, đóng gói cẩn thận, 5 sao cho shop, sẽ ủng hộ tiếp nha mọi người" +Kết quả: {"ref": 2, "points": []} +Review ref 3: "máy chạy mượt, chơi liên quân không giật, nhưng sạc lâu quá, gần 2 tiếng mới đầy" +Kết quả: {"ref": 3, "points": [ + {"kind": "pro", "criterion_key": "performance", "body": "Chơi game mượt, không giật", "evidence": "chơi liên quân không giật"}, + {"kind": "con", "criterion_key": "battery", "body": "Sạc đầy mất gần 2 tiếng", "evidence": "sạc lâu quá, gần 2 tiếng mới đầy"}]} +Review ref 4: "good phone for the price, camera is decent but the speaker is quiet" +Kết quả: {"ref": 4, "points": [ + {"kind": "pro", "criterion_key": "value", "body": "Đáng tiền so với giá", "evidence": "good phone for the price"}, + {"kind": "pro", "criterion_key": "camera", "body": "Camera ở mức khá", "evidence": "camera is decent"}, + {"kind": "con", "criterion_key": null, "body": "Loa nhỏ", "evidence": "the speaker is quiet"}]}`; + +/** The cached instruction block of a category (design 2.1). */ +export function reviewAiInstructions(categorySlug: string): string { + const criteria = getRatingCriteria(categorySlug) + .map( + c => + `- ${c.key}: ${c.label.vi}${CRITERION_HINTS[c.key] ? ` (${CRITERION_HINTS[c.key]})` : ''}` + ) + .join('\n'); + return `Bạn đọc review sản phẩm do người mua viết trên sàn thương mại điện tử và rút ra ưu điểm, nhược điểm của chính sản phẩm. + +Tiêu chí của ngành hàng này (criterion_key: tên): +${criteria} + +Luật: +1. Chỉ lấy điều review nói ra. Không suy ra từ số sao, không đoán, không thêm ý. +2. Mỗi review tối đa ${AI_POINTS_PER_KIND} ưu và ${AI_POINTS_PER_KIND} nhược. Không có ý rõ ràng về sản phẩm thì trả "points": [] (đây là kết quả bình thường). +3. body: viết lại gọn bằng tiếng Việt, ${REVIEW_POINT_MIN} đến ${REVIEW_POINT_MAX} ký tự, giữ đúng ý và đúng mức độ ("pin hơi yếu" không thành "pin rất tệ"). +4. criterion_key: một khoá trong danh sách trên, hoặc null nếu là ý chung không thuộc tiêu chí nào. +5. evidence: đoạn trích nguyên văn, ngắn, chép đúng từng chữ từ chính review đó (giữ nguyên chữ không dấu, viết tắt, lỗi chính tả). Ít nhất 2 từ. +6. Bỏ qua chuyện giao hàng, đóng gói, người bán, khuyến mãi, quà tặng, xu. +7. Không có dòng nào vừa là ưu vừa là nhược. Không lặp ý. +8. Trả đúng một mục cho mỗi ref nhận được. + +${EXAMPLES}`; +} + +/** The user message: the reviews with their temporary refs, as JSON. */ +export function reviewAiUserMessage(reviews: readonly { ref: number; text: string }[]): string { + return `Các review cần đọc (JSON):\n${JSON.stringify( + reviews.map(r => ({ ref: r.ref, text: r.text.normalize('NFC') })) + )}`; +} + +/** The structured output schema of a category: criterion_key limited to its keys. */ +export function reviewAiOutputSchema(categorySlug: string): Record { + const keys = getRatingCriteria(categorySlug).map(c => c.key); + return { + type: 'object', + properties: { + reviews: { + type: 'array', + items: { + type: 'object', + properties: { + ref: { type: 'integer' }, + points: { + type: 'array', + items: { + type: 'object', + properties: { + kind: { type: 'string', enum: ['pro', 'con'] }, + criterion_key: { anyOf: [{ type: 'string', enum: keys }, { type: 'null' }] }, + body: { type: 'string' }, + evidence: { type: 'string' }, + }, + required: ['kind', 'criterion_key', 'body', 'evidence'], + additionalProperties: false, + }, + }, + }, + required: ['ref', 'points'], + additionalProperties: false, + }, + }, + }, + required: ['reviews'], + additionalProperties: false, + }; +} + +/** One line as rate.store_review_ai_points takes it. */ +export interface AiPoint { + kind: 'pro' | 'con'; + criterion_key: string | null; + body: string; + evidence: string; + match: 'exact' | 'unaccented'; +} + +/** Why the job dropped a line (counted, never logged with text). */ +export type AiRejectReason = + 'shape' | 'evidence' | 'length' | 'criterion' | 'evasive' | 'duplicate' | 'too_many'; + +/** + * What the evidence check compares: the compared form of a line + * (comparablePoint: lower case, punctuation as spaces, spaces collapsed), the + * same form rate.store_review_ai_points compares with + * rate.review_point_compare, so a quote the job calls exact the writer finds. + */ +export function evidenceForm(text: string): string { + return comparablePoint(storedPoint(text)); +} + +/** The same without diacritics (đ as d): the fallback for unaccented reviews. */ +export function unaccentedForm(text: string): string { + return evidenceForm(text) + .normalize('NFD') + .replace(/\p{M}+/gu, '') + .replace(/đ/gu, 'd') + .normalize('NFC'); +} + +/** Where an evidence quote is found in its own review, or null (design 7.2). */ +export function evidenceMatch(review: string, evidence: string): 'exact' | 'unaccented' | null { + const quote = evidenceForm(evidence); + // Two words in the compared form, as the writer counts them ("đẹp :)" is one). + if (!quote.includes(' ')) return null; + if (evidenceForm(review).includes(quote)) return 'exact'; + if (unaccentedForm(review).includes(unaccentedForm(evidence))) return 'unaccented'; + return null; +} + +/** What the checks made of one review's lines. */ +export interface CheckedReview { + points: AiPoint[]; + rejected: Record; +} + +const emptyRejects = (): Record => ({ + shape: 0, + evidence: 0, + length: 0, + criterion: 0, + evasive: 0, + duplicate: 0, + too_many: 0, +}); + +/** + * The lines of one review that pass (design 7.2): a known kind; a key of the + * category or null; a body of 5 to 150 characters with words, as the writer + * stores it; no evasive con; no line twice, nor as a pro and a con; evidence + * found in this review's own text; at most 3 a side, in the model's order. + */ +export function checkReviewPoints( + reviewText: string, + categorySlug: string, + raw: unknown +): CheckedReview { + const rejected = emptyRejects(); + const keys = new Set(getRatingCriteria(categorySlug).map(c => c.key)); + const points: AiPoint[] = []; + const seen = new Set(); + const perKind = { pro: 0, con: 0 }; + for (const item of Array.isArray(raw) ? raw : []) { + const line = item as Record | null; + if ( + !line || + typeof line !== 'object' || + (line.kind !== 'pro' && line.kind !== 'con') || + typeof line.body !== 'string' || + typeof line.evidence !== 'string' || + (line.criterion_key !== null && typeof line.criterion_key !== 'string') + ) { + rejected.shape += 1; + continue; + } + const kind = line.kind; + const key = line.criterion_key as string | null; + if (key !== null && !keys.has(key)) { + rejected.criterion += 1; + continue; + } + const body = storedPoint(line.body); + const length = [...body].length; + if (length < REVIEW_POINT_MIN || length > REVIEW_POINT_MAX || !hasContent(body)) { + rejected.length += 1; + continue; + } + if (kind === 'con' && isEvasiveCon(body)) { + rejected.evasive += 1; + continue; + } + const evidence = storedPoint(line.evidence); + const match = [...evidence].length <= EVIDENCE_MAX ? evidenceMatch(reviewText, evidence) : null; + if (match === null) { + rejected.evidence += 1; + continue; + } + const compared = comparablePoint(body); + if (seen.has(compared)) { + rejected.duplicate += 1; + continue; + } + if (perKind[kind] >= AI_POINTS_PER_KIND) { + rejected.too_many += 1; + continue; + } + seen.add(compared); + perKind[kind] += 1; + points.push({ kind, criterion_key: key, body, evidence, match }); + } + return { points, rejected }; +} + +/** USD for a usage at a model's batch prices, cache reads and writes included. */ +export function batchCostUsd( + model: ReviewAiModel, + usage: { + input_tokens?: number | null; + output_tokens?: number | null; + cache_read_input_tokens?: number | null; + cache_creation_input_tokens?: number | null; + } +): number { + const price = REVIEW_AI_MODELS[model]; + const n = (v: number | null | undefined) => (typeof v === 'number' && v > 0 ? v : 0); + const usd = + (n(usage.input_tokens) * price.input + + n(usage.cache_read_input_tokens) * price.input * CACHE_READ_FACTOR + + n(usage.cache_creation_input_tokens) * price.input * CACHE_WRITE_FACTOR + + n(usage.output_tokens) * price.output) / + 1_000_000; + return usd * BATCH_DISCOUNT; +} + +/** + * The most a batch can cost before it is sent (design 8): every request at its + * output ceiling, its input counted at 2 characters a token (Vietnamese runs + * dense; real counts are lower) with no cache hit + * (the instruction block written to the cache each time). + */ +export function reserveUsd( + model: ReviewAiModel, + requests: readonly { instructions: string; message: string }[] +): number { + let total = 0; + for (const r of requests) { + const instructionTokens = Math.ceil(r.instructions.length / 2); + const messageTokens = Math.ceil(r.message.length / 2); + total += batchCostUsd(model, { + input_tokens: messageTokens, + cache_creation_input_tokens: instructionTokens, + output_tokens: REQUEST_MAX_TOKENS, + }); + } + return Math.ceil(total * 10_000) / 10_000; +} diff --git a/apps/web/locales/cs.json b/apps/web/locales/cs.json index 14ef6d74c..bd829dc46 100644 --- a/apps/web/locales/cs.json +++ b/apps/web/locales/cs.json @@ -1534,7 +1534,7 @@ "privacyS4Title": "4. Soukromé údaje prodejců", "privacyS4Desc": "Soukromé údaje prodejce (telefonní číslo, bankovní účet) jsou v polích, která veřejnost nemůže číst. Web i API mohou číst jen veřejná pole.", "privacyS5Title": "5. Rozšíření prohlížeče Rate.vn", - "privacyS5Desc": "Na stránkách produktů 10 podporovaných obchodů rozšíření posílá Rate.vn obchod, URL produktu bez sledovacích parametrů, název produktu, ID položky a obchodu, zvolenou variantu, cenu a původní cenu, aby vyhledalo historii cen; toto vyhledávání je zapnuté ve výchozím stavu, jen čte a nepotřebuje účet. Jen když zapnete \"Đóng góp phát hiện sản phẩm mới (Opt-In)\" (ve výchozím stavu vypnuto), posílá také danou nabídku s hodnocením a počtem hodnocení do fronty ke kontrole; tato data sama veřejné ceny nemění. Rozšíření neposílá účet ani osobní údaje, neotevírá stránky, které jste neotevřeli, a ve vašem prohlížeči ukládá jen adresu serveru, volbu opt-in a mezipaměť platnou po dobu relace s produkty, které jste si prohlédli, vaším posledním hledáním a výsledky vyhledávání. Text, který napíšete do vyhledávacího pole rozšíření, se také odešle k vyhledání.", + "privacyS5Desc": "Na stránkách produktů 10 podporovaných obchodů rozšíření posílá Rate.vn obchod, URL produktu bez parametrů, název produktu, ID položky a obchodu, zvolenou variantu, cenu a původní cenu, aby vyhledalo historii cen; toto vyhledávání je zapnuté ve výchozím stavu, jen čte a nepotřebuje účet. Text, který napíšete do vyhledávacího pole rozšíření, se také odešle k vyhledání. Jen když zapnete \"Pomoc s hledáním nových produktů (jen když ji zapnete)\" (ve výchozím stavu vypnuto), posílá také danou nabídku s hodnocením a počtem hodnocení do fronty ke kontrole, uloženou s časem, kdy ji server přijal; servery Rate.vn mohou nabídku v obchodě znovu ověřit a tato data sama veřejné ceny nemění. Rozšíření neposílá účet, jméno, e-mail ani cookies, neotevírá stránky, které jste neotevřeli, a ve vašem prohlížeči ukládá jen adresu serveru, volbu opt-in a mezipaměť platnou po dobu relace s produkty, které jste si prohlédli, vaším posledním hledáním, výsledky vyhledávání a dobou čekání na další vyhledání. Jako u každého webového požadavku server obdrží vaši IP adresu: kód Rate.vn ji používá jen v paměti pro limit požadavků za minutu a náš poskytovatel hostingu uchovává běžné přístupové záznamy po krátkou dobu.", "privacySide1Title": "Ověřte si otevřený kód", "privacySide1Desc": "Pravidla přístupu k datům a způsob ukládání hlasů jsou veřejné v repozitáři na GitHubu.", "privacySide1Btn": "Technická a API dokumentace", diff --git a/apps/web/locales/en.json b/apps/web/locales/en.json index 183324159..b7d1b41df 100644 --- a/apps/web/locales/en.json +++ b/apps/web/locales/en.json @@ -1534,7 +1534,7 @@ "privacyS4Title": "4. Sellers' private details", "privacyS4Desc": "A seller's private details (phone number, bank account) are kept in fields the public cannot read. The website and the API can only read the public fields.", "privacyS5Title": "5. The Rate.vn browser extension", - "privacyS5Desc": "On product pages of the 10 supported stores, the extension sends Rate.vn the store, the product URL with tracking parameters removed, the product title, the item and shop ids, the selected variant, the price and the original price to look up price history; this lookup is on by default, read-only and needs no account. Only if you turn on \"Đóng góp phát hiện sản phẩm mới (Opt-In)\" (off by default) does it also send that listing, with its rating and rating count, to a moderation queue; this data does not change public prices by itself. The extension sends no account or personal data, opens no page you did not open, and keeps only the server address, the opt-in choice and a session-only cache of the products you viewed, your latest search and their lookup results in your browser. Text you type into the extension's search box is also sent to look it up.", + "privacyS5Desc": "On product pages of the 10 supported stores, the extension sends Rate.vn the store, the product URL without its parameters, the product title, the item and shop ids, the selected variant, the price and the original price to look up price history; this lookup is on by default, read-only and needs no account. Text you type into the extension's search box is also sent to look it up. Only if you turn on \"Help find new products (only when you turn it on)\" (off by default) does it also send that listing, with its rating and rating count, to a moderation queue, stored with the time the server received it; Rate.vn's servers may re-check that listing on the store, and this data does not change public prices by itself. The extension sends no account, name, email or cookies, opens no page you did not open, and keeps only the server address, the opt-in choice and a session-only cache of the products you viewed, your latest search, their lookup results and a lookup wait time in your browser. Like any web request, the server receives your IP address: Rate.vn's code uses it only in memory for a per-minute request limit, and our hosting provider keeps standard access logs for a short time.", "privacySide1Title": "Check the open code", "privacySide1Desc": "The data access rules and how votes are stored are public in the GitHub repository.", "privacySide1Btn": "Technical and API docs", diff --git a/apps/web/locales/vi.json b/apps/web/locales/vi.json index 7d35d1d36..a7009f971 100644 --- a/apps/web/locales/vi.json +++ b/apps/web/locales/vi.json @@ -1534,7 +1534,7 @@ "privacyS4Title": "4. Thông tin riêng của gian hàng", "privacyS4Desc": "Thông tin riêng của gian hàng (số điện thoại, tài khoản ngân hàng) nằm ở các trường mà công chúng không đọc được. Trang web và API chỉ đọc được các trường công khai.", "privacyS5Title": "5. Tiện ích trình duyệt Rate.vn", - "privacyS5Desc": "Trên trang sản phẩm của 10 sàn được hỗ trợ, tiện ích gửi cho Rate.vn sàn, URL sản phẩm đã cắt tham số theo dõi, tên sản phẩm, mã sản phẩm và mã gian hàng, phiên bản đang chọn, giá bán và giá gốc để tra lịch sử giá; việc tra cứu này bật sẵn, chỉ đọc và không cần tài khoản. Chỉ khi bạn bật \"Đóng góp phát hiện sản phẩm mới (Opt-In)\" (mặc định tắt), tiện ích còn gửi tin bán đó kèm số sao và số lượt đánh giá vào hàng đợi kiểm duyệt; dữ liệu này không tự đổi giá công khai. Tiện ích không gửi tài khoản hay thông tin cá nhân, không mở trang bạn không mở, và chỉ lưu trên trình duyệt của bạn địa chỉ máy chủ, lựa chọn có đóng góp hay không, cùng bộ nhớ tạm theo phiên gồm các sản phẩm bạn vừa xem, lần tìm kiếm gần nhất và kết quả tra cứu. Nội dung bạn gõ vào ô tìm kiếm của tiện ích cũng được gửi để tra cứu.", + "privacyS5Desc": "Trên trang sản phẩm của 10 sàn được hỗ trợ, tiện ích gửi cho Rate.vn sàn, URL sản phẩm đã bỏ tham số, tên sản phẩm, mã sản phẩm và mã gian hàng, phiên bản đang chọn, giá bán và giá gốc để tra lịch sử giá; việc tra cứu này bật sẵn, chỉ đọc và không cần tài khoản. Nội dung bạn gõ vào ô tìm kiếm của tiện ích cũng được gửi để tra cứu. Chỉ khi bạn bật \"Đóng góp phát hiện sản phẩm mới (chỉ khi bạn bật)\" (mặc định tắt), tiện ích còn gửi tin bán đó kèm số sao và số lượt đánh giá vào hàng đợi kiểm duyệt, lưu kèm thời điểm máy chủ nhận; máy chủ Rate.vn có thể kiểm tra lại tin bán đó trên sàn, và dữ liệu này không tự đổi giá công khai. Tiện ích không gửi tài khoản, tên, email hay cookie, không mở trang bạn không mở, và chỉ lưu trên trình duyệt của bạn địa chỉ máy chủ, lựa chọn có đóng góp hay không, cùng bộ nhớ tạm theo phiên gồm các sản phẩm bạn vừa xem, lần tìm kiếm gần nhất, kết quả tra cứu và thời gian chờ khi bị giới hạn tra cứu. Như mọi yêu cầu web, máy chủ nhận địa chỉ IP của bạn: mã của Rate.vn chỉ dùng nó trong bộ nhớ để giới hạn số yêu cầu mỗi phút, còn nhà cung cấp hosting giữ nhật ký truy cập thông thường trong thời gian ngắn.", "privacySide1Title": "Kiểm tra mã nguồn công khai", "privacySide1Desc": "Quy tắc phân quyền dữ liệu và cách lưu bình chọn đều công khai trong repository trên GitHub.", "privacySide1Btn": "Xem tài liệu kỹ thuật và API", diff --git a/packages/shared-data/src/__tests__/shared-data.test.ts b/packages/shared-data/src/__tests__/shared-data.test.ts index 8ddfadf0e..34fe87ffc 100644 --- a/packages/shared-data/src/__tests__/shared-data.test.ts +++ b/packages/shared-data/src/__tests__/shared-data.test.ts @@ -2035,6 +2035,120 @@ function testProductPoints(): void { reversed.clusters.map((cl) => [cl.key, cl.support, cl.lineIds]), crowded.clusters.map((cl) => [cl.key, cl.support, cl.lineIds]) ); + + // Lines AI extracted from marketplace reviews (DOS/Rate#1531, design + // 2026-11-03 section 9, D4 option b): each review is one writer, adding + // 0.5 x min(decayed weight, 1). + const ai = ( + writerId: string, + body: string, + extra: Partial = {} + ) => line(writerId, 'pro', body, { method: 'ai', ...extra }); + assert.equal(rules.POINT_AI_WEIGHT_SHARE, 0.5); + // Two AI lines, weight 3 each: 0.5 + 0.5 = 1, under 1.5; capped before the + // half, so heavy reviews do not reach it with two. + const twoAi = run([ + ai('shop-a', 'Pin dùng được cả ngày', { weight: 3 }), + ai('shop-b', 'Pin dùng được cả ngày', { weight: 3 }), + ]); + assert.equal(twoAi.clusters.length, 1); + assert.equal(twoAi.clusters[0]!.support, 1); + assert.equal(twoAi.clusters[0]!.reviewerCount, 2); + assert.equal(twoAi.clusters[0]!.visible, false); + // Three AI lines reach 1.5 and show. + const threeAi = run([ + ai('shop-a', 'Pin dùng được cả ngày'), + ai('shop-b', 'Pin dùng được cả ngày'), + ai('shop-c', 'Pin dùng được cả ngày'), + ]); + assert.equal(threeAi.clusters[0]!.support, 1.5); + assert.equal(threeAi.clusters[0]!.visible, true); + // One wrong extraction never shows alone, however heavy its review. + assert.equal( + run([ai('shop-a', 'Camera chụp đêm rất đẹp', { weight: 3 })]).clusters[0]! + .visible, + false + ); + // An AI line adds to a writer's point: one writer (1) plus one AI line + // (0.5) reaches 1.5 with two writers. + const mixed = run([ + line('user-1', 'pro', 'Pin dùng được cả ngày'), + ai('shop-a', 'Pin dùng được cả ngày'), + ]); + assert.equal(mixed.clusters[0]!.support, 1.5); + assert.equal(mixed.clusters[0]!.visible, true); + // Decay applies before the cap and the half: a two-year-old AI line adds 0.13. + assert.equal( + run([ + ai('shop-a', 'Pin dùng được cả ngày', { + writtenAt: monthsAgo(24), + weight: 1, + }), + ]).clusters[0]!.support, + 0.13 + ); + // An AI line is never an expert's, so it cannot show a point alone. + const aiExpert = run([ + ai('shop-a', 'Pin dùng được cả ngày', { isExpert: true }), + ]); + assert.equal(aiExpert.clusters[0]!.hasExpert, false); + assert.equal(aiExpert.clusters[0]!.visible, false); + // The hide switch drops every AI line, as if none were stored. + const hidden = rules.aggregateProductPoints( + 'smartphones', + [ + line('user-1', 'pro', 'Pin dùng được cả ngày'), + ai('shop-a', 'Pin dùng được cả ngày'), + ai('shop-b', 'Pin dùng được cả ngày'), + ai('shop-c', 'Camera chụp đêm rất đẹp'), + ], + now, + { hideAiLines: true } + ); + assert.equal(hidden.clusters.length, 1); + assert.equal(hidden.clusters[0]!.support, 1); + assert.equal(hidden.clusters[0]!.reviewerCount, 1); + assert.equal(hidden.reviewCount, 1); + // A writer's words label the point, even when AI lines outnumber them. + const labelled = run([ + ai('shop-a', 'Pin dùng được cả ngày luôn'), + ai('shop-b', 'Pin dùng được cả ngày luôn'), + line('user-1', 'pro', 'Pin dùng được cả ngày'), + ]); + assert.equal(labelled.clusters.length, 1); + assert.equal(labelled.clusters[0]!.label, 'Pin dùng được cả ngày'); + // Points with a writer's own line rank before AI-only ones, whatever the + // support; AI-only points stay visible below them (PO decision, #1598). + const ranked = run([ + ...['a', 'b', 'c', 'd', 'e', 'f'].map((w) => + ai(`shop-${w}`, 'Màn hình sáng rõ ngoài trời') + ), + line('user-1', 'pro', 'Pin dùng được cả ngày'), + line('user-2', 'pro', 'Pin dùng được cả ngày'), + ]); + assert.deepEqual( + ranked.summary.pros.map((p) => [ + p.label, + p.support, + p.reviewerCount, + p.aiReviewerCount, + ]), + [ + ['Pin dùng được cả ngày', 2, 2, 0], + ['Màn hình sáng rõ ngoài trời', 3, 6, 6], + ] + ); + assert.equal(ranked.reviewCount, 8); + assert.equal(ranked.aiReviewCount, 6); + // A mixed point counts its marketplace reviews and ranks as writer-backed. + assert.equal(mixed.clusters[0]!.aiReviewerCount, 1); + assert.equal(mixed.aiReviewCount, 1); + // A line without a method is the writer's own, at full weight. + assert.equal( + run([line('user-1', 'pro', 'Pin dùng được cả ngày', { weight: 2 })]) + .clusters[0]!.support, + 2 + ); } function testReviewDepth(): void { diff --git a/packages/shared-data/src/database.types.ts b/packages/shared-data/src/database.types.ts index a45ea342c..f54114ed0 100644 --- a/packages/shared-data/src/database.types.ts +++ b/packages/shared-data/src/database.types.ts @@ -777,6 +777,127 @@ export type Database = { }; Relationships: []; }; + review_ai_batches: { + Row: { + cache_read_tokens: number; + cache_write_tokens: number; + cost_usd: number | null; + ended_at: string | null; + id: string; + input_tokens: number; + manifest: Json; + model: string; + output_tokens: number; + prompt_version: string; + request_count: number; + reserved_usd: number; + review_count: number; + status: string; + stored_at: string | null; + submitted_at: string; + }; + Insert: { + cache_read_tokens?: number; + cache_write_tokens?: number; + cost_usd?: number | null; + ended_at?: string | null; + id: string; + input_tokens?: number; + manifest: Json; + model: string; + output_tokens?: number; + prompt_version: string; + request_count: number; + reserved_usd: number; + review_count: number; + status?: string; + stored_at?: string | null; + submitted_at?: string; + }; + Update: { + cache_read_tokens?: number; + cache_write_tokens?: number; + cost_usd?: number | null; + ended_at?: string | null; + id?: string; + input_tokens?: number; + manifest?: Json; + model?: string; + output_tokens?: number; + prompt_version?: string; + request_count?: number; + reserved_usd?: number; + review_count?: number; + status?: string; + stored_at?: string | null; + submitted_at?: string; + }; + Relationships: []; + }; + review_ai_extractions: { + Row: { + attempts: number; + batch_id: string | null; + category_slug: string; + criteria_version: number; + evidence: Json; + lines_prompt_version: string | null; + model: string; + outcome: string; + point_count: number; + processed_at: string; + prompt_version: string; + rejected_count: number; + review_id: string; + text_hash: string; + }; + Insert: { + attempts?: number; + batch_id?: string | null; + category_slug: string; + criteria_version: number; + evidence?: Json; + lines_prompt_version?: string | null; + model: string; + outcome: string; + point_count?: number; + processed_at?: string; + prompt_version: string; + rejected_count?: number; + review_id: string; + text_hash: string; + }; + Update: { + attempts?: number; + batch_id?: string | null; + category_slug?: string; + criteria_version?: number; + evidence?: Json; + lines_prompt_version?: string | null; + model?: string; + outcome?: string; + point_count?: number; + processed_at?: string; + prompt_version?: string; + rejected_count?: number; + review_id?: string; + text_hash?: string; + }; + Relationships: [ + { + foreignKeyName: 'review_ai_extractions_batch_id_fkey'; + columns: ['batch_id']; + referencedRelation: 'review_ai_batches'; + referencedColumns: ['id']; + }, + { + foreignKeyName: 'review_ai_extractions_review_id_fkey'; + columns: ['review_id']; + referencedRelation: 'reviews'; + referencedColumns: ['id']; + }, + ]; + }; review_analysis: { Row: { ai_summary: string | null; @@ -1404,6 +1525,10 @@ export type Database = { Args: { p_retention_days?: number }; Returns: number; }; + purge_review_ai_points: { + Args: { p_prompt_version: string }; + Returns: Json; + }; record_price_observation: { Args: { p_observation_id?: string; @@ -1446,6 +1571,7 @@ export type Database = { input_hash: string; }[]; }; + review_ai_text_hash: { Args: { p_text: string }; Returns: string }; review_completeness_factor: { Args: { p_breakdown: Json }; Returns: number; @@ -1566,6 +1692,7 @@ export type Database = { Args: { p_params?: Json; p_source: string }; Returns: string; }; + store_review_ai_points: { Args: { p_rows: Json }; Returns: Json }; store_review_values: { Args: { p_values: Json }; Returns: number }; submit_community_review: { Args: { diff --git a/packages/shared-data/src/rules/product-points.ts b/packages/shared-data/src/rules/product-points.ts index 5fa334d05..4ed7b1694 100644 --- a/packages/shared-data/src/rules/product-points.ts +++ b/packages/shared-data/src/rules/product-points.ts @@ -2,8 +2,10 @@ * A product's pros and cons, built from its reviews' pro and con lines * (DOS/Rate#1546, design #1531 section 5.1, D9). Code only, no AI: lines are * normalised, given a criterion, clustered by shared words, counted once per - * writer by trust weight, decayed by age and filtered for display. AI later - * rewords a cluster's label; it never decides what is shown. + * writer by trust weight, decayed by age and filtered for display. Lines AI + * extracted from marketplace reviews count at half weight + * (POINT_AI_WEIGHT_SHARE), so they can add to a point but one wrong + * extraction never shows alone. * * Plain data and functions only, like the rest of `rules`, and no database: * the importer job reads `rate.review_points` into PointLine values and @@ -35,6 +37,15 @@ export const POINT_MIN_SUPPORT = 1.5; export const POINT_WRITER_SHARE_CAP = 1; /** A writer's weight counts at most this, as in the product rating. */ export const POINT_WEIGHT_CAP = 3; +/** + * A line AI extracted from a marketplace review (`method: 'ai'`) counts this + * share of its writer's decayed weight, after the per-writer cap + * (POINT_WRITER_SHARE_CAP): 0.5 x min(weight, 1). Its review still counts as + * one writer, so a point from marketplace reviews alone needs three of them + * to reach POINT_MIN_SUPPORT, and one wrong extraction never shows on its own + * (DOS/Rate#1531, `2026-11-03-ai-review-points.html` section 9, D4 option b). + */ +export const POINT_AI_WEIGHT_SHARE = 0.5; /** Pros and cons shown per criterion, each side. */ export const PRODUCT_CRITERION_POINTS_MAX = 3; /** Pros and cons shown in the product summary, each side. */ @@ -66,6 +77,21 @@ export interface PointLine { isExpert: boolean; /** When the review was written (ISO string or Date). */ writtenAt: string | Date; + /** + * `rate.review_points.method`: 'ai' for a line extracted by AI from a + * marketplace review, which counts at POINT_AI_WEIGHT_SHARE and never as + * an expert's; the writer's own lines otherwise. + */ + method?: 'writer' | 'ai'; +} + +/** Options of aggregateProductPoints. */ +export interface ProductPointsOptions { + /** + * Leave out every line AI extracted (`method: 'ai'`), as if none were + * stored: the switch that hides them without deleting anything. + */ + hideAiLines?: boolean; } /** A cluster of lines, as `rate.product_points` stores it. */ @@ -79,6 +105,12 @@ export interface PointCluster { /** Distinct writers' decayed, capped weights, at two decimals. */ support: number; reviewerCount: number; + /** + * Of reviewerCount, the marketplace reviews counted only through lines AI + * extracted ("trong đó N review trên sàn"). A point with a writer's own line + * has reviewerCount > aiReviewerCount and ranks before AI-only points. + */ + aiReviewerCount: number; hasExpert: boolean; firstSeen: Date; lastSeen: Date; @@ -102,6 +134,8 @@ export interface ProductPoints { summary: PointSides; /** Distinct reviews whose lines were counted ("Tóm tắt từ N review"). */ reviewCount: number; + /** Of reviewCount, those counted only through AI lines ("trong đó N review trên sàn"). */ + aiReviewCount: number; /** The run's clock, shown as "Cập nhật ngày ...". */ updatedAt: Date; } @@ -432,7 +466,8 @@ function matchScore(a: readonly string[], b: readonly string[]): number { function countedLines( categorySlug: string | null | undefined, lines: readonly PointLine[], - now: Date + now: Date, + hideAiLines: boolean ): Line[] { const counted: Line[] = []; const seen = new Set(); @@ -440,6 +475,7 @@ function countedLines( // A line read twice counts once. if (seen.has(source.id)) continue; seen.add(source.id); + if (hideAiLines && isAiLine(source)) continue; if (source.kind !== 'pro' && source.kind !== 'con') continue; if (!Number.isFinite(source.weight) || source.weight <= 0) continue; if (typeof source.writerId !== 'string' || !source.writerId.trim()) @@ -508,8 +544,12 @@ function clusterLines(lines: readonly Line[]): Line[][] { /** The label: the form most writers used, in its first writer's words. */ function clusterLabel(lines: readonly Line[]): string { + // The writers' own words when the cluster has any; AI lines label a + // cluster only of AI lines (DOS/Rate#1531). + const written = lines.filter((line) => !isAiLine(line.source)); + const source = written.length > 0 ? written : lines; const forms = new Map; first: Line }>(); - for (const line of lines) { + for (const line of source) { const entry = forms.get(line.form); if (entry) entry.writers.add(line.writer); else @@ -524,18 +564,42 @@ function clusterLabel(lines: readonly Line[]): string { return best!.first.source.body.trim().replace(/\s+/g, ' '); } +function isAiLine(line: PointLine): boolean { + return line.method === AI_METHOD; +} + +// A constant, because the rules import check reads a quote on an export +// line as a module path. +const AI_METHOD = 'ai'; + +/** + * What a line adds for its writer: its decayed weight, or for an AI line + * POINT_AI_WEIGHT_SHARE of that weight capped at POINT_WRITER_SHARE_CAP. + */ +function lineSupport(line: Line, now: Date): number { + const weight = decayedWeight(line.source.weight, line.writtenAt, now); + return isAiLine(line.source) + ? POINT_AI_WEIGHT_SHARE * Math.min(weight, POINT_WRITER_SHARE_CAP) + : weight; +} + function summarise(lines: readonly Line[], now: Date): PointCluster { const leader = lines[0]!; // Each writer counts once, with their strongest line in the cluster. const writers = new Map(); + // Writers with a line of their own; the rest are AI-only. + const ownWords = new Set(); let hasExpert = false; for (const line of lines) { - const weight = decayedWeight(line.source.weight, line.writtenAt, now); + if (!isAiLine(line.source)) ownWords.add(line.writer); + const weight = lineSupport(line, now); writers.set(line.writer, Math.max(writers.get(line.writer) ?? 0, weight)); // An expert counts while their line is fresh: an old expert line does not - // lift a new single writer's point (design 5.3). + // lift a new single writer's point (design 5.3). An AI line is never an + // expert's own words. if ( line.source.isExpert && + !isAiLine(line.source) && now.getTime() - line.writtenAt.getTime() <= POINT_SILENT_MS ) hasExpert = true; @@ -560,6 +624,7 @@ function summarise(lines: readonly Line[], now: Date): PointCluster { label: clusterLabel(lines), support, reviewerCount: writers.size, + aiReviewerCount: writers.size - ownWords.size, hasExpert, firstSeen, lastSeen, @@ -568,9 +633,14 @@ function summarise(lines: readonly Line[], now: Date): PointCluster { }; } -/** Display order: more support, more writers, an expert, then newer. */ +/** + * Display order: a point with at least one writer's own line before an + * AI-only one (DOS/Rate#1531), then more support, more writers, an expert, + * then newer. AI-only points stay eligible below them. + */ function byRank(a: PointCluster, b: PointCluster): number { return ( + Number(hasOwnWords(b)) - Number(hasOwnWords(a)) || b.support - a.support || b.reviewerCount - a.reviewerCount || Number(b.hasExpert) - Number(a.hasExpert) || @@ -579,6 +649,10 @@ function byRank(a: PointCluster, b: PointCluster): number { ); } +function hasOwnWords(cluster: PointCluster): boolean { + return cluster.reviewerCount > cluster.aiReviewerCount; +} + function sides(clusters: readonly PointCluster[], max: number): PointSides { return { pros: clusters.filter((cl) => cl.kind === 'pro').slice(0, max), @@ -586,19 +660,37 @@ function sides(clusters: readonly PointCluster[], max: number): PointSides { }; } +/** Reviews counted only through AI lines. */ +function aiOnlyReviews(lines: readonly Line[]): number { + const own = new Set(); + const ai = new Set(); + for (const line of lines) + (isAiLine(line.source) ? ai : own).add(line.source.reviewId); + let count = 0; + for (const reviewId of ai) if (!own.has(reviewId)) count += 1; + return count; +} + /** * A product's pros and cons from its review lines (design 5.1 and 5.3): * lines of gated reviews (weight 0), dodging cons and lines with no words * are left out; the rest are clustered per criterion and kind, counted once - * per writer and shown under D9. `now` is the run's clock, so a run is - * repeatable. + * per writer and shown under D9. Lines AI extracted count at + * POINT_AI_WEIGHT_SHARE, or not at all with `hideAiLines`. `now` is the run's + * clock, so a run is repeatable. */ export function aggregateProductPoints( categorySlug: string | null | undefined, lines: readonly PointLine[], - now: Date + now: Date, + options: ProductPointsOptions = {} ): ProductPoints { - const counted = countedLines(categorySlug, lines, now); + const counted = countedLines( + categorySlug, + lines, + now, + options?.hideAiLines === true + ); const clusters = clusterLines(counted) .map((cluster) => summarise(cluster, now)) .sort(byRank); @@ -619,6 +711,7 @@ export function aggregateProductPoints( byCriterion, summary, reviewCount: new Set(counted.map((line) => line.source.reviewId)).size, + aiReviewCount: aiOnlyReviews(counted), updatedAt: now, }; } diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index f91e3db04..98f965971 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -130,6 +130,9 @@ importers: apps/importer: dependencies: + '@anthropic-ai/sdk': + specifier: 1.3.0 + version: 1.3.0(zod@4.6.7) '@repo/shared-data': specifier: workspace:* version: link:../../packages/shared-data @@ -347,6 +350,15 @@ packages: resolution: {integrity: sha512-U4+70Pc5ZS9osnCBCE5Jha/ciHM+Yp+CNMNC/7HvYbNRk1Ldd+f7qO65W5qfhu/TCv+/ozljlXXe9Nj8419DMA==} engines: {node: '>=10'} + '@anthropic-ai/sdk@1.3.0': + resolution: {integrity: sha512-k7z9BlLLPxfZH+ov7dS6EnaN/wiZeV8YNT1LZfrpAHIy7oY7pHr4UzirF9f2OlHtFEnRBOX+AZQ91Y5fK+TPTw==} + hasBin: true + peerDependencies: + zod: ^3.25.0 || ^4.0.0 + peerDependenciesMeta: + zod: + optional: true + '@axe-core/playwright@4.13.0': resolution: {integrity: sha512-6YLx+kxXu5GJceG4ozFg+33a2EMTdjYwWGloJ3sb9Kta5pp+ZNS53uxGVog5JetIY8s++P5UrtX+cri+u0VAVg==} peerDependencies: @@ -1639,6 +1651,9 @@ packages: resolution: {integrity: sha512-fCTuZK4QBa+39Oz9l4OGfJfz+GpwCp3AqO7Zch3to99xHPgstVsRFpeQ8LNd2o1Gv8raL2mCFwiaHh7bFSp5DQ==} engines: {node: '>=22'} + '@stablelib/base64@1.0.1': + resolution: {integrity: sha512-1bnPQqSxSuc3Ii6MhBysoWCg58j97aUjuCSZrGSmDxNqtytIi0k8utUenAwTZN4V5mXXYGsVUI9zeBqy+jBOSQ==} + '@supabase/auth-js@2.117.2': resolution: {integrity: sha512-Z3WnGwrphYZubrLbxp5Iv0tLA9A5GhvKzJ/ZXglxqwd2QEH2R4dXRltjVJ8sIn4xEW2BSFIGT0yALVJIgZeDYw==} engines: {node: '>=22.0.0'} @@ -2710,6 +2725,9 @@ packages: fast-levenshtein@2.0.6: resolution: {integrity: sha512-DCXu6Ifhqcks7TZKY3Hxp3y6qphY5SJZmrWMDrKcERSOXWQdMhU9Ig/PYrzyw/ul9jOIyh0N4M0tbC5hodg8dw==} + fast-sha256@1.3.0: + resolution: {integrity: sha512-n11RGP/lrWEFI/bWdygLxhI+pVeo1ZYIVwvvPkW7azl/rOy+F3HYRZ2K5zeE9mmkhQppyv9sQFx0JM9UabnpPQ==} + fast-uri@3.1.9: resolution: {integrity: sha512-hS/0Wt3Ya043oKXs0sIxrMpKQITXhMLIrVAKKXi4gozGC9v7YRWVnXIZzjNPTO02M9BdxLCWqK1yRXgNziVyJQ==} @@ -3139,6 +3157,10 @@ packages: json-parse-even-better-errors@2.3.1: resolution: {integrity: sha512-xyFwyhro/JEof6Ghe2iz2NcXoj2sloNsWr/XsERDK/oiPCfaNhl5ONfp+jQdAZRQQ0IJWNzH9zIZF7li91kh2w==} + json-schema-to-ts@3.1.1: + resolution: {integrity: sha512-+DWg8jCJG2TEnpy7kOm/7/AxaYoaRbjVB4LFZLySZlWn8exGs3A4OLJR966cVvU26N7X9TWxl+Jsw7dzAqKT6g==} + engines: {node: '>=16'} + json-schema-traverse@0.4.1: resolution: {integrity: sha512-xbbCH5dCYU5T8LcEhhuh7HJ88HXuW3qsI3Y0zOZFKfZEHcpWiHU/Jxzk629Brsab/mMiHQti9wMP+845RPe3Vg==} @@ -3838,6 +3860,9 @@ packages: resolution: {integrity: sha512-WjlahMgHmCJpqzU8bIBy4qtsZdU9lRlcZE3Lvyej6t4tuOuv1vk57OW3MBrj6hXBFx/nNoC9MPMTcr5YA7NQbg==} engines: {node: '>=6'} + standardwebhooks@1.1.1: + resolution: {integrity: sha512-bCbX9ZEyFkWPsRz7Bl3NuQUJohmwGSev/yhr7vhaGPlc4AfIrspIRa6cPTBuI1ItmrTDJ4d/S2hCsfe4+vQGnQ==} + stop-iteration-iterator@1.1.0: resolution: {integrity: sha512-eLoXW/DHyl62zxY4SCaIgnRhuMr6ri4juEYARS8E6sCEqzKpOiE521Ucofdx+KnDZl5xmvGYaaKCk5FEOxJCoQ==} engines: {node: '>= 0.4'} @@ -3956,6 +3981,9 @@ packages: resolution: {integrity: sha512-65P7iz6X5yEr1cwcgvQxbbIw7Uk3gOy5dIdtZ4rDveLqhrdJP+Li/Hx6tyK0NEb+2GCyneCMJiGqrADCSNk8sQ==} engines: {node: '>=8.0'} + ts-algebra@2.0.0: + resolution: {integrity: sha512-FPAhNPFMrkwz76P7cdjdmiShwMynZYN6SgOujD1urY4oNm80Ou9oMdmbR45LotcKOXoy7wSmHkRFE6Mxbrhefw==} + ts-api-utils@2.5.0: resolution: {integrity: sha512-OJ/ibxhPlqrMM0UiNHJ/0CKQkoKF243/AEmplt3qpRgkW8VG7IfOS41h7V8TjITqdByHzrjcS/2si+y4lIh8NA==} engines: {node: '>=18.12'} @@ -4158,6 +4186,14 @@ snapshots: '@alloc/quick-lru@5.3.0': {} + '@anthropic-ai/sdk@1.3.0(zod@4.6.7)': + dependencies: + '@opentelemetry/api': 1.9.1 + json-schema-to-ts: 3.1.1 + standardwebhooks: 1.1.1 + optionalDependencies: + zod: 4.6.7 + '@axe-core/playwright@4.13.0(playwright-core@1.64.1)': dependencies: axe-core: 4.13.0 @@ -5225,6 +5261,8 @@ snapshots: '@simple-libs/stream-utils@2.0.0': {} + '@stablelib/base64@1.0.1': {} + '@supabase/auth-js@2.117.2': dependencies: tslib: 2.8.1 @@ -6423,6 +6461,8 @@ snapshots: fast-levenshtein@2.0.6: {} + fast-sha256@1.3.0: {} + fast-uri@3.1.9: {} fastq@1.20.3: @@ -6860,6 +6900,11 @@ snapshots: json-parse-even-better-errors@2.3.1: {} + json-schema-to-ts@3.1.1: + dependencies: + '@babel/runtime': 7.29.7 + ts-algebra: 2.0.0 + json-schema-traverse@0.4.1: {} json-schema-traverse@1.0.0: {} @@ -7590,6 +7635,11 @@ snapshots: dependencies: type-fest: 0.7.1 + standardwebhooks@1.1.1: + dependencies: + '@stablelib/base64': 1.0.1 + fast-sha256: 1.3.0 + stop-iteration-iterator@1.1.0: dependencies: es-errors: 1.3.0 @@ -7727,6 +7777,8 @@ snapshots: dependencies: is-number: 7.0.0 + ts-algebra@2.0.0: {} + ts-api-utils@2.5.0: dependencies: typescript: 6.0.3 diff --git a/scripts/test-review-ai-points-proposal.mjs b/scripts/test-review-ai-points-proposal.mjs new file mode 100644 index 000000000..6e7d93b87 --- /dev/null +++ b/scripts/test-review-ai-points-proposal.mjs @@ -0,0 +1,614 @@ +#!/usr/bin/env node +// Replays the DOS-Me review AI points PROPOSAL (DOS/Rate#1531, D6 of +// Docs/Proposals/2026-11-03-ai-review-points.html) on the migration replay stack: the proposal +// twice, the text hash equal to the job's (NFC, sha256), 'ai' lines and markers stored as +// rate_worker, the skips (missing, writer, stale), the failed retries, the refusals, the grants, +// the purge, every verification query, the rollback twice and the proposal again. Nothing +// touches a real database. +// +// Usage: node scripts/test-review-ai-points-proposal.mjs (needs Docker) + +import assert from 'node:assert/strict'; +import { createHash } from 'node:crypto'; +import { readFileSync } from 'node:fs'; +import path from 'node:path'; +import { spawnSync } from 'node:child_process'; + +import { repoRoot, startRateDb } from './lib/rate-db.mjs'; + +const read = (name) => + readFileSync(path.join(repoRoot, 'Docs', 'Proposals', name), 'utf8'); +const proposal = read('2026-11-03-dos-me-review-ai-points.sql'); +const rollback = read('2026-11-03-dos-me-review-ai-points-rollback.sql'); +const verify = read('2026-11-03-dos-me-review-ai-points-verify.sql'); + +const container = `rate-review-ai-points-proposal-${process.pid}`; +const removeContainer = () => + spawnSync('docker', ['rm', '--force', container], { timeout: 30_000 }); +for (const signal of ['SIGINT', 'SIGTERM']) { + process.on(signal, () => { + removeContainer(); + process.exit(130); + }); +} + +const checks = [ + ...verify.matchAll( + /^-- (V\d+) [^\n]*(?:\n--[^\n]*)*expect 0\n((?:SELECT|WITH)[\s\S]*?;)/gm + ), +].map(([, id, query]) => ({ id, query })); +assert.deepEqual( + checks.map((c) => c.id), + ['V1', 'V2', 'V3', 'V4', 'V5', 'V6', 'V7'], + 'the verification lists its expect-0 queries' +); +const info = verify.match(/^-- V8 [^\n]*\n((?:SELECT|WITH)[\s\S]*?;)/m)?.[1]; +assert.ok(info, 'V8 is there'); + +// The job's hash (apps/importer review-ai-points). +const jobHash = (text) => + createHash('sha256').update(text.normalize('NFC'), 'utf8').digest('hex'); + +const PHONES = 'c0000000-0000-4000-8000-000000000001'; +const LAPTOPS = 'c0000000-0000-4000-8000-000000000002'; +const MERCHANT = 'e0000000-0000-4000-8000-000000000001'; +const MASTER = 'a0000000-0000-4000-8000-000000000001'; +const LAPTOP = 'a0000000-0000-4000-8000-000000000002'; +const LISTING = 'b0000000-0000-4000-8000-000000000001'; +const LAPTOP_LISTING = 'b0000000-0000-4000-8000-000000000002'; +const review = (n) => `d0000000-0000-4000-8000-00000000000${n}`; +// Decomposed diacritics on purpose: the hash must normalize them. +const COMMENT = + 'Pin xài cả ngày vẫn còn, ra nắng hơi khó nhìn. Camera chụp đêm ổn.'.normalize( + 'NFD' + ); +// review: [comment, listing, reviewer_type, platform_review_id] +const REVIEWS = { + 1: [COMMENT, LISTING, 'user', 'p1'], + 2: ['Máy chạy mượt, chơi game không nóng lắm.', LISTING, 'user', 'p2'], + 3: [ + 'Đánh giá viết trên Rate, có ưu nhược riêng.', + LISTING, + 'user', + 'rate_comm_3', + ], + 4: ['Bài đánh giá chuyên gia về máy này.', LISTING, 'expert', 'p4'], + 5: ['Bàn phím gõ êm, màn hình hơi tối.', LAPTOP_LISTING, 'user', 'p5'], + 6: [ + 'Loa to rõ, sạc nhanh, giá hợp lý so với cấu hình.', + LISTING, + 'user', + 'p6', + ], +}; + +const line = (kind, criterion_key, body, evidence, match = 'exact') => ({ + kind, + criterion_key, + body, + evidence, + match, +}); +const row = (n, points, extra = {}) => ({ + id: review(n), + text_hash: jobHash(REVIEWS[n][0]), + prompt_version: 'p1', + criteria_version: 1, + category: 'smartphones', + model: 'claude-haiku-5-5', + batch_id: null, + failed: false, + points, + ...extra, +}); +const POINTS_1 = [ + line('pro', 'battery', 'Pin dùng hết ngày', 'pin xài cả ngày vẫn còn'), + line( + 'con', + 'display', + 'Màn hình khó nhìn ngoài nắng', + 'ra nắng hơi khó nhìn' + ), + line('pro', 'camera', 'Chụp đêm ổn', 'Camera chụp đêm ổn', 'unaccented'), +]; + +try { + const { sql, asRole } = startRateDb({ name: container }); + // Once DOS/Rate mirrors the migration, the reference stack has it: roll it back first. + if ( + sql(`SELECT to_regclass('rate.review_ai_extractions') IS NOT NULL`) === 't' + ) + sql(rollback); + + const checkAll = (when) => { + for (const { id, query } of checks) + assert.equal(sql(query), '0', `${id} ${when}`); + }; + const store = (rows, role = 'rate_worker') => + JSON.parse( + asRole( + role, + `SELECT rate.store_review_ai_points($v$${JSON.stringify(rows)}$v$::jsonb)` + ) + ); + const lines = (n) => + sql( + `SELECT coalesce(string_agg(kind || ':' || coalesce(criterion_key, '-') || ':' || position || ':' || body, '|' + ORDER BY kind DESC, position), '') + FROM rate.review_points WHERE review_id = '${review(n)}' AND method = 'ai'` + ); + const marker = (n) => + sql( + `SELECT coalesce((SELECT outcome || ',' || point_count || ',' || rejected_count || ',' || attempts + || ',' || prompt_version || ',' || coalesce(lines_prompt_version, '-') + || ',' || coalesce(batch_id, '-') + FROM rate.review_ai_extractions WHERE review_id = '${review(n)}'), 'none')` + ); + const counts = (over) => ({ + stored: 0, + empty: 0, + rejected: 0, + failed: 0, + missing: 0, + not_crawl: 0, + writer: 0, + stale: 0, + unquoted: 0, + done: 0, + exhausted: 0, + ...over, + }); + + const reviewRows = Object.entries(REVIEWS) + .map( + ([n, [comment, listing, type, platformId]]) => + `('${review(n)}', '${listing}', 4, $c$${comment}$c$, '${type}', '${platformId}')` + ) + .join(',\n'); + sql(` + INSERT INTO rate.categories (id, name, slug) VALUES + ('${PHONES}', 'Điện thoại', 'smartphones'), ('${LAPTOPS}', 'Laptop', 'laptops'); + INSERT INTO rate.merchants (id, platform, platform_shop_id, name, slug) + VALUES ('${MERCHANT}', 'shopee', 's1', 'Shop 1', 'shop-1'); + INSERT INTO rate.master_products (id, brand, model, model_key, name, slug, category_id, status, + storage_gb, min_price, max_price) + VALUES ('${MASTER}', 'Acme', 'Phone X', 'acme:phone-x:128gb', 'Acme Phone X 128GB', + 'acme-phone-x-128gb', '${PHONES}', 'published', 128, 10000000, 10000000), + ('${LAPTOP}', 'Acme', 'Book 14', 'acme:book-14', 'Acme Book 14', + 'acme-book-14', '${LAPTOPS}', 'published', NULL, 20000000, 20000000); + INSERT INTO rate.products (id, merchant_id, platform, platform_item_id, title, price_current, url, cluster_id) + VALUES ('${LISTING}', '${MERCHANT}', 'shopee', 'x128', 'Acme Phone X 128GB', 10000000, + 'https://shopee.vn/x128', '${MASTER}'), + ('${LAPTOP_LISTING}', '${MERCHANT}', 'shopee', 'b14', 'Acme Book 14', 20000000, + 'https://shopee.vn/b14', '${LAPTOP}'); + INSERT INTO rate.reviews (id, product_id, rating_star, comment, reviewer_type, platform_review_id) + VALUES ${reviewRows}; + INSERT INTO rate.review_points (review_id, criterion_key, kind, body, position, method) + VALUES ('${review(2)}', NULL, 'pro', 'Máy chạy mượt', 0, 'writer'); + `); + assert.equal( + sql(`SELECT string_agg(origin, ',' ORDER BY id) FROM rate.reviews`), + 'crawl,crawl,community,crawl,crawl,crawl' + ); + const ratingBefore = sql( + `SELECT coalesce(rating_score::text, '-') || ',' || coalesce(rating_weight_sum::text, '-') + FROM rate.master_products WHERE id = '${MASTER}'` + ); + + // Applied twice; nothing reads 'ai' lines, so no rating moves. + sql(proposal); + sql(proposal); + checkAll('after the proposal'); + + // The hash equals the job's, for decomposed and composed text, emoji and the empty string. + for (const text of [ + COMMENT, + COMMENT.normalize('NFC'), + '', + 'Máy đẹp 😍👍 giao nhanh', + 'é ế', + ]) + assert.equal( + sql(`SELECT rate.review_ai_text_hash($t$${text}$t$)`), + jobHash(text), + `hash of ${JSON.stringify(text)}` + ); + assert.equal( + sql( + `SELECT rate.review_ai_text_hash(comment) FROM rate.reviews WHERE id = '${review(1)}'` + ), + jobHash(COMMENT.normalize('NFC')) + ); + + // A batch row as the job writes it, as rate_worker. + asRole( + 'rate_worker', + `INSERT INTO rate.review_ai_batches (id, model, prompt_version, request_count, review_count, + manifest, reserved_usd) + VALUES ('msgbatch_replay01', 'claude-haiku-5-5', 'p1', 1, 3, + '[{"custom_id": "r0", "category": "smartphones", "criteria_version": 1, "reviews": []}]', 0.05)` + ); + + // Stored as rate_worker: lines, positions per kind, the marker and its evidence. + assert.deepEqual( + store([ + row(1, POINTS_1, { batch_id: 'msgbatch_replay01', rejected: 1 }), + row(6, []), + row(6, [], { rejected: 2 }), + ]), + counts({ stored: 1, rejected: 1 }), + 'the last row of a review wins' + ); + assert.equal( + lines(1), + 'pro:battery:0:Pin dùng hết ngày|pro:camera:1:Chụp đêm ổn|con:display:0:Màn hình khó nhìn ngoài nắng' + ); + assert.equal(marker(1), 'stored,3,1,1,p1,p1,msgbatch_replay01'); + assert.equal(marker(6), 'rejected,0,2,1,p1,-,-'); + assert.equal( + sql( + `SELECT string_agg((e ->> 'position') || ':' || (e ->> 'match') || ':' || (e ->> 'evidence'), '|' ORDER BY o) + FROM rate.review_ai_extractions x, jsonb_array_elements(x.evidence) WITH ORDINALITY AS t(e, o) + WHERE x.review_id = '${review(1)}'` + ), + '0:exact:pin xài cả ngày vẫn còn|0:exact:ra nắng hơi khó nhìn|1:unaccented:Camera chụp đêm ổn' + ); + checkAll('after storing'); + + // Stored again with fewer lines: replaced, not added. Empty: lines gone, marker empty. + assert.deepEqual( + store([row(1, POINTS_1.slice(0, 1), { prompt_version: 'p2' })]), + counts({ stored: 1 }) + ); + assert.equal(lines(1), 'pro:battery:0:Pin dùng hết ngày'); + assert.equal(marker(1), 'stored,1,0,1,p2,p2,-'); + checkAll('after a re-store'); + + // Skips: a missing review, a community and an expert review, a review with a writer line, a + // changed comment, an exact quote not in the comment; another category is stored. + assert.deepEqual( + store([ + { ...row(1, POINTS_1), id: 'd0000000-0000-4000-8000-0000000000ff' }, + row(3, []), + row(4, []), + row(2, POINTS_1.slice(0, 1)), + row(6, POINTS_1.slice(0, 1), { text_hash: jobHash('another text') }), + row(5, [line('pro', 'keyboard', 'Bàn phím gõ êm', 'Bàn phím gõ êm')], { + category: 'laptops', + }), + ]), + counts({ missing: 1, not_crawl: 2, writer: 1, stale: 1, stored: 1 }) + ); + assert.equal(lines(2), '', 'writer lines are never mixed with ai lines'); + assert.equal(marker(2), 'none'); + assert.equal(marker(3), 'none'); + assert.equal( + marker(6), + 'rejected,0,2,1,p1,-,-', + 'a stale row changes nothing' + ); + assert.equal(lines(5), 'pro:keyboard:0:Bàn phím gõ êm'); + // A made-up quote marked exact is not stored; an unaccented one is the job's to check. + assert.deepEqual( + store([row(6, [line('pro', null, 'Loa nghe rất to', 'loa nghe to lắm')])]), + counts({ unquoted: 1 }) + ); + assert.equal(marker(6), 'rejected,0,2,1,p1,-,-'); + // The laptop listing moves to the phone master: the laptop row is now stale, and V5 still + // reads the keys against the category the marker was processed with. + sql( + `UPDATE rate.products SET cluster_id = '${MASTER}' WHERE id = '${LAPTOP_LISTING}'` + ); + assert.deepEqual( + store([row(5, [], { category: 'laptops' })]), + counts({ stale: 1 }) + ); + checkAll('after a category move'); + sql( + `UPDATE rate.products SET cluster_id = '${LAPTOP}' WHERE id = '${LAPTOP_LISTING}'` + ); + + // Failed: a late failure never replaces a result of the same key; under a new key it counts + // attempts and keeps the lines with the version that wrote them; a fourth attempt is refused. + assert.deepEqual( + store([row(1, [], { prompt_version: 'p2', failed: true })]), + counts({ done: 1 }) + ); + assert.equal(marker(1), 'stored,1,0,1,p2,p2,-'); + assert.deepEqual( + store([row(1, [], { prompt_version: 'p3', failed: true })]), + counts({ failed: 1 }) + ); + assert.equal(marker(1), 'failed,1,0,1,p3,p2,-'); + store([row(1, [], { prompt_version: 'p3', failed: true })]); + store([row(1, [], { prompt_version: 'p3', failed: true })]); + assert.equal(marker(1), 'failed,1,0,3,p3,p2,-'); + assert.deepEqual( + store([row(1, [], { prompt_version: 'p3', failed: true })]), + counts({ exhausted: 1 }) + ); + assert.equal(marker(1), 'failed,1,0,3,p3,p2,-'); + assert.equal( + lines(1), + 'pro:battery:0:Pin dùng hết ngày', + 'a failure keeps the lines' + ); + checkAll('after failures'); + // Purging p2 removes the p2 lines kept under the p3 failure, with their marker. + assert.deepEqual( + JSON.parse( + asRole('rate_worker', `SELECT rate.purge_review_ai_points('p2')`) + ), + { lines: 1, markers: 1 } + ); + assert.equal(lines(1), ''); + assert.equal(marker(1), 'none'); + assert.deepEqual( + store([row(1, [], { prompt_version: 'p3' })]), + counts({ empty: 1 }) + ); + assert.equal(marker(1), 'empty,0,0,1,p3,-,-'); + checkAll('after a purge of a kept version'); + + // Refusals: one bad row refuses the call and writes nothing. + const refuse = (rows, pattern, note) => + assert.throws(() => store(rows), pattern, note); + const shape = /row 1 is not a review AI result/; + const lineShape = /row 1 has a line that is not a review AI line/; + const range = /row 1 has a line out of range/; + const good = row(6, POINTS_1.slice(0, 1)); + const bad = (change) => [good, { ...good, ...change }]; + const badLine = (change) => bad({ points: [{ ...POINTS_1[0], ...change }] }); + refuse({ id: 1 }, /p_rows must be a JSON array/); + refuse( + Array.from({ length: 501 }, () => good), + /at most 500 rows and 2 MB/ + ); + refuse(bad({ id: 'not-a-uuid' }), shape); + refuse(bad({ text_hash: 'ABC' }), shape); + refuse(bad({ prompt_version: 'P 1' }), shape); + refuse(bad({ criteria_version: 1.5 }), shape); + refuse(bad({ criteria_version: '1' }), shape); + refuse(bad({ model: 'Claude Haiku' }), shape); + refuse(bad({ batch_id: 'x' }), shape); + refuse(bad({ failed: 'yes' }), shape); + refuse(bad({ rejected: 21 }), shape); + refuse(bad({ note: 'x' }), shape); + refuse(bad({ failed: true }), shape, 'a failed row carries no points'); + refuse(bad({ points: Array.from({ length: 7 }, () => POINTS_1[0]) }), shape); + refuse( + bad({ criteria_version: 9 }), + /row 1 has no criteria list smartphones\/9/ + ); + refuse(bad({ category: 'services_x' }), /row 1 has no criteria list/); + refuse(badLine({ kind: 'neutral' }), lineShape); + refuse(badLine({ match: 'fuzzy' }), lineShape); + refuse(badLine({ body: 7 }), lineShape); + refuse(badLine({ extra: 1 }), lineShape); + refuse( + bad({ + points: [ + { + kind: 'pro', + body: 'Pin dùng hết ngày', + evidence: 'a b', + match: 'exact', + }, + ], + }), + lineShape + ); + refuse( + badLine({ criterion_key: 'keyboard' }), + range, + 'a key of another category' + ); + refuse(badLine({ body: 'Tốt' }), range); + refuse(badLine({ body: 'x'.repeat(151) }), range); + refuse(badLine({ body: 'aaaaaaa' }), range); + refuse( + badLine({ kind: 'con', body: 'Không có gì để chê' }), + range, + 'an evasive con' + ); + refuse(badLine({ evidence: 'pin' }), range, 'one word of evidence'); + refuse(badLine({ evidence: 'p '.repeat(200) }), range); + refuse( + bad({ + points: [ + POINTS_1[0], + { ...POINTS_1[0], kind: 'con', criterion_key: null }, + ], + }), + range, + 'one line as a pro and a con' + ); + refuse( + bad({ + points: Array.from({ length: 4 }, (_, i) => ({ + ...POINTS_1[0], + body: `Pin dùng tốt số ${i + 1}`, + })), + }), + range, + 'four pros' + ); + const noFlag = { ...good }; + delete noFlag.failed; + refuse( + [good, { ...noFlag, points: [] }], + shape, + 'a result without its failed flag' + ); + refuse(bad({ failed: null }), shape); + refuse(bad({ prompt_version: 'all' }), shape); + assert.throws( + () => + store([row(1, POINTS_1.slice(0, 1), { batch_id: 'msgbatch_unknown' })]), + /review_ai_extractions_batch_id_fkey/, + 'an unknown batch id' + ); + for (const role of ['anon', 'authenticated']) + assert.throws( + () => store([good], role), + /permission denied for function store_review_ai_points/ + ); + assert.equal( + marker(6), + 'rejected,0,2,1,p1,-,-', + 'a refused call writes nothing' + ); + + // Direct writes: rate_worker writes neither lines nor markers; the public reads neither table. + assert.throws( + () => + asRole( + 'rate_worker', + `INSERT INTO rate.review_points (review_id, kind, body, method) VALUES ('${review(6)}', 'pro', 'Loa to rõ ràng', 'ai')` + ), + /permission denied for table review_points/ + ); + assert.throws( + () => + asRole( + 'rate_worker', + `UPDATE rate.review_ai_extractions SET outcome = 'empty'` + ), + /permission denied for table review_ai_extractions/ + ); + assert.throws( + () => asRole('rate_worker', `DELETE FROM rate.review_ai_batches`), + /permission denied for table review_ai_batches/ + ); + assert.equal( + asRole( + 'rate_worker', + `UPDATE rate.review_ai_batches SET status = 'stored', cost_usd = 0.0123, output_tokens = 900 + WHERE id = 'msgbatch_replay01' RETURNING status` + ), + 'stored' + ); + // A stored batch is closed to rate_worker; service_role still cannot drop its cost. + assert.equal( + asRole( + 'rate_worker', + `UPDATE rate.review_ai_batches SET cost_usd = 0 WHERE id = 'msgbatch_replay01' RETURNING id` + ), + '' + ); + assert.equal( + sql( + `SELECT cost_usd FROM rate.review_ai_batches WHERE id = 'msgbatch_replay01'` + ), + '0.0123' + ); + assert.throws( + () => + sql( + `UPDATE rate.review_ai_batches SET cost_usd = NULL WHERE id = 'msgbatch_replay01'` + ), + /review_ai_batches_stored_cost/ + ); + for (const role of ['anon', 'authenticated']) + for (const table of ['review_ai_batches', 'review_ai_extractions']) + assert.throws( + () => asRole(role, `SELECT count(*) FROM rate.${table}`), + new RegExp(`permission denied for table ${table}`) + ); + assert.equal( + asRole( + 'anon', + `SELECT count(*) FROM rate.review_points WHERE method = 'ai'` + ), + '1', + 'the public reads ai lines like writer lines' + ); + checkAll('after the refusals'); + assert.match(sql(info), /spend\|claude-haiku-5-5\|\d{4}-\d{2}\|0\.0123/); + + // Purge one prompt version, then everything; writer lines stay. + store([row(1, POINTS_1, { prompt_version: 'p4' })]); + assert.deepEqual( + JSON.parse( + asRole('rate_worker', `SELECT rate.purge_review_ai_points('p4')`) + ), + { lines: 3, markers: 1 } + ); + assert.equal(lines(1), ''); + assert.equal( + lines(5), + 'pro:keyboard:0:Bàn phím gõ êm', + 'another version stays' + ); + assert.throws( + () => asRole('rate_worker', `SELECT rate.purge_review_ai_points('P 4')`), + /'all' or a prompt version/ + ); + assert.throws( + () => asRole('rate_worker', `SELECT rate.purge_review_ai_points(NULL)`), + /'all' or a prompt version/ + ); + assert.throws( + () => asRole('rate_worker', `SELECT rate.purge_review_ai_points()`), + /function rate.purge_review_ai_points\(\) does not exist/, + 'a bare call purges nothing' + ); + assert.throws( + () => asRole('anon', `SELECT rate.purge_review_ai_points('all')`), + /permission denied for function purge_review_ai_points/ + ); + assert.deepEqual( + JSON.parse( + asRole('rate_worker', `SELECT rate.purge_review_ai_points('all')`) + ), + { lines: 1, markers: 2 } + ); + assert.equal( + sql(`SELECT count(*) FROM rate.review_points WHERE method = 'writer'`), + '1' + ); + checkAll('after the purge'); + + // Rollback twice: every object and every ai line is gone; writer lines and ratings stay. + store([row(1, POINTS_1)]); + sql(rollback); + sql(rollback); + assert.equal( + sql( + `SELECT count(*) FROM pg_proc p JOIN pg_namespace n ON n.oid = p.pronamespace + WHERE n.nspname = 'rate' AND p.proname IN ('store_review_ai_points', 'purge_review_ai_points', + 'review_ai_text_hash')` + ), + '0' + ); + assert.equal( + sql( + `SELECT (to_regclass('rate.review_ai_extractions') IS NULL AND to_regclass('rate.review_ai_batches') IS NULL)::text` + ), + 'true' + ); + assert.equal( + sql(`SELECT string_agg(method, ',') FROM rate.review_points`), + 'writer' + ); + assert.equal( + sql( + `SELECT coalesce(rating_score::text, '-') || ',' || coalesce(rating_weight_sum::text, '-') + FROM rate.master_products WHERE id = '${MASTER}'` + ), + ratingBefore, + 'no rating moved' + ); + + // Applied again: empty tables, the writer works. + sql(proposal); + checkAll('after the re-apply'); + assert.deepEqual(store([row(1, POINTS_1)]), counts({ stored: 1 })); + checkAll('after a store on the re-apply'); + + console.log( + 'review AI points proposal: hash parity, lines and markers stored, skipped, retried and refused, grants, purge, verify and rollback replayed' + ); +} finally { + removeContainer(); +} diff --git a/supabase/migrations/20261106120000_rate_review_ai_points.sql b/supabase/migrations/20261106120000_rate_review_ai_points.sql new file mode 100644 index 000000000..500c97c85 --- /dev/null +++ b/supabase/migrations/20261106120000_rate_review_ai_points.sql @@ -0,0 +1,513 @@ +-- ============================================================================= +-- Migration: 20261106120000_rate_review_ai_points.sql +-- Purpose (DOS/Rate#1531): D6 of the DOS/Rate AI review points design +-- (Docs/Proposals/2026-11-03-ai-review-points.html; all six decisions approved by JOY on +-- 2026-11-03: "Duyệt theo khuyến nghị"). It needs P2 (20261101120000_rate_criteria_ratings.sql) +-- and P3 (20261102090000_rate_review_submission_v2.sql). Copied unchanged from the reviewed +-- DOS/Rate proposal Docs/Proposals/2026-11-03-dos-me-review-ai-points.sql (DOS/Rate#1597) +-- apart from this header. One transaction. +-- Verify: scripts/ops/rate-review-ai-points-verify.sql (V1 to V7 return 0). +-- Rollback: scripts/ops/rate-review-ai-points-rollback.sql. +-- +-- The marker (rate.review_ai_extractions), the batch record (rate.review_ai_batches) and the +-- writer (rate.store_review_ai_points) of the review AI points job; rate_worker still has no +-- direct write on rate.review_points. No rating changes at apply: nothing reads 'ai' lines +-- yet. Details in the proposal's header, below. +-- ============================================================================= +-- +-- ============================================================================= +-- PROPOSAL (DOS/Rate#1531, D6 of Docs/Proposals/2026-11-03-ai-review-points.html; all six +-- decisions approved by JOY on 2026-11-03 as recommended): the database side of the AI +-- extraction of pros and cons from marketplace reviews (rate.review_points method 'ai'). A +-- DOS-Me migration to review and apply; nothing here is applied by DOS/Rate. +-- Verify: 2026-11-03-dos-me-review-ai-points-verify.sql (V1 to V7 return 0). +-- Rollback: 2026-11-03-dos-me-review-ai-points-rollback.sql. +-- +-- What it adds: +-- rate.review_ai_batches one row per Batch API batch the job sends: its id (kept the +-- moment the batch is created, so a crashed run collects it +-- instead of paying twice), the manifest (review ids, refs and +-- text hashes; never review text), the cost reserved before +-- sending and the cost the API reported. The month's spend per +-- model is read from here (design section 8). rate_worker reads, +-- inserts and updates it until it is stored; nobody deletes +-- through the API. +-- rate.review_ai_extractions the process-once marker, one row per review: the NFC text hash, +-- prompt version, criteria version, category and model it was +-- processed with, the outcome (stored, empty, rejected, failed), +-- the attempts of a failed result (at most 3), the version that +-- wrote its lines, and the evidence quote of each line. +-- rate_worker reads it; only the functions write. +-- rate.review_ai_text_hash(text) sha256 hex of the text in Unicode NFC; the job hashes the +-- same way (the replay compares them). +-- rate.store_review_ai_points(p_rows) the writer for the job (rate_worker) and service_role: +-- the 'ai' lines of a review and its marker in one transaction. +-- Only crawled user reviews with no writer lines; at most 3 pros +-- and 3 cons, lines 5 to 150 characters with words, no evasive +-- con, no line twice, criterion keys of the review's category +-- list, an evidence quote per line found in the comment. +-- rate.purge_review_ai_points(p_prompt_version) removes every 'ai' line and marker ('all'), +-- or those of one prompt version (the job's --purge). +-- +-- rate_worker still has only SELECT on rate.review_points: 'ai' lines come only through the +-- writer, as writer lines come only through the review RPC. +-- +-- Depends on P2 (20261101120000) and P3 (20261102090000, rate.rating_criteria and the line +-- helpers); both are in production. +-- +-- Rating impact at apply time: none. Nothing reads 'ai' lines yet: the P2 rating never read +-- review_points, P6 counts only 'writer' lines, and product-points reads 'writer' lines until +-- its 'ai' change ships (D4). +-- +-- Idempotent: IF NOT EXISTS for tables and indexes, CREATE OR REPLACE for functions, DROP ... +-- IF EXISTS before each policy. One transaction. +-- ============================================================================= + +BEGIN; + +-- 1. rate.review_ai_batches -------------------------------------------------------------------- + +CREATE TABLE IF NOT EXISTS rate.review_ai_batches ( + -- The Batch API's id (msgbatch_...). + id text PRIMARY KEY CHECK (id ~ '^[A-Za-z0-9_-]{8,128}$'), + model text NOT NULL CHECK (model ~ '^[a-z0-9][a-z0-9.-]{1,63}$'), + prompt_version text NOT NULL CHECK (prompt_version ~ '^[a-z0-9][a-z0-9._-]{0,31}$'), + status text NOT NULL DEFAULT 'submitted' + CHECK (status IN ('submitted', 'ended', 'stored', 'canceled', 'expired')), + request_count integer NOT NULL CHECK (request_count BETWEEN 1 AND 10000), + review_count integer NOT NULL CHECK (review_count BETWEEN 1 AND 100000), + -- [{custom_id, category, criteria_version, reviews: [{ref, id, text_hash}]}]: what each + -- request carried, so results map back to reviews after a crash. No review text. + manifest jsonb NOT NULL + CHECK (jsonb_typeof(manifest) = 'array' AND pg_column_size(manifest) <= 1048576), + -- The ceiling the job reserved before sending (requests x max output, at list price), then + -- the cost from the usage the API reported. The month's spend counts cost, else the ceiling. + reserved_usd numeric(10, 4) NOT NULL CHECK (reserved_usd >= 0), + cost_usd numeric(10, 4) CHECK (cost_usd >= 0), + input_tokens bigint NOT NULL DEFAULT 0 CHECK (input_tokens >= 0), + output_tokens bigint NOT NULL DEFAULT 0 CHECK (output_tokens >= 0), + cache_read_tokens bigint NOT NULL DEFAULT 0 CHECK (cache_read_tokens >= 0), + cache_write_tokens bigint NOT NULL DEFAULT 0 CHECK (cache_write_tokens >= 0), + submitted_at timestamptz NOT NULL DEFAULT now(), + ended_at timestamptz, + stored_at timestamptz, + CONSTRAINT review_ai_batches_stored_cost CHECK (status <> 'stored' OR cost_usd IS NOT NULL) +); +CREATE INDEX IF NOT EXISTS review_ai_batches_month_idx + ON rate.review_ai_batches (model, submitted_at); +CREATE INDEX IF NOT EXISTS review_ai_batches_open_idx + ON rate.review_ai_batches (submitted_at) WHERE status IN ('submitted', 'ended'); + +COMMENT ON TABLE rate.review_ai_batches IS + 'Batches the review AI points job sent (DOS/Rate#1531, AI design D6): id, manifest, reserved and reported cost. No review text.'; + +ALTER TABLE rate.review_ai_batches ENABLE ROW LEVEL SECURITY; +REVOKE ALL ON rate.review_ai_batches FROM PUBLIC, anon, authenticated, rate_worker; +GRANT ALL ON rate.review_ai_batches TO service_role; +GRANT SELECT, INSERT, UPDATE ON rate.review_ai_batches TO rate_worker; +DROP POLICY IF EXISTS "Service role access review_ai_batches" ON rate.review_ai_batches; +CREATE POLICY "Service role access review_ai_batches" ON rate.review_ai_batches + FOR ALL TO service_role USING (true) WITH CHECK (true); +DROP POLICY IF EXISTS "Rate worker read review_ai_batches" ON rate.review_ai_batches; +CREATE POLICY "Rate worker read review_ai_batches" ON rate.review_ai_batches + FOR SELECT TO rate_worker USING (true); +DROP POLICY IF EXISTS "Rate worker insert review_ai_batches" ON rate.review_ai_batches; +CREATE POLICY "Rate worker insert review_ai_batches" ON rate.review_ai_batches + FOR INSERT TO rate_worker WITH CHECK (true); +DROP POLICY IF EXISTS "Rate worker update review_ai_batches" ON rate.review_ai_batches; +-- A stored batch is closed: its cost, which the month's cap reads, cannot be rewritten. +CREATE POLICY "Rate worker update review_ai_batches" ON rate.review_ai_batches + FOR UPDATE TO rate_worker USING (status <> 'stored') WITH CHECK (true); + +-- 2. rate.review_ai_extractions ---------------------------------------------------------------- +-- A review is done while its marker has the same text hash, prompt version, criteria version +-- and category, unless the outcome is 'failed' with fewer than 3 attempts (a cut-off or refused +-- result is never recorded as 'empty'; it is retried at most twice, and the writer refuses a +-- fourth attempt). + +CREATE TABLE IF NOT EXISTS rate.review_ai_extractions ( + review_id uuid PRIMARY KEY REFERENCES rate.reviews(id) ON DELETE CASCADE, + text_hash text NOT NULL CHECK (text_hash ~ '^[0-9a-f]{64}$'), + -- 'all' is the purge's word for every version, so no version is called that. + prompt_version text NOT NULL CHECK (prompt_version ~ '^[a-z0-9][a-z0-9._-]{0,31}$' + AND prompt_version <> 'all'), + criteria_version integer NOT NULL CHECK (criteria_version >= 1), + -- The category of the review's master when it was processed: a move to another category + -- makes the review due again. + category_slug text NOT NULL CHECK (category_slug ~ '^[a-z_-]{2,64}$'), + model text NOT NULL CHECK (model ~ '^[a-z0-9][a-z0-9.-]{1,63}$'), + -- stored: lines written; empty: the model found nothing; rejected: every line it returned + -- failed the job's checks; failed: cut off (max_tokens), refused or errored, to retry. + outcome text NOT NULL CHECK (outcome IN ('stored', 'empty', 'rejected', 'failed')), + -- The 'ai' lines of the review now, and the prompt version that wrote them (a failed retry + -- under a new version keeps the lines of the old one; the purge reads this). + point_count smallint NOT NULL DEFAULT 0 CHECK (point_count BETWEEN 0 AND 6), + lines_prompt_version text, + -- Lines the job dropped (evidence not found, too short, a key of another category...). + rejected_count smallint NOT NULL DEFAULT 0 CHECK (rejected_count BETWEEN 0 AND 20), + attempts smallint NOT NULL DEFAULT 1 CHECK (attempts BETWEEN 1 AND 3), + -- [{kind, criterion_key, position, evidence, match}] of the review's lines; match is + -- 'exact' or 'unaccented' (found only without diacritics, counted apart by the job). + evidence jsonb NOT NULL DEFAULT '[]'::jsonb CHECK (jsonb_typeof(evidence) = 'array'), + batch_id text REFERENCES rate.review_ai_batches(id) ON DELETE SET NULL, + processed_at timestamptz NOT NULL DEFAULT now(), + CONSTRAINT review_ai_extractions_stored CHECK (outcome <> 'stored' OR point_count > 0), + CONSTRAINT review_ai_extractions_failed CHECK (outcome = 'failed' OR attempts = 1), + CONSTRAINT review_ai_extractions_lines CHECK ((point_count > 0) = (lines_prompt_version IS NOT NULL)) +); +CREATE INDEX IF NOT EXISTS review_ai_extractions_version_idx + ON rate.review_ai_extractions (prompt_version); +CREATE INDEX IF NOT EXISTS review_ai_extractions_lines_version_idx + ON rate.review_ai_extractions (lines_prompt_version) WHERE lines_prompt_version IS NOT NULL; +CREATE INDEX IF NOT EXISTS review_ai_extractions_batch_idx + ON rate.review_ai_extractions (batch_id) WHERE batch_id IS NOT NULL; + +COMMENT ON TABLE rate.review_ai_extractions IS + 'Process-once marker of the review AI points job (DOS/Rate#1531, AI design D6). Written only by rate.store_review_ai_points and rate.purge_review_ai_points.'; + +ALTER TABLE rate.review_ai_extractions ENABLE ROW LEVEL SECURITY; +REVOKE ALL ON rate.review_ai_extractions FROM PUBLIC, anon, authenticated, rate_worker; +GRANT ALL ON rate.review_ai_extractions TO service_role; +GRANT SELECT ON rate.review_ai_extractions TO rate_worker; +DROP POLICY IF EXISTS "Service role access review_ai_extractions" ON rate.review_ai_extractions; +CREATE POLICY "Service role access review_ai_extractions" ON rate.review_ai_extractions + FOR ALL TO service_role USING (true) WITH CHECK (true); +DROP POLICY IF EXISTS "Rate worker read review_ai_extractions" ON rate.review_ai_extractions; +CREATE POLICY "Rate worker read review_ai_extractions" ON rate.review_ai_extractions + FOR SELECT TO rate_worker USING (true); + +-- 3. The text hash ----------------------------------------------------------------------------- +-- The review's comment as the job sent it: Unicode NFC, UTF-8, sha256, lower-case hex. The job +-- computes createHash('sha256').update(comment.normalize('NFC'), 'utf8'). + +CREATE OR REPLACE FUNCTION rate.review_ai_text_hash(p_text text) +RETURNS text +LANGUAGE sql +IMMUTABLE +PARALLEL SAFE +SET search_path = '' +AS $$ + SELECT pg_catalog.encode(pg_catalog.sha256( + pg_catalog.convert_to(normalize(COALESCE(p_text, ''), NFC), 'UTF8')), 'hex') +$$; + +REVOKE ALL ON FUNCTION rate.review_ai_text_hash(text) FROM PUBLIC, anon, authenticated; +GRANT EXECUTE ON FUNCTION rate.review_ai_text_hash(text) TO service_role, rate_worker; + +-- 4. The writer -------------------------------------------------------------------------------- +-- p_rows: a JSON array of at most 500 rows and 2 MB, each +-- {"id": review uuid, "text_hash": hex, "prompt_version": "...", "criteria_version": n, +-- "category": slug, "model": "...", "batch_id": "..." | null, "failed": true | false, +-- "rejected": 0-20 (optional), +-- "points": [{"kind": "pro" | "con", "criterion_key": key | null, "body": "...", +-- "evidence": "...", "match": "exact" | "unaccented"}]} +-- "failed" is required, so a cut-off or refused result can never pass as an empty one. +-- Checked for every row before anything is written; one malformed row fails the call (22023, +-- with its index): +-- the shape and types above; a failed row carries no points; +-- (category, criteria_version) a list in rate.rating_criteria, every criterion key in it; +-- at most 3 pros and 3 cons; each body 5 to 150 characters as stored (review_text_stored) +-- with words in it, a con not evasive, no line twice across pros and cons (compared form); +-- each evidence 2 to 300 characters as stored, at least two words. +-- A batch_id must be a row of rate.review_ai_batches (else 23503, from its foreign key). +-- Then, per review (a review listed twice: the last row wins), in id order, after the reviews +-- are locked FOR NO KEY UPDATE in id order (two calls never interleave on one review), each +-- row is one of: +-- 'missing' the review no longer exists; +-- 'not_crawl' it is not a crawled user review (origin 'crawl', reviewer_type 'user'); +-- 'writer' it has a writer line: writer lines are never mixed with 'ai'; +-- 'stale' its comment no longer hashes to text_hash, or its master is in another +-- category now: the job sends it again with what it is now; +-- 'unquoted' an 'exact' evidence quote is not in the comment (compared form, as P3 +-- compares lines): nothing is written, and the job reports it; +-- 'done' a failed row for a review whose marker already holds a result of the same +-- text, prompt version, criteria version and category: the result stays; +-- 'exhausted' a failed row for a review already failed 3 times with the same key: refused +-- as a fourth attempt; +-- 'failed' the marker says failed and counts the attempt (1 under a new key); the +-- review's lines and their evidence stay, with the version that wrote them; +-- 'stored', 'rejected', 'empty' the review's 'ai' lines are replaced by its points +-- (position 0 to 2 per kind), and the marker says which. +-- Returns the count of each. +CREATE OR REPLACE FUNCTION rate.store_review_ai_points(p_rows jsonb) +RETURNS jsonb +LANGUAGE plpgsql +SECURITY DEFINER +SET search_path = '' +AS $$ +DECLARE + v_row jsonb; + v_point jsonb; + v_index integer := 0; + v_pros integer; + v_cons integer; + v_seen text[]; + v_compared text; + v_body text; + v_len integer; + v_evidence text; + v_review record; + v_marker rate.review_ai_extractions%ROWTYPE; + v_same_key boolean; + v_outcome text; + v_count integer; + v_result jsonb := '{"stored": 0, "empty": 0, "rejected": 0, "failed": 0, "missing": 0, "not_crawl": 0, "writer": 0, "stale": 0, "unquoted": 0, "done": 0, "exhausted": 0}'::jsonb; +BEGIN + IF jsonb_typeof(p_rows) IS DISTINCT FROM 'array' THEN + RAISE EXCEPTION 'p_rows must be a JSON array' USING ERRCODE = '22023'; + END IF; + IF jsonb_array_length(p_rows) > 500 OR octet_length(p_rows::text) > 2097152 THEN + RAISE EXCEPTION 'at most 500 rows and 2 MB per call' USING ERRCODE = '22023'; + END IF; + + FOR v_row IN SELECT value FROM jsonb_array_elements(p_rows) LOOP + IF jsonb_typeof(v_row) IS DISTINCT FROM 'object' + OR v_row - ARRAY['id', 'text_hash', 'prompt_version', 'criteria_version', 'category', + 'model', 'batch_id', 'failed', 'rejected', 'points'] <> '{}'::jsonb + OR NOT v_row ?& ARRAY['id', 'text_hash', 'prompt_version', 'criteria_version', 'category', + 'model', 'failed', 'points'] + OR jsonb_typeof(v_row -> 'id') IS DISTINCT FROM 'string' + OR (v_row ->> 'id') !~ '^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$' + OR jsonb_typeof(v_row -> 'text_hash') IS DISTINCT FROM 'string' + OR (v_row ->> 'text_hash') !~ '^[0-9a-f]{64}$' + OR jsonb_typeof(v_row -> 'prompt_version') IS DISTINCT FROM 'string' + OR (v_row ->> 'prompt_version') !~ '^[a-z0-9][a-z0-9._-]{0,31}$' + OR v_row ->> 'prompt_version' = 'all' + OR jsonb_typeof(v_row -> 'criteria_version') IS DISTINCT FROM 'number' + OR (v_row ->> 'criteria_version') !~ '^[1-9][0-9]{0,5}$' + OR jsonb_typeof(v_row -> 'category') IS DISTINCT FROM 'string' + OR (v_row ->> 'category') !~ '^[a-z_-]{2,64}$' + OR jsonb_typeof(v_row -> 'model') IS DISTINCT FROM 'string' + OR (v_row ->> 'model') !~ '^[a-z0-9][a-z0-9.-]{1,63}$' + OR jsonb_typeof(v_row -> 'batch_id') NOT IN ('string', 'null') + OR (jsonb_typeof(v_row -> 'batch_id') = 'string' + AND (v_row ->> 'batch_id') !~ '^[A-Za-z0-9_-]{8,128}$') + OR jsonb_typeof(v_row -> 'failed') IS DISTINCT FROM 'boolean' + OR jsonb_typeof(v_row -> 'rejected') NOT IN ('number', 'null') + OR (jsonb_typeof(v_row -> 'rejected') = 'number' + AND (v_row ->> 'rejected') !~ '^([0-9]|1[0-9]|20)$') + OR jsonb_typeof(v_row -> 'points') IS DISTINCT FROM 'array' + OR jsonb_array_length(v_row -> 'points') > 6 + OR ((v_row ->> 'failed')::boolean AND jsonb_array_length(v_row -> 'points') > 0) THEN + RAISE EXCEPTION 'row % is not a review AI result', v_index USING ERRCODE = '22023'; + END IF; + IF NOT EXISTS (SELECT 1 FROM rate.rating_criteria + WHERE criteria_version = (v_row ->> 'criteria_version')::integer + AND category_slug = v_row ->> 'category') THEN + RAISE EXCEPTION 'row % has no criteria list %/%', v_index, v_row ->> 'category', + v_row ->> 'criteria_version' USING ERRCODE = '22023'; + END IF; + + v_pros := 0; + v_cons := 0; + v_seen := '{}'; + FOR v_point IN SELECT value FROM jsonb_array_elements(v_row -> 'points') LOOP + IF jsonb_typeof(v_point) IS DISTINCT FROM 'object' + OR v_point - ARRAY['kind', 'criterion_key', 'body', 'evidence', 'match'] <> '{}'::jsonb + OR NOT v_point ?& ARRAY['kind', 'criterion_key', 'body', 'evidence', 'match'] + OR jsonb_typeof(v_point -> 'kind') IS DISTINCT FROM 'string' + OR (v_point ->> 'kind') NOT IN ('pro', 'con') + OR jsonb_typeof(v_point -> 'criterion_key') NOT IN ('string', 'null') + OR jsonb_typeof(v_point -> 'body') IS DISTINCT FROM 'string' + OR jsonb_typeof(v_point -> 'evidence') IS DISTINCT FROM 'string' + OR jsonb_typeof(v_point -> 'match') IS DISTINCT FROM 'string' + OR (v_point ->> 'match') NOT IN ('exact', 'unaccented') THEN + RAISE EXCEPTION 'row % has a line that is not a review AI line', v_index + USING ERRCODE = '22023'; + END IF; + IF v_point ->> 'kind' = 'pro' THEN + v_pros := v_pros + 1; + ELSE + v_cons := v_cons + 1; + END IF; + v_body := rate.review_text_stored(v_point ->> 'body'); + v_len := rate.review_text_length(v_body); + v_evidence := rate.review_text_stored(v_point ->> 'evidence'); + v_compared := rate.review_point_compare(v_body); + IF v_pros > 3 OR v_cons > 3 + OR v_len < 5 OR v_len > 150 + OR NOT rate.review_point_has_content(v_body) + OR (v_point ->> 'kind' = 'con' AND rate.review_point_is_evasive_con(v_body)) + OR v_compared = ANY (v_seen) + OR rate.review_text_length(v_evidence) NOT BETWEEN 2 AND 300 + OR rate.review_point_compare(v_evidence) !~ ' ' + OR (jsonb_typeof(v_point -> 'criterion_key') = 'string' + AND NOT EXISTS (SELECT 1 FROM rate.rating_criteria + WHERE criteria_version = (v_row ->> 'criteria_version')::integer + AND category_slug = v_row ->> 'category' + AND key = v_point ->> 'criterion_key')) THEN + RAISE EXCEPTION 'row % has a line out of range', v_index USING ERRCODE = '22023'; + END IF; + v_seen := v_seen || v_compared; + END LOOP; + v_index := v_index + 1; + END LOOP; + + PERFORM 1 FROM rate.reviews r + WHERE r.id IN (SELECT (e ->> 'id')::uuid FROM jsonb_array_elements(p_rows) e) + ORDER BY r.id + FOR NO KEY UPDATE; + + FOR v_row IN + SELECT DISTINCT ON ((e ->> 'id')::uuid) e + FROM jsonb_array_elements(p_rows) WITH ORDINALITY AS a(e, n) + ORDER BY (e ->> 'id')::uuid, n DESC + LOOP + SELECT r.id, r.comment, r.origin, r.reviewer_type, c.slug AS category, + EXISTS (SELECT 1 FROM rate.review_points rp + WHERE rp.review_id = r.id AND rp.method = 'writer') AS has_writer + INTO v_review + FROM rate.reviews r + LEFT JOIN rate.products p ON p.id = r.product_id AND p.origin = 'crawl' + LEFT JOIN rate.master_products m ON m.id = p.cluster_id + LEFT JOIN rate.categories c ON c.id = m.category_id + WHERE r.id = (v_row ->> 'id')::uuid; + v_outcome := CASE + WHEN NOT FOUND THEN 'missing' + WHEN v_review.origin IS DISTINCT FROM 'crawl' + OR v_review.reviewer_type IS DISTINCT FROM 'user' THEN 'not_crawl' + WHEN v_review.has_writer THEN 'writer' + WHEN rate.review_ai_text_hash(v_review.comment) <> v_row ->> 'text_hash' + OR v_review.category IS DISTINCT FROM v_row ->> 'category' THEN 'stale' + WHEN EXISTS (SELECT 1 FROM jsonb_array_elements(v_row -> 'points') p + WHERE p ->> 'match' = 'exact' + AND strpos(rate.review_point_compare(v_review.comment), + rate.review_point_compare(p ->> 'evidence')) = 0) THEN 'unquoted' + END; + IF v_outcome IS NOT NULL THEN + v_result := jsonb_set(v_result, ARRAY[v_outcome], to_jsonb((v_result ->> v_outcome)::integer + 1)); + CONTINUE; + END IF; + + SELECT * INTO v_marker FROM rate.review_ai_extractions WHERE review_id = v_review.id; + v_same_key := FOUND + AND v_marker.text_hash = v_row ->> 'text_hash' + AND v_marker.prompt_version = v_row ->> 'prompt_version' + AND v_marker.criteria_version = (v_row ->> 'criteria_version')::integer + AND v_marker.category_slug = v_row ->> 'category'; + + IF (v_row ->> 'failed')::boolean THEN + v_outcome := CASE WHEN v_same_key AND v_marker.outcome <> 'failed' THEN 'done' + WHEN v_same_key AND v_marker.attempts >= 3 THEN 'exhausted' + ELSE 'failed' END; + IF v_outcome = 'failed' THEN + INSERT INTO rate.review_ai_extractions AS x ( + review_id, text_hash, prompt_version, criteria_version, category_slug, model, outcome, + point_count, lines_prompt_version, rejected_count, attempts, evidence, batch_id, + processed_at) + VALUES (v_review.id, v_row ->> 'text_hash', v_row ->> 'prompt_version', + (v_row ->> 'criteria_version')::integer, v_row ->> 'category', v_row ->> 'model', + 'failed', 0, NULL, 0, 1, '[]'::jsonb, v_row ->> 'batch_id', now()) + ON CONFLICT (review_id) DO UPDATE + SET attempts = CASE WHEN v_same_key THEN x.attempts + 1 ELSE 1 END, + text_hash = EXCLUDED.text_hash, + prompt_version = EXCLUDED.prompt_version, + criteria_version = EXCLUDED.criteria_version, + category_slug = EXCLUDED.category_slug, + model = EXCLUDED.model, + outcome = 'failed', + rejected_count = 0, + batch_id = EXCLUDED.batch_id, + processed_at = EXCLUDED.processed_at; + END IF; + v_result := jsonb_set(v_result, ARRAY[v_outcome], to_jsonb((v_result ->> v_outcome)::integer + 1)); + CONTINUE; + END IF; + + DELETE FROM rate.review_points WHERE review_id = v_review.id AND method = 'ai'; + INSERT INTO rate.review_points (review_id, criterion_key, kind, body, position, method) + SELECT v_review.id, p ->> 'criterion_key', p ->> 'kind', rate.review_text_stored(p ->> 'body'), + (row_number() OVER (PARTITION BY p ->> 'kind' ORDER BY n) - 1)::smallint, 'ai' + FROM jsonb_array_elements(v_row -> 'points') WITH ORDINALITY AS t(p, n); + GET DIAGNOSTICS v_count = ROW_COUNT; + v_outcome := CASE WHEN v_count > 0 THEN 'stored' + WHEN COALESCE((v_row ->> 'rejected')::integer, 0) > 0 THEN 'rejected' + ELSE 'empty' END; + INSERT INTO rate.review_ai_extractions AS x ( + review_id, text_hash, prompt_version, criteria_version, category_slug, model, outcome, + point_count, lines_prompt_version, rejected_count, attempts, evidence, batch_id, + processed_at) + VALUES (v_review.id, v_row ->> 'text_hash', v_row ->> 'prompt_version', + (v_row ->> 'criteria_version')::integer, v_row ->> 'category', v_row ->> 'model', + v_outcome, v_count, CASE WHEN v_count > 0 THEN v_row ->> 'prompt_version' END, + COALESCE((v_row ->> 'rejected')::integer, 0), 1, + COALESCE((SELECT jsonb_agg(s.line ORDER BY s.n) + FROM (SELECT n, jsonb_build_object( + 'kind', p ->> 'kind', 'criterion_key', p -> 'criterion_key', + 'position', row_number() OVER (PARTITION BY p ->> 'kind' ORDER BY n) - 1, + 'evidence', rate.review_text_stored(p ->> 'evidence'), + 'match', p ->> 'match') AS line + FROM jsonb_array_elements(v_row -> 'points') WITH ORDINALITY AS t(p, n)) s), + '[]'::jsonb), + v_row ->> 'batch_id', now()) + ON CONFLICT (review_id) DO UPDATE + SET text_hash = EXCLUDED.text_hash, + prompt_version = EXCLUDED.prompt_version, + criteria_version = EXCLUDED.criteria_version, + category_slug = EXCLUDED.category_slug, + model = EXCLUDED.model, + outcome = EXCLUDED.outcome, + point_count = EXCLUDED.point_count, + lines_prompt_version = EXCLUDED.lines_prompt_version, + rejected_count = EXCLUDED.rejected_count, + attempts = 1, + evidence = EXCLUDED.evidence, + batch_id = EXCLUDED.batch_id, + processed_at = EXCLUDED.processed_at; + v_result := jsonb_set(v_result, ARRAY[v_outcome], to_jsonb((v_result ->> v_outcome)::integer + 1)); + END LOOP; + + RETURN v_result; +END $$; + +REVOKE EXECUTE ON FUNCTION rate.store_review_ai_points(jsonb) FROM PUBLIC, anon, authenticated; +GRANT EXECUTE ON FUNCTION rate.store_review_ai_points(jsonb) TO service_role, rate_worker; + +-- 5. The purge --------------------------------------------------------------------------------- +-- 'all': every 'ai' line and every marker. A version: every marker of that version or holding +-- lines it wrote (lines_prompt_version, so lines kept through a failed retry under a later +-- version go too), with all the 'ai' lines of those reviews; they are processed again on the +-- next run. There is no default: a bare call is refused. The reviews concerned are locked in +-- id order first, as the writer locks them, so a purge and a store never leave a line without +-- its marker. Batches stay (they hold the month's spend). The next product-points run +-- aggregates without the lines; no writer line is touched. Returns {"lines": n, "markers": n}. +CREATE OR REPLACE FUNCTION rate.purge_review_ai_points(p_prompt_version text) +RETURNS jsonb +LANGUAGE plpgsql +SECURITY DEFINER +SET search_path = '' +AS $$ +DECLARE + v_all boolean := p_prompt_version = 'all'; + v_lines integer; + v_markers integer; +BEGIN + IF p_prompt_version IS NULL + OR (NOT v_all AND p_prompt_version !~ '^[a-z0-9][a-z0-9._-]{0,31}$') THEN + RAISE EXCEPTION 'p_prompt_version must be ''all'' or a prompt version' USING ERRCODE = '22023'; + END IF; + PERFORM 1 FROM rate.reviews r + WHERE r.id IN (SELECT x.review_id FROM rate.review_ai_extractions x + WHERE v_all OR x.prompt_version = p_prompt_version + OR x.lines_prompt_version = p_prompt_version + UNION + SELECT rp.review_id FROM rate.review_points rp WHERE v_all AND rp.method = 'ai') + ORDER BY r.id + FOR NO KEY UPDATE; + + DELETE FROM rate.review_points rp + WHERE rp.method = 'ai' + AND (v_all OR rp.review_id IN (SELECT x.review_id FROM rate.review_ai_extractions x + WHERE x.prompt_version = p_prompt_version + OR x.lines_prompt_version = p_prompt_version)); + GET DIAGNOSTICS v_lines = ROW_COUNT; + DELETE FROM rate.review_ai_extractions x + WHERE v_all OR x.prompt_version = p_prompt_version OR x.lines_prompt_version = p_prompt_version; + GET DIAGNOSTICS v_markers = ROW_COUNT; + RETURN jsonb_build_object('lines', v_lines, 'markers', v_markers); +END $$; + +REVOKE EXECUTE ON FUNCTION rate.purge_review_ai_points(text) FROM PUBLIC, anon, authenticated; +GRANT EXECUTE ON FUNCTION rate.purge_review_ai_points(text) TO service_role, rate_worker; + +COMMIT;