Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,9 @@ jobs:
- name: Replay the DOS-Me review submission v2 proposal (Docker)
run: pnpm --filter @repo/shared-data build && node scripts/test-review-submission-v2-proposal.mjs

- name: Replay the DOS-Me search rating proposal (Docker)
run: node scripts/test-search-rating-score-proposal.mjs

- name: Check generated rate schema types match the migrations
run: pnpm check:db-types

Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,9 @@ playwright-report-item-model/
!Docs/Proposals/2026-11-01-dos-me-review-submission-v2.sql
!Docs/Proposals/2026-11-01-dos-me-review-submission-v2-rollback.sql
!Docs/Proposals/2026-11-01-dos-me-review-submission-v2-verify.sql
!Docs/Proposals/2026-11-02-dos-me-search-rating-score.sql
!Docs/Proposals/2026-11-02-dos-me-search-rating-score-rollback.sql
!Docs/Proposals/2026-11-02-dos-me-search-rating-score-verify.sql
* - Copy*
* - BαΊ£n sao*
*.dump
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ DROP FUNCTION IF EXISTS rate.review_point_list(jsonb, text, integer, integer, te
DROP FUNCTION IF EXISTS rate.review_point_is_evasive_con(text);
DROP FUNCTION IF EXISTS rate.review_point_has_content(text);
DROP FUNCTION IF EXISTS rate.review_text_length(text);
DROP FUNCTION IF EXISTS rate.review_text_stored(text);
DROP FUNCTION IF EXISTS rate.review_point_compare(text);
DROP INDEX IF EXISTS rate.review_points_position_key;
DROP TABLE IF EXISTS rate.rating_criteria;
Expand Down
131 changes: 131 additions & 0 deletions Docs/Proposals/2026-11-02-dos-me-search-rating-score-rollback.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
-- ROLLBACK of the PROPOSAL 2026-11-02-dos-me-search-rating-score.sql (DOS/Rate#1545). NOT APPLIED.
--
-- Puts back rate.search_master_products as DOS/DOS.Me#1497 (head 8ccfbf2e) defines it: the
-- "highest rated" order reads user_rating_weighted then average_rating, and p_min_rating falls
-- back to expert_rating. The function body below is #1497's, byte for byte. No table, index or
-- data change, so nothing is lost. Idempotent: CREATE OR REPLACE FUNCTION.

CREATE OR REPLACE FUNCTION rate.search_master_products(
p_category_id UUID DEFAULT NULL,
p_brands TEXT[] DEFAULT NULL,
p_search TEXT DEFAULT NULL,
p_min_price NUMERIC DEFAULT NULL,
p_max_price NUMERIC DEFAULT NULL,
p_min_rating NUMERIC DEFAULT NULL,
p_ram_gb INTEGER[] DEFAULT NULL,
p_storage_gb INTEGER[] DEFAULT NULL,
p_min_battery_mah INTEGER DEFAULT NULL,
p_min_refresh_rate_hz INTEGER DEFAULT NULL,
p_min_fast_charging_w INTEGER DEFAULT NULL,
p_has_5g BOOLEAN DEFAULT NULL,
p_has_nfc BOOLEAN DEFAULT NULL,
p_max_screen_size_inch NUMERIC DEFAULT NULL,
p_sort TEXT DEFAULT 'default',
p_limit INTEGER DEFAULT 200,
p_offset INTEGER DEFAULT 0
)
RETURNS SETOF rate.master_products
LANGUAGE plpgsql
STABLE
SECURITY INVOKER
SET search_path = ''
SET plan_cache_mode = 'force_custom_plan'
AS $$
BEGIN
RETURN QUERY
SELECT m.*
FROM rate.master_products m
CROSS JOIN LATERAL (
SELECT
pg_catalog.regexp_replace(COALESCE(m.specifications #>> '{display,refresh_rate_hz}', ''), '[^0-9.]', '', 'g') AS refresh_rate_hz,
pg_catalog.regexp_replace(COALESCE(m.specifications #>> '{battery,fast_charging_w}', ''), '[^0-9.]', '', 'g') AS fast_charging_w,
pg_catalog.regexp_replace(COALESCE(m.specifications #>> '{display,size_inch}', ''), '[^0-9.]', '', 'g') AS size_inch
) spec
WHERE m.status = 'published'
AND (p_category_id IS NULL OR m.category_id = p_category_id)
AND (
p_brands IS NULL
OR cardinality(p_brands) = 0
OR pg_catalog.lower(pg_catalog.btrim(m.brand)) = ANY (
SELECT pg_catalog.lower(pg_catalog.btrim(b)) FROM unnest(p_brands) AS b
)
)
AND (
p_search IS NULL
OR pg_catalog.btrim(p_search) = ''
OR (
-- Same expression as idx_master_products_search_full_trgm.
(
COALESCE(m.brand, '') || ' ' ||
COALESCE(m.name, '') || ' ' ||
COALESCE(m.model, '') || ' ' ||
COALESCE(m.model_key, '') || ' ' ||
COALESCE(m.variant_name, '') || ' ' ||
COALESCE(m.key_specs ->> 'chip', '') || ' ' ||
COALESCE(m.specifications #>> '{hardware,chipset}', '')
) ILIKE '%' || pg_catalog.regexp_replace(pg_catalog.btrim(p_search), '([%_\\])', '\\\1', 'g') || '%' ESCAPE '\'
)
)
AND (p_min_price IS NULL OR p_min_price <= 0 OR m.min_price >= p_min_price)
AND (p_max_price IS NULL OR p_max_price <= 0 OR m.min_price <= p_max_price)
AND (
p_min_rating IS NULL
OR p_min_rating <= 0
OR COALESCE(
NULLIF(m.user_rating_weighted, 0),
NULLIF(m.expert_rating, 0),
m.average_rating,
0
) >= p_min_rating
)
AND (p_ram_gb IS NULL OR cardinality(p_ram_gb) = 0 OR m.ram_gb = ANY (p_ram_gb))
AND (p_storage_gb IS NULL OR cardinality(p_storage_gb) = 0 OR m.storage_gb = ANY (p_storage_gb))
AND (p_min_battery_mah IS NULL OR p_min_battery_mah <= 0 OR m.battery_mah >= p_min_battery_mah)
AND (
p_min_refresh_rate_hz IS NULL
OR p_min_refresh_rate_hz <= 0
OR COALESCE(
CASE WHEN spec.refresh_rate_hz ~ '^([0-9]+\.?[0-9]*|\.[0-9]+)$' THEN spec.refresh_rate_hz::NUMERIC END,
0
) >= p_min_refresh_rate_hz
OR COALESCE(m.key_specs ->> 'screen', '') ILIKE '%120hz%'
OR COALESCE(m.key_specs ->> 'screen', '') ILIKE '%144hz%'
)
AND (
p_min_fast_charging_w IS NULL
OR p_min_fast_charging_w <= 0
OR COALESCE(
CASE WHEN spec.fast_charging_w ~ '^([0-9]+\.?[0-9]*|\.[0-9]+)$' THEN spec.fast_charging_w::NUMERIC END,
0
) >= p_min_fast_charging_w
)
AND (p_has_5g IS NULL OR p_has_5g = false OR m.has_5g = true)
AND (
p_has_nfc IS NULL
OR p_has_nfc = false
OR (m.specifications -> 'connectivity' ->> 'nfc') = 'true'
)
AND (
p_max_screen_size_inch IS NULL
OR p_max_screen_size_inch <= 0
OR COALESCE(
m.screen_size_inch,
CASE WHEN spec.size_inch ~ '^([0-9]+\.?[0-9]*|\.[0-9]+)$' THEN spec.size_inch::NUMERIC END
) <= p_max_screen_size_inch
)
ORDER BY
CASE WHEN p_sort = 'price_asc' THEN m.min_price END ASC NULLS LAST,
CASE WHEN p_sort = 'price_desc' THEN m.min_price END DESC NULLS LAST,
CASE WHEN p_sort = 'merchants_desc' THEN m.merchant_count END DESC NULLS LAST,
CASE WHEN p_sort = 'rating_desc' THEN COALESCE(NULLIF(m.user_rating_weighted, 0), m.average_rating, 0) END DESC NULLS LAST,
m.created_at DESC,
m.id ASC
LIMIT GREATEST(1, LEAST(COALESCE(p_limit, 200), 500))
OFFSET GREATEST(0, COALESCE(p_offset, 0));
END;
$$;

GRANT EXECUTE ON FUNCTION rate.search_master_products(
UUID, TEXT[], TEXT, NUMERIC, NUMERIC, NUMERIC, INTEGER[], INTEGER[],
INTEGER, INTEGER, INTEGER, BOOLEAN, BOOLEAN, NUMERIC, TEXT, INTEGER, INTEGER
) TO anon, authenticated, service_role;
79 changes: 79 additions & 0 deletions Docs/Proposals/2026-11-02-dos-me-search-rating-score-verify.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
-- VERIFY the PROPOSAL 2026-11-02-dos-me-search-rating-score.sql (DOS/Rate#1545). Read-only.
-- Run after the migration on preview, beta and production. Every check returns 0.
-- The shown rating below is the migration's expression (resolveCompareEffectiveUserRating).

-- V1 the function is #1497's plpgsql, invoker-rights, search_path '' and custom plans; expect 0
SELECT count(*) FROM pg_proc p JOIN pg_namespace n ON n.oid = p.pronamespace
WHERE n.nspname = 'rate' AND p.proname = 'search_master_products'
AND NOT (
p.prolang = (SELECT oid FROM pg_language WHERE lanname = 'plpgsql')
AND NOT p.prosecdef
AND p.provolatile = 's'
AND p.proconfig @> ARRAY['search_path=""', 'plan_cache_mode=force_custom_plan']
);

-- V2 the order and the filter read rating_score and no longer read expert_rating; expect 0
SELECT count(*) FROM pg_proc p JOIN pg_namespace n ON n.oid = p.pronamespace
WHERE n.nspname = 'rate' AND p.proname = 'search_master_products'
AND (p.prosrc NOT LIKE '%m.rating_score%' OR p.prosrc LIKE '%expert_rating%');

-- V3 anon, authenticated and service_role can still call it; expect 0
SELECT count(*) FROM unnest(ARRAY['anon', 'authenticated', 'service_role']) AS r(role)
WHERE NOT has_function_privilege(
r.role,
'rate.search_master_products(uuid, text[], text, numeric, numeric, numeric, integer[], integer[], integer, integer, integer, boolean, boolean, numeric, text, integer, integer)',
'EXECUTE'
);

-- V4 "highest rated" never puts a product above one with a higher shown rating; expect 0
SELECT count(*) FROM (
SELECT lag(r.rating_5) OVER (ORDER BY s.ordinality) AS above, r.rating_5 AS here
FROM rate.search_master_products(p_sort => 'rating_desc', p_limit => 500)
WITH ORDINALITY AS s
CROSS JOIN LATERAL (
SELECT COALESCE(CASE
WHEN s.rating_score > 0 AND s.rating_score <= 5 THEN s.rating_score
ELSE COALESCE(
CASE WHEN s.user_rating_weighted > 0 AND s.user_rating_weighted <= 100 THEN round(CASE
WHEN s.user_rating_weighted > 10 THEN s.user_rating_weighted / 20
WHEN s.user_rating_weighted > 5 THEN s.user_rating_weighted / 2
ELSE s.user_rating_weighted END, 2) END,
CASE WHEN s.average_rating > 0 AND s.average_rating <= 100 THEN round(CASE
WHEN s.average_rating > 10 THEN s.average_rating / 20
WHEN s.average_rating > 5 THEN s.average_rating / 2
ELSE s.average_rating END, 2) END)
END, 0) AS rating_5
) r
) o
WHERE o.here > o.above;

-- V5 a minimum rating returns exactly the products whose shown rating reaches it; expect 0
-- (the shown rating at one decimal, as the pages show it;
-- per threshold: products returned below it, plus products at or above it that are missing).
WITH shown AS (
SELECT m.id, COALESCE(CASE
WHEN m.rating_score > 0 AND m.rating_score <= 5 THEN m.rating_score
ELSE COALESCE(
CASE WHEN m.user_rating_weighted > 0 AND m.user_rating_weighted <= 100 THEN round(CASE
WHEN m.user_rating_weighted > 10 THEN m.user_rating_weighted / 20
WHEN m.user_rating_weighted > 5 THEN m.user_rating_weighted / 2
ELSE m.user_rating_weighted END, 2) END,
CASE WHEN m.average_rating > 0 AND m.average_rating <= 100 THEN round(CASE
WHEN m.average_rating > 10 THEN m.average_rating / 20
WHEN m.average_rating > 5 THEN m.average_rating / 2
ELSE m.average_rating END, 2) END)
END, 0) AS rating_5
FROM rate.master_products m
WHERE m.status = 'published'
),
thresholds AS (SELECT unnest(ARRAY[3.5, 4.0, 4.5, 4.8]::numeric[]) AS t),
returned AS (
SELECT t.t, s.id
FROM thresholds t
CROSS JOIN LATERAL rate.search_master_products(p_min_rating => t.t, p_limit => 500) s
)
SELECT
(SELECT count(*) FROM returned r JOIN shown s ON s.id = r.id WHERE round(s.rating_5, 1) < r.t)
+ (SELECT count(*) FROM thresholds t
WHERE (SELECT count(*) FROM returned r WHERE r.t = t.t)
< LEAST(500, (SELECT count(*) FROM shown s WHERE round(s.rating_5, 1) >= t.t)));
100 changes: 100 additions & 0 deletions Docs/Proposals/2026-11-02-dos-me-search-rating-score.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# DOS-Me migration proposal: catalog search sorts and filters on the shown rating

**Not applied.** This is a follow-up of DOS/Rate#1545, which is phase P4 of the criteria ratings design (`2026-10-31-criteria-stars-pros-cons.html`, D3: one rating everywhere). JOY reviews it and applies it through DOS-Me: preview, then beta, then production. DOS/Rate runs no DDL against production.

## Problem

Since PR #1571, every page shows the same rating. It is `master_products.rating_score` when the product has one, and otherwise the weighted user rating, then the marketplace average (`resolveCompareEffectiveUserRating`, `apps/web/src/lib/rate-domain.ts`).

The catalog RPC `rate.search_master_products` still uses older values for two things:

- **The order `p_sort = 'rating_desc'`** reads `user_rating_weighted`, then `average_rating`. It ignores `rating_score`, and it does not bring a 0-10 or 0-100 average to 0-5.
- **The filter `p_min_rating`** reads `user_rating_weighted`, then `expert_rating`, then `average_rating`. No page shows `expert_rating` as the product's rating.

So three things can disagree with the rating on screen:

- the catalog's "Điểm Δ‘Γ‘nh giΓ‘ cao nhαΊ₯t" order;
- `?rating=` on the catalog and in `GET /api/v1/rate/products`;
- the leaderboard's top-100 load, which calls the RPC with `rating_desc`.

## Change

The migration is one `CREATE OR REPLACE FUNCTION`. The order and the filter read a new `shown.rating_5`, worked out exactly as the web does:

- `rating_score`, when it is above 0 and at most 5;
- otherwise `user_rating_weighted`, then `average_rating`. Each one counts only when it is above 0 and at most 100. A value above 10 is divided by 20, and a value above 5 is divided by 2. The result is rounded to 2 decimals (`normalizeCompareFiveStarRating`);
- otherwise nothing. Such a product sorts last and never passes a minimum rating, as before.

The order uses this rating unrounded. The filter compares it at one decimal, as every page shows it (`productRatingScore5`): a stored 4.46 shows 4.5, so it passes a 4.5 minimum.

**Nothing else changes.** The rest is DOS/DOS.Me#1497's function (head `8ccfbf2e`), byte for byte: the signature, the return type, `SECURITY INVOKER`, `search_path = ''`, `plan_cache_mode = force_custom_plan`, the other filters, the trigram search expression, the `created_at` and `id` tiebreak, the paging and the grants. There is no table, index or data change.

**Diff against #1497's function:**

```diff
) spec
+ CROSS JOIN LATERAL (
+ SELECT CASE
+ WHEN m.rating_score > 0 AND m.rating_score <= 5 THEN m.rating_score
+ ELSE COALESCE(<user_rating_weighted on 0-5>, <average_rating on 0-5>)
+ END AS rating_5
+ ) shown
...
- OR COALESCE(
- NULLIF(m.user_rating_weighted, 0),
- NULLIF(m.expert_rating, 0),
- m.average_rating,
- 0
- ) >= p_min_rating
+ OR pg_catalog.round(COALESCE(shown.rating_5, 0), 1) >= p_min_rating
...
- CASE WHEN p_sort = 'rating_desc' THEN COALESCE(NULLIF(m.user_rating_weighted, 0), m.average_rating, 0) END DESC NULLS LAST,
+ CASE WHEN p_sort = 'rating_desc' THEN COALESCE(shown.rating_5, 0) END DESC NULLS LAST,
```

### What changes for shoppers

- **The rating order follows the shown rating.** A product with a stored score of 4.6 no longer ranks above one shown at 4.8 because of a higher weighted rating.
- **`?rating=4.5` returns exactly the products shown at 4.5 or more**, counting the rating at one decimal as the pages do. A product with only an expert score, and no rating on screen, no longer passes.
- **A 0-10 average no longer ranks first.** Before, an average stored as 9.0 sorted above every 0-5 rating.

## Order (needs, in this order)

1. **P2** (`2026-10-31-dos-me-criteria-ratings.sql`, live on production) provides `rating_score`.
2. **DOS/DOS.Me#1497** (open; `20261101170000_rate_search_master_products_trgm_index_and_numeric_guard`) rewrites the function in plpgsql and adds the trigram index.
3. **This proposal**, as a DOS-Me migration with a timestamp after `20261101170000`.

This proposal must land **after #1497**. If it is applied first, #1497 would later put the old rating expressions back, and verification check V2 below would catch that. If #1497 changes its function before it merges, rebase this function on its final body: only the three hunks above are this proposal's.

## Files

- `2026-11-02-dos-me-search-rating-score.sql`: the migration. It is re-runnable.
- `-verify.sql`: read-only checks. V1 to V5 must each return 0:
- **V1:** the function is #1497's: plpgsql, invoker rights, stable, `search_path = ''`, custom plans.
- **V2:** the function reads `m.rating_score` and no longer reads `expert_rating`.
- **V3:** `anon`, `authenticated` and `service_role` can still call it.
- **V4:** in the `rating_desc` order, no product sits above one with a higher shown rating.
- **V5:** at minimum ratings of 3.5, 4.0, 4.5 and 4.8, the RPC returns exactly the published products whose shown rating, at one decimal, reaches the threshold.
- `-rollback.sql`: #1497's function, byte for byte. There is no data to restore.
- **Replay test:** `scripts/test-search-rating-score-proposal.mjs`, on Docker, in DOS/Rate CI. It runs these steps:
1. Apply P2, then #1497's function (from the rollback file).
2. Check the old order and filter on a small catalog: a stored score, weighted only, a 0-10 average, expert only, a zero stored score, and a draft.
3. Check that V2, V4 and V5 flag #1497's behaviour, so the checks have teeth.
4. Apply the proposal twice, then check the new order, the filter thresholds, the three roles, search, the draft, and V1 to V5 at 0.
5. Roll back twice, then apply the proposal again.

## Known limits

- **Rounding:** the fallback rounds with PostgreSQL `round(numeric, 2)` (half away from zero), and the web uses `toFixed(2)`. They can differ by 0.01 only on a binary halfway value of a fallback rating. `rating_score` itself is not rounded before that; both sides round the result once more to one decimal for the filter, half up.
- **No index on the rating:** the shown rating is computed per row, so the rating order and filter have no index. `rate.master_products` has under 40 rows on production. If the catalog grows past a few thousand published rows, a generated column or an expression index is a later step.
- **Web follow-ups (separate web PR, not this proposal):**
- **The JS fallback path:** `getEffectiveRating` in `apps/web/src/lib/rate-api.ts` runs only when the RPC is unavailable. It still tries `expert_rating` before `average_rating`, does not bring a 0-10 value to 0-5, and accepts a `rating_score` above 5. It should use `resolveCompareEffectiveUserRating`, with the one-decimal filter. A test (`core-rate-architecture.test.ts`, "rating_desc sort must use expert_rating") pins the old behaviour.
- **The OpenAPI text** for `min_rating` and `sort` (`server/rate/openapi.ts`) could say they use `rating_score_5`, the shown rating.
- **The leaderboard engine's `userScore5`** fallback reads a value above 10 as 0, where `rate-domain.ts` divides it by 20. It only matters for edge rows, which the leaderboard re-ranks itself.

## Rollout

1. **Preview:** a DOS-Me PR with this migration, after #1497 has merged. Supabase Preview applies it; then run the verify file there, read-only.
2. **Beta:** the GitHub Integration applies it on merge to `dev`. Run the verify file again.
3. **Production:** a selective promote PR into `main`, after JOY's approval. Run the verify file.
4. **Rollback:** `-rollback.sql`, at any time. No data depends on it.
Loading
Loading