Repository navigation
feat(pricing): check official display pricing drift - #136
Conversation
pierreraby
left a comment
There was a problem hiding this comment.
Verified: 41/41 offline tests green, typecheck clean; ran the live checker locally and it correctly flags the four missing prices from #135 (expected — this branch predates it) plus real findings (Qwen3.6-Plus tier, Step-3.5-Flash 0.10→0.09, expired Qwen deal), so the tool bites. Read-only, no runtime or pricing changes. Leaving the merge to the maintainers — will merge tomorrow if no objections.
pierreraby
left a comment
There was a problem hiding this comment.
Re-verified on cb38cf1 (merge of main + findings): typecheck, checker tests 42/42 green locally. Merging once CI is green.
|
Remaining drift findings after #135 (for follow-up, out of this PR's scope): |
|
Heads-up, @sandexzx: #138 just merged into main, so this PR now conflicts on two files. Both resolutions are mechanical — keeping both sides each time:
I verified the merged result locally (typecheck, release tests 46/46, pricing-check 42/42, transport 16/16, prettier clean), so a fresh |
pierreraby
left a comment
There was a problem hiding this comment.
Final head b702076 verified locally: typecheck, checker 42/42, transport 16/16, release 46/46, pi-local PASS, prettier clean. CI + benchmark green; catalog-sync check reports only the 3 known residual drifts (already tracked in comments), no missing prices. Merging.
|
Almost there — but the branch policy requires the pricing check green, and it can't pass while 3 residual drifts remain. Could you fold these into the PR? I verified each against the official page just now:
Plus the matching fixture updates ( |
pierreraby
left a comment
There was a problem hiding this comment.
Final head 90b8e00 verified: typecheck, checker 44/44, pricing 9/9, cost 9/9 green locally — and the live checker now reports PASS (no drift). The resolvedExpired mechanism for the stale Qwen badge is cleaner than my removal suggestion. Merging.
Summary
Add a read-only
npm run check:commandcode-pricingcommand to detect pricing drift before newly advertised models silently display zero cost.MODEL_COSTS.props.rowspayload without executing JavaScript or adding dependencies.0(matches),1(manual review required), and2(check could not complete).shell: bashpreserves checker failures throughteeusing pipefail.This PR is based directly on main, independent of #135. No runtime files, manual prices, verification dates, or existing model/pricing fixtures are changed. The checker never writes prices or credentials and does not introduce runtime network requests.
Validation
npm test— passed, including TypeScript and real Pi mock-API integration. OMP compatibility skipped locally becauseompis not installed.npm run format:check— passed.git diff --check— passed.actionlint1.7.7 on.github/workflows/model-metadata.yml— passed.npm run check:commandcode-pricing— successfully parsed 85 API models and 89 page rows, exited 1 / REVIEW REQUIRED, as expected for current main.Expected live-check failures
The new pricing step will currently fail intentionally; these are pre-existing data/runtime-policy discrepancies, not parser failures:
MODEL_COSTSentries forclaude-sonnet-5-5,gpt-6.1-sol,deepseek/deepseek-v4.1-flash-fast, andinclusionai/ling-3.1-flash:free(addressed separately by fix(pricing): add October catalog display prices #135).> 256Kcontext tier absent locally.0.09, local rate0.1.2026-06-22date; this needs manual review of the upstream information.The checker must not rewrite rates merely to make this job green. Actual prices remain manually reviewed, and the billing Usage page is authoritative.
Source-format limitation
The pricing page is not a public pricing API. This parser depends on its private Next.js Flight format and deliberately fails with exit code 2 if the required structure becomes unrecognizable, rather than reporting a false pass.