Skip to content

test(quota): correct the contract comment and close the coverage holes from #3200 - #3327

Draft
olddonkey wants to merge 1 commit into
lidge-jun:devfrom
olddonkey:fix/quota-test-uncalibrated-contract
Draft

test(quota): correct the contract comment and close the coverage holes from #3200#3327
olddonkey wants to merge 1 commit into
lidge-jun:devfrom
olddonkey:fix/quota-test-uncalibrated-contract

Conversation

@olddonkey

@olddonkey olddonkey commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #3200, which landed the contract move itself (fcf0da257).

An adversarial multi-agent review of that change produced eleven findings seventeen minutes after it merged, so none of them made that train. No wrong assertion values — three reviewers independently re-derived all four numbers by executing the aggregator — but one factual error in the comment I wrote, now sitting on dev, plus several coverage holes the contract move opened.

The comment on dev is wrong, in two ways

It claims an unlisted plan name and a malformed non-string "both normalize to undefined via codexPlanKey". Neither half survives contact with the code:

  • codexPlanKey("edu_plus") returns "edu_plus" — a defined key that then fails the Object.hasOwn lookup in configuredWeight. Only a non-string normalizes to undefined.
  • On this test's integration path the malformed value never reaches codexPlanKey at all: poolAccountDto (src/codex/auth-api.ts:265) strips it via codexPlanValue, so the aggregation sees an absent plan.

The second point has a corollary the test's own name got wrong: it cannot distinguish "malformed plan" from "no plan at all". Rewritten to state both routes, and renamed to what it actually pins — counted at baseline, kept out of the public shape. Reference normalized to #3155 (PR #3198), matching the ten sibling citations; the fourth prose copy of the policy rationale replaced with a pointer to CODEX_DEFAULT_CAPACITY_WEIGHT's own doc comment.

Assertions added where a regression would now ship green

added catches
weekly: { includedAccounts, excludedAccounts, incomplete } the per-window flags are what ProviderCapacityQuota.tsx renders per bar — a finalizeWindow totalAccounts regression leaves the weekly bar struck through while the top-level flag stays clean
JSON.stringify(refreshed) free of "tier" with exclusion gone, nothing proved the malformed value cannot escape to the wire or persisted config — loosening nonEmptyPlan corrupts config.json while every prior assertion passes
the sibling's weight-field anti-leak regex publicCapacityWindow dropping its destructure leaks totalWeight/consumedWeight here silently; the sibling catches it, this test did not
toBe on the aggregation object the cached half re-read a field of the object asserted one line up — the cache fast path returns it by identity, so a clone-introducing change was invisible

The load-bearing mocked plan_type is now documented at the mock: a fetched plan outranks the persisted one, so tidying it to a plain string silently recalibrates the weight and breaks four assertions at once with only a numeric mismatch to explain it.

GUI: the one missing pairing

gui/tests/provider-capacity-shell.test.tsx had no fixture with incomplete: false and unknownPlanAccounts > 0 — both uncalibrated-notice tests rode on aggregations also incomplete for another reason. That pairing is the #3155 reporter's own fully-included pool, where the notice is the only remaining uncertainty signal.

Mutation-checked rather than asserted: gating the notice under the aggregation.incomplete branch — a plausible tidy-up of three similar warning blocks sharing a CSS class — fails only the new test; all eleven prior fixtures pass. That is exactly the silent hide the review predicted.

Verification

  • bun run test17419 pass, 0 fail, exit 0; all six serial lanes green.
  • tests/provider-quota.test.ts 110/0, gui/tests/provider-capacity-shell.test.tsx 12/0.
  • bun run typecheck / bun run privacy:scan — passed.

Checklist

  • Scope stays focused and avoids unrelated cleanup.
  • Docs or release notes were updated when needed.
  • Security-sensitive changes were reviewed for secrets, auth, and unsafe defaults.

Review readiness checklist

This PR stays in draft until every box below is ticked. Tick all four boxes once the requirements are met:

  • All CI tests are green on my local testing.

  • I pushed my PR to the latest dev commit.

  • I resolved all correct Codex and CodeRabbit findings.

  • My PR is ready for review.

Summary by CodeRabbit

  • Tests
    • Added regression coverage to ensure complete aggregates continue showing the uncalibrated-plan notice while preserving usage percentages.
    • Expanded quota reporting tests to verify baseline weighting, unknown-plan handling, per-window aggregation details, sanitized public results, and consistent cached aggregation behavior when plan data is malformed.

The review's headline finding was that the new comment stated a wrong mechanism
at a wrong layer: on this test's integration path the malformed plan never
reaches codexPlanKey — poolAccountDto strips it via codexPlanValue, so the
aggregation sees an ABSENT plan — and an unlisted string name normalizes to a
defined key that fails the weight-map lookup, not to undefined. The comment now
states both routes correctly, cites lidge-jun#3155 (PR lidge-jun#3198) the way the ten sibling
references do, and replaces its fourth copy of the policy rationale with a
pointer to CODEX_DEFAULT_CAPACITY_WEIGHT's own doc.

The assertions grew where the review showed regressions could ship green:

- weekly: { includedAccounts, excludedAccounts, incomplete } — the per-window
  flags are what the dashboard renders per bar, and nothing pinned them.
- JSON.stringify(refreshed) must not contain "tier" — with exclusion gone,
  nothing else proved the malformed value cannot escape into the public shape.
- the sibling's anti-leak regex over the aggregation, closing that asymmetry.
- the cached half now asserts the identity contract itself (toBe on the
  aggregation object) instead of re-reading a field of the object checked one
  line up, with a comment saying why content is only covered once.
- the load-bearing mock is documented: a fetched plan_type outranks the
  persisted plan, so it must stay non-string for every value below.

The test is renamed to what it now pins — counted at baseline, kept out of the
public shape — since "through cache and aggregation" was false on both nouns.

gui/tests/provider-capacity-shell.test.tsx gains the one missing pairing:
incomplete:false with unknownPlanAccounts>0, the lidge-jun#3155 reporter's own
fully-included pool. Mutation-checked: gating the uncalibrated notice under the
incomplete branch fails only the new test; every prior fixture passes, which is
exactly the silent-hide the review predicted.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@github-actions github-actions Bot added the chore Maintenance, CI, tests, refactors, or build changes (not a user-facing bug or feature). label Sep 3, 2026
@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Deterministic PR hygiene checks passed.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 3, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
🔒 Security Review Completed 2026-09-03T04:47:36.108258Z 73b8257 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

⏳ DRAFT

  • UI screenshot required.

What to do

  • Add a screenshot of the UI change to the PR description.

Review readiness checklist

  • ✅ All CI tests are green on my local testing.
  • ✅ I pushed my PR to the latest dev commit.
  • ✅ I resolved all correct Codex and CodeRabbit findings.
  • ✅ My PR is ready for review.

4/4 boxes ticked.

This pull request was already a draft. Its draft status will be preserved after every issue above is resolved.

@github-actions
github-actions Bot marked this pull request as draft September 3, 2026 04:45
@olddonkey

Copy link
Copy Markdown
Contributor Author

@lidge-jun @Ingwannu — requesting the GUI-screenshot waiver for this PR.

enforce-target fails on missing_ui_screenshot, which guiPathsChanged arms on any path under gui/. The only such path here is a test file:

gui/tests/provider-capacity-shell.test.tsx   +38

It adds one fixture-driven test (incomplete: false with unknownPlanAccounts > 0) to an existing suite. No component, style, string, or locale file is touched — ProviderCapacityQuota.tsx and the nine locale files are untouched — so there is no rendered change to capture. A screenshot of this PR would be a screenshot of unchanged UI.

The waiver regex looks for a maintainer comment, so a reply along the lines of "this does not change the GUI" is enough to clear it. Happy to answer anything about the test itself first — it is mutation-checked: gating the uncalibrated-plan notice under the aggregation.incomplete branch fails only this new test while all eleven pre-existing fixtures pass.

@coderabbitai

coderabbitai Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Team

Run ID: 003fa2f2-8b87-4091-8093-d6f46c6951bb

📥 Commits

Reviewing files that changed from the base of the PR and between ff1ac6b and 73b8257.

📒 Files selected for processing (2)
  • gui/tests/provider-capacity-shell.test.tsx
  • tests/provider-quota.test.ts

Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review.


📝 Walkthrough

Walkthrough

The tests expand provider quota coverage for malformed persisted plans and complete aggregates with uncalibrated plans. They validate baseline weighting, sanitized aggregation metadata, cached identity, notice rendering, and preserved aggregate usage.

Changes

Provider quota behavior

Layer / File(s) Summary
Quota aggregation and public-shape assertions
tests/provider-quota.test.ts:1862-1871, tests/provider-quota.test.ts:1892-1926
The pool test verifies baseline weighting for malformed persisted plans, unknownPlanAccounts, per-window completeness, removal of plan and internal weight data from the public shape, and cached aggregation object identity.
Provider quota notice rendering
gui/tests/provider-capacity-shell.test.tsx:331-374
The rendering test verifies that a complete aggregate with one uncalibrated plan shows the baseline-weight notice without an incomplete-coverage warning and retains the 44% usage display.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Merge Risk: ⚪ Minimal · up to 73b82

This change strengthens quota aggregation and provider-capacity UI regression coverage without changing production behavior. The covered malformed-plan and uncalibrated-plan scenarios are ready to merge.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately identifies the main changes: correcting the quota contract comment and adding coverage for gaps related to #3200. It is specific and clear.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 2…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 2 files.

✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@lidge-jun

Copy link
Copy Markdown
Owner

리뷰 · 우선순위 68 / 80

이 PR은 이미 dev에 올라간 #3200(fcf0da257)의 후속 테스트·주석 수리입니다. 제품 코드를 바꾸지 않고, tests/provider-quota.test.tsgui/tests/provider-capacity-shell.test.tsx만 손봅니다. 지금 dev HEAD는 ff1ac6b8c(Meta Model API direct provider #3321)이고, 쿼터 대시보드 쪽 계약은 #3198/#3155에서 “미보정 요금제는 제외하지 말고 기본 좌석 가중치로 센다”로 이미 바뀌어 있습니다. #3200은 그 계약에 맞게 테스트를 옮겼는데, 그 직후 리뷰에서 지적된 잘못된 주석아직 안 잡히는 회귀 구멍이 남아 있었고, 이 PR이 그걸 메웁니다.

설명부터 하면, dev에 남아 있던 주석은 “목록에 없는 요금제 이름”과 “문자열이 아닌 깨진 plan 값”이 둘 다 codexPlanKey를 거쳐 undefined로 같다고 적혀 있었습니다. 지금 체크아웃을 보면 그게 사실이 아닙니다. 목록에 없는 이름(예: edu_plus)은 codexPlanKey정의된 키를 돌려주고, configuredWeight/isCalibratedPlan 쪽에서 weight 맵 lookup이 실패해 CODEX_DEFAULT_CAPACITY_WEIGHT(=src/providers/codex-capacity.ts1)로 떨어집니다. 반면 { tier: "pro" }처럼 깨진 값은 이 테스트의 통합 경로에서는 애초에 codexPlanKey까지 안 갑니다. src/codex/auth-api.tspoolAccountDtocodexPlanValue로 plan을 걸러서, 집계 쪽에는 plan이 없는 계정으로 보입니다. 그래서 “malformed”와 “no plan”을 이 테스트만으로 구분할 수 없고, 이름·주석을 “baseline으로 세고 public shape 밖으로 막는다”로 고친 방향이 맞습니다. 정책 근거도 네 번째 복붙 대신 CODEX_DEFAULT_CAPACITY_WEIGHT 문서 주석을 가리키게 해서 #3155(PR #3198) 형제들과 citation 스타일도 맞춰 두었습니다.

추가 assertion들도 역할이 분명합니다. weekly: { includedAccounts, excludedAccounts, incomplete }는 GUI가 막대마다 그리는 창 단위 플래그를 고정합니다. 상위 incomplete만 깨끗하고 weekly 쪽이 깨지면 ProviderCapacityQuota.tsx에서 weekly 막대만 이상해질 수 있는데, 예전 테스트는 그걸 못 잡았습니다. JSON.stringify(refreshed)"tier"가 없어야 한다는 검사와 weight 필드 anti-leak 정규식은, exclusion을 걷어낸 뒤 malformed 값이 wire/캐시 밖으로 새거나 내부 weight가 public aggregation에 붙는 회귀를 막습니다. 캐시 경로는 필드 재검증 대신 toBe동일 객체 반환(identity) 계약을 직접 고정했고, mock의 plan_type이 비문자열이어야 하는 이유도 freshPlan이 persisted plan을 이긴다는 사실과 함께 mock 옆에 적어 두었습니다.

GUI 쪽은 gui/src/components/provider-workspace/ProviderCapacityQuota.tsx에서 uncalibrated 안내가 aggregation.unknownPlanAccounts > 0일 때 incomplete 게이트와 분리되어 렌더됩니다(주석도 #3155를 명시). 그런데 기존 shell 픽스처들은 uncalibrated를 켜면서 동시에 incomplete인 경우가 많아서, “완전 포함 + incomplete:false + unknownPlanAccounts>0” 조합이 없었습니다. 그 조합이 #3155 제보자의 실제 풀 모양이고, 안내가 남는 유일한 불확실성 신호입니다. 새 테스트가 그 구멍을 막고, incomplete 분기로 접으면 이 테스트만 깨진다는 mutation 설명도 설득력 있습니다. 제품 코드 변경 없이 회귀 울타리를 두껍게 하는 작업이라 현재 dev 방향(프로바이더/릴리즈 열차)과 충돌하지 않습니다.

라인 문제:

tests/provider-quota.test.ts / PR 본문 - 본문은 poolAccountDtoauth-api.ts:265로 적었지만, 지금 dev HEAD에서는 poolAccountDto·codexPlanValue 호출이 대략 277–284 근처다. 코드 버그는 아니고 설명 줄번호만 어긋난다.
enforce-target CI - gui/tests/...를 건드려 “missing UI screenshot”으로 quality gate가 빨간 상태다. 테스트만 바꾼 PR인데 스크린샷 게이트가 걸린다.
JSON.stringify(refreshed).not.toContain("tier") - 의도된 넓은 그물이라 지금은 맞지만, 나중에 무관한 필드에 tier 문자열이 들어오면 오탐할 수 있다. 형제 테스트와 같은 스타일이면 유지해도 된다.
gui/tests/provider-capacity-shell.test.tsx 새 expect - 영문 i18n 문장 앞부분만 toContain한다. 같은 파일의 기존 incomplete 문자열 검사와 같은 패턴이라 일관되지만, 문구가 크게 바뀌면 약해질 수 있다.

메인테이너의 판단이 필요한 지점

  • GUI 테스트만 추가된 PR에 enforce-target의 UI 스크린샷 요구를 웨이버할지, 아니면 형식상 스크린샷을 붙일지
  • "tier" 전체 JSON 문자열 금지 assertion을 그대로 둘지, 더 좁은 경로(예: aggregation/currentAccount)로 줄일지
  • test(quota): move the malformed-plan pool test onto the #3198 contract #3200 후속로 이 울타리까지 같이 합칠지, 아니면 주석 수리만이라도 먼저 빨리 넣을지(범위는 이미 테스트-only로 작음)

너의 추천
스크린샷 게이트만 정리되면(웨이버 또는 최소 첨부) 머지 추천입니다. 제품 경로를 건드리지 않고, dev에 남아 있던 틀린 계약 주석을 고치고, #3155 대시보드 회귀(완전 포함 풀에서 uncalibrated 안내가 사라지는 침묵 숨김)와 weekly 창 플래그·public shape 누수·캐시 identity를 실제로 고정합니다. types/config 분할과 무관하니 close-don't-rebase 대상도 아닙니다. CI 초록 확인 후 랜딩하면 됩니다.

이 댓글은 grok-bot이 작성했습니다

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

chore Maintenance, CI, tests, refactors, or build changes (not a user-facing bug or feature).

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants