Skip to content

Tweaks for AI ratings#139

Merged
jmgasper merged 13 commits into
masterfrom
develop
Jun 24, 2026
Merged

Tweaks for AI ratings#139
jmgasper merged 13 commits into
masterfrom
develop

Conversation

@jmgasper

Copy link
Copy Markdown
Contributor

jmgasper added 13 commits June 17, 2026 12:28
What was broken
QA Challenge winners could rerate into the public Data Science Challenge bucket and update the member's main rating, but winner-only QA challenge cards could still be absent from profile stats history when no persisted history row was available.

Root cause (if identifiable)
The previous fix mapped QA Challenge rows for rating replay, but the stats history fallback still resolved challenge-backed history rows from the source challenge track. QA source rows therefore resolved to a Quality Assurance track, which is not one of the public history groups exposed by member-api.

What was changed
Normalized QA Challenge source rows to the public DATA_SCIENCE / Challenge dimension in the shared stats dimension resolver used by review-result and ChallengeWinner history fallbacks.

Any added/updated tests
Added StatisticsService coverage showing a winner-only QA Challenge is returned under DATA_SCIENCE.Challenge history when the member has the Data Science Challenge aggregate but no persisted history row.
PM-5223: show QA winner history under Data Science
What was broken
QA Challenge rows were being replayed and displayed under DATA_SCIENCE / Challenge, so profile history showed QA wins in the Data Science Challenge section instead of the Testing section.

Root cause (if identifiable)
The previous follow-up normalized Quality Assurance Challenge rows into the Data Science Challenge bucket for both rerating and history fallback. That made the rating update visible, but it used the wrong public profile bucket for QA challenges.

What was changed
Routed Quality Assurance Challenge rerates to DEVELOP / BUG_HUNT, the legacy testing subtrack used by the profiles app for Testing. Data Science Challenge rerates now replay only Data Science source challenges. History reads also remap persisted QA Challenge rows out of DATA_SCIENCE / Challenge and into DEVELOP / BUG_HUNT when challenge metadata identifies the source as Quality Assurance. The member stats recalculation helper applies the same QA dimension normalization for future backfills.

Any added/updated tests
Updated Development rating engine and StatisticsService coverage for QA Challenge winners under Testing. Added StatisticsService coverage for remapping previously persisted Data Science QA history rows into Testing. Added recalculation coverage for QA aggregate normalization into the Testing bucket.
PM-5223: route QA challenge stats to Testing
What was broken
QA Challenge ratings and history were previously being forced into existing public buckets instead of using a QA stats dimension. The latest QA feedback asked for the QA track/type to be supported directly instead of using DEVELOP.

Root cause
The stats dimension resolver and unified stats response helpers did not treat QA as a supported canonical track, so follow-up fixes mapped Quality Assurance Challenge rows to DATA_SCIENCE or DEVELOP-based buckets.

What was changed
Added QA as a canonical stats track, routed Quality Assurance Challenge rerates and history normalization to QA / Challenge, exposed QA in unified stats/history/search responses, and updated the recalculation path to normalize QA Challenge aggregates to QA / Challenge.

Any added/updated tests
Updated rerating and history tests to expect QA / Challenge, added recalculate coverage for QA Challenge aggregates, and added formatter coverage for QA stats and QA history responses.
PM-5223: route QA challenge ratings to QA
What was broken
AI Engineering stats could show challenge counts, but the profile challenge details view could still be empty because the member stats history response did not expose those rows under the configured AI Engineering rating path.

Root cause
When history rows were enriched with Challenge API metadata, configured rating-path rows kept their challenge id and name but had their stored track/type overwritten by the source challenge's native Challenge or Marathon Match dimensions.

What was changed
Preserve configured rating-path dimensions during history metadata enrichment so AI Engineering history remains under DATA_SCIENCE.AI Engineering while still receiving canonical challenge metadata.

Any added/updated tests
Added a StatisticsService regression test that verifies an AI Engineering rating-path history row stays under DATA_SCIENCE.AI Engineering after challenge metadata enrichment.
PM-5397: Preserve AI Engineering history dimensions
@jmgasper jmgasper merged commit 6c9fa3c into master Jun 24, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant