Repository navigation
feat: explain benchmark model choices - #441
mattrossman merged 19 commits into
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
raulb
left a comment
There was a problem hiding this comment.
@mattrossman Code LGTM, however I would love to hear @SaxonF's thoughts if he has cycles.
For instance, I'm not sure I love having to click on the arrow to expand the Experiment upload dates, considering there are only a handful of them. Approving to unblock, but I would like to include @SaxonF in the loop.
|
@raulb I just changed it to show it flattened w/o the collapsible label, and if we want to make further UI follow ups that's fine!
Note I also explored having this info embedded inside of the "Why these experiments" table but it looked noisy to me, hence the smaller separate table below it. It also isn't relevant just the justifications, but also the results tables above. |

Adds a "Why these experiments" section to the results page to shed light on why we picked these model / reasoning effort loadouts, since that was questioned here.
Also adds upload dates for the published benchmark experiments to communicate freshness of these explanations (which will typically update at the same time as experiment pins).
Rationales are maintained alongside the pinned Braintrust experiment IDs so they stay in sync with results shown on the site.
Review in the preview build: https://evals-git-mattrossman-ai-1285-explain-why-each-10afd2-supabase.vercel.app/
Closes AI-1285