Adjust fixtures for cross platform tolerances

[kurkle]

Fixture tolerances were hand-tuned against a single environment and had drifted out of sync with the browsers CI actually runs. As Chrome and Firefox updated, fixtures started failing one at a time — each fix was calibrated from the failure report, which meant guessing at a margin rather than measuring one.

Measurement

Every fixture was measured with tolerance forced to 0 in four environments:

• macOS Chrome 152 / Firefox 156
• Linux Chrome 153 / Firefox 156, in a container reproducing the CI image

The container was validated against three pixel counts observed in CI, all reproduced exactly:

fixture	browser	CI	container
point/crossShadow	Firefox	1565px	1565px
point/starShadow	Firefox	1476px	1476px
doughnutLabel/contentMultiline	Chrome	919px	919px

Native arm64 was tried first and rejected: it differs from x86_64 (starShadow 1474px vs 1476px, line/labelShadowColors 111px vs 193px), so the numbers would not have been CI-faithful.

Rule

condition	tolerance	fixtures
delta 0 in all four environments	0	73
otherwise	max(1.5 × linux, 1.05 × macOS)	153

Linux gets a real margin because CI is the gate. macOS gets just enough to keep local runs green — it renders text 3-4x further from the reference than Linux does, so sizing everything for macOS is what made the old values loose.

Effect

The suite gets stricter, not looser: total allowed differing pixels drop 35% (247,975 → 159,947). 175 fixtures are tighter than before; 51 are looser, in each case because the old value did not actually cover the measured rendering.

Verified green by real runs in all four environments: macOS 1048 specs, Linux Firefox 524, Linux Chrome 524.

Known trade-offs

• The macOS margin is 1.05x, so a macOS browser update will redden local runs without affecting CI. Re-measuring takes minutes.
• The 73 zero-tolerance fixtures fail on a single differing pixel. Measurement showed 80 fixtures that are zero in one environment but not another, so zero is environment-specific — these 73 are zero in all four, but Windows is not in the sample and future browser versions cannot be measured in advance.

🤖 Generated with Claude Code