diff screenshot: the default --threshold 0.1 reports a real soft-shadow regression as match: true on iOS and Android #2579

agent-device 0.21.0, macOS, Node 24.18. iPhone 17 Pro simulator (iOS 26.5, 3x) and Pixel_10_Pro emulator (API 37, 480 dpi). Found while dogfooding agent-device as a visual regression tool for react-native-paper (callstack/react-native-paper#5115).

A realistic UI regression, a Material Surface at elevation level 1 rendering the level-2 shadow (one step off), is reported as a perfect match by diff screenshot at the default threshold, on both platforms:

npx agent-device@0.21.0 diff screenshot --baseline baseline.png current.png --out diff.png --threshold 0.1 --json
→ { "totalPixels": 774252, "differentPixels": 0, "mismatchPercentage": 0, "match": true }

The same two images at --threshold 0.02:

The 0.02 result is the same on three independent captures per platform, and the noise floor of the same crop (four captures of an unchanged build, including a relaunch) is 0 px at every threshold down to 0.01. So the miss is the threshold, not capture noise. Diff image at 0.02: https://github.com/callstack/react-native-paper/raw/poc/agent-device-visual/example/visual/evidence/diff-images/ios-realistic-1-t0.02.png

A gross break (every level forced to the level-5 shadow) is also under-reported at 0.1 on iOS: 4,277 px / 0.55 % in one thin band, versus 116,292 px / 15 % at 0.02, for a change that is obvious to the eye across the whole row.

Soft shadows and gradients are low-contrast by design, so a per-pixel colour distance of 0.1 filters them out entirely. Suggestions, any of: lower the default; document that 0.1 is unsuitable for anti-aliased or low-contrast content and that callers should sweep thresholds against a known break before trusting a green run; echo the threshold actually used in the JSON output (it is not today).