ShapePipe verification test suite (live science guardrails) #765

Activity

1. cailmdaley

   added sub-issues on Jun 20, 2026

   • Star shear-response test (R1=R2=0±0.03) #763
   • m-bias guardrail test #764
2. cailmdaley

   changed the title [-][Tracking] ShapePipe verification test suite (live science guardrails)[/-] [+]ShapePipe verification test suite (live science guardrails)[/+] on Jun 30, 2026
3. cailmdaley

   added a sub-issue on Jun 30, 2026

   • Finalise the star shear-response test definition (star cutout vs input PSF) #772
4. cailmdaley

   added

   tests

   on Jul 6, 2026
5. cailmdaley commented on Jul 10, 2026

   

   cailmdaley

   on Jul 10, 2026

   ContributorAuthor

   More actions

   Status update on the in-memory breakdown grid (the resolution × S/N panel on the status board) — relevant to the concern about strong S/N-dependence of m and c in the toy sims.

   The alarming trends were artifacts of the v1 harness statistics, not the pipeline. The v1 grid ran 16 seeds per cell with single-signed shear arms and reported no uncertainties; its noise floor at moderate S/N was σ_m ≈ 0.5, and the one catastrophic value (m₂ = −58.6 at the point-source/low-S/N corner) was division by a response consistent with zero. The v1 numbers (e.g. m = +0.31 at S/N ≈ 50) were displays of that noise floor.

   The v2 harness (PR #804: paired ±γ arms on identical noise realizations — the same cancellation idea as #796 — joint bootstrap errors, degeneracy guards, ~100k fits) measures, on the same live do_ngmix_metacal path:

   • every resolved cell (r ≥ 0.37) in every band consistent with m = 0 — precision ~1×10⁻³ at S/N ≈ 1000, ~1×10⁻² at S/N ≈ 50, ~1.5×10⁻² at S/N ≈ 15;
   • |c₁| < 1×10⁻³ and PSF-leakage α ≤ 0.2% everywhere, all bands;
   • R₁₁ ≈ R₂₂ on resolved rungs (the apparent v1 asymmetry came from one recycled seed set correlating all cells);
   • small negative m hints only at the unresolved rungs (r ≤ 0.27, ≤2σ), where the point-source response collapse is expected and the CI tripwires deliberately don't assert.

   The refreshed board panel now shows error bars on every point (new board rule: no number without its uncertainty), and the degenerate cell renders as an open marker instead of an exploded value.

   Caveat for scope: this grid runs n_epochs=1, exact-true-PSF, trivial pixel-scale WCS, and the legacy HSM centroid — the next round checks the pass survives production settings (WCS centroid per #767, non-trivial jacobians per esheldon/ngmix#72, multi-epoch).

   — Claude Fable on behalf of Cail