On stars, reduced-χ² runs too large at the bright end (climbing to ~8 for the brightest) and too small at the faint end (~0.2, against an expected ~1). χ² naturally improves for faint objects (noise dominates the denominator), but ~0.2 is well below 1 and points at a mis-estimated error model or an over-flexible fit. The faint-star ≈0.2 is the genuinely concerning piece: it implies the assumed variance is ~5× too large (σ inflated ~2.2×).
The sign flips with brightness, so a single noise-mis-scaling can't explain both tails — the answer likely names more than one effect:
Checkable: χ²_red vs magnitude re-derived from a controlled rerun, split by branch; the noise/weight path the fit actually consumes audited end-to-end (prepare_ngmix_weights ngmix.py:986, get_noise ngmix.py:941); and, where fixable, the assumed errors corrected so χ²_red → ~1, or the residual named as an intrinsic model-inadequacy floor.
χ² is not exported today — the fit result dict surfaces only s2n (ngmix.py:505–512); χ² lives inside the ngmix LM result. A reduced-χ² column needs wiring out per object (the prototype feat(ngmix): write reduced chi2 …, commit ba384428 on investigate/star-response, is the clean seed). That export is what unblocks the χ²-vs-magnitude panel the verification suite (#765) wants. Also fix the χ²-vs-mag plot's "Galaxy Fit" y-axis mislabel — it sits on a star fit.
Longer-term (not blocking the ngmix v2.0 merge — under #773). Coupled to the dropped PSF noise-injection (its own issue under #762), which is a leading candidate for the error-model mis-specification this anomaly measures. Related: #749, #765 (consumes the χ² export).
— Claude on behalf of Cail
On stars, reduced-χ² runs too large at the bright end (climbing to ~8 for the brightest) and too small at the faint end (~0.2, against an expected ~1). χ² naturally improves for faint objects (noise dominates the denominator), but ~0.2 is well below 1 and points at a mis-estimated error model or an over-flexible fit. The faint-star ≈0.2 is the genuinely concerning piece: it implies the assumed variance is ~5× too large (σ inflated ~2.2×).
The sign flips with brightness, so a single noise-mis-scaling can't explain both tails — the answer likely names more than one effect:
Checkable: χ²_red vs magnitude re-derived from a controlled rerun, split by branch; the noise/weight path the fit actually consumes audited end-to-end (
prepare_ngmix_weightsngmix.py:986,get_noisengmix.py:941); and, where fixable, the assumed errors corrected so χ²_red → ~1, or the residual named as an intrinsic model-inadequacy floor.χ² is not exported today — the fit result dict surfaces only
s2n(ngmix.py:505–512); χ² lives inside the ngmix LM result. A reduced-χ² column needs wiring out per object (the prototypefeat(ngmix): write reduced chi2 …, commitba384428oninvestigate/star-response, is the clean seed). That export is what unblocks the χ²-vs-magnitude panel the verification suite (#765) wants. Also fix the χ²-vs-mag plot's "Galaxy Fit" y-axis mislabel — it sits on a star fit.Longer-term (not blocking the ngmix v2.0 merge — under #773). Coupled to the dropped PSF noise-injection (its own issue under #762), which is a leading candidate for the error-model mis-specification this anomaly measures. Related: #749, #765 (consumes the χ² export).
— Claude on behalf of Cail