Repository navigation
Additive-bias subtraction in calibration zeroes the signal on constant-shear sims (m = −1) #226
Description
Found while running the image-sim m-bias chain end-to-end (#225): the m-bias came back at m ≈ −1 (no shear response at all) even after the estimator itself was fixed. The cause is a calibration step that is correct for data and silently fatal for constant-shear simulations.
Mechanism (exact, not just empirical)
scripts/calibration/calibrate_comprehensive_cat.py builds the science shear via calibration.get_calibrated_m_c():
# get_calibrated_quantities: correct ensemble metacal
g_corr = np.linalg.inv(R).dot(g_uncorr) # R = ensemble 2x2 (shear + selection)
# get_calibrated_m_c: additive-bias removal — the problem for sims
c = np.mean(g_uncorr[comp]) # "additive bias" := mean uncalibrated shear
g_corr_mc = g_corr - np.linalg.inv(R).dot(c) # subtracted from every objectThe multiplicative step is right. But the additive step subtracts each catalogue's own mean — and since ⟨g_corr⟩ = R⁻¹⟨g_uncorr⟩ = R⁻¹c, we get ⟨g_corr_mc⟩ = 0 exactly, for every sim. For real data this is the intended additive-systematics removal (mean cosmic shear ≈ 0, so the mean is PSF leakage / additive bias). For a constant-shear sim the mean is the injected signal, so the subtraction deletes exactly what the m-bias measures: each of 1p2z/1m2z/1z2p/1z2m ends up with ⟨e⟩ = 0 by construction, the response is a difference of zeros, and m = −1.
Evidence (grid_1, 1 tile, inverse-variance weighted, paired estimator)
| shear column | response 1+m₁ | response 1+m₂ |
|---|---|---|
e_uncal (raw) |
+0.84 | +0.92 |
e = g_corr_mc (what the m-bias currently reads) |
+0.20 | +0.02 ← signal gone |
ensemble R⁻¹ on paired e_uncal, no additive step |
+1.14 → m₁ = +0.14 ± 0.10 | +1.27 → m₂ = +0.27 ± 0.11 |
The last row is a hand-diagnostic using only the per-object R_g** columns (it omits the selection-response part of the ensemble R, a few-% term), so it's the cross-check that the pipeline is otherwise healthy — not the final number.
Possible fixes — design call needed, since this touches the shared calibration path
- Sim-mode flag (my leaning): an
additive_correction: true/falseknob oncalibrate_comprehensive_cat.py, defaulttrueso the data path is provably unchanged; the image-sim mask config sets itfalseand the calibration writesg_corr(multiplicative-only) ase1/e2. Full ensemble R preserved; minimal diff. - Always expose
g_corras extra columns (e1_mult_only/e2_mult_only); the sim m-bias reads those. Non-invasive but adds columns for everyone. - The m-bias estimator applies R itself from
e_uncal+R_g**. No shared-code change, but it can only apply the per-object shear response, not the global selection response — a good cross-check, not the final number.
Leaving the choice open — @-free ping via the PR thread. Note this finding is the image-sim gate working: it caught a real calibration/sim incompatibility before merge. Whether the data path's additive step is doing the right thing there is probably fine (mean shear ≈ 0 on data) but may deserve a look while we're here.
— Fable, on behalf of Cail
Activity
martinkilbinger commentedon Jul 6, 2026
Not sure why this is a problem for image sims or constant shear. For real data we were able to compute and output the additive bias, independent of subsequent code that subtracts this bisa to get (m- and c-)calibrated shears. Why is this different here?
It is true that, for constant shear simulations, if we want to estimate this constant shear we need to use the non-c-calibrated (non-subtracted) shear estimates.
cailmdaley commentedon Jul 6, 2026
There is no issue calculating the additive bias, but it's nulling the downstream constant-shear m-bias calculation because
<e> = e_inputin that case. As you said the solution is to not use the c-calibrated shear, but there are multiple ways to do this in practice, laid out as option 1/2/3 above. Which would you prefer?martinkilbinger commentedon Jul 7, 2026
I've implemented option 1:
the data path is unchanged). When False, the c subtraction is skipped; c and c_err are still
computed and returned, so they remain available as diagnostics of what would have been
subtracted.
mask config and passes it through, printing a notice when the correction is disabled.
mask_v1.X.9_im_sim.yaml set additive_correction: False with a comment pointing to this
issue. Real-data configs are untouched and keep the current behaviour via the default.
Catalogues calibrated before this change have the signal subtracted, so any image-sim m-bias
derived from them is invalid; I'm rerunning calibrate_all _ mbias on the current grid run
to validate. Will report the recalibrated m here, and suggest we close the issue once it
comes out at the per-cent level.
Unrelated to the fix but noticed while reading the code: c_err is currently np.std(g_uncorr)
(population scatter, not the error on the mean) _ there's already a MKDEBUG TODO on it;
worth a separate small issue.
martinkilbinger commentedon Jul 7, 2026
Implemented in #225
martinkilbinger commentedon Jul 7, 2026
Preliminary results on 15 tiles without the subtracted additive-bias:
m1 = +0.1028 +-0.0605
c1 = -0.0003 +-0.0015
m2 = +0.1538 +-0.0633
c2 = -0.0027 +-0.0015
The estimates of the constant shear look good:
1p2z: = +0.02609 = -0.00188
1m2z: = -0.02506 = -0.00319
1z2p: = -0.00069 = +0.02507
1z2m: = +0.00027 = -0.02971
feat(calibration): mirror Martin's #226 additive_correction flag into…
cailmdaley commentedon Jul 10, 2026
Heads-up: the
additive_correctionfix (e38ace9) is onimage_sims_val, not on the #225 branch — #225 moved calibration toscripts/calibration/, so the keyword wasn't reaching the workflow there. Mirrored into #225's layout with identical semantics (64ec1d7) and exercised end-to-end; with all weights also removed per #227, the first fully-repaired one-tile number is m1 = +0.274 ± 0.028, m2 = +0.231 ± 0.022 (details on #225 and shapepipe#803). The ±0.02 errors come from shapepipe#803's position-seeded fixnoise — percent-level σ_m now needs only ~4–7 tiles, so your percent-level close condition is within reach as soon as the mechanism behind the +0.25 central value is understood.— Claude (Fable) on behalf of Cail
Merge image_sims_val (Martin's parallel sim-validation stack) into the …
cailmdaley commentedon Jul 10, 2026
Closing as resolved — the
additive_correctionflag (Martin'simage_sims_valstack, mirrored into the image-sims branch layout) makes the additive-bias subtraction opt-out, and the sims configuration shipsadditive_correction: Falsefor constant-shear runs.Closing evidence from the corrected end-to-end chain (tile 233.293, grid_1, true amplitude g = 0.025): m1 = +0.019 ± 0.022, m2 = −0.015 ± 0.017 — the m ≈ −1 signal-zeroing this issue documents is gone, while the additive terms stay clean (c1 = −0.0026 ± 0.0084, c2 = −0.0033 ± 0.0075), which is exactly the split the fix was designed to preserve.
— Fable on behalf of Cail
Image-simulation m-bias: extracted core + snakemake orchestration (#225)
Record truth pass: image-simulation shear bias
Record truth pass: image-simulation shear bias
Record truth pass: image-simulation shear bias
Record truth pass: image-simulation shear bias