Weighted vs unweighted m-bias disagree in the latest image-sim runs #885

Activity

1. cailmdaley

   assigned

   fabianhervaspeters

   and

   martinkilbinger

   on Sep 8, 2026
2. cailmdaley commented on Sep 16, 2026

   

   cailmdaley

   on Sep 16, 2026

   ContributorAuthor

   More actions

   Likely cause, from reading sp_validation/calibration.py and image_sims.py on develop: the calibrated e1/e2 that ImageSimMBias reads carry a single global R computed with global_R_weight: null (unweighted). For each weighted scheme in w_cols, the estimator then weights the numerator ⟨e⟩, but the R inside those e's is still the unweighted one — numerator and response use different weights, so m_w ≠ m_unw by construction. Only the "none" scheme is self-consistent.

   Prediction to check against the numbers here: Δm ≈ (R_unw − R_w)/R_w per component, same sign for both.

   Proposed fix: make the weighting scheme one configurable choice applied to both ⟨e⟩ and R (none / DES-style shape-noise+σ_e / inverse-variance), computing R per scheme rather than baking one R into e1/e2. Related: _selection_response is always an unweighted mean even on the data path where global_R_weight: w, a smaller instance of the same inconsistency.

   — Fable on behalf of Cail
3. fabianhervaspeters commented on Sep 16, 2026

   

   fabianhervaspeters

   on Sep 16, 2026

   Contributor

   More actions

   First ;) . Well if it really is global_R_weight: null we have a 2% bias indeed, so here we go again. The first thing for me would be to rule out the two previous 2% bias sources which were SExtractor flags selection biases and offsets. If it really reproduced the 1.6.x selection, these have the SExtractor==0 so that could still be the same bias source. I can not locate mask_v1.X.6_im_sim.yaml, on any branch, so which calibration file did Claude read in that global_R_weight: null ?
4. martinkilbinger commented on Sep 17, 2026

   

   martinkilbinger

   on Sep 17, 2026

   Contributor

   More actions

   First ;) . Well if it really is global_R_weight: null we have a 2% bias indeed, so here we go again. The first thing for me would be to rule out the two previous 2% bias sources which were SExtractor flags selection biases and offsets. If it really reproduced the 1.6.x selection, these have the SExtractor==0 so that could still be the same bias source. I can not locate mask_v1.X.6_im_sim.yaml, on any branch, so which calibration file did Claude read in that global_R_weight: null ?

   I checked again, see nibi:/home/mkilbing/astro/Runs/shapepipe/smk-g6-base (not sure though that you can access this dir).
   The run was a type .6, with FLAGS == 0.

   This is the config:

   # Copied and edited from sp_validation/config/calibration/mask_v1.X.6.yaml.
   # Standard cuts without halo masks, type v1.X.6.
   
   # General parameters (can also given on command line)
   params:
     # TODO: point at the *masked* comprehensive catalogue -- i.e. the output of
     # demo_apply_hsp_masks.py, run on
     # sp_output/shape_catalog_comprehensive_ngmix.hdf5 (extract_info.py's
     # output). Not produced yet: demo_apply_hsp_masks.py needs HealSparse
     # structural/coverage mask files (mask_r_*.hsp, coverage_*.hsp) that have
     # not been located on this cluster -- flagged, unresolved.
     input_path: ./sp_output/shape_catalog_comprehensive_struc_ngmix.hdf5
     cmatrices: False
     sky_regions: False
     verbose: True
   
   # Masks
   ## Using columns in 'dat' group (ShapePipe flags)
   dat:
     # SExtractor flags
     - col_name: FLAGS
       label: SE FLAGS
       kind: equal
       value: 0```
   
   confirmed in the output masks.txt: `[SE FLAGS]                      FLAGS = 0`
   
5. cailmdaley commented on Sep 17, 2026

   

   cailmdaley

   on Sep 17, 2026

   Last edited by cailmdaley

   ContributorAuthor

   More actions

   On develop in sp_validation, global_R_weight: null is set in config/calibration/mask_v1.X.4_im_sim.yaml and config/calibration/mask_v1.X.9_im_sim.yaml (line 73). There is no .6_im_sim file. The .9_im_sim config also sets FLAGS ≤ 2 for the sims. Agreed that a FLAGS == 0-type selection bias should be ruled out first — the weighting mismatch only predicts a difference between weighted and unweighted m, not the level of either.

   For the v2 data runs, the selection will be committed rather than hand-copied: a v2 mask config with FLAGS ≤ 2 and the MASK_n* cut goes into CosmoStat/sp_validation#343.

   — Fable on behalf of Cail