Repository navigation
Masking unification: external healsparse masks throughout ShapePipe #845
Description
Activity
- added sub-issues
on Jul 16, 2026 We (myself, Ludo and Hendrick) are going to discuss updating these masks next week (week of July 20).
For now, you can use the old masks for testing purposes.
Reacted by Cail McLean DaleyPolicy update from the 2026-07-21 mask-force telecon
Meeting notes: wiki — 21/07/2026 Masking Telecon (Mike, Ludo, Hendrik, Cail).
Three points change or sharpen the plan recorded in this issue:
- Measure everything; flag, don't cut. Shapes are measured for all objects. Mask information propagates as flags (pixel flags →
IMAFLAGS_ISO→ catalogue columns). Selection happens at the catalogue level, after the fact. This matches Consistent use of the external healsparse masks #847's mechanism — the rasterizer only writes flags — but changes the policy on who consumes them. - PSF star selection must not reject stars in star-halo masks (bits 1, 2). PSF stars are bright; a faint halo background does not bias them, and halo rejection costs sky area and risks an under-constrained PSF model. This walks back the earlier thread consensus that the external masks "also feed PSF star selection." Consequence for Consistent use of the external healsparse masks #847: the exposure-level
BIT_FLAG_MAPbecomes consumer-aware — defect bits (e.g. MaxiMask) may enter the star-selection cut; halo bits ride along as flags only. Config change, not a redesign. - This is provisional on evidence. Axel's concern stands (bad pixels on the stamp bias PSFEx, which consumes neither masks nor weights); the group's counter is that magnitude cuts + ellipticity outlier rejection + sigma clipping were validated years ago. The arbiter, actioned to Cail: overlay the PSF stars surviving the full selection on the Old-vs-new mask comparison on real data (tiles + exposures) #850 comparison figures. If survivors sit on unmasked defects or in halos, we revisit.
Also settled: MaxiMask stays — it alone catches satellite trails and (often) dead columns (Axel concurs for streaks, suggesting the mask be enlarged/smoothed before use — #850). And the mask assembly moves to PhotoPipe (UNIONS-WL/PhotoPipeUNIONS,
CANFARbranch): per-tile*_final.mask.fits.gznext to the DR6 catalogues, healsparse conversion after; Ludo→Hendrik scripts by Jul 27.— Claude (fable) on behalf of Cail
- Measure everything; flag, don't cut. Shapes are measured for all objects. Mask information propagates as flags (pixel flags →
- added a commit that references this issue
on Aug 31, 2026
Goal. Replace ShapePipe's internal mask generation (GSC-2.3 star halos/spikes rasterized by WeightWatcher, Messier/NGC patches — THELI-vintage, cf. #15) with the external healsparse masks produced by the maskforce pipeline (Hendrik / Ludo / Mike). Design: query-everything — no rasterization of external masks. Every healsparse bit is queried once per object (RA, Dec) and stored as catalogue flag columns; rejection happens downstream at the catalogue level. The instrument flag images remain the only per-exposure, pixel-level path. PSF star selection cuts on the star bit + MaxiMask, but halo bits do not reject PSF stars (2026-07-21 mask-force telecon: measure everything, flag, reject at catalogue level). The old masking code stays config-selectable until the real-data comparison validates the switch (#850).
Key agreements (Slack 2026-07-14→16 + 07-21 telecon):
Mask provenance. Ludo assembles pixel/FITS masks tile-by-tile → Mike converts to healsparse (
Healsparse_mask_pool.py, nside 131072, conservatively fattened); regeneration tooling at lvanwaerbeke/UNIONS_mask_processing. Products are boolean healsparse maps, one file per reason bit (1faint star halo /2bright star halo /4bright stars /8manual /1024maximask;512out-of-bounds); the combined r mask =OR{1,2,4,8,64,1024}, verified empirically on P3. Current products: the post-2026-07-31 rerun (fixed, de-duplicated reg files), announced in #mask-force with paths in the Mask_Assembly canvas; images on the wiki masking page.— Fable, on behalf of Cail