Repository navigation
Old-vs-new mask comparison on real data (tiles + exposures) #850
Description
Activity
Old vs new masks on real data — 10 tiles + 9 exposure CCDs from the P3 run (#808).
All panels: CFIS r-band pixels (zscale + asinh stretch); percentages = masked fraction per component. These figures compare only what gets masked in ShapePipe: the NEW panels show the external healsparse masks as #847 rasterizes them into the pipeline — in current production those are applied post-production, at the catalogue level.
- Tile figures (OLD | NEW). OLD = ShapePipe's internal tile
pipeline_flagfrom this run: GSC-2.3 star halos + diffraction spikes rasterized by WeightWatcher (Messier/NGC masking enabled but empty on these tiles; no instrument content at tile level). NEW = the external healsparse reason maps (nside 131072 ≈ 1.6″) queried at each pixel's sky position: faint star halo (bit 1), bright star halo (2), bright stars (4), manual galaxy mask (8), MaxiMask (1024). - Exposure figures (INSTRUMENT | OLD | NEW). One full 2112×4644 MegaCam CCD each. INSTRUMENT = the CFHT/Elixir flag file (bad pixels, saturation, bleed trails); in production it is summed into both old and new
pipeline_flag, so it is shown separately — the OLD/NEW panels isolate what each masking system itself adds. OLD caveat: bit 2 is ambiguous (halo=2 collides with instrument saturated=2 under summing).
Tiles
Exposure CCDs
— Fable, on behalf of Cail
- Tile figures (OLD | NEW). OLD = ShapePipe's internal tile
If I may make a few comments:
- Using MaxiMask on the coadds is hard to justify. MaxiMask was not trained on coadds but on single exposures.
- More generally, I don't see a justification to use MaxiMask at all. It was trained on old and outdated CFHT images and the outcome is redundant in some case, or just bad.
- We don't need pixel level masks on the coadds.
- The bright stars/objects can be constructed at the single exposure level and combined to a full sky mask at posteriori.
A mask like this is the best way to inject systematics in the data
Something went very wrong with the ShapePipe tile masks, none of the star masks and actual star coordinates match.
@martinkilbinger i don't think that's true, there are stars under the old ShapePipe tile masks. it's just that not all stars are masked.
Reacted by Mike HudsonLudo identified one case where the external (galaxy mask) did not look good.
Can you @cailmdaley / Claude maybe draw 20 tiles or CCDs randomly from P3 that have at least one external mask present, and overlay just the mask with the greyscale so we can see how well or poorly it fits? (It looks better if its just an outline not a filled-in coloured polygon).
I will let you turn this into a separate subtask on github - I claim ignorance.
@martinkilbinger i don't think that's true, there are stars under the old ShapePipe tile masks. it's just that not all stars are masked.
I believe the GSC (guide star catalogue) intentionally rejected close pairs of stars (these not good for HST to guide on), so its far from complete.
More generally, I don't see a justification to use MaxiMask at all.
@aguinot what should we do for the streaks (satellites?) that appear occasionally then? the instrument mask doesn't seem to capture them:
Can you @cailmdaley / Claude maybe draw 20 tiles or CCDs randomly from P3 that have at least one external mask present, and overlay just the mask with the greyscale so we can see how well or poorly it fits? (It looks better if its just an outline not a filled-in coloured polygon).
sure, check out #855 @AstroMike
Satellite strickes could be a reason to use MaxiMMask but just for this purpose only. Also, maybe the mask would need to be rework a bit before use to enlarge and "smooth" it. The dented pattern is probably not great in terms of systematic.
Reacted by Cail McLean DaleyFollowing the telecon action, here are updated versions of the mask-comparison CCD figures with the PSF stars added. Each setools PSF-star candidate is drawn as an open circle on both the old pipeline_flag and the new healsparse (n1|n2|n4) panels: blue = survives the full ShapePipe PSF selection (setools cuts -> PSFEx sample selection -> PSFEx sigma clipping / FLAGS_PSF), so it constrains the PSF model; red = candidate rejected somewhere in that chain. Markers are open so the star stays visible underneath.
— Claude on behalf of Cail
It looks like the mask of the spikes is not aligned with the spikes. Do you understand why?
Refreshed the comparison against the regenerated August masks (
UNIONS_DR6_finalmask; same 10 P3 tiles plus the CCDs from the earlier posts; masks now drawn as outlines + hatching rather than fills). Two things stand out:- Bits 0 and 1 have exchanged content relative to the July set: July
n1≡ Augustn2pixel-for-pixel and vice versa, while the bit-table header still says bit 0 = faint halo, bit 1 = bright halo — so one vintage disagrees with its own header. The two bits are a subset/superset pair, not concentric rings: where both are present the halos coincide; the larger set adds same-radius halos around more stars. The Aug-2025 products had the same layout as July (the cumulative-area table on the wiki Masking page:n2924 deg²,n1adds 1.8 deg²), so August 2026 is the vintage that changed. The extra stars in the larger set skew fainter (GSC: brightest 5% at V≈12 vs V≈10 for stars in both), which suggests the larger set is the faint-halo selection — i.e. the August labelling is the header-consistent one and the earlier products had the bits swapped, but the GSC magnitudes are too sparse to call it. @AstroMike, is the swap intended, and are the two halo bits meant to be nested (same radius, deeper cut) rather than distinct populations? - Otherwise the change is a ~30% growth in
n4(bright stars, 8.5% → 11.1% of tile area); the other bits move by a few percent or less.
The CCD panels also carry the PSFEx star fate from a rerun on the #847 branch (○ kept, ✕ clipped by PSFEx outlier rejection, marker size ∝ CHI2_PSF; faint dots are the 20% validation hold-out). Over the 14 CCDs: 531 kept, 22 clipped, 145 held out.
All 10 tile panels
All 14 CCD panels
Full set: https://gist.github.com/cailmdaley/db592afba3553207d8f679029b581a15
— Claude, on behalf of Cail
- Bits 0 and 1 have exchanged content relative to the July set: July
It looks like the mask of the spikes is not aligned with the spikes. Do you understand why?
i don't, i was waiting for the above comparison to see the misalignment went way, looks like it hasn't.. the star spike mask seems to be big enough that it's not an issue.
The current DR6 halo labels are settled:
MASK_n1= faint-star halo (value 1, bit index 0);MASK_n2= bright-star halo (value 2, bit index 1).MASK_n4is the separate star-body mask. This follows the producer schema, Mike's identification of n2 as the bright-halo panel at the 29 September call, and Cail's adoption of that convention.The descriptions now agree in ShapePipe #886, sp_validation #343, calibration labels and the mask wiki page. Independent fixtures verify that distinct n1/n2 maps retain their identities through catalogue writing, adaptation and separate cuts (5 ShapePipe tests and 55 validation grammar tests passed in containers). No mask files, numerical column identities or default selections were exchanged.
The measured July–August content exchange remains a historical provenance difference; its cause is unknown. Each replacement ladder should carry its release, source revision when available, checksums and bit schema. This closes the remaining label question in this already-closed issue; it does not claim a new validation of mask radii or a with/without-mask rho campaign.
— GPT-6 on behalf of Cail
The gate for retiring ShapePipe's internal masking (#847 keeps the old path config-selectable until this lands). Part of epic #845.
Goal. Run both mask paths on the same real data — the Nibi P3 run (#808: 10 tiles, ~1960 single-CCD exposures) is the natural testbed — and understand, visually and quantitatively, what each mask piece does and where it comes from, before the old path is removed.
Deliverables:
USE_EXT_FLAG), cosmic rays (Stephen's masking), bright-star halos + spikes (old: GSC-2.3 + WeightWatcher; new: PhotoPipe bright-star masks), extended galaxies (old: NGC/Messier region files; new: 2MASS manual masks), MaxiMask. Per Axel (Slack 2026-07-16), the exposure level is where pixel masks matter — the audit should make explicit what the exposure-level flag image contains under each path.IMAFLAGS_ISOpopulation changes, PSF star-selection differences (counts + size–magnitude locus), and PSF model quality on the affected CCDs.Non-goal: deciding halo/spike sizes — that validation is the tangential-shear test (separate sub-issue), though the two share infrastructure.
— Fable, on behalf of Cail