Skip to content

Estimating NILC covariances + weights correctly in subset footprints - #16

Closed
jaejoonk wants to merge 6 commits into
mainfrom
nilc-weight-maps
Closed

jaejoonk wants to merge 6 commits into
mainfrom
nilc-weight-maps

Conversation

@jaejoonk

@jaejoonk jaejoonk commented Oct 2, 2026

Copy link
Copy Markdown
Collaborator

Changes include the following (most of heavy lifting done by Claude Opus 5.5):

  • Include options for the coberus coadder to evaluate weight maps, write them to disk, and (for pipeline.py) check that the sum of all weights is 1 for each wavelet scale
  • For Gaussian and tophat smoothing:
    • Correctly normalize covariance maps such that means (implemented as an application of the smoothing filter) involving subset footprints are not biased low near apodized and/or smoothed edges.
    • Apply these normalizations in a non-block-local way that ensures positive-semidefinite covariance matrices for the coberus coadder to invert safely (the bitwise encoding implementation for tagging overlapping regions really impressed me, recommend checking that out!)
    • N_modes_eff correctly evaluated (@samgolds)

jaejoonk and others added 6 commits September 29, 2026 16:13
- coadd_maps_pixels now also returns the per-pixel ILC weights, and
  coadd(..., return_weights=True) returns them alongside the map.
- needlet_coadd writes the weights estimated for 'coadd' to
  {out_root}wavelet_weights_scale_{k}_{tag}.fits (removed with
  delete_intermediate like the covariance maps).
- New 'cmb_weights' nmap label applies the weights to ones maps and
  returns the per-scale weight sums (no wave2map) in
  outmaps['cmb_weights_coadd'].

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Runs a block-smoothed CMB NILC on the synthetic test_pipeline dataset
with nmap_labels=["cmb_weights"] and prints, per wavelet scale, the
range of cmb_weights_coadd and the sum over tags of the written weight
maps (both should be 1 wherever any map contributes).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
needlet_coadd no longer accepts 'cmb_weights' in nmap_labels. Per-scale
ILC weight maps are still written to disk; with check_cmb_weights=True,
each scale's weights are checked to sum to 1 in every covered pixel
(CMB solution), raising ValueError otherwise.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
For gaussian and tophat covariance smoothing, the smoothed products and
local means averaged in zeros from outside each map's footprint, so the
covariance was biased low (by up to ~50%) near mask edges. Normalizing
each pair by its own smoothed overlap fixes the bias but makes the
per-pixel covariance indefinite when more than two footprints overlap.

Instead, at each pixel every entry is now estimated over the common mask
M_T of the set T of maps covering that pixel,
C_ij = S[M_T d_i d_j] / S[M_T], which is unbiased at edges and stays
positive semi-definite for a non-negative kernel. The local mean used
for smooth_mean_cov is normalized by each map's own mask. Block
smoothing is unchanged.

Add tests/test_cov_edges.py, which checks that the covariance is
unbiased and positive definite for three maps with overlapping edges.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The coadder returns zero wherever no mask covers a pixel, so the
taper region and source holes of the base map were zeroed in each
wavelet scale. The sharp cut leaked into the footprint on
reconstruction, giving a ~6% rms signal error at low ell even far
from the mask edge. Those pixels now take the base map's own wavelet
coefficients; the final base mask is still applied afterwards.

Add tests/check_two_map_subset.py, a two-map test (wide noisy base map
plus a deeper map on a subset footprint) that checks per-scale ILC
weights against the analytic inverse-noise-variance solution, signal
preservation and noise of the coadd for block, gaussian and tophat
covariance smoothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants