Skip to content

Merge develop into the tomography branch #374

Description

@cailmdaley

Step 3 of the epic: merge develop (D) into feature/sp_validation-extend-to-tomography (T) on an integration branch, with a PR into T.

Resolution: keep T's API (bin-pair loops, renamed functions). Bring in D's:

  • I/O: SACC writers, open_entry/_CatalogueLoader, mandatory patch_number;
  • pure-E/B operator API;
  • uv.lock.

Ports, as separate commits:

Gate: the container test suite passes. Non-tomographic outputs (pseudo-Cℓ, ξ±, ρ/τ, COSEBIs, pure-E/B) on SP_v1.4.6.3 match D, apart from those ports.

Check shear_psf_leakage: confirm that the merged uv.lock pins a version that has CovTauTh mask support and the two-argument compute_xi_psf_sys. D's 649edf4 has both; T's 0b3c175 has neither. Then rerun the tomographic ρ/τ on the merged branch, rather than trusting earlier T outputs.

Review: Sacha. Lisa reviews the real-space/ξ± part.

Traps git won't flag (found by reading the code):

  • pol_factor changed from a bool to ±1. D's config.yaml value true would silently drop the e2 flip, so set it to −1.
  • summarize_bmodes swallows the KeyError from the new result shapes.
  • Taking D's side in cosmo_val/pseudo_cl.py creates duplicate method definitions.
  • Several workflow scripts and cv_basename still assume the old names.

— Claude on behalf of Cail

Activity

  1. cailmdaley commented on Oct 2, 2026

    @cailmdaley
    CollaboratorAuthor

    These regression tests from the sp_validation audit should pass on the merged branch. They are on branch assay/decision-record (audit epic #377). The tests in the first group already pass on develop, so they guard against the merge reintroducing the tomography branch's versions.

    Develop is right; the tomography branch regresses

    • Joint xi+^B/xi-^B chi2 uses Hartlap p=n instead of 2n, so the joint PTE is too low (b_modes.py:410). Keep develop's hartlap_factor(N, 2*nbins_eff). src/sp_validation/tests/regression/test_tomo_joint_pure_b_hartlap_dimension.py
    • treecorr_config drops min_top=6 and num_threads, so xi± depends on the node's CPU count: 0.14 sigma between 16 and 48 CPUs on a jackknife fixture, 0.66 sigma at 256. src/sp_validation/tests/regression/test_tomo_drops_min_top_pin.py
    • The MC pure-E/B covariance rebins fine xi with a uniform mean instead of TreeCorr pair weights. sigma(xi+^B) shifts by 3-5%, and the [12,83]' B-mode PTE moves from 0.0385 to 0.0428. src/sp_validation/tests/regression/test_tomo_mc_covariance_uniform_rebinning.py
    • Covariance and tau rules call scripts in an absolute external checkout, and GLASS outputs land outside the launch directory. src/sp_validation/tests/regression/test_tomo_covariance_rules_call_external_checkout.py
    • run_rho_tau.py chdirs into a hard-coded external checkout and reads its cat_config.yaml and output/. src/sp_validation/tests/regression/test_tomo_rho_tau_driver_external_checkout.py

    The tomography branch is right; develop is wrong

    • rho/tau ignore the configured HSM_FLAG_PSF/HSM_FLAG_STAR cut. Keep the tomography branch's _get_star_mask, forwarded to rho, tau and CovTauTh. src/sp_validation/tests/regression/test_rho_tau_ignores_hsm_flags.py
    • The rho/tau least-squares sample cache is not keyed on cov_estimate_method. Keep the tomography branch's cov_type key. src/sp_validation/tests/regression/test_lsq_sample_cache_ignores_cov_type.py

    Defects in the tomography branch's own code

    • plot_pure_eb passes the bin-keyed result to calculate_eb_statistics and raises KeyError('gg'). Keep tomographic iteration. src/sp_validation/tests/regression/test_tomo_pure_eb_mapping_keyerror.py
    • plot_cosebis hardcodes compute_tomography=False, so the flag has no effect. src/sp_validation/tests/regression/test_tomo_plot_cosebis_ignores_tomography.py
    • _merge_iNKA_covariance fills lower cross-polarisation blocks with COVAR_{pa}_{pb}.T instead of COVAR_{pb}_{pa}.T. All 12 cross-pol blocks are wrong. src/sp_validation/tests/regression/test_tomo_inka_merge_crosspol_transpose.py
    • Gaussian-sim workers write ..._seed_{seed}.npz, but the covariance reader looks for the unsuffixed name. It fails in a fresh directory and reads stale spectra otherwise. src/sp_validation/tests/regression/test_tomo_gaussian_sims_seed_filename_mismatch.py
    • Per-bin tau: Catalogs.build_catalog in shear_psf_leakage removes the full-catalogue mean before applying the bin mask, leaving a tau_0/2/5 offset in each bin. The fix belongs in shear_psf_leakage; the test also fails on develop. src/sp_validation/tests/regression/test_tomo_tau_centering_before_bin_mask.py
    • The pseudo-C_ell covariance producer is broken on both branches. On develop, get_theo_c_ell now returns a dict (TypeError); see also the ell=0 convention in Fix one-multipole shift in the pseudo-Cℓ Gaussian covariance theory #372. The tomography branch calls the removed calculate_pseudo_cl_eb_cov, and cv_pseudo_cl.py omits pol_list. src/sp_validation/tests/regression/test_pseudo_cl_covariance_producer_crashes.py

    Claude (Opus 5.5) on behalf of Cail

  2. cailmdaley commented on Oct 5, 2026

    @cailmdaley
    CollaboratorAuthor

    Done in #394 (merge commit 2ff1b50 on the tomography branch).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions