Skip to content

Migrate cosmo_val + Snakemake to the SACC writers #261

Closed as not planned
Closed as not planned

Description

@cailmdaley

Make SACC the sole native output format of the cosmology-validation pipeline. Every two-point product (ξ± coarse and fine, pseudo-Cℓ, COSEBIs, pure-E/B, ρ/τ, n(z)) is written through the sacc_io builders into single-statistic SACC part files, and the Snakemake workflow assembles those parts into the two terminal SACC files per catalogue version. No bespoke writer remains on the data-vector path.

Desired end state

  • Each cosmo_val statistic's write step is replaced by a call to the matching sacc_io builder, producing a part file that holds exactly one statistic plus the self-describing n(z) tracers it references. A producer writes a part with a single save(s, path) call; it never assembles a terminal file and never emits a bespoke format alongside.
  • A dedicated Snakemake assembly rule reads the parts, concatenates them in canonical order, attaches the assembled covariance, and writes the two terminal files: {version}.sacc (analysis vector, FullCovariance) and {version}_xi_fine.sacc (fine ξ± grid, DiagonalCovariance from TreeCorr variances).
  • The bespoke .txt/custom-FITS/.npz writers on the data-vector path are deleted; nothing downstream reads them.

sacc_io is a fixed dependency and the single source of truth for every builder signature. Metadata keys required on every file: catalogue_version, sp_validation_version, created, npatch.

Acceptance

The workflow produces both terminal files per version from parts; the deleted bespoke writers have no remaining readers; the assembled files match the sacc_io layout contract.

— Opus on behalf of Cail.

Activity

  1. cailmdaley commented on Jul 10, 2026

    @cailmdaley
    CollaboratorAuthor

    Filed in error — content folded into #247.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions