Package UK area lists and certify UK regional releases

[juaristi22]

Part of #553 (the infrastructure half; the certification PR will close it)

The infrastructure half of #553. The UK default stays the enhanced FRS 1.58.0 for now, so manifest.json and both TRO sidecars are unchanged. This PR adds what a later certification PR needs to switch the default to a Microcosm national release. That works with a Microcosm local-area release, or, if the national release is ready first, with the enhanced FRS serving constituency and local-authority runs.

Packaged area lists

• Lists: src/policyengine/data/uk_geography/ ships constituencies_2024.csv and local_authorities_2023.csv. Their sources and hashes are in SOURCES.md.
  • Constituencies: 650 on the 2024 boundaries, including Northern Ireland, copied from policyengine-uk-data.
  • Local authorities: 361 on the April 2023 frame, with ONS names and sim-api's hex layout. This adds Ards and North Down (N09000011), and the four 2023 unitaries get their own hexes instead of a shared (0, 0).
• Regions: constituency and local-authority regions read these lists. On main the opt-in build_uk_region_registry(include_constituencies=True, include_local_authorities=True) returns no areas: the GCS helper it imports no longer exists in policyengine-core 3.33.0, and the loader swallows the ImportError.
• Outputs: compute_uk_constituency_impacts and compute_uk_local_authority_impacts label areas from the packaged lists. An explicit CSV path still wins. They no longer search local folders or download from GCS, because the local search included the old download cache and would prefer a stale 360-row file.
• Matrix lookups unchanged: the lookup CSVs paired with the legacy weight matrices (local_authorities_2021.csv) are untouched. A matrix indexes areas by that file's row order, and sim-api's fallback depends on it.

UK regional certification (certify-data --regional-manifest-uri)

#552's regional path was US-only. It now covers both UK cases:

• Ship together: a Microcosm non_default_local_area release routes region_datasets.constituency and .local_authority to its H5. This records regional_release_kind: local_area.

• National first: --regional-release-kind area_coded_national takes a certified policyengine-uk-data release, such as the enhanced FRS 1.58.0 manifest. It carries:

  • that release's national dataset, with area codes;
  • its weights artifacts: the constituency and local-authority matrices and local_geography_weights.

  All of them keep their own repo and revision, so sim-api's same-release matrix binding still holds. The constituency and LA region types route to the enhanced FRS, and the run records regional_release_kind: area_coded_national.

• Either way:

  • Both area types are routed, which satisfies sim-api's uk_area_regions_routed.
  • The regional release must serve the pinned model version, by its build or its own claim.
  • Every carried artifact is HEAD-checked. Microcosm UK's vendored artifacts are HEAD-checked too, as US Populace's already were.

• Guard: re-certifying a bundle that routes regions, without --regional-manifest-uri, is refused instead of silently dropping the routing. --drop-regional-release removes it on purpose. This applies to the US as well and automates the runbook's "still pass --regional-manifest-uri".

Cut tag as build identity

Microcosm keeps build.build_id equal to the line name on every cut and records the cut in build.cut_tag. Certification and the import-time compatibility check both use the cut tag when there is one, through release_build_id. It feeds version, data_build_id and the changelog fragment name, so two cuts of one line no longer look identical. Releases without a cut tag, such as the enhanced FRS, are unchanged.

UK region routing

• Constituency and local-authority regions take dataset_path=resolve_region_dataset_path("uk", …) and keep their row filters. That is None today, so behaviour is unchanged.
• build_uk_region_registry() adds the area regions by default only when regional_release_kind == "local_area". Explicit True/False still wins.

Universal Credit coverage floor (#491)

scripts/check_uk_uc_floor.py checks two Great Britain measures, each within 10% of its benchmark:

• benefit units with Universal Credit, against DWP's households on UC (6.7m in 2025);
• UC spend, against the OBR's March 2026 EFO Table 4.9 (£79.3bn in 2025-26).

These are the figures policyengine-uk-data calibrates to. Sources are in data/qualification/uk_uc_floor.json. The receipt holds weighted national totals only. On the enhanced FRS 1.58.0 at 2025 it passes: benefit units at 0.925 of DWP's figure and spend at 0.979 of the OBR's.

Checks

• Tests:
  • New: test_uk_area_lists.py, test_uk_uc_floor.py, UK merge and end-to-end certification tests (both modes, the compatibility refusal, the re-certification guard, cut-tag identity, the sim-api predicate), UK routing tests, and a runtime cut-tag test.
  • Updated: three US certification tests now use a temporary bundle instead of the repo's manifest, and the label-lookup tests now expect the packaged lists.
• Bundle: bundle.py check --include-tros passes, and src/policyengine/data/bundle/ is unchanged.
• Live dry run (scratch copy, nothing committed): the published Microcosm national cut …5c6b3f68 was certified national-first against the enhanced FRS 1.58.0 manifest.
  • It is refused at pe-uk 2.100.0 (the cut's build version) on the enhanced FRS's own claim, as it should be.
  • Merging the live manifests routes both area types to enhanced_frs_2024_25.h5, with the matrices at policyengine-uk-data-private@1.58.0, and names the cut tag as the build.
  • All 12 dataset references and the carried artifacts answer HEAD requests.

Follow-up (certification PR, not this one)

• Certify a Microcosm national cut built on the pinned pe-uk, using one of the two modes above.
• Add dataset_overlays.uk entries for the enhanced-FRS names no longer certified.
• Update the 14 tests pinned to the enhanced FRS, plus the docs and examples.
• Attach the Qualify UK model and data candidates automatically before promotion #574 qualification receipt and the UC floor receipt.
• Coordinate with sim-api#728.

🤖 Generated with Claude Code