Repository navigation
Register the ONS / NRS / NISRA small-area geography lookups and census counts behind microcosm's UK atomic-area supports (raw-only, no facts) #269
Description
Why. microcosm is rebuilding its UK atomic-area supports (E&W 2021 Output Areas, Scotland 2022 Output Areas, NI 2021 Data Zones with their LSOA/MSOA, local authority, ward, constituency, region and ITL layers) from the latest published lookups, for the geography-first reordering tracked in PolicyEngine/microcosm#931. The supports are derived build artifacts (the CONGRESSIONAL_DISTRICT_VINTAGE_CROSSWALK.md rule: regenerable, sidecar provenance, never a Ledger fact), so they stay in microcosm. Their publisher inputs are exactly what Chronicle's governance says it may hold: "register raw publisher artifacts and checksums" (docs/chronicle-governance.md), with the bytes in git plus ledger-raw R2 and the source URL, sha256, size, vintage and retrieval date in the package manifest. The precedent is statbel-nis-2025-commune-crosswalk (db/data/statbel/nis_2025_commune_crosswalk/manifest.yaml): a manifest-only registration with no source_package.yaml. docs/pe-uk-source-checklist.md line 97 already records that "Microcosm's sha-pinned OA ladder keeps its own path until consumption switches"; this issue registers the path's inputs without changing who consumes them.
Pairing rule. Each file is fetched once into microcosm's cache (~/.cache/populace-uk-geography), registered in Chronicle from those exact bytes with chronicle fetch-artifact, and consumed by microcosm's support tool through a --source-manifest pointing at the same files, so both repos pin one sha256 per file by construction. microcosm's provenance JSON carries each file's chronicle_source_id, chronicle_package_id and chronicle_year; a later cross-repo check diffs the two registers.
Ask: sixteen raw-only packages (source_id / package_id / year / source_url / vintage). Sha256s are as fetched by microcosm (2026-08-19, 2026-09-09 and 2026-09-15) and pinned in its uk/uk_atomic_area_supports.provenance.json; registration from the same bytes should reproduce them.
England and Wales (source_id ons):
ons-oa21-lsoa-msoa-lad22-lookup/ 2021 — Output Area (2021) to LSOA to MSOA to LAD (December 2021) Exact Fit Lookup, ONS Open Geography Portal itemb9ca90c10aaa4b8d9791e9859a38ca67; vintage2021_census; sha2561b7e715936374c586efbe669d82e3f3d6d093dc5295c2d99ce6ea2349f85153a(18,193,283 bytes, cached copyew_oa21_lsoa_msoa_lad22_hierarchy.csv).ons-oa21-parncp-lad-rgn-ctry-dec2024-lookup/ 2024 — Output Area (2021) to Parish/NCP to LAD to Region to Country (December 2024) Lookup V2, item7507c0292db546ed83e4ba60f1115b1d; carries the April 2023 LAD code set asLAD24CDandRGN24CD; vintages2023_april_lad,2024_rgn; sha256e01842a57c26036641fa5d6c2769dd5ac67af03a64d6790d4f35883f04eb2694(27,249,403 bytes, cached copyew_oa21_parncp_lad24_rgn24_dec2024.csv).ons-oa21-pcon-jul2024-lookup/ 2024 — Output Area (2021) to Westminster Parliamentary Constituency (July 2024) Best Fit Lookup, item5968b5b2c0f14dd29ba277beaae6dec3(fields now labelledPCON25CD, boundaries unchanged); vintage2024_pcon; sha256e4ec577416bde73d5718ef133af4adeaae9b95a41867aeb006b5fc0260c790c0(12,879,604 bytes; unchanged since the ladder build despite the field rename).ons-oa21-wd24-lad24-lookup/ 2024 — Output Area (2021) to Ward (2024) to LAD (May 2024) Best Fit Lookup V2, itemb21c632804094c149ccdfea1ae59c8c8; vintage2024_wd; sha256f2d2df158c6e3e29da09a96d456c718a065e1229590d27f6f5349e45689d7b67(12,628,147 bytes, cached copyew_oa21_ward24_lad24_bestfit.csv).ons-lad-dec2024-itl-jan2025-lookup/ 2025 — Local Authority District (December 2024) to LAU1 to ITL3 to ITL2 to ITL1 (January 2025) Lookup in the UK, item15bdb6b0ff1c4a64b34a64e2e39f8caf; vintage2025_itl; covers E/W/S12/N09 codes (361 LADs; North Ayrshire appears on two ITL3 rows); sha2562db528a486fdefebfb03a3fb89d706d037cd10f06692c98cdef60a9b7796b284(45,097 bytes, cached copyuk_lad24_itl25_lookup.csv).ons-census2021-ts001-oa/ 2021 — Census 2021 TS001 usual resident population by Output Area (Nomis bulk zipcensus2021-ts001.zip); sha256b724c1208b7628c7c558156fcc619446486353a12d0b6e0bf388add225653dc6(1,870,128 bytes).ons-census2021-ts041-oa/ 2021 — Census 2021 TS041 number of households by Output Area (Nomis bulk zipcensus2021-ts041.zip); sha256e4f11fc0ea22393fe3be444027878e67b51d2a44687e122c37439d19a548cb9f(1,511,641 bytes).
Scotland (source_id nrs):
nrs-oa22-dz22-iz22-lookup/ 2022 —oa22_dz22_iz22.zip(OA 2022 to Data Zone 2022 to Intermediate Zone 2022); vintage2022_census; sha2563565794561306479b6c618aaa3915f1f29fdce31ee798fdcdab45f880a7ba4aa(237,841 bytes; the server copy was re-uploaded on 3 Nov 2025 and the cached bytes still match).nrs-oa22-lau25-itl25-lookup/ 2025 —oa22_lau25_itl25.zip(OA 2022 to LAU 2025 to ITL 2025, bridged to Council Area 2019); vintages2019_council_area,2025_itl; sha2564d94e959dfcc83ce6bcd0c09cfc073ad597fb89812eab1d8ab664f4ccbc6ba77(369,739 bytes).nrs-oa22-ukpc24-lookup/ 2024 —oa22_ukpc24.zip(OA 2022 to UK Parliamentary Constituency 2024); vintage2024_pcon; sha256b63829d28cdf93606d17ffd12999ec12f54a8bc3065cc40fbf9e975629e4689a(181,009 bytes).nrs-census2022-oa22-usual-resident-population/ 2022 —outputarea2022_usualresidentpopulation.csv; sha256f7af756710c56f335d9332c08f95a68a5775aec4f25f17b21ab979257ed40148(678,338 bytes).nrs-census2022-index/ 2022 —census_2022_index.zip(OA to higher areas incl. electoral ward 2022; Postcode_To_OA household counts, cell-key perturbed); vintages2022_census,2022_ew; sha2560e4096a321cba2318dc5d2e35c197bf815ac333aebdf4cb0ddce82112b57a261(2,922,015 bytes).
Northern Ireland (source_id nisra):
nisra-dz2021-geojson/ 2021 —geography-dz2021-geojson.zip(Data Zone 2021 boundaries with SDZ2021, LGD2014, DEA2014); sha2563acb357ebdecae64dbc2fb645e62c68f529a00f7ae9c19684bd3b7a10e62676b(26,823,947 bytes).nisra-census2021-dz21-people/ 2021 — NISRA table buildertable.csv?d=PEOPLE&v=DZ21; sha2565fea5eb724034b4d8d749c67114d903383074dca090e8203a8e819a0d6dd445b(108,660 bytes; dynamic URL; NISRA flags MS-A01 as [UPDATED]; the R2 copy is the durable bytes).nisra-census2021-dz21-households/ 2021 — NISRA table buildertable.csv?d=HOUSEHOLD&v=DZ21; sha256fa97a7f61036f9a3ec40ea6e3812249d87d0010375f0ff7df0b5d9f77864563d(108,641 bytes; same caveat, MS-E01).nisra-dz-sdz-lookups-v3/ 2025 —geography-data-zone-and-super-data-zone-lookups-v3.xlsx(sheetDZ2021_Admin_geog_lookup, PARLCON2024); vintage2024_pcon; sha2568e2e1f6daaddd2f5b6887ccab1bf9d0bf179f6d0d2ba60ef8e0ebb495a3e1999(603,543 bytes; already the pin in Bind UK census households from Chronicle and map NI Data Zones with NISRA's published lookup (#887) microcosm#900 / #887).
Retired inputs, not to register: OA21→LAD (April 2023) item 83982ff4…, LAD22→Region 2022 item 78b348cd…, OA21→Ward 2022 item 7207b517…, LAD (April 2023)→ITL (January 2021) item 02b49429… (superseded by 2, 4 and 5 above).
What this issue does not ask. No facts, no source_package.yaml, no consumer-artifact change, no consumer channel for factless artifacts (microcosm keeps reading its own provenance register), and no change to which repo owns the derived supports.
Microcosm side: the supports built from exactly these bytes, the per-file register and the pairing are in PolicyEngine/microcosm#931's first PR (tools/build_uk_atomic_area_supports.py, uk/uk_atomic_area_supports.provenance.json).
Activity