Repository navigation
Load certified annual projection datasets by year #523
Description
Microcosm's static-aging operator now supports independent annual cross-sections (PolicyEngine/microcosm#957; design tracking in PolicyEngine/microcosm#333). The production data interface will publish one native single-year H5 per supported year.
The wrapper currently extracts future-year inputs from one base file and reuses derived cache files by dataset basename and year. It needs to select explicitly published annual artifacts and preserve their projection provenance.
Required behavior:
- Validate producer
metadata.dataset_yearsmappings during bundle certification and retain exact revision/hash pins. - Load the requested annual H5 and verify its embedded year, entity layout, and source identity.
- Keep cache entries separate across data revisions and relevant model/runtime identities.
- Prevent country auto-extension from substituting unpublished years; preserve genuine internal historical dependencies.
- Load requested years plus the earlier published input years required for those dependencies, without fetching future years by default.
- Preserve native-ID regional filtering and projected household weights.
This issue covers consumer integration. A separate certification change will select the published annual release and the model version containing the Medicaid/SNAP cycle fix (PolicyEngine/policyengine-us#9539).
Activity
MaxGhenis commentedon Oct 9, 2026
A concrete UK case for "preserve genuine internal historical dependencies", from #556/#557.
policyengine-uk's
basic_state_pension,new_state_pensionandadditional_state_pensiontake the data year asmin(simulation.dataset.years). They splitstate_pension_reportedat that year against its legislated rates, then scale to the simulated year's rates.If a published UK annual H5 for 2026 were loaded on its own, 2026 would become the data year. The State Pension would then follow whatever uprating the producer applied to
state_pension_reported(CPI in policyengine-uk's own projection) instead of the triple lock. That was the bug fixed in #557: −£6.2bn of State Pension in 2026, and rate reforms scored £0 under policyengine-uk 2.102.3.#557 handles this for wrapper-cut year files: each one keeps the observed year's tables, and
run()rebuilds the chain from that year. Consumer integration of published UK annual files needs the same: load the observed base year with the requested year, or have the producer record the observed year and itsstate_pension_reported. The differential test pattern intests/test_uk_year_file_data_year.pywould catch a regression.