You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Tie UK year files to the source dataset and package versions they were cut from #600
uk.create_datasets() names each UK year file {data_folder}/{stem}_year_{year}.h5, where the stem is dataset_logical_name(...), which drops the @revision. uk.ensure_datasets() reuses a year file whenever it exists and records its data year (DATA_YEAR_KEY), and uk.load_datasets() refuses one only when it lacks the data year. Nothing records which source a year file was cut from, or under which package versions:
A re-pinned managed dataset. When the bundle points enhanced_frs_2024_25, populace_uk_2023 or another logical name at a new revision or sha256, year files cut from the old release keep their names and are reused.
An edited local source. A local dataset rewritten in place (allow_unmanaged=True) keeps its year files, which still hold the old data. Two local sources with one file name in different folders share year files too.
Another policyengine-uk or policyengine-core version. policyengine-uk projects a single-year source to later years with extend_single_year_dataset, which uprates each year by policyengine-uk parameters read through policyengine-core's parameter tree. Year files cut under one version are reused under another.
Stamp each UK year file after its tables, as #593 does for the US, with the source identity (dataset_materialization.dataset_source_identity) and the installed policyengine-core and policyengine-uk versions. ensure_datasets() cuts the files again and load_datasets() refuses them unless the stamp equals the one a file cut now from the requested dataset would carry, computed without downloading or reading the source (requested_dataset_source_identity). The check reads only the stamp (and the data-year key) from each year file. Mirror #593's guards: re-check the source before stamping, refuse two sources that share a stem, and treat link cycles and malformed stamps as out of date. Share the stamp code between the countries where it stays simple.
Activity
Check the source again only when year files are cut from it
Stamp UK year files with their source dataset and package versions