Stamp saved US outputs with the rename register they were calculated under

[MaxGhenis]

Fixes #592

Follow-up to #590 (issue #532), which stamped create_datasets year files with the rename register and left saved simulation outputs out of scope.

The defect

MicrosimulationModelVersion.load() refused a saved US output only when it had no renames record (policyengine_legacy_input_renames), and save() required only that record. The record shows that an output was calculated with the legacy-input mapping, not under which register. Once LEGACY_INPUT_RENAMES gains, loses or retargets an entry, outputs saved earlier still carry a record, so Simulation.load() and Simulation.ensure() reuse them. An output calculated before the WIC entry existed records {} and passed that check, though every WIC-eligible person in it took WIC up (PolicyEngine/microcosm#1026).

There was a second gap. A run over a year file maps only what the file still stores under a legacy name. So an output also depends on the register the year file was cut under, and on its format revision, and nothing in the output recorded either. An output calculated from a stale year file opened directly (PolicyEngineUSDataset(filepath=...), which load_datasets refuses) was reused indefinitely. Outputs over year files would also be reused after YEAR_FILE_STAMP_REVISION is raised, for example when #563 removes the temporary ACS WIC assumption.

What changes

The stamp. A US run() records, before it applies the mapping, metadata["output_stamp"] = legacy_inputs.output_stamp(year_file):

{"register": {"would_claim_wic": "takes_up_wic_if_eligible"}, "revision": 1, "year_file": null}

• register is the whole LEGACY_INPUT_RENAMES, read when the run starts, as for year files.
• revision is the new OUTPUT_STAMP_REVISION. Its comment says to raise it with YEAR_FILE_STAMP_REVISION when the temporary ACS WIC assumption is removed, because a run straight over the ACS source applies that assumption too.
• year_file is the stamp of the year file the input data was cut into, or null when the input dataset keeps none (in-memory data, a source file). This answers the "carry the year file's stamp into the output's provenance" question: yes, because the output inherits whatever the year file lost.

Where the year-file stamp comes from. PolicyEngineUSDataset.load() keeps a file's year-file stamp in metadata["year_file_stamp"], and drops the key when the file has none. create_datasets() sets the key on the datasets it returns, so a run over ensure_datasets() output records it on both the load and the create paths. legacy_inputs.year_file_stamp_metadata(path) reads the stamp. It returns None only when no stamp is stored. A stamp stored as JSON null, or one that cannot be read, becomes the string "unreadable", so a corrupt year-file stamp can never pass as "no year file".

Save. MicrosimulationModelVersion.save() requires the recorded stamp, as it already required the record. It writes the stamp last, as H5 dataset policyengine_output_stamp, after the tables, the record and the SPM receipt. It writes the stamp the run recorded, not one made at save time. PolicyEngineUSDataset.save() truncates the file first, so a save that stops part-way leaves no stamp.

Load. load() calls legacy_inputs.output_stamp_problem(path) first and raises ValueError("Saved US simulation output {path} is out of date: {reason}. Run the simulation again (Simulation.ensure() does so)."). Simulation.ensure() already reruns on any load error. A missing file still raises FileNotFoundError, so ensure() reruns without logging an unexpected error. load() restores the stamp into metadata, so saving a loaded output again keeps it.

The check. It mirrors year_file_stamp_problem. An output is current only when all of these hold:

• the stamp has exactly the keys register, revision and year_file;
• its revision is an int (not a bool) equal to OUTPUT_STAMP_REVISION;
• its register is a str→str map equal to this release's;
• its year_file is null, or is exactly this release's year_file_stamp(), compared with the same strict rules (true is not 1, and 1.0 is not 1);
• the renames record is a str→str map that lies within the stamped register.

Put another way, the stamp equals what this release would write for a run over a dataset with no year-file stamp, or over a current year file. The reasons come from the register alone, with no WIC special case, and name each disagreeing entry as old -> new. For example:

• "it was calculated before these renames were registered, so it may have been calculated without them: would_claim_wic -> takes_up_wic_if_eligible";
• "it was calculated from a year file cut before these renames were registered, which may have lost them: …";
• "it has no stamp of the register of renamed inputs it was calculated under, …" (outputs saved by Map the stored WIC take-up draw onto takes_up_wic_if_eligible when loading US data #531 to this PR, which are calculated again once);
• "it has no record of the renamed stored inputs mapped when it was calculated, so it was saved by an older policyengine.py or not saved completely, …".

Distinct stamps. The output stamp's H5 name and key set both differ from the year file's. So an output never passes as a year file, nor a year file as an output, even when one stamp is stored under the other's name.

The _register_differences helper is factored out of year_file_stamp_problem, with no change to its behaviour (all 71 year-file tests pass unchanged).

Invariants

All of these are executed, not just stated. The Hypothesis oracles are stated on the stored JSON (and, for raw nodes, on an independent decoding of how HDF5 stores them), not by calling the code under test.

Invariant	Property tests (Hypothesis)	Example and integration tests
Exactness. An output is current iff its stamp has this release's output revision and register, its year_file is null or exactly this release's year-file stamp, and its record lies within the stamped register. Equivalently, the stamp equals output_stamp(None) or output_stamp(year_file_stamp()).	test_an_output_is_current_only_under_this_releases_exact_stamp (300 examples, weighted so about 44% are current, 22% are stale only through their year file, and 30% are stale otherwise; registers come from a four-name alphabet)	test_a_future_register_entry_…, test_retiring_or_retargeting_an_entry_…, test_raising_the_output_revision_…, test_raising_the_year_file_revision_makes_outputs_from_year_files_out_of_date
Totality. For any stamp and record content, the check returns a reason or None and agrees with the oracle in both directions. It raises only FileNotFoundError and the three errors unrelated to content.	test_the_output_problem_is_total_and_exact_for_any_content (300 examples: arbitrary JSON, raw text, raw bytes, numbers, arrays, groups)	test_a_malformed_stamp_is_out_of_date (16 cases), the out-of-date year-file cases (15), raw-node records, cyclic soft links, dangling links, deep JSON, forced RecursionError, and errors raised on open and on read
Refinement. The stamp never accepts an output the old record check refused.	inside the exactness property	test_an_output_saved_before_the_mapping_is_not_reused (existing test, now with record and stamp both removed)
Round trip. A stamp reads back as written, writing again replaces it, and the tables, the record and any year-file stamp are untouched. A loaded output saved again keeps its stamp.	test_an_output_stamp_reads_back_as_written (50 examples)	test_the_stamp_is_stored_as_utf8_json, test_a_saved_output_is_stamped_with_the_register_it_was_calculated_under
Named renames. The reason names every entry on which the output's own register or its year file's register disagrees with this release, as old -> new, and no entry they agree on.	test_a_stale_output_names_each_rename_it_may_lack (200 examples over the output's register and its year file's register together)
No partial output is current. This covers every truncation, a save that stops before the stamp, a save without a recorded stamp, and re-saving the tables.	test_a_truncated_output_is_never_current (100 prefix lengths)	test_save_stamps_the_output_last (call order, and the record and SPM receipt already present when stamping), test_an_output_whose_save_stopped_before_the_stamp_is_calculated_again, test_an_output_without_a_stamp_is_not_saved, test_saving_the_tables_again_drops_the_stamp
Provenance. The stamp the run recorded is the one saved, from the dataset the run was given.		test_save_writes_the_stamp_the_run_recorded, test_a_run_over_a_year_file_records_its_stamp (both the create and load paths), test_a_region_scoped_run_records_its_year_files_stamp, test_a_dataset_keeps_its_year_files_stamp (11 stored forms), test_reloading_a_resaved_year_file_drops_the_stamp_it_kept
Distinct stamps. A current output is never a current year file, and the reverse, even with one stamp stored under the other's name.	test_an_output_never_passes_as_a_year_file_nor_the_reverse (100 examples)	test_an_output_is_never_loaded_as_a_year_file_nor_the_reverse (Simulation.load() on a year file, and load_datasets on an output, over the real engine)
Load and ensure agree with the check. Simulation.load() refuses exactly when output_stamp_problem gives a reason, and its message carries that reason. ensure() runs again exactly then, and otherwise reuses the output.	test_load_refuses_and_ensure_runs_again_exactly_when_an_output_is_out_of_date (60 examples over 3 registers × 14 stamps × 8 records, applied to a real saved output; about 30% are current)
Stamp-only decision. Deciding never opens a table.		test_the_decision_reads_only_the_stamp_and_the_record

Integration over the real policyengine-us 2.2.1 engine uses the three-person Los Angeles WIC fixture.

• test_an_output_calculated_before_a_register_entry_is_calculated_again replays the WIC history with no fake engine. It runs and saves under an empty register: the infant gets WIC, and the output records {}, which the old check accepted. Under the real register, load() refuses it with a reason naming would_claim_wic -> takes_up_wic_if_eligible. ensure() runs again, the infant's WIC is 0, and the next ensure() reuses the output.
• test_an_output_from_an_out_of_date_year_file_is_not_reused cuts a year file under an empty register and opens it directly. The run over it is calculated under the current register but gives the infant WIC, and load() refuses it on the year-file reason alone. After ensure_datasets cuts the file again, ensure() over it recalculates, and the infant's WIC is 0.
• test_a_future_register_entry_makes_saved_outputs_stale adds a fictitious entry that the engine does not define. Outputs go stale and are calculated again with unchanged results, and they go stale once more when the register reverts.

Mutation testing

All 44 mutants were killed. They ran in three parallel copies of the source, with the Hypothesis database cleared before each mutant. Unit tests ran first, and the engine integration tests ran only for a mutant the unit tests left alive. The harness (mutate.py), the per-worker logs and mutation-summary.txt are in ~/PolicyEngine/_reviews/pepy-592/. Each line of the summary names the first test that failed.

• output_stamp_problem:
  • newer or older register accepted;
  • revision check dropped, or a newer revision accepted;
  • isinstance revision accepting True;
  • unstamped output counted current;
  • absent record treated as {};
  • record-within-register check dropped;
  • extra stamp keys accepted;
  • year_file key optional;
  • year file ignored;
  • malformed register accepted.
• The year-file part:
  • revision check dropped;
  • isinstance revision;
  • extra keys accepted;
  • a malformed stamp accepted;
  • newer or older year-file register accepted;
  • loose == comparison, which accepts true for 1.
• Errors:
  • a missing output not raised;
  • environmental errors swallowed on open;
  • an unreadable output raising;
  • the decision reading a table node;
  • a cyclic link not caught (only RecursionError caught).
• Readers and writers:
  • register frozen at import;
  • revision hardcoded;
  • year-file stamp not copied;
  • JSON null year-file stamp kept as None;
  • an unreadable year-file stamp kept as None;
  • environmental errors swallowed in year_file_stamp_metadata;
  • the output stamp sharing the year-file stamp's H5 name.
• datasets.py:
  • load() keeping no year-file stamp;
  • load() not dropping a stale one;
  • create_datasets keeping none;
  • load() reading through read_year_file_stamp, which conflates null with absent.
• model.py: the run recording no year file; a scoped run reading the scoped copy's empty metadata.
• model_version.py:
  • save() stamping with the register at save time;
  • save() not stamping;
  • stamping before the SPM receipt;
  • saving without a recorded stamp;
  • load() ignoring the check, or keeping the record-only check;
  • load() not restoring the stamp.

Review

Three independent reviews ran on read-only Subfleet lanes. Their reports are in ~/PolicyEngine/_reviews/pepy-592/.

1. GPT-6.1 Sol (hard tier) at 66e97bb4: APPROVE (review-1.md), with three minor findings and a nit, all fixed in e7261779:

   • cyclic HDF5 soft links raised RuntimeError instead of reading as unreadable content. The fix is in the shared JSON reader, so year files gain it too;
   • the totality oracle got raw-node descriptors such as ("text", "{}"), which decode to a valid record. It now decodes them independently and asserts both directions;
   • region-scoped year-file provenance had no regression test;
   • the changelog overstated the stale-year-file guarantee.

   It also noted that repeated st.one_of branches don't weight a draw. Hypothesis drops duplicate branches, which I confirmed. The generator now weights with sampled_from.

2. Opus 5.5 on 66e97bb4..e7261779: APPROVE (review-2.md; no Codex lane was free). Both of its nits, an exact-reason assertion and a comment, were fixed in 90fa5037.

3. Opus 5.5 on e7261779..a51f31c0: APPROVE (review-3.md). This delta adds the nits and str(...) around the output path, which removes a new mypy arg-type error and cannot change behaviour. Its optional comment nit was fixed in f06017bc, a comment-only commit. 1e107e09 merges origin/main (48ba01a5, a version bump).

Not covered here

• storage_id is not content-addressed and carries no model or data version (Simulation cache is write-only: uuid4 keys mean ensure() never reloads results, and data_folder accumulates orphaned h5 files #455). The stamp makes outputs stale when the register changes, not when the engine or the data does.
• Source dataset identity. dataset_logical_name drops @revision, so a year file re-cut from a new release of the same logical dataset carries the same stamp. That is the separate follow-up Stamp US year files with the rename register they were cut under #590 named.
• Year files cut before the stamp existed, opened directly. They keep no stamp, so an output calculated from one records null and is reused. load_datasets refuses such files, and ensure_datasets cuts them again.
• Run records and release_bundle are unchanged. A run record already binds the SHA-256 of the input and output files, and those bytes include both stamps.
• Remove temporary ACS WIC assumption after replacement certification #563 (removing the temporary ACS WIC assumption) must raise OUTPUT_STAMP_REVISION as well as YEAR_FILE_STAMP_REVISION. Both constants' comments say so.

What I verified

Locally with Python 3.14.4 on macOS, HF_HUB_OFFLINE=1, no downloads. The host was under heavy load (load average 50–80).

• At e7261779:
  • tests/test_us_output_stamp.py: 105 passed;
  • tests/test_us_year_file_stamp.py: 71 passed, unchanged by the _register_differences refactor;
  • tests/test_us_legacy_inputs_integration.py and tests/test_us_legacy_inputs.py: 119 passed, peak RSS 4.0 GiB.
• At 66e97bb4, 154 passed in the related suites: test_cache, test_dataset_persistence, test_run_record, test_spm_model, test_small_follow_ups, test_us_native_alignment, test_us_long_term_datasets, test_extra_variables, test_us_microsim_structural_reforms, test_us_program_statistics, test_poverty_run, test_uk_year_file_data_year and test_release_manifests. Peak RSS was 7.1 GiB.
• Later commits:
  • the save-path integration tests (11) pass at a51f31c0;
  • the cyclic-link tests pass at f06017bc;
  • 1e107e09 only merges the version bump.
• Lint and types: ruff 0.17.0 format --check and check are clean. mypy (informational in CI) reports no new errors in the changed modules compared with origin/main.

CI: pending at 1e107e09 (the runs queued behind a runner backlog; earlier heads were cancelled by later pushes).

axiom: n/a: infra (US saved-output cache staleness; no policy logic)

🤖 Generated with Claude Code