Stamp saved US outputs with the rename register they were calculated under

changelog.d/592.fixed.md

@@ -0,0 +1 @@
+A US `Simulation.run()` stamps its output with the register of renamed stored inputs it was calculated under, an output format revision, and the stamp of the year file its input data was cut into, if any. `Simulation.load()` refuses a saved US output, and `Simulation.ensure()` runs it again, unless that stamp matches this release exactly. An output calculated before a rename was registered, or from a year file carrying an out-of-date stamp, is therefore not reused. The error says why, naming the renames the output may lack. Outputs saved before the stamp existed are calculated again once. A stamp or renames record behind a soft link that leads back to itself now counts as unreadable, for year files too, rather than raising.

docs/microsim.md

@@ -401,11 +401,9 @@ The renames applied are recorded as `{old: new}` (`{}` when none applied):
 carried the draw: data that stores neither name runs with the new input's
 default, which for WIC is full take-up.
 
-A saved US output written before this mapping existed has no record and may
-have been calculated without the mapped inputs, so `load()` refuses it, and
-`ensure()` runs the simulation again and saves the new output. Saved outputs
-are checked only for the record, not for the register they were calculated
-under.
+Year files and saved outputs are each stamped with the register they were
+cut or calculated under, so neither is reused once the register changes (see
+below).
 
 ### Year files
 
@@ -443,7 +441,56 @@ than cutting the files again, when a year file cannot be opened for reasons
 unrelated to its contents: no permission, a directory at its path, or a lock
 held by another process. A year file opened directly, as
 `PolicyEngineUSDataset(filepath=...)`, is not checked, and saving it again with
-`save()` drops its stamp.
+`save()` drops its stamp. A dataset read from a year file, or returned by
+`create_datasets`, keeps the file's stamp as `metadata["year_file_stamp"]`, so
+an output calculated from it records the stamp (see below).
+
+### Saved outputs
+
+A US `Simulation.run()` records a stamp in its output's
+`metadata["output_stamp"]`, read before it applies the mapping: UTF-8 JSON
+`{"register": {old: new, ...}, "revision": 1, "year_file": ...}`. It holds
+the whole register, an output format revision, and the stamp of the year file
+the input data was cut into, or `null` when the input dataset keeps none (an
+in-memory dataset or a source file, for example). `save()` writes it into the
+output file, after everything else, as
+`legacy_inputs.OUTPUT_STAMP_H5_DATASET`; `load()` restores it. Its name and
+keys differ from the year-file stamp's, so an output never passes as a year
+file, nor a year file as an output.
+
+`Simulation.load()` refuses a saved US output unless its stamp is exactly what
+this release would write for a run over a dataset with no year-file stamp, or
+over a current year file, and its renames record lists only renames in the
+stamped register. `Simulation.ensure()` then runs the simulation again and
+saves the new output. `legacy_inputs.output_stamp_problem(path)` returns the
+reason, or `None` for a current output; it reads only the stamp and the
+record. A saved output is out of date when:
+
+- it has no stamp. Outputs saved before the mapping existed have no record
+  either, and may have been calculated with every WIC-eligible person taking
+  WIC up. Outputs saved with the record but before the stamp are calculated
+  again once, and so is an output whose save did not finish;
+- its register differs from this release's. An output calculated before an
+  entry was registered may lack that entry's input, and one calculated under
+  an entry this release lacks, or maps to another new name, used a mapping
+  this release does not;
+- it was calculated from a year file whose stamp is not this release's: a
+  run over a year file maps only what the file still stores under an old
+  name, so the output inherits whatever the file lost. That happens for a
+  year file opened directly when `load_datasets` would refuse it, and for
+  every year file once the register or the year-file format revision
+  changes;
+- it has another output format revision, or its stamp or record cannot be
+  read.
+
+So any change to `LEGACY_INPUT_RENAMES` makes every saved US output out of
+date too. A missing output file raises `FileNotFoundError`, as before, so
+`ensure()` runs the simulation. An output file that cannot be opened for
+reasons unrelated to its contents (no permission, a directory, a lock held by
+another process) raises that error rather than counting as out of date. A
+year file cut before the stamp existed and opened directly keeps no stamp, so
+an output calculated from it records `null` and is reused; `load_datasets`
+refuses such a file.
 
 ## Pinned model versions
 

src/policyengine/tax_benefit_models/common/model_version.py

@@ -356,9 +356,11 @@ def save(self, simulation: Simulation) -> None:
                 "something to persist."
             )
         serialized_spm = None
+        stamp = None
         if self.country_code == "us":
             from policyengine.core.spm import SPMProvenance
             from policyengine.tax_benefit_models.us.legacy_inputs import (
+                OUTPUT_STAMP_KEY,
                 RENAMES_RECORD_KEY,
             )
 
@@ -379,15 +381,23 @@ def save(self, simulation: Simulation) -> None:
             )
             # ``PolicyEngineUSDataset.save()`` writes this record into the
             # file, and ``load()`` refuses an output without it.
-            renames = (getattr(simulation.output_dataset, "metadata", None) or {}).get(
-                RENAMES_RECORD_KEY
-            )
+            metadata = getattr(simulation.output_dataset, "metadata", None) or {}
+            renames = metadata.get(RENAMES_RECORD_KEY)
             if renames is None:
                 raise ValueError(
                     "This US output does not record which renamed stored "
                     "inputs were mapped when it was calculated; run again "
                     "before saving"
                 )
+            # The stamp the run recorded, not one made now: the register the
+            # mapping used when the output was calculated.
+            stamp = metadata.get(OUTPUT_STAMP_KEY)
+            if stamp is None:
+                raise ValueError(
+                    "This US output does not record the register of renamed "
+                    "stored inputs it was calculated under; run again before "
+                    "saving"
+                )
         simulation.output_dataset.save()
         if serialized_spm is not None:
             # Store UTF-8 JSON in a dataset rather than an attribute: the
@@ -399,6 +409,16 @@ def save(self, simulation: Simulation) -> None:
                     data=serialized_spm,
                     dtype=h5py.string_dtype("utf-8"),
                 )
+        if stamp is not None:
+            from policyengine.tax_benefit_models.us.legacy_inputs import (
+                write_output_stamp,
+            )
+
+            # Stamp last. ``PolicyEngineUSDataset.save()`` truncates the file
+            # first, so an output whose save stopped part-way has no stamp,
+            # and ``load()`` refuses it. It also refuses to save without a
+            # filepath, so there is one here.
+            write_output_stamp(str(simulation.output_dataset.filepath), stamp)
 
     def load(self, simulation: Simulation) -> None:
         """Rehydrate the simulation's output dataset from disk.
@@ -414,17 +434,36 @@ def load(self, simulation: Simulation) -> None:
         if self.country_code == "us":
             from policyengine.core.spm import SPMProvenance
             from policyengine.tax_benefit_models.us.legacy_inputs import (
+                OUTPUT_STAMP_KEY,
                 RENAMES_RECORD_KEY,
+                output_stamp_problem,
+                read_output_stamp,
                 read_renames_record,
             )
 
+            # Outputs calculated under another register of renamed stored
+            # inputs, or from a year file cut under one, may have been
+            # calculated without a mapped input (for example with every
+            # WIC-eligible person taking WIC up), so they are not reused.
+            # ``Simulation.ensure()`` runs such a simulation again. A missing
+            # file raises ``FileNotFoundError`` here.
+            problem = output_stamp_problem(filepath)
+            if problem is not None:
+                raise ValueError(
+                    f"Saved US simulation output {filepath} is out of date: "
+                    f"{problem}. Run the simulation again (Simulation.ensure() "
+                    "does so)."
+                )
             with h5py.File(filepath, "r") as stream:
                 raw = (
                     stream["policyengine_spm"].asstr()[()]
                     if "policyengine_spm" in stream
                     else None
                 )
+            # Both are current: ``output_stamp_problem`` accepts an output
+            # only with a stamp and a well-formed record.
             recorded_renames = read_renames_record(filepath)
+            recorded_stamp = read_output_stamp(filepath)
             if raw is None:
                 raise ValueError(
                     "Saved US simulation has no SPM configuration or receipt"
@@ -433,15 +472,6 @@ def load(self, simulation: Simulation) -> None:
             if recorded["config"] != simulation.spm_config:
                 raise ValueError("Saved US simulation uses different SPM settings")
             receipt = SPMProvenance.model_validate(recorded["provenance"])
-            # Outputs saved before stored inputs were mapped onto renamed
-            # live inputs were calculated without them (for example with
-            # every WIC-eligible person taking WIC up), so they are not
-            # reused. ``Simulation.ensure()`` runs such a simulation again.
-            if recorded_renames is None:
-                raise ValueError(
-                    "Saved US simulation predates the mapping of renamed "
-                    "stored inputs (it has no record of them); run it again"
-                )
 
         if self.country_code == "uk":
             from policyengine.tax_benefit_models.uk.datasets import (
@@ -471,8 +501,13 @@ def load(self, simulation: Simulation) -> None:
 
             simulation.spm_receipt = receipt
             simulation.spm = SPMSelection.model_validate(recorded["config"])
+            # Kept so that saving the loaded output again writes its stamp.
             simulation.output_dataset.metadata.update(
-                {"spm_config": recorded["config"], RENAMES_RECORD_KEY: recorded_renames}
+                {
+                    "spm_config": recorded["config"],
+                    RENAMES_RECORD_KEY: recorded_renames,
+                    OUTPUT_STAMP_KEY: recorded_stamp,
+                }
             )
 
         if os.path.exists(filepath):

src/policyengine/tax_benefit_models/us/datasets.py

@@ -1,3 +1,4 @@
+import copy
 import hashlib
 import importlib.util
 import json
@@ -27,13 +28,15 @@
 )
 from policyengine.tax_benefit_models.us.legacy_inputs import (
     RENAMES_RECORD_KEY,
+    YEAR_FILE_STAMP_KEY,
     apply_legacy_input_renames_to_microsimulation,
     check_yearly_periods,
     pending_legacy_input_renames,
     read_renames_record,
     write_renames_record,
     write_year_file_stamp,
     year_file_stamp,
+    year_file_stamp_metadata,
     year_file_stamp_problem,
 )
 from policyengine.utils.hashing import sha256_file
@@ -127,20 +130,29 @@ def load(self) -> None:
             self.metadata[RENAMES_RECORD_KEY] = renames
         if _is_policyengine_core_h5(path):
             self.data = _load_policyengine_core_h5(path, self.year)
-            return
-
-        with pd.HDFStore(filepath, mode="r") as store:
-            data = {entity: store[entity] for entity in US_ENTITY_KEYS}
-        # Native producer files store calibrated household weights. Project
-        # missing weights through membership IDs on these in-memory copies,
-        # before constructing MicroDataFrames; the source HDF remains read-only.
-        _assign_missing_entity_weights(data)
-        self.data = USYearData(
-            **{
-                entity: MicroDataFrame(frame, weights=f"{entity}_weight")
-                for entity, frame in data.items()
-            }
-        )
+        else:
+            with pd.HDFStore(filepath, mode="r") as store:
+                data = {entity: store[entity] for entity in US_ENTITY_KEYS}
+            # Native producer files store calibrated household weights. Project
+            # missing weights through membership IDs on these in-memory copies,
+            # before constructing MicroDataFrames; the source HDF remains
+            # read-only.
+            _assign_missing_entity_weights(data)
+            self.data = USYearData(
+                **{
+                    entity: MicroDataFrame(frame, weights=f"{entity}_weight")
+                    for entity, frame in data.items()
+                }
+            )
+        # The stamp of the year file this data was cut into, if it is one. A
+        # run records it in its output's stamp, so an output calculated from
+        # a year file that goes out of date is not reused (see
+        # ``legacy_inputs.OUTPUT_STAMP_H5_DATASET``).
+        stamp = year_file_stamp_metadata(path)
+        if stamp is None:
+            self.metadata.pop(YEAR_FILE_STAMP_KEY, None)
+        else:
+            self.metadata[YEAR_FILE_STAMP_KEY] = stamp
 
     def __repr__(self) -> str:
         if self.data is None:
@@ -572,6 +584,9 @@ def create_datasets(
             # Stamp last. ``save()`` truncates the file first, so a year file
             # whose write stopped part-way has no stamp and is cut again.
             write_year_file_stamp(us_dataset.filepath, stamp)
+            # Keep it, as loading the file would, so a run over the returned
+            # dataset records it in its output's stamp.
+            us_dataset.metadata[YEAR_FILE_STAMP_KEY] = copy.deepcopy(stamp)
 
             dataset_key = f"{dataset_stem}_{year}"
             result[dataset_key] = us_dataset