diff --git a/changelog.d/602-cgt-provenance.added.md b/changelog.d/602-cgt-provenance.added.md new file mode 100644 index 00000000..81efa567 --- /dev/null +++ b/changelog.d/602-cgt-provenance.added.md @@ -0,0 +1 @@ +Added a record of the capital gains realisation response to UK runs (`policyengine.tax_benefit_models.uk.capital_gains_responses`). Each population run records the policyengine-uk release and where it was installed from when that is not a package index, the caller's explicit `gov.simulation.capital_gains_responses` settings in the order they apply and, where the installed policyengine-uk reports it, the response it resolved after every reform and modifier. The record is kept with the output dataset (`metadata`), in the release bundle, the run record and the saved output file; household results carry it under `provenance`, and managed microsimulations under `policyengine_bundle["capital_gains_responses"]` for each year. UK cache and saved-output identity now includes a digest of the explicit settings, and a saved UK output is reused only under the same settings and the same policyengine-uk release from the same source (never from a local directory install). Each saved UK output is therefore calculated once more after upgrading, so a job that relies on a warm UK cache takes as long as a cold run the first time. Added `StaleOutputError`, a `ValueError` that `load()` raises for a saved output it refuses (UK and US alike); `Simulation.ensure()` logs the reason at INFO and runs the simulation again, instead of logging a warning with a traceback. diff --git a/docs/dev.md b/docs/dev.md index 62a3e398..d246ebf7 100644 --- a/docs/dev.md +++ b/docs/dev.md @@ -106,6 +106,6 @@ src/policyengine/ **Country-specific model classes**: `PolicyEngineUSLatest` and `PolicyEngineUKLatest` inherit from a shared `MicrosimulationModelVersion` base (variable/parameter loading, manifest certification, `save`/`load`). Each subclass only implements `run()` and a handful of country hooks (`_load_system`, `_load_region_registry`, `_dataset_class`, `_get_runtime_data_build_metadata`). The US `run` applies reforms as a dict at `Microsimulation(reform=...)` construction time; the UK `run` wraps inputs as `UKSingleYearDataset` and applies reforms via a modifier after construction. -**LRU cache + file caching**: `Simulation.ensure()` checks an in-process LRU cache (max 100 entries), then tries loading from disk, then falls back to `run()` + `save()`. +**LRU cache + file caching**: `Simulation.ensure()` checks an in-process LRU cache (max 100 entries), then tries loading from disk, then falls back to `run()` + `save()`. When `load()` refuses an out-of-date saved output it raises `StaleOutputError` (a `ValueError`), which `ensure()` logs at INFO as an expected rerun. Any other load error is logged as a warning. **Output pattern**: All output types inherit from `Output`, implement `.run()`, and populate result fields. Convenience functions (e.g., `calculate_us_poverty_rates()`) create, run, and return collections of output objects. diff --git a/docs/households.md b/docs/households.md index f83e3fc7..c963a63b 100644 --- a/docs/households.md +++ b/docs/households.md @@ -165,6 +165,8 @@ For UK households, a `YYYY-MM-DD` key is effective from that date until the next UK households reject bare years such as `"2027"` or `2027`, because the underlying packages assign different meanings to them. Use `"2027-04-06.2028-04-05"` for one UK fiscal year or `"2027-01-01"` for an effective date. See [Reforms](reforms.md) for the shared `Simulation(policy=...)` and `Simulation(dynamic=...)` date contract. The US household route retains its country package's period-key handling. +UK household results record the capital gains realisation response the calculation applied, under `result["provenance"]["capital_gains_responses"]`: the policyengine-uk release, the reform's explicit `gov.simulation.capital_gains_responses` settings, and the assumption policyengine-uk resolved (see [The UK capital gains realisation response](microsim.md#the-uk-capital-gains-realisation-response)). `result.write(...)` includes it. + Structural reforms (new variables, formula swaps) require the `Simulation` path — see [Reforms](reforms.md). ## Year diff --git a/docs/microsim.md b/docs/microsim.md index 98dd0244..14d5c6c1 100644 --- a/docs/microsim.md +++ b/docs/microsim.md @@ -30,7 +30,7 @@ total_snap.run() total_snap.result ``` -`Simulation.ensure()` loads a cached result if one exists, or runs and caches on miss. Call `Simulation.run()` explicitly if you want to bypass the cache. +`Simulation.ensure()` loads a cached result if one exists, or runs and caches on miss. Call `Simulation.run()` explicitly if you want to bypass the cache. A UK simulation's explicit capital gains response settings are part of its cache and saved-output identity; see [The UK capital gains realisation response](#the-uk-capital-gains-realisation-response). ### Canonical US SPM settings and provenance @@ -288,6 +288,51 @@ checked. A saved UK simulation output records the data year its run anchored on; `Simulation.load()` refuses an output saved without one, and `Simulation.ensure()` runs it again. +### The UK capital gains realisation response + +policyengine-uk scales realised capital gains in response to a reform, at an +elasticity set under `gov.simulation.capital_gains_responses`. Its default is +the model's, so it can change between policyengine-uk releases. Each UK run +records the response it applied, read after every reform and modifier. + +- **Where it is kept.** It is in `simulation.release_bundle["capital_gains_responses"]`, + the run record's `results.json` and the saved output file. +- **What it holds.** + - `model`: the policyengine-uk release. Its `source` is `null` for a + release from a package index; otherwise it says where the install came + from: a version control commit, an archive's SHA-256, or a local + directory and whether it is editable. + - `request`: the caller's explicit settings under that path, in the order + they apply (policy, then dynamic), and their digest. + - `resolved`: the assumption the release reports. That is the convention, + the elasticities for main-rate gains and for gains qualifying for Business + Asset Disposal Relief, the MTR elasticity, the separate relief switch, and + whether each is the default, an override or set to zero for an MTR + elasticity. It is `null` with a release that doesn't report it. +- **Modifiers.** Changes a simulation modifier makes aren't listed under + `request`, as run records don't describe modifiers. `resolved` reflects them. + Neither the `storage_id` nor `Simulation.load()` sees them, so two + simulations with the same `id` whose modifiers set the response differently + share a saved output. Give such simulations different ids. +- **Identity.** A simulation with explicit settings adds `-cgr-` to its + `storage_id`, so its cache entry and saved output are kept apart from runs + with other settings. The digest covers each setting as given, including + whether it came from the policy or the dynamic and its dates. So the same + value set another way, or with an explicit end date instead of none, is + saved separately. +- **Reuse.** `Simulation.load()` reuses a saved UK output only when its record + shows the same policyengine-uk release, installed from the same source, and + the same settings. A branch installed from git or a local directory can + carry the version number of a published release with another default. + With policyengine-uk installed from a local directory, editable or not, no + saved UK output is reused, as its code can change without its version + changing. +- **Reruns.** Otherwise `load()` raises `StaleOutputError`, and + `Simulation.ensure()` runs the simulation again and logs the reason at INFO. + Outputs saved before outputs recorded the response are calculated once + more, so a job that relies on a warm UK cache takes as long as a cold run + the first time it runs after the upgrade. + ## Simulations A `Simulation` needs a dataset, a tax-benefit model version, and optionally a policy (reform): @@ -379,6 +424,14 @@ Pass `allow_unmanaged=True` with a custom `dataset=` to opt out of the release bundle. Explicit local paths and Hugging Face URIs remain supported in this mode. GCS dataset URIs are not supported. +A UK managed microsimulation also records, in +`sim.policyengine_bundle["capital_gains_responses"]`, the capital gains +realisation response it applies in each year of its dataset (see +[The UK capital gains realisation response](#the-uk-capital-gains-realisation-response)). +Its `reform=` goes to policyengine-uk unchanged, so no request is listed. +Settings that conflict are recorded as an error for that year, and raise when +the response is calculated. + For managed simulations, `sim.policyengine_bundle` records the actual source package, repository type, revision, verified SHA-256, and local path. diff --git a/docs/run-records.md b/docs/run-records.md index efb0a510..3f88bd26 100644 --- a/docs/run-records.md +++ b/docs/run-records.md @@ -46,7 +46,7 @@ The directory contains: | `bundle.trace.tro.jsonld` | The certified bundle TRO (model + data pins) | | `reform.json` | The reform as parameter values with effective dates | | `input.json` | Input dataset hash, dynamics, scoping, extra variables | -| `results.json` | Output dataset hash and per-entity table summaries; for US runs, the SPM receipt and the renamed stored inputs mapped onto live ones (see [Renamed inputs in stored data](microsim.md#renamed-inputs-in-stored-data)) | +| `results.json` | Output dataset hash and per-entity table summaries; for US runs, the SPM receipt and the renamed stored inputs mapped onto live ones (see [Renamed inputs in stored data](microsim.md#renamed-inputs-in-stored-data)); for UK runs, the capital gains realisation response the run applied (see [The UK capital gains realisation response](microsim.md#the-uk-capital-gains-realisation-response)) | All payload files are written with the same canonical JSON used for hashing, so the record verifies offline exactly as written. diff --git a/src/policyengine/core/__init__.py b/src/policyengine/core/__init__.py index 4f749de4..70391746 100644 --- a/src/policyengine/core/__init__.py +++ b/src/policyengine/core/__init__.py @@ -27,6 +27,7 @@ ) from .simulation import Simulation as Simulation from .tax_benefit_model import TaxBenefitModel as TaxBenefitModel +from .tax_benefit_model_version import StaleOutputError as StaleOutputError from .tax_benefit_model_version import ( TaxBenefitModelVersion as TaxBenefitModelVersion, ) diff --git a/src/policyengine/core/run_record.py b/src/policyengine/core/run_record.py index 126e49f5..d4564863 100644 --- a/src/policyengine/core/run_record.py +++ b/src/policyengine/core/run_record.py @@ -20,6 +20,7 @@ from __future__ import annotations +import copy from dataclasses import dataclass, field from pathlib import Path from typing import TYPE_CHECKING, Any, Mapping, Optional, Union @@ -202,6 +203,29 @@ def build_simulation_run_record_payloads( if legacy_spm_declarations is not None: results_payload["legacy_spm_declarations"] = dict(legacy_spm_declarations) + # The capital gains realisation response a UK run applied: the model + # resolves its default, so the record binds the assumption itself. See + # ``policyengine.tax_benefit_models.uk.capital_gains_responses``. + capital_gains_responses = ( + getattr(simulation.output_dataset, "metadata", None) or {} + ).get("capital_gains_responses") + if capital_gains_responses is not None: + from policyengine.tax_benefit_models.uk.capital_gains_responses import ( + simulation_request, + ) + + if ( + capital_gains_responses["request"]["digest"] + != simulation_request(simulation)["digest"] + ): + raise ValueError( + "Capital gains response settings changed since this output was " + "calculated; run again before writing a run record" + ) + results_payload["capital_gains_responses"] = copy.deepcopy( + capital_gains_responses + ) + return {"reform": reform, "input": input_payload, "results": results_payload} diff --git a/src/policyengine/core/simulation.py b/src/policyengine/core/simulation.py index 93762b88..071856d3 100644 --- a/src/policyengine/core/simulation.py +++ b/src/policyengine/core/simulation.py @@ -1,3 +1,4 @@ +import copy import hashlib import json import logging @@ -13,7 +14,7 @@ from .policy import Policy from .scoping_strategy import ScopingStrategy from .spm import SPMProvenance, SPMSelection -from .tax_benefit_model_version import TaxBenefitModelVersion +from .tax_benefit_model_version import StaleOutputError, TaxBenefitModelVersion logger = logging.getLogger(__name__) @@ -124,12 +125,22 @@ def spm_provenance(self) -> Optional[dict[str, Any]]: @property def storage_id(self) -> str: - """Include resolved SPM settings in cache and saved-result identity.""" + """Include resolved SPM settings, and a UK run's explicit capital + gains response settings, in cache and saved-result identity.""" + storage_id = self.id config = self.spm_config - if config is None: - return self.id - encoded = json.dumps(config, sort_keys=True, separators=(",", ":")).encode() - return f"{self.id}-spm-{hashlib.sha256(encoded).hexdigest()}" + if config is not None: + encoded = json.dumps(config, sort_keys=True, separators=(",", ":")) + storage_id += f"-spm-{hashlib.sha256(encoded.encode()).hexdigest()}" + if getattr(self.tax_benefit_model_version, "country_code", None) == "uk": + from policyengine.tax_benefit_models.uk.capital_gains_responses import ( + simulation_request, + ) + + digest = simulation_request(self)["digest"] + if digest is not None: + storage_id += f"-cgr-{digest}" + return storage_id @model_validator(mode="after") def _compile_dict_reforms(self) -> "Simulation": @@ -192,6 +203,12 @@ def ensure(self): except FileNotFoundError: self.run() self.save() + except StaleOutputError as error: + # An expected rerun, such as the first after an upgrade that + # changes what saved outputs record. + logger.info("Running simulation %s again: %s", self.id, error) + self.run() + self.save() except Exception: logger.warning( "Unexpected error loading simulation %s; falling back to run()", @@ -266,4 +283,11 @@ def release_bundle(self) -> dict[str, Any]: ).get("legacy_spm_declarations") if legacy_spm_declarations is not None: result["legacy_spm_declarations"] = dict(legacy_spm_declarations) + # The capital gains realisation response a UK run applied. See + # ``policyengine.tax_benefit_models.uk.capital_gains_responses``. + capital_gains_responses = ( + getattr(self.output_dataset, "metadata", None) or {} + ).get("capital_gains_responses") + if capital_gains_responses is not None: + result["capital_gains_responses"] = copy.deepcopy(capital_gains_responses) return result diff --git a/src/policyengine/core/tax_benefit_model_version.py b/src/policyengine/core/tax_benefit_model_version.py index d6aead49..977561c1 100644 --- a/src/policyengine/core/tax_benefit_model_version.py +++ b/src/policyengine/core/tax_benefit_model_version.py @@ -24,6 +24,16 @@ from .variable import Variable +class StaleOutputError(ValueError): + """A saved output that ``load()`` refuses to reuse, and why. + + The output was calculated under something that has since changed (a + model release, settings, or how outputs are recorded). + ``Simulation.ensure()`` runs the simulation again and logs the reason at + INFO, as an expected rerun rather than an error. + """ + + class TaxBenefitModelVersion(BaseModel): model_config = {"arbitrary_types_allowed": True} @@ -77,6 +87,8 @@ def save(self, simulation: "Simulation"): ) def load(self, simulation: "Simulation"): + """Load a saved output: ``FileNotFoundError`` when there is none, + ``StaleOutputError`` when it must not be reused.""" raise NotImplementedError( "The TaxBenefitModel class must define a method to load simulations." ) diff --git a/src/policyengine/tax_benefit_models/common/model_version.py b/src/policyengine/tax_benefit_models/common/model_version.py index 849f7478..3b72ad2c 100644 --- a/src/policyengine/tax_benefit_models/common/model_version.py +++ b/src/policyengine/tax_benefit_models/common/model_version.py @@ -32,6 +32,7 @@ from policyengine.core import ( Parameter, ParameterNode, + StaleOutputError, TaxBenefitModelVersion, Variable, ) @@ -393,6 +394,28 @@ def save(self, simulation: Simulation) -> None: "inputs were mapped when it was calculated; run again " "before saving" ) + if self.country_code == "uk": + from policyengine.tax_benefit_models.uk.capital_gains_responses import ( + RECORD_KEY, + simulation_request, + ) + + # ``PolicyEngineUKDataset.save()`` writes the record into the + # file, and ``load()`` refuses an output without it. + record = (getattr(simulation.output_dataset, "metadata", None) or {}).get( + RECORD_KEY + ) + if record is None: + raise ValueError( + "This UK output does not record the capital gains " + "realisation response it was calculated under; run again " + "before saving" + ) + if record["request"]["digest"] != simulation_request(simulation)["digest"]: + raise ValueError( + "Capital gains response settings changed since this output " + "was calculated; run again before saving" + ) simulation.output_dataset.save() if serialized_spm is not None: # Store UTF-8 JSON in a dataset rather than an attribute: the @@ -439,19 +462,21 @@ def load(self, simulation: Simulation) -> None: ) recorded_renames = read_renames_record(filepath) if raw is None: - raise ValueError( + raise StaleOutputError( "Saved US simulation has no SPM configuration or receipt" ) recorded = json.loads(raw) if recorded["config"] != simulation.spm_config: - raise ValueError("Saved US simulation uses different SPM settings") + raise StaleOutputError( + "Saved US simulation uses different SPM settings" + ) receipt = SPMProvenance.model_validate(recorded["provenance"]) # Outputs saved before stored inputs were mapped onto renamed # live inputs were calculated without them (for example with # every WIC-eligible person taking WIC up), so they are not # reused. ``Simulation.ensure()`` runs such a simulation again. if recorded_renames is None: - raise ValueError( + raise StaleOutputError( "Saved US simulation predates the mapping of renamed " "stored inputs (it has no record of them); run it again" ) @@ -462,7 +487,7 @@ def load(self, simulation: Simulation) -> None: recorded_legacy_spm, expected_profile_for_dataset(simulation.dataset) ) if legacy_problem is not None: - raise ValueError( + raise StaleOutputError( f"Saved US simulation output is out of date: {legacy_problem}; " "run it again" ) @@ -477,10 +502,30 @@ def load(self, simulation: Simulation) -> None: # observed and uprated the State Pension by CPI, so they are not # reused. ``Simulation.ensure()`` runs such a simulation again. if not _year_file_records_data_year(Path(filepath)): - raise ValueError( + raise StaleOutputError( "Saved UK simulation predates anchoring on the observed " "data year (it records none); run it again" ) + from policyengine.tax_benefit_models.uk.capital_gains_responses import ( + read_record, + saved_record_problem, + simulation_request, + ) + + # An output is reused only under the capital gains realisation + # response it was calculated with: the same policyengine-uk + # release (whose default may differ from another's) and the same + # explicit settings. + with pd.HDFStore(filepath, mode="r") as store: + response_record = read_record(store) + problem = saved_record_problem( + response_record, simulation_request(simulation) + ) + if problem is not None: + raise StaleOutputError( + f"Saved UK simulation output is out of date: {problem}; " + "run it again" + ) simulation.output_dataset = self._dataset_class( id=simulation.id, @@ -502,6 +547,12 @@ def load(self, simulation: Simulation) -> None: simulation.output_dataset.metadata[LEGACY_SPM_RECORD_KEY] = ( recorded_legacy_spm ) + if self.country_code == "uk": + from policyengine.tax_benefit_models.uk.capital_gains_responses import ( + RECORD_KEY, + ) + + simulation.output_dataset.metadata[RECORD_KEY] = response_record if os.path.exists(filepath): simulation.created_at = datetime.datetime.fromtimestamp( diff --git a/src/policyengine/tax_benefit_models/uk/capital_gains_responses.py b/src/policyengine/tax_benefit_models/uk/capital_gains_responses.py new file mode 100644 index 00000000..231dcfe0 --- /dev/null +++ b/src/policyengine/tax_benefit_models/uk/capital_gains_responses.py @@ -0,0 +1,321 @@ +"""The capital gains realisation response a UK run applies. + +policyengine-uk scales realised capital gains in response to a reform, at an +elasticity set under ``gov.simulation.capital_gains_responses``. The default +is the model's, so it can change between model releases, and a caller can +change it through a policy, a dynamic or a reform dict. Each UK run records +the response it applied, so a result says which assumption it was +calculated under and a saved result is not reused under another: + +- ``model``: the policyengine-uk release that resolved it, and where it was + installed from when that is not a package index (``source``); +- ``resolved``: the assumption itself, as policyengine-uk's + ``capital_gains_response_assumption`` reports it after every reform and + modifier (the convention, the elasticities for main-rate gains and for + gains qualifying for Business Asset Disposal Relief, the MTR elasticity, + the separate relief switch, and whether each is the default, the caller's + override, or set to zero for an MTR elasticity); ``None`` where the + installed release has no such report; +- ``request``: the caller's explicit settings under that path, in the order + they apply, and their digest. Changes a simulation modifier makes are not + listed, as run records do not describe modifiers. So two runs with the same + ``id`` that differ only in a modifier are not told apart for reuse. + +The record is stored with the output dataset, written into a saved output +file, and returned with a run's release bundle, run record, household +result (``provenance``) and managed microsimulation bundle. +""" + +from __future__ import annotations + +import datetime +import hashlib +import importlib.metadata +import json +from collections.abc import Iterable +from typing import Any, Optional + +import pandas as pd + +RECORD_KEY = "capital_gains_responses" +#: The output file key the record is written under. +FILE_KEY = "policyengine_uk_capital_gains_responses" +SCHEMA_VERSION = 1 +RESPONSE_SETTINGS = "gov.simulation.capital_gains_responses" +# Changes to how settings are listed or digested change this, so outputs +# saved under an earlier listing are not reused. +REQUEST_REVISION = 1 + + +def _date(value) -> Optional[str]: + if value is None: + return None + if isinstance(value, datetime.datetime): + value = value.date() + return value.isoformat() + + +def _value(value) -> Any: + if isinstance(value, bool): + return value + if hasattr(value, "item"): # numpy scalars + value = value.item() + if isinstance(value, (int, float)) and not isinstance(value, bool): + return float(value) + return value + + +def requested_settings(sources: Iterable[tuple[str, Any]]) -> list[list]: + """The caller's explicit response settings, in the order they apply. + + ``sources`` gives each ``(name, parameter_values)`` applied in turn (a + policy's before a dynamic's). Each setting is listed as + ``[parameter, start, end, value, source]``. + """ + settings = [] + for source, parameter_values in sources: + for parameter_value in parameter_values or []: + name = parameter_value.parameter.name + if name != RESPONSE_SETTINGS and not name.startswith( + RESPONSE_SETTINGS + "." + ): + continue + settings.append( + [ + name, + _date(parameter_value.start_date), + _date(parameter_value.end_date), + _value(parameter_value.value), + source, + ] + ) + return settings + + +def request_digest(settings: list[list]) -> Optional[str]: + """The SHA-256 of the settings, or None when there are none. + + It covers each setting as given, source and dates included, so the same + value set through a dynamic instead of a policy, or with an explicit end + date instead of none, is another request with its own saved output. The + record says how the response was asked for, not only what it came to. + """ + if not settings: + return None + payload = json.dumps( + {"revision": REQUEST_REVISION, "settings": settings}, + sort_keys=True, + separators=(",", ":"), + ) + return hashlib.sha256(payload.encode()).hexdigest() + + +def request(sources: Iterable[tuple[str, Any]]) -> dict[str, Any]: + """The explicit settings ``sources`` request, with their digest.""" + settings = requested_settings(sources) + return {"settings": settings, "digest": request_digest(settings)} + + +def simulation_request(simulation) -> dict[str, Any]: + """The explicit settings a population simulation requests.""" + return request( + ( + ("policy", getattr(simulation.policy, "parameter_values", None)), + ("dynamic", getattr(simulation.dynamic, "parameter_values", None)), + ) + ) + + +def installed_model_version() -> Optional[str]: + try: + return importlib.metadata.version("policyengine-uk") + except importlib.metadata.PackageNotFoundError: + return None + + +def source_from_direct_url(text: Optional[str]) -> Optional[dict[str, Any]]: + """An install's source, from its ``direct_url.json`` (PEP 610). + + None for an install from a package index, which has no such file. + Otherwise the kind of install and what pins its code: the commit of a + version control install, the SHA-256 of an archive, or whether a local + directory install is editable. URLs are left out, as a local path names + the machine a run was on. + """ + if not text: + return None + try: + direct_url = json.loads(text) + except ValueError: + direct_url = None + if not isinstance(direct_url, dict): + return {"kind": "unknown"} + vcs_info = direct_url.get("vcs_info") + if isinstance(vcs_info, dict): + return { + "kind": "vcs", + "vcs": vcs_info.get("vcs"), + "commit": vcs_info.get("commit_id"), + } + archive_info = direct_url.get("archive_info") + if isinstance(archive_info, dict): + hashes = archive_info.get("hashes") + sha256 = hashes.get("sha256") if isinstance(hashes, dict) else None + legacy_hash = str(archive_info.get("hash") or "") + if sha256 is None and legacy_hash.startswith("sha256="): + sha256 = legacy_hash.removeprefix("sha256=") + return {"kind": "archive", "sha256": sha256} + dir_info = direct_url.get("dir_info") + if isinstance(dir_info, dict): + return {"kind": "directory", "editable": bool(dir_info.get("editable"))} + return {"kind": "unknown"} + + +def installed_model_source() -> Optional[dict[str, Any]]: + """Where the installed policyengine-uk came from; None for an index.""" + try: + distribution = importlib.metadata.distribution("policyengine-uk") + except importlib.metadata.PackageNotFoundError: + return None + return source_from_direct_url(distribution.read_text("direct_url.json")) + + +def _source_label(source: Optional[dict[str, Any]]) -> str: + if source is None: + return "a package index" + kind = source.get("kind") + if kind == "vcs": + vcs = source.get("vcs") or "version control" + commit = source.get("commit") + return f"{vcs} commit {commit}" if commit else f"{vcs} with no commit" + if kind == "archive": + sha256 = source.get("sha256") + return f"an archive with SHA-256 {sha256}" if sha256 else "an unhashed archive" + if kind == "directory": + if source.get("editable"): + return "an editable local directory" + return "a local directory" + return "an unrecognised source" + + +def unpinned_source_problem(source: Optional[dict[str, Any]]) -> Optional[str]: + """Why an install's code can change under the same version, or None. + + A package index release, a commit and a hashed archive each pin the + code. A local directory does not: its code can change, editable or + reinstalled, while its version stays what its ``pyproject.toml`` says. + """ + if source is None: + return None + if source.get("kind") == "vcs" and source.get("commit"): + return None + if source.get("kind") == "archive" and source.get("sha256"): + return None + return ( + f"the installed policyengine-uk comes from {_source_label(source)}, " + "whose code can change without its version changing, so saved " + "outputs are not reused" + ) + + +def resolve(microsimulation, year) -> Optional[dict[str, Any]]: + """The response a policyengine-uk simulation applies in ``year``. + + None where the installed policyengine-uk does not report it. Raises + policyengine-uk's ``ValueError`` on settings that conflict. + """ + try: + from policyengine_uk.utils.capital_gains import ( + capital_gains_response_assumption, + ) + except ImportError: + return None + return capital_gains_response_assumption(microsimulation, int(year)).to_dict() + + +def build_record( + microsimulation, year, request: Optional[dict[str, Any]] +) -> dict[str, Any]: + """The record of the response a run applies, after every modifier.""" + return { + "schema_version": SCHEMA_VERSION, + "model": { + "package": "policyengine-uk", + "version": installed_model_version(), + "source": installed_model_source(), + }, + "period": str(int(year)), + "resolved": resolve(microsimulation, year), + "request": request, + } + + +def managed_records(microsimulation) -> dict[str, Any]: + """Records for each year of a managed microsimulation's dataset. + + The managed helper passes ``reform=`` to policyengine-uk unchanged, so no + request is listed. Settings that conflict are recorded as an error here, + and raise when the response is calculated, as in a direct run. + """ + years = getattr(getattr(microsimulation, "dataset", None), "years", None) or [ + microsimulation.default_calculation_period + ] + records = {} + for year in years: + try: + records[str(int(year))] = build_record(microsimulation, year, None) + except ValueError as error: + records[str(int(year))] = {"error": str(error)} + return records + + +def write_record(store: pd.HDFStore, record: dict[str, Any]) -> None: + """Write the record into an open output file.""" + store.put(FILE_KEY, pd.Series([json.dumps(record, sort_keys=True)])) + + +def read_record(store: pd.HDFStore) -> Optional[dict[str, Any]]: + """The record an open output file holds, or None.""" + if f"/{FILE_KEY}" not in store.keys(): + return None + return json.loads(store[FILE_KEY].iloc[0]) + + +def saved_record_problem( + record: Optional[dict[str, Any]], request: dict[str, Any] +) -> Optional[str]: + """Why a saved output's record rules out reusing it, or None. + + The installed policyengine-uk must be the same release from the same + source: a branch installed from git or a local directory can carry the + version number of a published release with another default. + """ + if record is None: + return ( + "it predates the record of its capital gains realisation response " + "(it has none)" + ) + if record.get("schema_version") != SCHEMA_VERSION: + return "its capital gains response record has another schema version" + installed_source = installed_model_source() + unpinned = unpinned_source_problem(installed_source) + if unpinned is not None: + return unpinned + saved_model = record.get("model") or {} + saved_version = saved_model.get("version") + if saved_version != installed_model_version(): + return ( + f"it was calculated with policyengine-uk {saved_version}, not the " + f"installed {installed_model_version()}, whose default capital " + "gains response may differ" + ) + if saved_model.get("source") != installed_source: + return ( + f"it was calculated with policyengine-uk {saved_version} from " + f"{_source_label(saved_model.get('source'))}, not from " + f"{_source_label(installed_source)} as installed, whose default " + "capital gains response may differ" + ) + if (record.get("request") or {}).get("digest") != request["digest"]: + return "it was calculated under other capital gains response settings" + return None diff --git a/src/policyengine/tax_benefit_models/uk/datasets.py b/src/policyengine/tax_benefit_models/uk/datasets.py index dbe31dbc..04937d85 100644 --- a/src/policyengine/tax_benefit_models/uk/datasets.py +++ b/src/policyengine/tax_benefit_models/uk/datasets.py @@ -1,9 +1,9 @@ from pathlib import Path -from typing import Optional +from typing import Any, Optional import pandas as pd from microdf import MicroDataFrame -from pydantic import ConfigDict +from pydantic import ConfigDict, Field from policyengine.core import Dataset, YearData from policyengine.provenance.dataset_materialization import ( @@ -74,6 +74,11 @@ class PolicyEngineUKDataset(Dataset): instead of the triple lock. """ + metadata: dict[str, Any] = Field(default_factory=dict) + """Records about how an output was calculated, such as the capital gains + realisation response a run applied (see ``capital_gains_responses``). + ``save()`` writes them into the file and ``load()`` reads them back.""" + def model_post_init(self, __context): """Called after Pydantic initialization. @@ -140,20 +145,35 @@ def save(self) -> None: if not filepath.parent.exists(): filepath.parent.mkdir(parents=True, exist_ok=True) + from policyengine.tax_benefit_models.uk.capital_gains_responses import ( + RECORD_KEY, + write_record, + ) + with pd.HDFStore(filepath, mode="w") as store: _put_year_data(store, self.data) if self.data_year_data is not None: _put_year_data(store, self.data_year_data, DATA_YEAR_TABLE_PREFIX) + if RECORD_KEY in self.metadata: + write_record(store, self.metadata[RECORD_KEY]) # Written last: its presence marks a complete file. if self.data_year is not None: store.put(DATA_YEAR_KEY, pd.Series([int(self.data_year)])) def load(self) -> None: """Load dataset from HDF5 file into this instance.""" + from policyengine.tax_benefit_models.uk.capital_gains_responses import ( + RECORD_KEY, + read_record, + ) + filepath = self.filepath with pd.HDFStore(filepath, mode="r") as store: keys = set(store.keys()) self.data = _read_year_data(store) + record = read_record(store) + if record is not None: + self.metadata[RECORD_KEY] = record self.data_year = ( int(store[DATA_YEAR_KEY].iloc[0]) if f"/{DATA_YEAR_KEY}" in keys diff --git a/src/policyengine/tax_benefit_models/uk/household.py b/src/policyengine/tax_benefit_models/uk/household.py index 2c5f8d0c..776c3c32 100644 --- a/src/policyengine/tax_benefit_models/uk/household.py +++ b/src/policyengine/tax_benefit_models/uk/household.py @@ -186,11 +186,10 @@ def calculate_household( names=extra_variables or [], ) output_columns = _default_output_columns(extra_by_entity) - reform_dict = reform_dict_from_parameter_values( - _reform_dict_to_parameter_values( - reform or {}, year=year, model_version=uk_latest - ) + reform_values = _reform_dict_to_parameter_values( + reform or {}, year=year, model_version=uk_latest ) + reform_dict = reform_dict_from_parameter_values(reform_values) normalized_axes = normalize_axes(axes=axes, year=year, model_version=uk_latest) axes_active = normalized_axes is not None @@ -205,6 +204,15 @@ def calculate_household( reform=reform_dict, ) + from policyengine.tax_benefit_models.uk import capital_gains_responses + + # The capital gains realisation response the calculation applies. + response_record = capital_gains_responses.build_record( + simulation, + year, + capital_gains_responses.request((("reform", reform_values),)), + ) + result = HouseholdResult() for entity, columns in output_columns.items(): raw = { @@ -237,4 +245,5 @@ def calculate_household( for variable in columns } ) + result["provenance"] = {capital_gains_responses.RECORD_KEY: response_record} return result diff --git a/src/policyengine/tax_benefit_models/uk/model.py b/src/policyengine/tax_benefit_models/uk/model.py index d0d1ed65..9f5bf853 100644 --- a/src/policyengine/tax_benefit_models/uk/model.py +++ b/src/policyengine/tax_benefit_models/uk/model.py @@ -172,6 +172,7 @@ def _dataset_class(self): def run(self, simulation: "Simulation") -> "Simulation": from policyengine_uk import Microsimulation + from policyengine.tax_benefit_models.uk import capital_gains_responses from policyengine.utils.parametric_reforms import ( build_reform_dict, merge_reform_dicts, @@ -253,6 +254,14 @@ def run(self, simulation: "Simulation") -> "Simulation": if dynamic_modifier is not None: dynamic_modifier(microsim) + # The capital gains realisation response this run applies, read once + # every reform and modifier has run. + response_record = capital_gains_responses.build_record( + microsim, + simulation.dataset.year, + capital_gains_responses.simulation_request(simulation), + ) + data = { "person": pd.DataFrame(), "benunit": pd.DataFrame(), @@ -297,6 +306,7 @@ def run(self, simulation: "Simulation") -> "Simulation": benunit=data["benunit"], household=data["household"], ), + metadata={capital_gains_responses.RECORD_KEY: response_record}, ) @@ -467,6 +477,12 @@ def managed_microsimulation( source.bundle_dataset, ) ) + from policyengine.tax_benefit_models.uk import capital_gains_responses + + # The capital gains realisation response, for each year of the dataset. + microsim.policyengine_bundle[capital_gains_responses.RECORD_KEY] = ( + capital_gains_responses.managed_records(microsim) + ) return microsim diff --git a/tests/test_household_calculator_snapshot.py b/tests/test_household_calculator_snapshot.py index 170f7b4c..e2589f8e 100644 --- a/tests/test_household_calculator_snapshot.py +++ b/tests/test_household_calculator_snapshot.py @@ -213,7 +213,11 @@ def test_uk_household_snapshot(case_name: str) -> None: kwargs = UK_CASES[case_name] result = pe.uk.calculate_household(**kwargs) out: dict[str, float] = {} - _flatten("", result.to_dict(), out) + # Provenance (the capital gains response record) is tested in + # test_uk_cgt_responses.py; keep this snapshot to the numeric output. + values = result.to_dict() + values.pop("provenance", None) + _flatten("", values, out) _check_snapshot(case_name, out) diff --git a/tests/test_uk_cgt_response_records.py b/tests/test_uk_cgt_response_records.py new file mode 100644 index 00000000..1f2b0e67 --- /dev/null +++ b/tests/test_uk_cgt_response_records.py @@ -0,0 +1,231 @@ +"""The record of the capital gains realisation response a UK run applies. + +These tests check the record's parts without running the model: which +settings a request lists and in what order, the digest that keys a run's +cache and saved output, and when a saved output's record rules out reusing +it. ``test_uk_cgt_responses.py`` runs the model through each entry point. +""" + +import datetime + +import pytest + +from policyengine.tax_benefit_models.uk import capital_gains_responses as records + +ELASTICITY = "gov.simulation.capital_gains_responses.elasticity" +MTR_ELASTICITY = "gov.simulation.capital_gains_responses.mtr_elasticity" +SEPARATE = "gov.simulation.capital_gains_responses.separate_badr_elasticity" + + +class _Parameter: + def __init__(self, name): + self.name = name + + +class _Value: + def __init__(self, name, value, start, end=None): + self.parameter = _Parameter(name) + self.value = value + self.start_date = start + self.end_date = end + + +def _values(*settings): + return [_Value(*setting) for setting in settings] + + +@pytest.fixture(autouse=True) +def _index_install(monkeypatch): + """Run as if policyengine-uk came from a package index, however it is + installed here. Tests of other sources set their own.""" + monkeypatch.setattr(records, "installed_model_source", lambda: None) + + +START = datetime.datetime(2027, 1, 1) +END = datetime.date(2027, 12, 31) + + +def test_a_request_lists_response_settings_in_the_order_they_apply(): + policy = _values( + ("gov.hmrc.cgt.higher_rate", 0.45, START), + (ELASTICITY, 0.5, START, END), + ) + dynamic = _values((ELASTICITY, 2, START), (SEPARATE, True, START)) + + request = records.request((("policy", policy), ("dynamic", dynamic))) + + assert request["settings"] == [ + [ELASTICITY, "2027-01-01", "2027-12-31", 0.5, "policy"], + [ELASTICITY, "2027-01-01", None, 2.0, "dynamic"], + [SEPARATE, "2027-01-01", None, True, "dynamic"], + ] + assert len(request["digest"]) == 64 + + +def test_no_settings_means_no_digest(): + request = records.request( + (("policy", _values(("gov.hmrc.cgt.higher_rate", 0.45, START))),) + ) + + assert request == {"settings": [], "digest": None} + + +def test_the_digest_tells_requests_apart(): + def digest(*settings, source="dynamic"): + return records.request(((source, _values(*settings)),))["digest"] + + central = digest((ELASTICITY, 1.0, START)) + assert digest((ELASTICITY, 1, START)) == central + assert digest((ELASTICITY, 0.5, START)) != central + assert digest((ELASTICITY, 1.0, START), source="policy") != central + assert digest((MTR_ELASTICITY, -0.7, START)) != digest((ELASTICITY, -0.7, START)) + # The order the settings apply in decides which one wins. + assert digest((ELASTICITY, 0.5, START), (ELASTICITY, 2.0, START)) != digest( + (ELASTICITY, 2.0, START), (ELASTICITY, 0.5, START) + ) + + +def _record(**changes): + record = { + "schema_version": records.SCHEMA_VERSION, + "model": { + "package": "policyengine-uk", + "version": records.installed_model_version(), + "source": None, + }, + "period": "2027", + "resolved": None, + "request": {"settings": [], "digest": None}, + } + record.update(changes) + return record + + +@pytest.mark.parametrize( + "record, problem", + [ + (None, "predates the record"), + (_record(schema_version=0), "another schema version"), + ( + _record( + model={"package": "policyengine-uk", "version": "0.0.1", "source": None} + ), + "calculated with policyengine-uk 0.0.1", + ), + ( + _record(request={"settings": [], "digest": "0" * 64}), + "other capital gains response settings", + ), + ], +) +def test_a_saved_record_rules_out_reuse(record, problem): + assert problem in records.saved_record_problem( + record, {"settings": [], "digest": None} + ) + + +def test_a_matching_saved_record_allows_reuse(): + assert records.saved_record_problem(_record(), {"digest": None}) is None + + +def test_a_record_carries_the_resolved_assumption(monkeypatch): + resolved = {"convention": "retention", "elasticity": 1.0} + monkeypatch.setattr(records, "resolve", lambda simulation, year: resolved) + request = {"settings": [], "digest": None} + + record = records.build_record(object(), 2027, request) + + assert record == _record(resolved=resolved, request=request) + + +GIT = {"kind": "vcs", "vcs": "git", "commit": "c6c77ef3"} + + +@pytest.mark.parametrize( + "direct_url, source", + [ + (None, None), + ( + '{"url": "https://github.com/PolicyEngine/policyengine-uk", ' + '"vcs_info": {"vcs": "git", "commit_id": "c6c77ef3", ' + '"requested_revision": "cgt-retention-default-2396"}}', + GIT, + ), + ( + '{"url": "file:///home/user/policyengine-uk", ' + '"dir_info": {"editable": true}}', + {"kind": "directory", "editable": True}, + ), + ( + '{"url": "file:///home/user/policyengine-uk", "dir_info": {}}', + {"kind": "directory", "editable": False}, + ), + ( + '{"url": "https://example.org/policyengine_uk-2.131.0.whl", ' + '"archive_info": {"hashes": {"sha256": "ab12"}}}', + {"kind": "archive", "sha256": "ab12"}, + ), + ( + '{"url": "https://example.org/policyengine_uk-2.131.0.tar.gz", ' + '"archive_info": {"hash": "sha256=ab12"}}', + {"kind": "archive", "sha256": "ab12"}, + ), + ( + '{"url": "https://example.org/policyengine_uk-2.131.0.tar.gz", ' + '"archive_info": {}}', + {"kind": "archive", "sha256": None}, + ), + ("not json", {"kind": "unknown"}), + ], +) +def test_the_install_source_is_read_from_direct_url(direct_url, source): + # URLs are left out: a local path names the machine a run was on. + assert records.source_from_direct_url(direct_url) == source + + +@pytest.mark.parametrize( + "saved, installed, problem", + [ + (None, None, None), + (GIT, GIT, None), + # A branch can report the version of a published release. + (None, GIT, "from a package index, not from git commit c6c77ef3"), + (GIT, None, "from git commit c6c77ef3, not from a package index"), + (GIT, {**GIT, "commit": "d4e5f6a7"}, "not from git commit d4e5f6a7"), + (None, {"kind": "directory", "editable": True}, "an editable local"), + ( + {"kind": "directory", "editable": False}, + {"kind": "directory", "editable": False}, + "a local directory, whose code can change", + ), + (None, {"kind": "archive", "sha256": None}, "an unhashed archive"), + (None, {"kind": "unknown"}, "an unrecognised source"), + ], +) +def test_a_saved_output_is_reused_only_from_the_same_pinned_install( + monkeypatch, saved, installed, problem +): + monkeypatch.setattr(records, "installed_model_source", lambda: installed) + record = _record( + model={ + "package": "policyengine-uk", + "version": records.installed_model_version(), + "source": saved, + } + ) + + found = records.saved_record_problem(record, {"digest": None}) + + if problem is None: + assert found is None + else: + assert problem in found + + +def test_a_record_carries_the_install_source(monkeypatch): + monkeypatch.setattr(records, "installed_model_source", lambda: GIT) + monkeypatch.setattr(records, "resolve", lambda simulation, year: None) + + record = records.build_record(object(), 2027, None) + + assert record["model"]["source"] == GIT diff --git a/tests/test_uk_cgt_responses.py b/tests/test_uk_cgt_responses.py new file mode 100644 index 00000000..76a3288e --- /dev/null +++ b/tests/test_uk_cgt_responses.py @@ -0,0 +1,432 @@ +"""The capital gains realisation response through policyengine.py's UK routes. + +policyengine-uk scales realised gains in response to a reform, at an +elasticity under ``gov.simulation.capital_gains_responses`` whose default is +the model's (policyengine-uk#2396). These tests run a tiny in-memory 2027 +dataset through the population route with explicit settings, so they hold +whatever the installed default is, and check that: + +- explicit settings reach policyengine-uk unchanged: each retention-rate + elasticity scales gains by the retention ratio to its power, a static + setting holds them, the OBR comparison splits relief gains, and an MTR + elasticity is not read as a retention-rate one; +- the dynamic's settings win over the policy's, on both of the runner's + paths, and conflicting settings raise; +- each run records the response it applied, and the record survives saving, + loading, run records and JSON, while a saved output is not reused under + other settings or another model release or install, and + ``Simulation.ensure()`` runs it again as an expected rerun; +- the household route records the response too. +""" + +from __future__ import annotations + +import json +import logging +import math +import shutil +import uuid + +import pandas as pd +import pytest +from microdf import MicroDataFrame + +pytest.importorskip("policyengine_uk") + +import policyengine as pe # noqa: E402 +from policyengine.core import Policy, Simulation, StaleOutputError # noqa: E402 +from policyengine.core.run_record import ( # noqa: E402 + build_simulation_run_record_payloads, +) +from policyengine.tax_benefit_models.common.model_version import ( # noqa: E402 + output_dataset_filepath, +) +from policyengine.tax_benefit_models.uk import ( # noqa: E402 + capital_gains_responses as records, +) +from policyengine.tax_benefit_models.uk.datasets import ( # noqa: E402 + PolicyEngineUKDataset, + UKYearData, +) + +YEAR = 2027 +RESPONSES = "gov.simulation.capital_gains_responses" +FROM_2027 = "2027-04-06.2100-12-31" +# Higher and additional CGT rates of 45%, so the marginal rate on main-rate +# gains above the basic rate band rises from 24% to 45%. +RATE_RISE = { + "gov.hmrc.cgt.higher_rate": {FROM_2027: 0.45}, + "gov.hmrc.cgt.additional_rate": {FROM_2027: 0.45}, +} +RATIO = (1 - 0.45) / (1 - 0.24) +# 20% of 50% gains qualify for the relief at 18%: the share-weighted rate +# rises from 0.4 x 18% + 0.6 x 24% to 0.4 x 18% + 0.6 x 45%. +MIXED_RATIO = (1 - 0.342) / (1 - 0.216) +GAINS_VARIABLES = { + "person": [ + "capital_gains", + "capital_gains_before_response", + "capital_gains_behavioural_response", + "capital_gains_badr_behavioural_response", + ] +} + + +@pytest.fixture(autouse=True) +def _index_install(monkeypatch): + """Run as if policyengine-uk came from a package index, however it is + installed here: a local directory install refuses every saved output. + Tests of other sources set their own.""" + monkeypatch.setattr(records, "installed_model_source", lambda: None) + + +def settings(**values) -> dict: + return {f"{RESPONSES}.{name}": {FROM_2027: value} for name, value in values.items()} + + +def dataset(path=None) -> PolicyEngineUKDataset: + """Four people in their own households, observed in 2027: higher-rate + gainers with and without gains qualifying for the relief, a basic-rate + gainer, and someone without gains. Saved to ``path`` when given, as run + records and saved outputs need an input file.""" + ids = [1, 2, 3, 4] + person = pd.DataFrame( + { + "person_id": ids, + "person_benunit_id": ids, + "person_household_id": ids, + "age": [45, 45, 45, 45], + "employment_income": [80_000.0, 80_000.0, 20_000.0, 30_000.0], + "capital_gains": [50_000.0, 50_000.0, 10_000.0, 0.0], + "capital_gains_badr": [0.0, 20_000.0, 0.0, 0.0], + "person_weight": [1.0, 1.0, 1.0, 1.0], + } + ) + benunit = pd.DataFrame({"benunit_id": ids, "benunit_weight": [1.0] * 4}) + household = pd.DataFrame( + { + "household_id": ids, + "household_weight": [1.0] * 4, + "region": ["LONDON"] * 4, + "tenure_type": ["OWNED_OUTRIGHT"] * 4, + "council_tax": [0.0] * 4, + "rent": [0.0] * 4, + } + ) + built = PolicyEngineUKDataset( + name="cgt-responses", + description="capital gains responses", + year=YEAR, + data=UKYearData( + person=MicroDataFrame(person, weights="person_weight"), + benunit=MicroDataFrame(benunit, weights="benunit_weight"), + household=MicroDataFrame(household, weights="household_weight"), + ), + ) + if path is not None: + built.filepath = str(path) + built.save() + return built + + +def simulation(policy=None, dynamic=None, path=None, **kwargs) -> Simulation: + return Simulation( + dataset=dataset(path), + tax_benefit_model_version=pe.uk.model, + policy=policy, + dynamic=dynamic, + extra_variables=GAINS_VARIABLES, + **kwargs, + ) + + +def run(policy=None, dynamic=None, **kwargs) -> Simulation: + sim = simulation(policy, dynamic, **kwargs) + sim.run() + return sim + + +def person(sim, variable): + return list(sim.output_dataset.data.person[variable]) + + +def factors(sim) -> list: + before = person(sim, "capital_gains_before_response") + response = person(sim, "capital_gains_behavioural_response") + return [1 + r / b if b else 1.0 for r, b in zip(response, before)] + + +# Explicit settings reach the model unchanged. + + +@pytest.mark.parametrize("elasticity", [0.5, 1.0, 2.0]) +def test_a_retention_rate_elasticity_scales_gains_by_the_ratio(elasticity): + sim = run(policy=RATE_RISE, dynamic=settings(elasticity=elasticity)) + + assert factors(sim)[0] == pytest.approx(RATIO**elasticity, abs=1e-5) + # The basic-rate gainer's rate doesn't move, and the last has no gains. + assert factors(sim)[2:] == [1.0, 1.0] + + +def test_static_settings_hold_gains(): + static = settings(elasticity=0, mtr_elasticity=0, separate_badr_elasticity=False) + sim = run(policy=RATE_RISE, dynamic=static) + + assert person(sim, "capital_gains_behavioural_response") == [0.0] * 4 + + +def test_the_obr_comparison_splits_relief_gains(): + sim = run( + policy=RATE_RISE, + dynamic=settings(elasticity=3.6, separate_badr_elasticity=True), + ) + relief = 20_000 + relief_response = person(sim, "capital_gains_badr_behavioural_response")[1] + main_response = ( + person(sim, "capital_gains_behavioural_response")[1] - relief_response + ) + + assert 1 + relief_response / relief == pytest.approx(MIXED_RATIO**1.4, abs=1e-4) + assert 1 + main_response / 30_000 == pytest.approx(MIXED_RATIO**3.6, abs=1e-4) + + +def test_an_mtr_elasticity_is_not_a_retention_rate_elasticity(): + sim = run( + policy=RATE_RISE, + dynamic=settings(elasticity=0, mtr_elasticity=-0.7), + ) + + # (45% / 24%) ** -0.7, not the retention-rate 0.55 / 0.76. + assert factors(sim)[0] == pytest.approx( + math.exp(-0.7 * math.log(0.45 / 0.24)), abs=1e-5 + ) + + +def test_conflicting_settings_raise(): + with pytest.raises(ValueError, match="mtr_elasticity"): + run(policy=RATE_RISE, dynamic=settings(elasticity=0.5, mtr_elasticity=-0.7)) + + +def test_settings_split_between_policy_and_dynamic_conflict_too(): + with pytest.raises(ValueError, match="mtr_elasticity"): + run( + policy={**RATE_RISE, **settings(elasticity=0.5)}, + dynamic=settings(mtr_elasticity=-0.7), + ) + + +# Precedence. + + +def test_the_dynamic_wins_over_the_policy(): + sim = run( + policy={**RATE_RISE, **settings(elasticity=0.5)}, + dynamic=settings(elasticity=2.0), + ) + + assert factors(sim)[0] == pytest.approx(RATIO**2.0, abs=1e-5) + assert [ + setting[-1] + for setting in sim.output_dataset.metadata[records.RECORD_KEY]["request"][ + "settings" + ] + ] == ["policy", "dynamic"] + + +def test_the_dynamic_wins_after_a_policy_modifier(): + """With a policy modifier the dynamic's values are written after it, to + the annualised parameters; a value from 1 January covers the year.""" + rate_rise = Policy( + name="CGT rate rise", + simulation_modifier=lambda microsim: microsim, + ) + sim = run( + policy=rate_rise + simulation(policy=RATE_RISE).policy, + dynamic={f"{RESPONSES}.elasticity": {"2027-01-01.2100-12-31": 2.0}}, + ) + + assert factors(sim)[0] == pytest.approx(RATIO**2.0, abs=1e-5) + + +# The record of the response. + + +def test_a_run_records_the_response_it_applied(): + sim = run(policy=RATE_RISE, dynamic=settings(elasticity=1.0)) + record = sim.output_dataset.metadata[records.RECORD_KEY] + + assert record["model"] == { + "package": "policyengine-uk", + "version": records.installed_model_version(), + "source": None, + } + assert record["period"] == "2027" + assert record["request"]["settings"] == [ + [f"{RESPONSES}.elasticity", "2027-04-06", "2100-12-31", 1.0, "dynamic"] + ] + if record["resolved"] is not None: + assert record["resolved"]["convention"] == "retention" + assert record["resolved"]["elasticity"] == 1.0 + assert sim.release_bundle[records.RECORD_KEY] == record + assert json.loads(json.dumps(record)) == record + + +def test_storage_ids_tell_settings_apart(): + plain = simulation(id="cgt-run") + central = simulation(id="cgt-run", dynamic=settings(elasticity=1.0)) + lower = simulation(id="cgt-run", dynamic=settings(elasticity=0.5)) + + assert plain.storage_id == "cgt-run" + assert central.storage_id.startswith("cgt-run-cgr-") + assert len({plain.storage_id, central.storage_id, lower.storage_id}) == 3 + + +def saved(tmp_path, **kwargs) -> Simulation: + """A run of an input saved in ``tmp_path``, its output saved beside it.""" + sim = run(path=tmp_path / "input.h5", **kwargs) + sim.save() + return sim + + +def test_the_record_survives_saving_and_loading(tmp_path): + original = saved( + tmp_path, id="cgt-run", policy=RATE_RISE, dynamic=settings(elasticity=0.5) + ) + reloaded = simulation( + id="cgt-run", policy=RATE_RISE, dynamic=settings(elasticity=0.5) + ) + reloaded.dataset.filepath = original.dataset.filepath + reloaded.load() + + assert ( + reloaded.output_dataset.metadata[records.RECORD_KEY] + == original.output_dataset.metadata[records.RECORD_KEY] + ) + assert ( + reloaded.release_bundle[records.RECORD_KEY] + == (original.release_bundle[records.RECORD_KEY]) + ) + + +def test_a_saved_output_is_not_reused_under_other_settings(tmp_path): + original = saved( + tmp_path, id="cgt-run", policy=RATE_RISE, dynamic=settings(elasticity=0.5) + ) + other = simulation(id="cgt-run", policy=RATE_RISE, dynamic=settings(elasticity=2)) + other.dataset.filepath = original.dataset.filepath + # Another request saves elsewhere; copied into its place, the output is + # still refused. + shutil.copy(original.output_dataset.filepath, str(output_dataset_filepath(other))) + + with pytest.raises(ValueError, match="other capital gains response settings"): + other.load() + + +def test_a_saved_output_is_not_reused_under_another_model_release( + tmp_path, monkeypatch +): + original = saved(tmp_path, id="cgt-run", policy=RATE_RISE) + monkeypatch.setattr(records, "installed_model_version", lambda: "0.0.1") + reloaded = simulation(id="cgt-run", policy=RATE_RISE) + reloaded.dataset.filepath = original.dataset.filepath + + with pytest.raises(ValueError, match="calculated with policyengine-uk"): + reloaded.load() + + +def test_a_saved_output_is_not_reused_from_another_install(tmp_path, monkeypatch): + # A branch installed from git can report the version of a published + # release whose default differs. + original = saved(tmp_path, id="cgt-run", policy=RATE_RISE) + monkeypatch.setattr( + records, + "installed_model_source", + lambda: {"kind": "vcs", "vcs": "git", "commit": "c6c77ef3"}, + ) + reloaded = simulation(id="cgt-run", policy=RATE_RISE) + reloaded.dataset.filepath = original.dataset.filepath + + with pytest.raises( + StaleOutputError, match="from a package index, not from git commit c6c77ef3" + ): + reloaded.load() + + +def test_ensure_runs_an_out_of_date_output_again_without_a_warning( + tmp_path, monkeypatch, caplog +): + # An id of its own, so no earlier test's in-memory cache entry answers. + run_id = f"cgt-rerun-{uuid.uuid4().hex}" + original = saved(tmp_path, id=run_id, policy=RATE_RISE) + monkeypatch.setattr(records, "installed_model_version", lambda: "0.0.1") + reloaded = simulation(id=run_id, policy=RATE_RISE) + reloaded.dataset.filepath = original.dataset.filepath + + with caplog.at_level(logging.INFO, logger="policyengine.core.simulation"): + reloaded.ensure() + + assert reloaded.output_dataset.metadata[records.RECORD_KEY]["model"] == { + "package": "policyengine-uk", + "version": "0.0.1", + "source": None, + } + logged = [ + (record.levelno, record.getMessage()) + for record in caplog.records + if record.name == "policyengine.core.simulation" + ] + assert [level for level, _ in logged] == [logging.INFO] + assert "calculated with policyengine-uk" in logged[0][1] + + +def test_a_saved_output_without_a_record_is_not_reused(tmp_path): + original = saved(tmp_path, id="cgt-run", policy=RATE_RISE) + output = original.output_dataset + output.metadata.pop(records.RECORD_KEY) + output.save() + reloaded = simulation(id="cgt-run", policy=RATE_RISE) + reloaded.dataset.filepath = original.dataset.filepath + + with pytest.raises(ValueError, match="predates the record"): + reloaded.load() + + +def test_changed_settings_are_not_saved_or_recorded(tmp_path): + sim = saved(tmp_path, policy=RATE_RISE, dynamic=settings(elasticity=0.5)) + sim.dynamic = simulation(dynamic=settings(elasticity=2.0)).dynamic + + with pytest.raises(ValueError, match="changed since this output"): + sim.save() + with pytest.raises(ValueError, match="changed since this output"): + build_simulation_run_record_payloads(sim) + + +def test_a_run_record_binds_the_response(tmp_path): + sim = saved(tmp_path, policy=RATE_RISE, dynamic=settings(elasticity=1.0)) + payloads = build_simulation_run_record_payloads(sim) + + assert ( + payloads["results"][records.RECORD_KEY] + == (sim.output_dataset.metadata[records.RECORD_KEY]) + ) + + +# The household route. + + +def test_the_household_route_records_the_response(): + result = pe.uk.calculate_household( + people=[{"age": 45, "employment_income": 80_000, "capital_gains": 50_000}], + year=YEAR, + reform={**RATE_RISE, **settings(elasticity=1.0)}, + extra_variables=["capital_gains"], + ) + record = result["provenance"][records.RECORD_KEY] + + assert result.person[0]["capital_gains"] == pytest.approx(50_000 * RATIO, abs=0.5) + assert record["request"]["settings"] == [ + [f"{RESPONSES}.elasticity", "2027-04-06", "2100-12-31", 1.0, "reform"] + ] + assert json.loads(json.dumps(result.to_dict()))["provenance"] == { + records.RECORD_KEY: record + } diff --git a/tests/test_uk_dated_reforms.py b/tests/test_uk_dated_reforms.py index cfa27ab7..ddedba47 100644 --- a/tests/test_uk_dated_reforms.py +++ b/tests/test_uk_dated_reforms.py @@ -45,6 +45,11 @@ def calculate(self, variable, period, map_to): "_default_output_columns", lambda extra: {"person": ["income_tax"]}, ) + # The stub has no tax-benefit system to resolve a response from. + monkeypatch.setattr( + "policyengine.tax_benefit_models.uk.capital_gains_responses.resolve", + lambda simulation, year: None, + ) pe.uk.calculate_household( people=[{"age": 40, "employment_income": 30_000}], @@ -153,6 +158,10 @@ def calculate(self, *args, **kwargs): monkeypatch.setattr("policyengine_uk.Microsimulation", RecordingMicrosimulation) monkeypatch.setattr(uk_model, "_policyengine_uk_input", lambda dataset: None) + monkeypatch.setattr( + "policyengine.tax_benefit_models.uk.capital_gains_responses.resolve", + lambda microsimulation, year: events.append("resolve capital gains response"), + ) monkeypatch.setattr( parametric_reforms, "simulation_modifier_from_parameter_values", @@ -302,4 +311,6 @@ def make(cls, kind, value, name): make(Dynamic, dynamic_kind, 0.30, "dynamic modifier"), events, ) - assert events == expected + # The capital gains response the run records is read after every + # modifier and late value has run. + assert events == expected + ["resolve capital gains response"] diff --git a/tests/test_uk_year_file_data_year.py b/tests/test_uk_year_file_data_year.py index 816c11cc..5ecae60f 100644 --- a/tests/test_uk_year_file_data_year.py +++ b/tests/test_uk_year_file_data_year.py @@ -65,6 +65,16 @@ simulation_modifier_from_parameter_values, ) + +@pytest.fixture(autouse=True) +def _index_install(monkeypatch): + """Run as if policyengine-uk came from a package index, however it is + installed here: a local directory install refuses every saved output.""" + from policyengine.tax_benefit_models.uk import capital_gains_responses + + monkeypatch.setattr(capital_gains_responses, "installed_model_source", lambda: None) + + DATA_YEAR = 2024 STATE_PENSION = [ "basic_state_pension",