Record the capital gains realisation response each UK run applies

[juaristi22]

Part of #602.

PR set: CGT retention-rate default (PolicyEngine/policyengine-uk#2396, #602)

Step	PR	What	Depends on
0	PolicyEngine/policyengine-uk#2277	Measurement branches (Max)	—
1	#620	Budget contract: hold CGT realisations fixed in the rate-blending case; add a response case	—
2	PolicyEngine/policyengine-uk#2419	Retention-rate elasticity of 1.0 as the model default	step 0 merged; step 1 merged and released
3	this PR	Record the response each UK run applies	—
4	—	policyengine-uk release, and its compatibility claim for the certified policyengine-uk-data release (1.58.1)	step 2
5	not opened yet	Adopt the release: certify, pin, default tests, docs	steps 1, 3, 4
6	not opened yet	policyengine-app-v2 methodology page	step 5 released

Depends on: nothing. It works on the current pin.
Blocks: step 5.
Tracking checklist: checklist on #602

Why

policyengine-uk resolves the capital gains realisation response from a default that changes between releases: #2396 makes it a retention-rate elasticity of 1.0, where it was 0. A UK result should say which assumption it was calculated under, and a saved result shouldn't be reused under another (#602, item 5).

What

The record. policyengine.tax_benefit_models.uk.capital_gains_responses builds it after every reform and modifier has run. It holds:

• the policyengine-uk release, and where it was installed from when that isn't a package index (source: a version control commit, an archive's SHA-256, or a local directory and whether it is editable; never a URL);
• resolved: the assumption as the release reports it through capital_gains_response_assumption, added in Make a retention-rate elasticity of 1 the default CGT realisation response policyengine-uk#2419. That covers the convention, the elasticities for main-rate and relief gains, the MTR elasticity, the separate relief switch, and whether each is the default, an override or set to zero for an MTR elasticity. It is null on the current pin, which has no such report.
• request: the caller's explicit gov.simulation.capital_gains_responses settings in the order they apply (policy, then dynamic), and their SHA-256 digest.

Changes a simulation modifier makes aren't listed, as run records don't describe modifiers. So two simulations with the same id that differ only in a modifier share a saved output; the docs say to give them different ids.

Where it goes:

Route	Location
Population	output_dataset.metadata, release_bundle, run-record results, and the saved output file (one key in the UK HDF5 output)
Household	result["provenance"]["capital_gains_responses"], as the US route records SPM provenance
Managed	policyengine_bundle["capital_gains_responses"], one per dataset year; settings that conflict are recorded as an error there and raise when the response is calculated

Identity:

• Cache and file names. A UK simulation with explicit response settings adds -cgr-<digest> to its storage_id, as US simulations add their SPM settings. The in-process cache and saved-output file names then tell settings apart. The digest covers each setting as given, including its source (policy or dynamic) and dates, so the same value set another way is saved separately. That is deliberate: the record says how the response was asked for.
• Reuse. load() reuses a saved UK output only when its record shows the same policyengine-uk release from the same install source, and the same settings. A branch installed from git can report the version of a published release with another default. Under a local directory install, editable or not, no saved UK output is reused, as its code can change while its version stays the same. save() and run records refuse an output whose settings changed after it was calculated.
• Quiet reruns. load() raises StaleOutputError (a ValueError, new in policyengine.core) for every saved output it refuses, UK and US. ensure() logs the reason at INFO and runs the simulation again, instead of logging a warning with a traceback.
• One rerun. Every saved UK output from before this PR is calculated once more, so a job that relies on a warm UK cache takes as long as a cold run the first time after upgrading.

Order with #608. #608 (draft) restructures UK saving and loading, adds the same metadata field, and also refuses saved outputs without its stamp. This PR is expected first: #608 sits behind #596, #594, #593 and #601, and step 5 depends on this one. Whichever lands second moves its lines into the other's structure. If both land in the same policyengine.py release, saved UK outputs rerun once, not twice. The same note is on #608.

Checks

• tests/test_uk_cgt_response_records.py (no engine) covers:
  • which settings a request lists and in what order;
  • the digest;
  • when a saved record rules out reuse, including the install source read from direct_url.json for each kind of install;
  • the resolved assumption reaching the record.
• tests/test_uk_cgt_responses.py runs a four-person in-memory 2027 dataset with explicit settings, so it holds whatever the installed default is. It covers:
  • the retention ratio to the power of 0.5, 1 and 2;
  • static settings;
  • the OBR split of relief gains;
  • an MTR elasticity of −0.7 giving 0.6440, not 0.7237;
  • conflicts, including one split between policy and dynamic;
  • the dynamic winning on both of the runner's paths;
  • the record in the release bundle, after saving and loading, and in run records;
  • refusing reuse under other settings, another release, another install source or no record;
  • ensure() rerunning an out-of-date output with one INFO record and no warning;
  • the household route's provenance.
• The UK household snapshot test drops provenance before comparing, as the US one does.
• The population route's ordering matrix (test_uk_dated_reforms.py) now checks every case reads the response after the last modifier and late value. Tests that stand in for policyengine-uk's simulations replace the resolver.
• With Make a retention-rate elasticity of 1 the default CGT realisation response policyengine-uk#2419 installed locally, these tests also pass and resolved carries the assumption.
• Main is merged in, so the checks ran on policyengine-uk 2.130.1. The record, response, dated-reform, snapshot, data-year, run-record and cache tests give 117 passed.

🤖 Generated with Claude Code

[vahid-ahmadi]

@juaristi22

Good to merge once CI is green, ideally with point 1 fixed first. Points 2-4 are docs or coordination.

The design holds up. Explicit settings go into storage_id only when there are some, so default runs keep their file names and the change doesn't break anything for them. Each distinct set of settings adds one cache entry and one file, which is bounded by what callers ask for, so I don't see a risk of the cache blowing up. The policyengine-uk version is checked in the saved record instead of the file name, which covers a change of default between releases without renaming anything. I couldn't find a stale-reuse path for the cases the PR sets out to cover, apart from the two below.

1. should, capital_gains_responses.py:187-207 (saved_record_problem). Reuse is keyed on the version string from importlib.metadata. A git or editable install of a policyengine-uk branch reports whatever its pyproject says. Here the #2419 branch reports 2.131.0, which is also a published PyPI release with the static default. So an output saved under PyPI 2.131.0 would be reused by a session running the branch, with a different default, and the version check passes. That's exactly the set-up the step 5 parity and default tests will use. A cheap fix is to also record direct_url.json from the distribution, when present (the commit for a git install, and an "editable" flag), and treat a record whose install differs as a mismatch. Alternatively, refuse reuse whenever the installed policyengine-uk isn't from an index.

2. should, docs/microsim.md ("Modifiers") and capital_gains_responses.py:18-20. A simulation modifier that changes the response settings shows up in resolved but not in request, so neither storage_id nor load() can see it. Two simulations with the same caller-set id and different modifiers would share a saved output. That's true of modifiers in general, not new here, and fixing it needs a built simulation, so I'd document it rather than fix it. The docs say modifiers aren't listed under request. They don't yet say what follows: such outputs aren't told apart for reuse.

3. should, common/model_version.py:501-520 with core/simulation.py:190-205. When load() refuses an out-of-date output, ensure() catches it as an "Unexpected error loading simulation" and logs a WARNING with a traceback. Every saved UK output reruns once after this lands, so anyone with a warm cache gets one traceback per output. That's correct behaviour, and the US checks do the same, but a dedicated exception type that ensure() logs at INFO with the reason would make the one-off rerun read as expected. If any scheduled job relies on a warm UK cache, its first run after release will take as long as a cold run, so it's worth a line in the changelog.

4. should, coordination with Stamp saved UK outputs with the year file they were calculated from #608. Both PRs add metadata to the UK dataset and touch datasets.py and model_version.py. The PR says whichever lands second moves its lines into the other's structure. Could you note on both PRs which is expected first? Step 5 depends on this one. If Stamp saved UK outputs with the year file they were calculated from #608 also refuses outputs without its stamp, landing both in the same release means users pay for one rerun, not two.

5. nit, capital_gains_responses.py:96-106. The digest includes the source label and the literal start and end dates. So the same setting given through dynamic instead of policy, or with an explicit far-future end date instead of none, gets a different storage_id and a rerun. That's harmless and arguably right, since the record should say how it was asked for. Mentioning it so it's a choice.

On the questions in the brief:

• What changes outputs. The policyengine-uk version and the explicit settings are covered, apart from points 1 and 2.
• Data. The data release isn't in the record. Outputs sit beside the input dataset's file, so a different data release gets a different directory, and that's unchanged by this PR. Adding the dataset's release or revision to the record would make it fully say what a result was calculated under, but Adopt CenTax’s retention-rate elasticity of 1.0 as the default for UK CGT modelling #602 doesn't ask for that.
• Backwards compatibility. Every existing saved UK output reruns once, as the PR says, and nothing raises out of ensure().

What I checked

• On policyengine-uk 2.126.12, the PR's pin: test_uk_cgt_response_records.py, test_uk_cgt_responses.py, test_uk_dated_reforms.py and the household snapshot test, 56 passed. Every UK, run-record, cache, manifest and simulation test (-k "uk or run_record or cache or manifest or simulation", contract file excluded), 386 passed.
• The same six files (adding test_run_record.py and test_cache.py) with the #2419 branch (c6c77ef3) installed: 77 passed.
• capital_gains_response_assumption takes about 3 ms on a one-person simulation under that branch and measures nothing, so the per-year call in managed_records doesn't add runtime that matters.
• Conflicting settings through calculate_household raise policyengine-uk's ValueError on both installs, so the household record doesn't change how a conflict surfaces.
• Two mutations, each caught by the intended tests. Removing the version check in saved_record_problem fails 2 tests. Dropping the -cgr- suffix from storage_id fails 2 tests.
• The PR merges cleanly with main, which now pins policyengine-uk 2.130.1. I didn't rerun on 2.130.1.
• Not checked: the managed route against real data (that needs the private data release, which I didn't download), and Stamp saved UK outputs with the year file they were calculated from #608's branch.

🤖 Generated with Claude Code

[juaristi22]

Thanks @vahid-ahmadi. Changes are in 98de95c, after merging main (policyengine-uk 2.130.1).

1. The install source. The record's model now carries source, read from the distribution's direct_url.json. It's null for a package index. Otherwise it gives the version control commit, the archive's SHA-256, or a local directory and whether it's editable. It never includes a URL, since a local path names the machine a run was on.
   • load() refuses a saved output from another release or another source.
   • It refuses every saved output under a local directory install, editable or not, because its code can change while its version stays the same.
   • So your case is refused: an output saved under 2.131.0 from PyPI and reloaded with the #2419 branch installed from git gives "calculated with policyengine-uk 2.131.0 from a package index, not from git commit …".
   • Tests cover parsing direct_url.json for each kind of install, the reuse rule for each pair of sources, and a save and reload under a patched git source. The two test modules that reload saved UK outputs run as if installed from an index, so they pass under an editable install too.
2. Modifiers. Documented in the module and in docs/microsim.md. Neither storage_id nor load() sees settings a modifier makes, so two simulations with the same id that differ only in a modifier share a saved output. The docs say to give them different ids.
3. Stale outputs. New StaleOutputError(ValueError) in policyengine.core.
   • load() raises it for every refusal, UK and US: the data year, the response record, SPM settings, renamed inputs and legacy SPM declarations.
   • ensure() logs Running simulation <id> again: <reason> at INFO. Any other load error still logs a warning with the traceback.
   • A test checks that ensure() logs one INFO record and no warning for an out-of-date output.
   • The changelog says the first run after upgrading is a cold one for any job that relies on a warm UK cache.
4. Stamp saved UK outputs with the year file they were calculated from #608. This PR goes first. Stamp saved UK outputs with the year file they were calculated from #608 is a draft at the end of a stack (Read each US year file once in load_datasets #596, Stamp saved US outputs with the rename register they were calculated under #594, Tie US year files to the source dataset and package versions they were cut from #593, Tie UK year files to the source dataset and package versions they were cut from #601), and step 5 depends on this one. Both refuse saved UK outputs that predate them, so landing them in one policyengine.py release means one rerun. I've noted this on Stamp saved UK outputs with the year file they were calculated from #608, along with the two pieces it could reuse (the exception, and the install source for its package versions, which have the same gap), and in the body and the checklist on Adopt CenTax’s retention-rate elasticity of 1.0 as the default for UK CGT modelling #602.
5. The digest. Kept as it is, on purpose. The request_digest docstring and the docs now say the same value given another way is another request, because the record says how the response was asked for.

On the data release: agreed, it's outside #602. Outputs already sit beside their input file.

Locally, the record, response, dated-reform, snapshot, data-year, run-record and cache tests give 117 passed on 2.130.1.

🤖 Generated with Claude Code

[vahid-ahmadi]

@juaristi22

Good to merge. CI is green on 98de95c, and all five points are dealt with. One optional nit below.

Round 1 points

1. Fixed. capital_gains_responses.py:136-219 and :312. I installed policyengine-uk from git at c6c77ef3 into a throwaway venv. uv wrote a direct_url.json with vcs_info.commit_id set to the full commit, and the package reports 2.131.0. I copied that file into the PR venv's policyengine-uk 2.130.1 dist-info and ran it end to end, with nothing patched. An output saved under the index install was refused, with "calculated with policyengine-uk 2.130.1 from a package index, not from git commit c6c77ef3… as installed". A second run under the same commit reused it. An editable dir_info file got it refused every time. Removing the file brought it back to the index case, which reran, because the saved record then showed the editable source. No URL or path ends up in the record.
2. Fixed. docs/microsim.md:312-316 and the module docstring now say that a modifier's settings aren't seen for reuse, and tell users to give such simulations different ids.
3. Fixed. StaleOutputError(ValueError) is at core/tax_benefit_model_version.py:27, and ensure() catches it at core/simulation.py:206. In the end-to-end runs above, the git and editable refusals each logged one INFO line, Running simulation e2e again: …, with no warning and no traceback. All six refusals in load(), UK and US, now raise it. The changelog has the warm-cache line.
4. Fixed. The order (this PR first) is noted on Stamp saved UK outputs with the year file they were calculated from #608, here and on Adopt CenTax’s retention-rate elasticity of 1.0 as the default for UK CGT modelling #602.
5. Kept on purpose, and now documented in the request_digest docstring and docs/microsim.md:318-322. That's fine by me.

New

6. nit, core/simulation.py:206-211. With policyengine-uk installed from a local directory, ensure() still calls save() after the rerun. The saved file can never be reused, since every later load either refuses the directory install again or sees a source that differs. It also overwrites a reusable output saved under an index install. So a short debugging session on an editable install makes the next index-install run cold again. That's a fair price for being safe. Skipping save() when the install is unpinned would avoid it, but I'd leave it unless you think editable installs are common among the people who rely on the cache.

What I checked

• A fresh uv venv at 98de95c (policyengine-uk 2.130.1): the response-record, response, dated-reform, data-year, run-record, cache and household snapshot tests gave 117 passed. The wider -k "uk or run_record or cache or manifest or simulation or spm or stale" run, with the two contract files excluded, gave 742 passed and 5 skipped.
• Six mutations, each caught:
  • removing the version check fails 3 tests;
  • dropping the -cgr- suffix fails 2;
  • removing the source comparison fails 4;
  • removing the unpinned-source check fails 1;
  • catching something other than StaleOutputError in ensure() fails the INFO-logging test;
  • recording source as always None fails 1.
• The end-to-end direct_url.json runs and log output in points 1 and 3.
• ruff format --check is clean. ruff check with my local ruff 0.12 lists 49 UP038 hits across the repo, one of them in this PR's new module (line 63), unchanged in this round. CI's lint passes, so it's a version difference, not a problem.
• Not checked: the managed route against real data (that needs the private data release), and a real archive install. Your tests cover how it's parsed, and an unhashed archive is refused, so the untested cases fail safe.

🤖 Generated with Claude Code