Hold CGT realisations fixed in the Budget contract's rate-blending case, and check the response

[juaristi22]

Part of #602.

PR set: CGT retention-rate default (PolicyEngine/policyengine-uk#2396, #602)

Step	PR	What	Depends on
0	PolicyEngine/policyengine-uk#2277	Measurement branches (Max)	—
1	this PR	Budget contract: hold CGT realisations fixed in the rate-blending case; add response and default cases	—
2	PolicyEngine/policyengine-uk#2419	Retention-rate elasticity of 1.0 as the model default	step 0 merged; step 1 merged and released
3	#621	Record the response each UK run applies	—
4	—	policyengine-uk release, and its compatibility claim for the certified policyengine-uk-data release (1.58.1)	step 2
5	not opened yet	Adopt the release: certify, pin, default tests, docs	steps 1, 3, 4
6	not opened yet	policyengine-app-v2 methodology page	step 5 released

Blocks step 2, as a release. The contract's consumer today is policyengine-sim-api's deploy gate (test_budget_reform_contract_route.py imports CASES), which sees this change only once it pins a policyengine.py release that has it. So this PR has to be released, and that release taken by sim-api, before sim-api takes the policyengine-uk release that step 2 publishes. Merging it first is not enough on its own. #574's qualification is still a draft. Landing this first means it starts from a contract that holds under the new default.
This PR also fixes policyengine.py's releases. Since #571, uv.lock has carried two dependency markers (cffi → pycparser, pexpect → ptyprocess) written by a different uv than the 0.12.14 CI pins. uv lock --check accepts them. But when Versioning bumps the version, uv re-resolves and drops them, so release_lock.py --refresh stops the release with "Versioning changed the reviewed dependency graph". No merge since #571 has published. d1bd9a5 re-locks with uv 0.12.14 (the diff is those two lines) and adds a fixed fragment. Locally, the version bump followed by release_lock.py --refresh passes on this branch and fails on main with CI's error. So merging this PR publishes a release carrying it and everything merged since #571, which is the release step 2 needs.
Tracking checklist: checklist on #602

What

mid_year_blended_rate checks how a CGT rate change part-way through a year blends into the fiscal-year rate. Its oracle charges £10,000 of gains at the blended rate, so it assumes realisations don't respond. policyengine-uk#2396 makes a retention-rate elasticity of 1.0 the default. On that release the case would fail on every in-process route, and qualification would stop:

Year	Expected now	Under the new default
2027-28	£60.82	£40.83
2028-29	£140	£91.22

• The rate-blending case pins the response off. It sets the elasticity to 0, the MTR elasticity to 0 and the separate BADR switch off from 2025, through capital_gains_response(), so its expected values hold whatever the model's default. A parameter check confirms the setting reaches each route.
• A new case checks the response itself. cgt_retention_response sets a 20% CGT basic rate from 6 April 2027 with an explicit retention-rate elasticity of 1. Realised gains are £10,000 × 0.80 / 0.82 = £9,756.10, and the CGT change is £91.22. The expected values come from _realised_gains and _gains_response_uplift, which read current law as the other oracles do.
• A third case shows the default without gating. cgt_default_response runs the same reform with no response settings. Its checks expect the retention-rate elasticity of 1 that policyengine-uk#2396 makes the default. They carry pending (decision policyengine-uk#2396) until policyengine.py pins that release, so the report shows what the default does on every route without failing qualification. Step 5 makes them gating with the pin.
• Each case checks all three settings. Parameter rows for the elasticity, the MTR elasticity and the separate BADR switch confirm what reached each route's built system.
• Contract 1.0.0 → 1.1.0.

Checks

Main is merged in, so these ran on policyengine-uk 2.130.1, the current pin.

• Every in-process route. The two gating CGT cases pass every check on all five in-process routes: 151 pass, 0 fail. The 20 unsupported rows are parameter checks on the wrapper routes. cgt_default_response reports 19 pending rows on 2.130.1, whose default is still static. The 6 that hold are the MTR and switch settings. The 13 that don't yet hold are the elasticity rows and the response itself (£140 against the £91.22 expected).
• The contract tests. pytest tests/test_uk_budget_reform_contract.py gives 106 passed (about 7 minutes). The new tests cover:
  • the oracle at an elasticity of 0;
  • the pins in every route's representation;
  • the two gating CGT cases on every in-process route, and the default case on one real route, where it reports pending. Its rows on every route are in the golden rows.
• The golden rows. tests/fixtures/uk_contract/evaluate_case_rows.json is recorded again. The CGT cases add 216 rows, and no existing row changes.
• Qualify UK model and data candidates automatically before promotion #574's model gates against Make a retention-rate elasticity of 1 the default CGT realisation response policyengine-uk#2419. I ran Qualify UK model and data candidates automatically before promotion #574's own code (head d3b526e) locally: run_metrics_report with its pinned tool, then metrics_review and reform_responsiveness, for 2025-26 to 2031-32. It ran on one dataset for three installs: 2.130.1, #2419's merge base, and #2419's head. #2419 trips no review bound, and every baseline metric is unchanged. Reform responsiveness passes in all three. metrics_review fails identically in all three, from the tool pin rather than the model (see the reply below).
• The API route. policyengine-sim-api's test_budget_reform_contract_route.py picks up the new cases. Its _normalise_policy passes gov.simulation.* paths and boolean values through unchanged, as it does for structural_switch.

🤖 Generated with Claude Code

[vahid-ahmadi]

@juaristi22

Good to merge once CI is green. The shoulds below are about the contract and the merge order once the default flips, not about this diff.

On the main question: pinning the elasticity to 0 in mid_year_blended_rate is the right call. That case checks how a mid-year rate change blends into the fiscal-year rate, and its oracle is static arithmetic. A behavioural response in it would test two things at once. I reproduced the failure you describe: with the PolicyEngine/policyengine-uk#2419 branch installed, the case as it is on main fails on all five in-process routes, at £40.83 against £60.82 for 2027-28 and £91.22 against £140 for 2028-29. With this PR it passes. So the pin doesn't hide a wrong number. But it does mean the contract no longer sees the default (point 1).

1. should, budget_reform_contract.py:742-830. After this PR, both CGT cases set the elasticity explicitly. The contract, and so Qualify UK model and data candidates automatically before promotion #574's qualification, gives the same result whatever default policyengine-uk ships: 0, 1.0 or 2.0. Users will see the default, not the pinned value. The plan puts the default-response cases in contract 1.2.0 with the adoption PR (step 5). Until then, a release whose default silently moved would still qualify. Could you open the default case before #2419 merges, so that gap stays short? One way is a third case with no response settings, carrying pending until step 5. Its row then shows the default in the report without gating, and it can be flipped to gating with the pin. That's a suggestion; adding it in step 5 is fine too, if the window is accepted on purpose.

2. should, PR body and plan. The ordering note says this must merge before #2419 or "Qualify UK model and data candidates automatically before promotion #574's qualification fails every later UK candidate". Qualify UK model and data candidates automatically before promotion #574 is still a draft, so main doesn't run that qualification yet. The consumer that runs the contract today is the sim-api deploy gate (test_budget_reform_contract_route.py imports CASES). As I read it, that gate sees this PR only once a policyengine.py release with it is pinned there. So the real constraint looks like: this PR is released and picked up by sim-api before sim-api takes the new policyengine-uk, not just merged first. Worth stating that way in the checklist.

3. should, qualification beyond the contract. Qualify UK model and data candidates automatically before promotion #574's uk.json also runs the standard reforms (reform_responsiveness) and the metric replay against review bounds. None of the standard reforms is a CGT change, but #2419 says every reformed simulation now measures, and an income tax change can move gains between bands. Before #2419 merges, it would help to run Qualify UK model and data candidates automatically before promotion #574's qualification once against the #2419 branch, so a review-bound trip there doesn't surprise anyone. I haven't run it.

4. nit, budget_reform_contract.py:50, :625 and :1807. The module docstring, the pending oracle text and the expected_annual docstring still say "contract 1.0.0". If they mean "since 1.0.0", fine. If they mean the current contract, they need updating to 1.1.0.

5. nit, budget_reform_contract.py:768. The case checks only elasticity at the parameter level. mtr_elasticity and separate_badr_elasticity are covered by the representation test at tests/test_uk_budget_reform_contract.py:123, not on each route's built system. Two more parameter rows would make the report show all three settings reached the model.

The new oracle checks out by hand: £10,000 × 0.80 / 0.82 = £9,756.10, and (£9,756.10 − £3,000) × 20% − £7,000 × 18% = £91.22. That matches the 2028-29 figure under the new default, which is a good sign the response is the same on every route.

What I checked

• tests/test_uk_budget_reform_contract.py on policyengine-uk 2.126.12, the PR's pin: 36 passed.
• The full contract (all cases, five in-process routes, no property examples), with this PR and the #2419 branch head (c6c77ef3, reports 2.131.0) installed: 722 pass, 0 fail, 6 pending, 73 unsupported.
• Both properties with the same install, seed 0 and 15 examples each: both pass, so the routes still agree and baselines hold with the response on. Qualification asks for 20, and I didn't run 20.
• mid_year_blended_rate at the merge base under the same branch: 10 fails, the figures above. That shows the pin is what fixes it.
• ruff format --check is clean. The 10 ruff check findings (UP038) are in files this PR doesn't touch and come from my newer ruff; CI lint passes.
• The PR merges cleanly with main, which now pins policyengine-uk 2.130.1. I didn't rerun on 2.130.1.
• Not checked: the api.sim_api route, and Qualify UK model and data candidates automatically before promotion #574's qualification as a whole.

🤖 Generated with Claude Code

[juaristi22]

Thanks @vahid-ahmadi. Changes are in a5ca27c. I merged main first, so everything below ran on policyengine-uk 2.130.1.

1. The default case. Added cgt_default_response, which is the same reform with no response settings. Its checks expect the retention-rate elasticity of 1. They carry pending with decision policyengine-uk#2396, so the report shows the default on every route without gating it.
   • On 2.130.1 they report pending and don't hold yet: £140 against the expected £91.22.
   • Step 5 makes them gating along with the pin.
   • The integration test checks that the default case's rows are pending on every in-process route.
2. The ordering. Agreed: the constraint is a release. sim-api's deploy gate needs a policyengine.py release with this PR before it takes the new policyengine-uk. The PR body, the checklist on Adopt CenTax’s retention-rate elasticity of 1.0 as the default for UK CGT modelling #602 and Make a retention-rate elasticity of 1 the default CGT realisation response policyengine-uk#2419's body now say so. Merging this first still matters for Qualify UK model and data candidates automatically before promotion #574 once it lands.
3. Qualify UK model and data candidates automatically before promotion #574's qualification against #2419. I replicated its two model gates locally with Qualify UK model and data candidates automatically before promotion #574's own code (head d3b526e): run_metrics_report with the pinned tool, then metrics_review and reform_responsiveness, for 2025-26 to 2031-32. There were three installs on one dataset: 2.130.1 from PyPI, #2419's merge base, and #2419's head. The dataset was uk-data 1.58.0's eFRS. The certified 1.58.1 file wasn't on this machine, and these are model-only steps, so the data version doesn't change what they test.
   • Reform responsiveness passes for all three. #2419 leaves every standard reform's target change exactly as it was.
   • #2419 trips no review bound. Every baseline metric is bit-identical to its merge base, CGT receipts included, so the fast path holds on the full dataset. It moves 21 of 1,470 metrics, all of them reform costings, and all through capital gains tax. Each move is under £1m a year, well inside the reform bound.
     • The personal allowance and State Pension reforms move because they shift some gains across CGT bands.
     • The basic rate rise moves negligibly.
     • Income tax, VAT, other receipts and spending don't change.
   • metrics_review fails, but not because of #2419. It fails identically for 2.130.1, the merge base and #2419 on three hard checks: closure, identity:gov_spending and identity:household_benefits.
     • The cause is the partition at Qualify UK model and data candidates automatically before promotion #574's tool pin (9b33fe19). It predates healthy_start_vouchers, scottish_child_payment, statutory_paternity_pay, two_child_limit_payment and pawhp, so it fails for any current policyengine-uk.
     • Report each pull request's effect on national aggregates on the certified dataset policyengine-uk#2335's current head (a4b18de9) covers all of them, with no closure problems against any of the three installs. So moving the pin there should clear it. I haven't done a full rerun with that pin. cc @MaxGhenis for Qualify UK model and data candidates automatically before promotion #574.
4. The version wording. Where a docstring means "since", it now says "since contract 1.0.0". The prior-law docstrings name PRIOR_LAW_OPTION rather than a version.
5. All three settings. Each CGT case now has parameter rows for the elasticity, the MTR elasticity and the separate BADR switch on every route's built system. On the wrapper routes they're unsupported, as other parameter checks are.

The golden rows are recorded again: the CGT cases add 216 rows, and no existing row changes. pytest tests/test_uk_budget_reform_contract.py gives 106 passed. The two gating CGT cases pass on all five in-process routes (151 pass, 0 fail).

🤖 Generated with Claude Code

[juaristi22]

A correction to point 1 above, after CI. The Test job hit its 60-minute limit on Python 3.12 and 3.13, seconds after pytest finished; 3.11 and 3.14 passed. The CGT integration test was the slowest in the file. In 9372051 it runs the two gating cases on every in-process route as before, and the pending default case on country.reform_dict only. That adds a data check that every one of the default case's checks waits on policyengine-uk#2396 and that the case sets no response settings. The golden rows already cover its rows on every route under fake routes.

For what it's worth, main's own Test jobs now take 24-51 minutes against the workflow's 60-minute limit, whose comment still says 28 minutes was the slowest observed in early October, so other PRs may hit it too.

🤖 Generated with Claude Code

[juaristi22]

I've added the release-lock fix to this PR, so that its merge publishes a release.

• The cause. f68b63a (Add execution receipt schema v1 for published results #571) wrote two dependency markers into uv.lock (cffi → pycparser, pexpect → ptyprocess) that uv 0.12.14, the version CI pins, doesn't write. uv lock --check accepts them, so PR CI passes. But when Versioning bumps the version, uv re-resolves and drops them, and release_lock.py --refresh stops with "Versioning changed the reviewed dependency graph". Every merge since Add execution receipt schema v1 for published results #571 has failed at Versioning, and none has published.
• The fix. d1bd9a5 re-locks with uv 0.12.14 across a temporary version change, which writes the lock in that version's form. The diff is the two marker lines, plus a fixed fragment.
• Checked locally.
  • On this branch, bump_version.py followed by release_lock.py --refresh passes with uv 0.12.14, and the lock then differs only in its version line.
  • On main, the same steps fail with CI's error.
• The release. When this merges, its release carries everything merged since Add execution receipt schema v1 for published results #571.

cc @MaxGhenis, since the markers came in with #571 and your merges since then haven't released. If you'd rather land the lock fix separately and first, it's one commit and I can split it out.

🤖 Generated with Claude Code