Re-average fuel duty and other blended rates after mid-year reforms

[vahid-ahmadi]

Fixes #2167.

What was wrong

Parameters marked fiscal_year_blend: true (petrol and diesel, LPG and natural gas duty, and the CGT rates) are converted to one day-weighted average per model year, 6 April to 5 April. Reforms were applied to that converted tree and never re-averaged. A persistent reform dated part-way through a year therefore applied only from its calendar month, against a baseline average that still included the rates it replaces.

For the issue's example (60p a litre from 1 September 2026, 1,200 litres, model year 2026):

	Δ fuel_duty
Raw dated schedule, day-weighted over 6 April 2026 – 5 April 2027	+£38.26
Reform.from_dict({path: {"2026-09-01.2100-12-31": 0.60}}), before	+£24.19
Same reform, after	+£38.26

The £38.26 assumes equal daily litres, which is what the day weighting means.

Fix

• Recording updates. reblend_dated_reforms (utils/parameters.py) is a context manager around every place a reform changes the converted tree:

  • Simulation.apply_reform;
  • _apply_reform_class before data load;
  • the dict-reform modifier in Scenario.from_reform.

  Inside it, each update to a blended parameter is recorded as it is applied.

• Replaying them on the dated schedule. On exit, the recorded updates are replayed on the dated schedule the averages came from. That is the uprated parameters.baseline tree, or the schedule an earlier reform left. Each model year from 2015 to 2040 is then re-averaged.

• Model-year boundaries. On the converted tree, 1 January and 31 December bound model years. So those boundaries move to 6 April and 5 April when replayed, and any other date is kept as written.

• Whole-year reforms are left exactly as they are. A parameter is re-averaged only when some update starts or ends part-way through a year. Bare years, year: periods, scalars and ranges such as "2026-01-01.2100-12-31" produce the same model-year values as before, and a test pins this for 2015–2040.

• Chained reforms. Each re-averaged parameter keeps its replayed schedule, so a later reform in a tuple builds on an earlier mid-year change.

• Untouched: the baseline tree, unblended parameters and preserve_calendar_dates parameters.

• No policyengine-core API changes.

One behaviour change beyond the issue: a plain dictionary reform (reform={path: {"2026-09-01": v}}) reads a single date as one day, as Scenario does. It is now re-averaged as one day (+£0.23 in the example), where before the converted tree applied it to the whole September monthly reading.

Documentation

• A "Reforms dated part-way through a year" section in docs/engineering/excise-duties.md covers:
  • the fiscal-year semantics;
  • the accepted syntax for each API, including that a Scenario single-date key is a one-day period and range keys raise ValueError;
  • how to score a mid-year change, with the equal-daily-use assumption;
  • the preserve_calendar_dates exception.
• Limitation, documented there: a change that really takes effect on 1 January can't be written as a dated reform, because 1 January reads as a model-year boundary.
• The petrol_and_diesel.yaml header points to that section.

Tests

policyengine_uk/tests/test_mid_year_reforms.py has 15 cases. The 8 mid-year cases below fail on main; the 7 that check nothing else changes pass on both.

• Mid-year start: pinned at +£38.2636, and model year 2027 at the full-year change.
• Expiry and reversion within a fiscal year: 2027 is unchanged.
• Several changes within one fiscal year.
• A CGT basic rate change from 1 October 2026.
• The The temporary 0% VAT on domestic electricity in Great Britain (1 October 2026 to 31 March 2027) is not modelled, and VAT on domestic energy cannot be changed on its own #2166 electricity VAT dates: 0% from 1 October 2026 to 31 March 2027, then 5% from 1 April 2027. Model year 2026 is (178 × 52.95p + 5 × 5p) / 365. This runs on the fuel duty rate, because the electricity VAT rate isn't on main yet; a direct electricity test should follow once The temporary 0% VAT on domestic electricity in Great Britain (1 October 2026 to 31 March 2027) is not modelled, and VAT on domestic energy cannot be changed on its own #2166 lands.
• The same reform as a dict with a range key, and as a Reform class applied before data load.
• Chained reforms.
• Whole-year reforms in five forms, against the pre-fix values for every year 2015–2040.
• The baseline schedule and unblended parameters are untouched.

Suite results:

• full YAML suite: 2,658 passed;
• pytest outside tests/policy and tests/microsimulation: 3,217 passed, 8 skipped, 2 xfailed, 0 failed;
• ruff format and ruff check are clean;
• the Towncrier draft renders the fragment under "Fixed".

Performance

Median time to build a household Simulation and calculate fuel_duty, 7 runs:

Reform	main	This PR
None	0.66 s	0.53 s
Mid-year fuel duty (re-averaged)	1.14 s	1.04 s
Whole-year NI rate	0.95 s	0.97 s

The differences are within run-to-run noise on a busy machine. Recording wraps the six blended parameters for each reform, and re-averaging runs only for parameters a reform changes part-way through a year.

🤖 Generated with Claude Code

[juaristi22]

Review of #2185 — Re-average fuel duty and other blended rates after mid-year reforms

Base repository: PolicyEngine/policyengine-uk
PR number: 2185
Reviewed head SHA: 0210317
Merge base SHA: ba06cec
Mode: incremental from 0210317
Scope: changed behavior and affected dependencies
Source evidence: Official sources are linked below; local source copies and validation artifacts were retained.
Review status: COMPLETE

Source Documents

• policy-source-fuel-duty-amended.html.

Critical

None.

Should Address

None.

Suggestions

None.

Evidence Gaps

No material gaps reported.

Notes

• The context records opt-in fiscal-year-blended gov scalar updates, restores the update methods on exit, replays changes on a detached dated schedule and invalidates parameter caches. Existing stored schedules preserve earlier reforms; nested tuple application records updates at the outer context. Tests cover persistent/temporary starts, successive overrides, whole-year preservation, CGT rates, baseline isolation, dicts, before-data-load application, and unblended NI behavior. Scenario.parameter_changes rebuilds the dated tree through reset_parameters/process_parameters, so it does not need the converted-tree wrapper.
• Optional Hypothesis regression test_fuel_duty_schedule.py NOT RUN: Hypothesis absent. Runnable changed/regression batch passed after excluding that path.
• Evidence reused after confirming identical reviewed head and merge base. The fetched main branch has advanced; the review remains scoped to this unchanged head. Current CI was captured separately. Local tests were not rerun.
• Daily-use and model-year-boundary assumptions documented. Code reviewer owns any composition finding.
• Captured base branch: main. Actual fetched base SHA: f1a9a3c. Incremental scope contains no code changes.

Validation Summary

Reused exact-head validation (not rerun in this batch): 48 passed, 1 warning in 31.13s; snapshot import confirmed; raw command/paths in pr-2185-code-tests-config.json and pr-2185-code-tests.log;

Reused exact-head validation (not rerun in this batch): Official HMRC original and extract acquired; statutory fuel dates checked; hypothetical £38.26356 example independently derived; no duplicate model run.; bundled check_source_manifest.py passed under existing Python3.13; cache result pr-2185-review-source-cache-check.json

CI at 0210317: Install + smoke-import (py3.11)=SUCCESS, Test documentation builds=SUCCESS, Install + smoke-import (py3.12)=SUCCESS, Install + smoke-import (py3.13)=SUCCESS, Install + smoke-import (py3.14)=SUCCESS, Lint=SUCCESS, Test=SUCCESS.

Detailed code, policy, source and reproduction artifacts were retained locally.

Timing

elapsed seconds: 1151.70s; setup seconds: 270.16s

Review Severity

APPROVE. Open findings: 0 critical, 0 should address, 0 suggestions.

[MaxGhenis]

Suggested Reproducer: line for the new Fix evidence check; add it to the description, or narrow it to the test that reproduces the bug. It lists the tests this PR adds or changes, and the check (.github/check_fix_pr.py) accepts it against this PR's diff.

Reproducer: `policyengine_uk/tests/test_mid_year_reforms.py::test_mid_year_start_is_day_weighted_over_the_model_year`, `policyengine_uk/tests/test_mid_year_reforms.py::test_change_that_expires_within_the_year_reverts_to_the_schedule`, `policyengine_uk/tests/test_mid_year_reforms.py::test_several_changes_within_one_year`, `policyengine_uk/tests/test_mid_year_reforms.py::test_cgt_rate_change_part_way_through_the_year`, `policyengine_uk/tests/test_mid_year_reforms.py::test_temporary_zero_rate_with_the_electricity_vat_dates`, `policyengine_uk/tests/test_mid_year_reforms.py::test_whole_year_reforms_keep_their_values_exactly`

The check isn't required yet (decision d1368); it fails fix PRs whose description names no reproducer.

🤖 Generated with Claude Code

[vahid-ahmadi]

#2305 fixed this on the statutory timeline and closed #2167; the issue's +£38.26 case passes on main. Closing.