From 0410b497017ceeee76bfaded491d1261a0ec0716 Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Tue, 29 Sep 2026 13:26:18 -0400 Subject: [PATCH 1/6] Compute the State Pension triple lock from its statutory inputs The April uprating now uses September CPI (ONS D7G7) and May-July AWE total pay growth as used in each review (ONS KAC3, per the Up-rating Order explanatory memoranda), not calendar-year OBR growth. The inputs live in gov.economic_assumptions.statutory_uprating_inputs with ONS outturn, then forecasts: calendar-year growth plus the OBR's gap between its statutory-basis and calendar-year forecasts (September CPI from receipts Table 3.19, Q3 CPI and Q2 earnings from economy Tables 1.7 and 1.6), then calendar-year growth after the EFO horizon. Macro scenarios that edit calendar-year growth move them; scenarios can also set them directly. The rule reproduces every published uprating from April 2012 to April 2026. April 2011 (RPI) is the only override; April 2022 is include_earnings false. The rates run to the end of the economic assumptions. The active toggle now means the s150A statutory minimum, and an optional earnings-path guarantee models plans that keep the pension in line with earnings over time. The OBR importer regenerates the forecast gaps. April 2027 rises by 3.9% (published May-July 2026 earnings) rather than 3.4%. Co-Authored-By: Claude Opus 5.5 --- .../triple-lock-statutory-inputs.added.md | 1 + .../triple-lock-statutory-inputs.changed.md | 1 + .../book/assumptions/nowcasting-comparison.md | 2 +- docs/book/programs/gov/dwp/state-pension.md | 142 +++++++- .../dwp/state_pension/triple_lock/active.yaml | 7 +- .../triple_lock/create_triple_lock.py | 234 ++++++++++-- .../triple_lock/earnings_path_guarantee.yaml | 21 ++ .../triple_lock/include_earnings.yaml | 10 +- .../triple_lock/include_inflation.yaml | 2 +- .../state_pension/triple_lock/outturn.yaml | 25 +- .../gov/economic_assumptions/README.md | 26 ++ .../create_statutory_uprating_inputs.py | 76 ++++ .../awe_total_pay_may_july.yaml | 47 +++ .../cpi_september.yaml | 34 ++ .../forecast_gap/awe_total_pay_may_july.yaml | 28 ++ .../forecast_gap/cpi_september.yaml | 31 ++ .../scenarios/no_economic_assumptions.py | 22 +- policyengine_uk/tax_benefit_system.py | 5 + .../tests/test_import_obr_forecasts.py | 123 +++++++ .../tests/test_triple_lock_outturn.py | 146 ++++++-- .../tests/test_triple_lock_properties.py | 254 +++++++++++++ .../test_triple_lock_statutory_inputs.py | 339 ++++++++++++++++++ policyengine_uk/utils/create_triple_lock.py | 22 -- policyengine_uk/utils/import_obr_forecasts.py | 273 ++++++++++++++ 24 files changed, 1745 insertions(+), 126 deletions(-) create mode 100644 changelog.d/triple-lock-statutory-inputs.added.md create mode 100644 changelog.d/triple-lock-statutory-inputs.changed.md create mode 100644 policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/earnings_path_guarantee.yaml create mode 100644 policyengine_uk/parameters/gov/economic_assumptions/create_statutory_uprating_inputs.py create mode 100644 policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/awe_total_pay_may_july.yaml create mode 100644 policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/cpi_september.yaml create mode 100644 policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/forecast_gap/awe_total_pay_may_july.yaml create mode 100644 policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/forecast_gap/cpi_september.yaml create mode 100644 policyengine_uk/tests/test_triple_lock_properties.py create mode 100644 policyengine_uk/tests/test_triple_lock_statutory_inputs.py delete mode 100644 policyengine_uk/utils/create_triple_lock.py diff --git a/changelog.d/triple-lock-statutory-inputs.added.md b/changelog.d/triple-lock-statutory-inputs.added.md new file mode 100644 index 0000000000..49c80cd4cd --- /dev/null +++ b/changelog.d/triple-lock-statutory-inputs.added.md @@ -0,0 +1 @@ +Add an optional State Pension earnings-path guarantee (`gov.dwp.state_pension.triple_lock.earnings_path_guarantee`), so reforms that keep the pension in line with earnings over time, such as one reading of the plan announced in September 2026, can be modelled as parameter changes. Off under current law. diff --git a/changelog.d/triple-lock-statutory-inputs.changed.md b/changelog.d/triple-lock-statutory-inputs.changed.md new file mode 100644 index 0000000000..9cbc76543b --- /dev/null +++ b/changelog.d/triple-lock-statutory-inputs.changed.md @@ -0,0 +1 @@ +Compute the State Pension triple lock from its statutory inputs, September CPI and May-July average weekly earnings total pay growth, instead of calendar-year OBR growth. The inputs hold ONS outturn and OBR forecasts on the statutory basis, and macro scenarios that edit calendar-year growth move them. April 2027 now rises by 3.9% (published May-July 2026 earnings growth) rather than 3.4%, and the rates run to the end of the economic-assumption series instead of 2034. diff --git a/docs/book/assumptions/nowcasting-comparison.md b/docs/book/assumptions/nowcasting-comparison.md index 7b257cfad5..15d4fa795b 100644 --- a/docs/book/assumptions/nowcasting-comparison.md +++ b/docs/book/assumptions/nowcasting-comparison.md @@ -48,7 +48,7 @@ earnings index. | Aspect | Resolution Foundation | PolicyEngine UK | |-------|------------------------|-----------------| | Working-age benefits | Uses statutory uprating with explicit overrides for announced policy (e.g. CoL Payments, benefit freezes). | Same approach. Parameters under `gov/dwp/` and `gov/hmrc/child_benefit/` track legislated rates; ad hoc payments live under `gov/treasury/cost_of_living_support`. | -| State Pension | Models the triple lock explicitly, using its own internal earnings/CPI forecasts. | Models the triple lock via `gov/dwp/state_pension/triple_lock/*.yaml`; the uprating value tracks announced DWP rates with a fallback to the maximum of earnings, CPI and the 2.5% floor. | +| State Pension | Models the triple lock explicitly, using its own internal earnings/CPI forecasts. | Models the triple lock from its statutory inputs (September CPI, May-July AWE total pay growth, 2.5% floor) in `gov/economic_assumptions/statutory_uprating_inputs/` and `gov/dwp/state_pension/triple_lock/`, with OBR forecasts of those inputs; see the State Pension page. | ### Take-up diff --git a/docs/book/programs/gov/dwp/state-pension.md b/docs/book/programs/gov/dwp/state-pension.md index 24513d3d7c..5e9fc65a5b 100644 --- a/docs/book/programs/gov/dwp/state-pension.md +++ b/docs/book/programs/gov/dwp/state-pension.md @@ -36,19 +36,135 @@ before or on/after 6 April 2016. State Pension flat-rate parameters live in `gov/dwp/state_pension/basic_state_pension/amount.yaml` and -`gov/dwp/state_pension/new_state_pension/amount.yaml`. Both are uprated -under the **triple lock**: the maximum of earnings growth, CPI -inflation, or the 2.5% floor parameterised at -`gov/dwp/state_pension/triple_lock/minimum_rate.yaml`. Active components -of the triple lock are controlled by: - -- `triple_lock/active.yaml` — top-level toggle. -- `triple_lock/include_earnings.yaml` — whether the earnings limb is - active. -- `triple_lock/include_inflation.yaml` — whether the CPI limb is active. - -These flags exist so that policy reforms can disable individual limbs -(e.g. "double lock" scenarios that drop the earnings or inflation limb). +`gov/dwp/state_pension/new_state_pension/amount.yaml`. Published rates run +to 2026-27; later years are uprated by +`gov.economic_assumptions.indices.triple_lock`, built from the yearly rates +in `gov.economic_assumptions.yoy_growth.triple_lock`. + +### The triple lock + +The rise each April is the highest of: + +- **earnings growth**: average weekly earnings, total pay, whole economy, in + May to July of the previous year on a year earlier (ONS KAC3); +- **CPI inflation**: the 12-month rate in September of the previous year + (ONS D7G7); +- **2.5%**: `triple_lock/minimum_rate.yaml`. + +`create_triple_lock.py` computes each element from these statutory inputs, +rounded to the one decimal place the ONS publishes. From them it reproduces +every published rise from April 2012 to April 2026. April 2011 is the one +override (`triple_lock/outturn.yaml`): the basic State Pension rose by +September 2010 RPI (4.6%) during the switch to CPI. The April 2022 +suspension of the earnings element is `include_earnings` set to false for +that year. + +The yearly rates run from April 2011 to one year past the end of the +economic-assumption series. + +### The statutory inputs + +`gov.economic_assumptions.statutory_uprating_inputs` holds: + +- `cpi_september`: September CPI, keyed to 1 September; +- `awe_total_pay_may_july`: May-July earnings growth, keyed to 1 July, as + used in the uprating review. + +Each holds published figures and then a null. From the null onwards, +`create_statutory_uprating_inputs.py` fills in a forecast: calendar-year +growth in the matching OBR series (`yoy_growth.obr.consumer_price_index` or +`yoy_growth.obr.average_earnings`) plus `forecast_gap`. The gap is the OBR's +forecast of the statutory measure minus its calendar-year forecast, from the +same Economic and Fiscal Outlook: + +- September CPI: the OBR's September CPI forecast (receipts Table 3.19, the + CPI used to uprate tax thresholds), or Q3 CPI (economy Table 1.7) in years + that table does not cover; +- May-July earnings: Q2 average earnings growth (economy Table 1.6). The + OBR does not forecast the ONS AWE series; its measure is wages and + salaries per employee, and Q2 is the quarter nearest May to July. + +After the EFO horizon the gap is zero, so the inputs follow calendar-year +growth. `policyengine_uk/utils/import_obr_forecasts.py` regenerates the gaps +with the calendar-year series (pass `--receipts-file` or `--receipts-url` +for September CPI). + +### Scenarios + +A macro scenario applied before the data load that edits calendar-year +growth moves the forecast inputs one for one, and so the triple lock: + +```python +from policyengine_uk.model_api import Scenario + +scenario = Scenario( + parameter_changes={ + "gov.economic_assumptions.yoy_growth.obr.average_earnings": { + "year:2027-01-01:1": 0.05, + }, + }, + applied_before_data_load=True, +) +``` + +A scenario can also set the statutory inputs directly, for example from a +model of the monthly series, since the triple lock pays out on how September +CPI and May-July earnings differ from each other. A value replaces the +forecast for that year only; a bare year names the fiscal year from 6 April, +which contains both observation dates: + +```python +Scenario( + parameter_changes={ + "gov.economic_assumptions.statutory_uprating_inputs.cpi_september": { + "2027": 0.031, + }, + "gov.economic_assumptions.statutory_uprating_inputs.awe_total_pay_may_july": { + "2027": 0.024, + }, + }, + applied_before_data_load=True, +) +``` + +These changes must go through `Scenario(parameter_changes=...)`. The rates +are built when parameters are processed, and a `reform=` dictionary edits +parameters after that, so it changes the parameter but not the uprating. + +### Reform levers + +- `triple_lock/active.yaml` switches the triple lock off. The pension then + rises by the statutory minimum from the review under the Social Security + Administration Act 1992, section 150A: earnings growth, and nothing when + earnings fall. +- `triple_lock/include_earnings.yaml` and + `triple_lock/include_inflation.yaml` drop an element, e.g. for a double + lock. +- `triple_lock/minimum_rate.yaml` sets the floor. +- `triple_lock/earnings_path_guarantee.yaml` (off under current law) keeps + the pension on or above an earnings path started from its level in the + year before the guarantee first applies. With `include_earnings` false, + it models a plan that drops the earnings element of the triple lock but + keeps the pension in line with earnings over time. The plan announced on + 29 September 2026 gave no formula; one reading, from April 2030, is: + +```python +Scenario( + parameter_changes={ + "gov.dwp.state_pension.triple_lock.include_earnings": { + "year:2030-01-01:100": False, + }, + "gov.dwp.state_pension.triple_lock.earnings_path_guarantee": { + "year:2030-01-01:100": True, + }, + }, + applied_before_data_load=True, +) +``` + +Each guaranteed year, the rise is the highest of the triple lock elements +still included and the rise needed to reach the earnings path, rounded up to +0.1 percentage points. ## Known aggregate gap (#1632) diff --git a/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/active.yaml b/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/active.yaml index cec29c6f10..00413c770f 100644 --- a/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/active.yaml +++ b/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/active.yaml @@ -1,6 +1,9 @@ -description: The triple lock is active if this value is true. +description: Whether the triple lock applies to the April uprating of the State Pension; without it, the pension rises by the statutory minimum, earnings growth, and not at all when earnings fall. values: 2010-01-01: true metadata: unit: bool - label: Triple lock \ No newline at end of file + label: Triple lock + reference: + - title: Social Security Administration Act 1992, section 150A + href: https://www.legislation.gov.uk/ukpga/1992/5/section/150A diff --git a/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/create_triple_lock.py b/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/create_triple_lock.py index fc884db555..260715799e 100644 --- a/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/create_triple_lock.py +++ b/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/create_triple_lock.py @@ -1,47 +1,213 @@ -from policyengine_core.parameters import ( - ParameterNode, - Parameter, - get_parameter, +"""Build the State Pension uprating series from the statutory inputs. + +Each April the basic and new State Pension rise by the highest of: + +- earnings growth: average weekly earnings, total pay, whole economy, the + three months May to July of the previous year on a year earlier; +- CPI inflation: the 12-month rate in September of the previous year; +- the minimum rate (2.5%). + +The inputs live in ``gov.economic_assumptions.statutory_uprating_inputs``, +which ``create_statutory_uprating_inputs`` completes with forecasts before +this module runs. Each input is rounded to the one decimal place the ONS +publishes, as the uprating review uses the published figure. + +With ``active`` false there is no triple lock, only the statutory minimum +from the review under Social Security Administration Act 1992 s150A: the +pension rises by earnings growth, and not at all when earnings fall. + +Years with a published rate that the rule cannot reproduce are overridden in +``triple_lock/outturn.yaml``. Only April 2011, when the basic State Pension +rose by RPI during the switch to CPI, needs one; the April 2022 suspension of +the earnings element is expressed through ``include_earnings``. + +``earnings_path_guarantee`` adds an optional floor for modelling reforms: in +each year it applies, the pension also rises at least enough to stay on an +earnings path started from its level in the year before the guarantee first +applied. Current law leaves it off. + +The output parameter ``gov.economic_assumptions.yoy_growth.triple_lock`` holds +the rate taking effect in April of each year, keyed to 1 January as the other +growth series are, and runs to one year past the last year of the +economic-assumption series. +""" + +from dataclasses import dataclass +from decimal import ROUND_CEILING, ROUND_HALF_UP, Decimal +from typing import Dict, Iterable, Optional + +from policyengine_core.parameters import Parameter, ParameterNode + +from policyengine_uk.parameters.gov.economic_assumptions.create_statutory_uprating_inputs import ( + AWE_OBSERVATION_MONTH_DAY, + CPI_OBSERVATION_MONTH_DAY, + last_input_year, ) -YEARS = list(range(2022, 2035)) +FIRST_UPRATING_YEAR = 2011 +# Published statistics are quoted to one decimal place of a percentage. +PUBLISHED_PRECISION = Decimal("0.001") +# Guards the round-up of a guaranteed minimum against floating-point noise, +# so a ratio of 1.025000000000001 is not taken up to 2.6%. +RATIO_PRECISION = Decimal("0.000000001") -def add_triple_lock(parameters: ParameterNode) -> ParameterNode: - obr = parameters.gov.economic_assumptions.yoy_growth.obr - average_earnings = obr.average_earnings - cpi = obr.consumer_price_index - triple_lock = parameters.gov.dwp.state_pension.triple_lock - min_rate = triple_lock.minimum_rate - # Years with a published outturn or DWP-announced rate to use directly - # instead of the formula (the formula relies on OBR calendar-year averages, - # which don't match the May-July AWE window DWP actually uses). - outturn_years = {int(v.instant_str[:4]) for v in triple_lock.outturn.values_list} +def uprating_instant(year: int) -> str: + """Date at which policy parameters are read for the April ``year`` uprating. + + 30 April is the date fiscal-year conversion samples, so a reform keyed to + the bare year (the fiscal year from 6 April), to ``year:YYYY-01-01:1`` or + to 1 January all take effect in the uprating for that April. + """ + return f"{year}-04-30" + + +def round_to_published_precision(rate: float) -> float: + """Round a growth rate to 0.1 percentage points, halves away from zero.""" + return float(Decimal(repr(rate)).quantize(PUBLISHED_PRECISION, ROUND_HALF_UP)) + + +def round_up_to_published_precision(rate: float) -> float: + """Round a guaranteed minimum rate up to the next 0.1 percentage points.""" + cleaned = Decimal(repr(rate)).quantize(RATIO_PRECISION, ROUND_HALF_UP) + return float(cleaned.quantize(PUBLISHED_PRECISION, ROUND_CEILING)) + - values = {} +@dataclass(frozen=True) +class UpratingYear: + """Inputs to the April uprating in one year.""" - for year in YEARS: - if year in outturn_years: - triple_lock_increase = triple_lock.outturn(year) + earnings: float + cpi: float + minimum_rate: float + active: bool = True + include_earnings: bool = True + include_inflation: bool = True + earnings_path_guarantee: bool = False + outturn: Optional[float] = None + + +def triple_lock_rate( + earnings: float, + cpi: float, + minimum_rate: float, + include_earnings: bool = True, + include_inflation: bool = True, +) -> float: + """The highest of the included elements and the minimum rate.""" + candidates = [minimum_rate] + if include_earnings: + candidates.append(earnings) + if include_inflation: + candidates.append(cpi) + return max(candidates) + + +def rule_rate(inputs: UpratingYear) -> float: + """The year's rate before any earnings-path top-up or override.""" + if not inputs.active: + # s150A: at least the rise in earnings; no rise when earnings fall. + return max(inputs.earnings, 0.0) + return triple_lock_rate( + inputs.earnings, + inputs.cpi, + inputs.minimum_rate, + inputs.include_earnings, + inputs.include_inflation, + ) + + +def uprating_rates(years: Dict[int, UpratingYear]) -> Dict[int, float]: + """Uprating rate for each year, in year order. + + The earnings-path guarantee makes the rule path dependent. When it first + applies, it anchors an earnings path at the pension's level in the year + before; each guaranteed year the path grows by that year's earnings input, + and the rate is raised, rounding up, to keep the pension on or above it. + The path is dropped in any year the guarantee is off and re-anchored if it + returns. + """ + rates = {} + level = 1.0 + earnings_path = None + for year in sorted(years): + inputs = years[year] + rate = inputs.outturn if inputs.outturn is not None else rule_rate(inputs) + if inputs.earnings_path_guarantee: + if earnings_path is None: + earnings_path = level + earnings_path *= 1 + inputs.earnings + if inputs.outturn is None: + rate = max( + rate, + round_up_to_published_precision(earnings_path / level - 1), + ) else: - earnings_increase = average_earnings(year - 1) - cpi_increase = cpi(year - 1) - min_rate_y = min_rate(year) - if triple_lock.include_earnings(year) and triple_lock.include_inflation( - year - ): - triple_lock_increase = max(earnings_increase, cpi_increase, min_rate_y) - elif triple_lock.include_earnings(year): - triple_lock_increase = max(earnings_increase, min_rate_y) - elif triple_lock.include_inflation(year): - triple_lock_increase = max(cpi_increase, min_rate_y) - else: - triple_lock_increase = min_rate_y - values[f"{year}-01-01"] = round(triple_lock_increase, 3) + earnings_path = None + rates[year] = rate + level *= 1 + rate + return rates + +def _flag(parameter: Parameter, instant: str) -> bool: + value = parameter(instant) + return bool(value) if value is not None else False + + +def uprating_years(parameters: ParameterNode) -> Iterable[int]: + """Uprating years covered: from April 2011 to one year past the inputs.""" + return range(FIRST_UPRATING_YEAR, last_input_year(parameters) + 2) + + +def read_uprating_years(parameters: ParameterNode) -> Dict[int, UpratingYear]: + """Collect each year's rule inputs from a parameter tree.""" + inputs = parameters.gov.economic_assumptions.statutory_uprating_inputs + triple_lock = parameters.gov.dwp.state_pension.triple_lock + + years = {} + for year in uprating_years(parameters): + review_year = year - 1 + instant = uprating_instant(year) + earnings = inputs.awe_total_pay_may_july( + f"{review_year}-{AWE_OBSERVATION_MONTH_DAY}" + ) + cpi = inputs.cpi_september(f"{review_year}-{CPI_OBSERVATION_MONTH_DAY}") + # Null outside the overridden years. + outturn = triple_lock.outturn(instant) + if outturn is None and (earnings is None or cpi is None): + raise ValueError( + f"Missing statutory uprating input for April {year}: " + f"May-July {review_year} earnings = {earnings}, " + f"September {review_year} CPI = {cpi}." + ) + years[year] = UpratingYear( + earnings=round_to_published_precision(earnings or 0), + cpi=round_to_published_precision(cpi or 0), + minimum_rate=triple_lock.minimum_rate(instant), + active=_flag(triple_lock.active, instant), + include_earnings=_flag(triple_lock.include_earnings, instant), + include_inflation=_flag(triple_lock.include_inflation, instant), + earnings_path_guarantee=_flag(triple_lock.earnings_path_guarantee, instant), + outturn=outturn, + ) + return years + + +def add_triple_lock(parameters: ParameterNode) -> ParameterNode: + """Add ``gov.economic_assumptions.yoy_growth.triple_lock``.""" + rates = uprating_rates(read_uprating_years(parameters)) new_parameter = Parameter( "gov.economic_assumptions.yoy_growth.triple_lock", - data={"values": values, "metadata": {"unit": "/1"}}, + data={ + "description": ( + "State Pension uprating taking effect in April of each year." + ), + "values": {f"{year}-01-01": rate for year, rate in rates.items()}, + "metadata": { + "unit": "/1", + "label": "State Pension uprating rate", + }, + }, ) parameters.gov.economic_assumptions.yoy_growth.add_child( "triple_lock", new_parameter diff --git a/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/earnings_path_guarantee.yaml b/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/earnings_path_guarantee.yaml new file mode 100644 index 0000000000..0ab5e1c258 --- /dev/null +++ b/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/earnings_path_guarantee.yaml @@ -0,0 +1,21 @@ +description: Whether the State Pension also rises at least enough to stay on an earnings path, started from its level in the year before this guarantee first applies and uprated each year by the May-July earnings input. +values: + 2010-01-01: false +metadata: + unit: bool + label: State Pension earnings path guarantee + documentation: | + Not part of current law. It lets reforms that replace the triple lock + with a lower annual floor, but promise that the pension keeps pace with + earnings over time, be modelled as parameter changes. The plan announced + on 29 September 2026 drops the annual earnings link from April 2030 but + says the pension will hold its value relative to earnings over time. The + speech gave no formula; one reading (rises of at least CPI or 2.5%, and + the pension never below an earnings link from its 2029-30 level) is + include_earnings false and this guarantee true from 2030. + Each guaranteed year, the rise is the higher of the triple lock + elements still included and the rise needed to reach the earnings path, + rounded up to 0.1 percentage points. + reference: + - title: BBC News live, Labour conference speech, 29 September 2026 + href: https://www.bbc.co.uk/news/live/c6x2zrv774gvt diff --git a/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/include_earnings.yaml b/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/include_earnings.yaml index f8f0e838cd..88592646d7 100644 --- a/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/include_earnings.yaml +++ b/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/include_earnings.yaml @@ -1,8 +1,14 @@ description: Whether the triple lock should include average earnings. values: - 2012-01-01: true + 2011-01-01: true + # April 2022: the review for 2022-23 used prices instead of earnings, with + # a 2.5% minimum, under the Social Security (Up-rating of Benefits) Act + # 2021. 2022-01-01: false 2023-01-01: true metadata: unit: bool - label: Include average earnings in triple lock \ No newline at end of file + label: Include average earnings in triple lock + reference: + - title: Social Security (Up-rating of Benefits) Act 2021, section 1 + href: https://www.legislation.gov.uk/ukpga/2021/32/section/1/enacted diff --git a/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/include_inflation.yaml b/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/include_inflation.yaml index 84a2d176e9..cd825015f6 100644 --- a/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/include_inflation.yaml +++ b/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/include_inflation.yaml @@ -1,6 +1,6 @@ description: Whether the triple lock should include inflation. values: - 2012-01-01: true + 2011-01-01: true metadata: unit: bool label: Include inflation in triple lock \ No newline at end of file diff --git a/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/outturn.yaml b/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/outturn.yaml index 315a55e5dc..ef7e13aaed 100644 --- a/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/outturn.yaml +++ b/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/outturn.yaml @@ -1,21 +1,14 @@ -description: Published triple lock uprating rates (outturn or DWP-announced). Used in preference to the max-of(earnings, CPI, minimum) formula because the formula relies on calendar-year OBR averages, while the actual uprating uses May-July AWE growth. +description: Published State Pension upratings that the rule cannot reproduce from the statutory inputs, used in place of the rule in those years. values: - 2022-01-01: 0.031 - 2023-01-01: 0.101 - 2024-01-01: 0.085 - 2025-01-01: 0.041 - 2026-01-01: 0.048 + # April 2011: the basic State Pension rose by September 2010 RPI (4.6%), + # above CPI (3.1%) and earnings (1.3%), a one-off guarantee during the + # switch from RPI to CPI. + 2011-01-01: 0.046 + # No other year departs from the rule. + 2012-01-01: null metadata: unit: /1 label: Triple lock outturn rate reference: - - title: DWP benefit and pension rates 2022 to 2023 - href: https://www.gov.uk/government/publications/benefit-and-pension-rates-2022-to-2023 - - title: DWP benefit and pension rates 2023 to 2024 - href: https://www.gov.uk/government/publications/benefit-and-pension-rates-2023-to-2024 - - title: DWP benefit and pension rates 2024 to 2025 - href: https://www.gov.uk/government/publications/benefit-and-pension-rates-2024-to-2025 - - title: DWP benefit and pension rates 2025 to 2026 - href: https://www.gov.uk/government/publications/benefit-and-pension-rates-2025-to-2026 - - title: DWP benefit and pension rates 2026 to 2027 - href: https://www.gov.uk/government/news/over-12-million-pensioners-to-receive-575-state-pension-boost + - title: Explanatory Memorandum to the Social Security Benefits Up-rating Order 2011 (SI 2011/821), paragraph 7.4 + href: https://www.legislation.gov.uk/uksi/2011/821/contents/made diff --git a/policyengine_uk/parameters/gov/economic_assumptions/README.md b/policyengine_uk/parameters/gov/economic_assumptions/README.md index 5418feddc2..7effffc4be 100644 --- a/policyengine_uk/parameters/gov/economic_assumptions/README.md +++ b/policyengine_uk/parameters/gov/economic_assumptions/README.md @@ -58,6 +58,28 @@ new Fiscal Risks and Sustainability report. The convergence path itself is conservative interpolation and is not load-bearing for short-horizon analysis. +## Statutory uprating inputs + +[`statutory_uprating_inputs/`](./statutory_uprating_inputs/) holds the two +published statistics that set the April uprating of the State Pension: +September CPI (ONS D7G7) and May-July average weekly earnings total pay +growth (ONS KAC3). They are keyed to their observation month, not to +1 January, and carry `preserve_calendar_dates`. + +Each series holds published figures and then a null. From there, +[`create_statutory_uprating_inputs.py`](./create_statutory_uprating_inputs.py) +fills in calendar-year growth from `yoy_growth.obr` plus `forecast_gap`, the +OBR's forecast of the statutory measure minus its calendar-year forecast. +The gap is zero after the EFO horizon, so the forecasts fall back to +calendar-year growth. A scenario that edits calendar-year growth therefore +moves them, and a scenario can also set them directly. + +After each release, add the new published figure in place of the null and +move the null one year on: September CPI in October, May-July earnings in +September. Then replace May-July earnings with the October release's figure, +which the review uses: the April 2025 and April 2026 rises used October's +4.1% and 4.8%, where September's first estimates were 4.0% and 4.7%. + ## Refreshing after a new EFO 1. Replace the 2025-2030 block in each series with values from the new EFO @@ -68,5 +90,9 @@ analysis. to regenerate the cumulative `indices/` parameters that uprating depends on. 4. Update the EFO reference in each series' `metadata.reference`. +5. Regenerate `statutory_uprating_inputs/forecast_gap/` from the same EFO. + `policyengine_uk/utils/import_obr_forecasts.py` does this when it updates + `yoy_growth.yaml`; pass the receipts tables with `--receipts-file` or + `--receipts-url` so September CPI uses the OBR's September forecast. [rpi-cpi]: https://obr.uk/box/the-long-run-difference-between-rpi-and-cpi-inflation/ diff --git a/policyengine_uk/parameters/gov/economic_assumptions/create_statutory_uprating_inputs.py b/policyengine_uk/parameters/gov/economic_assumptions/create_statutory_uprating_inputs.py new file mode 100644 index 0000000000..d542f09f30 --- /dev/null +++ b/policyengine_uk/parameters/gov/economic_assumptions/create_statutory_uprating_inputs.py @@ -0,0 +1,76 @@ +"""Complete the statutory uprating inputs with forecasts. + +Two published statistics set the April uprating of the State Pension: + +- ``cpi_september``: the CPI 12-month rate in September (ONS D7G7). +- ``awe_total_pay_may_july``: average weekly earnings, total pay, whole + economy, the three months May to July on a year earlier, as first published + (ONS KAC3). + +Each is keyed to its observation month (1 September, 1 July) and carries +``preserve_calendar_dates`` so fiscal-year conversion leaves the dates alone. +The YAML holds the published figures and then a null: from there on, values +are forecasts. + +For a year without a published figure, the forecast is calendar-year growth +in the matching series of ``gov.economic_assumptions.yoy_growth.obr`` plus +``forecast_gap``: the OBR's forecast of the statutory-basis quarter minus its +calendar-year forecast, both from the same Economic and Fiscal Outlook. The +gap is zero after the EFO horizon, so the forecast falls back to +calendar-year growth. A macro scenario that edits calendar-year growth moves +the inputs one for one; a scenario that sets an input directly replaces the +forecast for the years it covers. +""" + +from policyengine_core.parameters import Parameter, ParameterNode + +CPI_OBSERVATION_MONTH_DAY = "09-01" +AWE_OBSERVATION_MONTH_DAY = "07-01" + +# Statutory input: (calendar-year series in yoy_growth.obr, observation date). +STATUTORY_INPUTS = { + "cpi_september": ("consumer_price_index", CPI_OBSERVATION_MONTH_DAY), + "awe_total_pay_may_july": ("average_earnings", AWE_OBSERVATION_MONTH_DAY), +} + + +def _last_year(parameter: Parameter) -> int: + return max( + int(value.instant_str[:4]) + for value in parameter.values_list + if value.value is not None + ) + + +def _first_year(parameter: Parameter) -> int: + return min(int(value.instant_str[:4]) for value in parameter.values_list) + + +def last_input_year(parameters: ParameterNode) -> int: + """Last year covered by the calendar-year series behind the inputs.""" + obr = parameters.gov.economic_assumptions.yoy_growth.obr + return min( + _last_year(getattr(obr, series)) for series, _ in STATUTORY_INPUTS.values() + ) + + +def add_statutory_uprating_inputs(parameters: ParameterNode) -> ParameterNode: + """Fill each input's unpublished years with its forecast. + + Runs after any scenario has edited the raw parameters and before the + uprating rules read the inputs. + """ + inputs = parameters.gov.economic_assumptions.statutory_uprating_inputs + obr = parameters.gov.economic_assumptions.yoy_growth.obr + end_year = last_input_year(parameters) + for name, (series, month_day) in STATUTORY_INPUTS.items(): + parameter = getattr(inputs, name) + gap = getattr(inputs.forecast_gap, name) + calendar_growth = getattr(obr, series) + for year in range(_first_year(parameter), end_year + 1): + observed = f"{year}-{month_day}" + if parameter(observed) is not None: + continue + forecast = calendar_growth(f"{year}-01-01") + (gap(observed) or 0) + parameter.update(period=f"year:{observed}:1", value=forecast) + return parameters diff --git a/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/awe_total_pay_may_july.yaml b/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/awe_total_pay_may_july.yaml new file mode 100644 index 0000000000..e7db53d4a0 --- /dev/null +++ b/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/awe_total_pay_may_july.yaml @@ -0,0 +1,47 @@ +description: Growth in average weekly earnings (total pay, whole economy) in May to July on a year earlier, the figure the Secretary of State uses in the review that sets the April uprating of the State Pension the following year. +values: + # Outturn: the figure each year's review used, as stated in the + # Explanatory Memorandum to the following April's Social Security Benefits + # Up-rating Order (for 2021, the Government Actuary's report on the 2022 + # Order). Reviews use the October labour market release, which the + # explanatory notes to the 2021 Up-rating of Benefits Act call the final + # figure. It can differ from the September first estimate (May-July 2024: + # 4.0% in September, 4.1% in October and used) and from the latest ONS + # vintage of KAC3 (4.4% for May-July 2024). + 2010-07-01: 0.013 # SI 2011/821 EM, footnote 7 + 2011-07-01: 0.028 # SI 2012/780 EM + 2012-07-01: 0.016 # SI 2013/574 EM + 2013-07-01: 0.012 # SI 2014/516 EM + 2014-07-01: 0.006 # SI 2015/457 EM + 2015-07-01: 0.029 # SI 2016/230 EM + 2016-07-01: 0.024 # SI 2017/260 EM + 2017-07-01: 0.022 # SI 2018/281 EM + 2018-07-01: 0.026 # SI 2019/480 EM + 2019-07-01: 0.039 # SI 2020/234 EM + 2020-07-01: -0.010 # SI 2021/162 EM + 2021-07-01: 0.083 # GAD report on the 2022 Order (the earnings element was suspended) + 2022-07-01: 0.055 # SI 2023/316 EM + 2023-07-01: 0.085 # SI 2024/242 EM + 2024-07-01: 0.041 # SI 2025/295 EM + 2025-07-01: 0.048 # SI 2026/148 EM + # First estimate (ONS labour market release, 15 September 2026). Replace + # with the October release, due 20 October 2026, once published. + 2026-07-01: 0.039 + # Not yet published. From here the value is calendar-year earnings growth + # (yoy_growth.obr.average_earnings) plus forecast_gap.awe_total_pay_may_july. + 2027-07-01: null +metadata: + unit: /1 + label: May-July earnings growth + preserve_calendar_dates: true + reference: + - title: ONS KAC3, AWE whole economy total pay, three-month average growth on a year earlier + href: https://www.ons.gov.uk/employmentandlabourmarket/peopleinwork/earningsandworkinghours/timeseries/kac3/lms + - title: Explanatory Memorandum to the Social Security Benefits Up-rating Order 2026 (SI 2026/148) + href: https://www.legislation.gov.uk/uksi/2026/148/contents/made + - title: Explanatory Memorandum to the Social Security Benefits Up-rating Order 2025 (SI 2025/295) + href: https://www.legislation.gov.uk/uksi/2025/295/contents/made + - title: Explanatory Notes to the Social Security (Up-rating of Benefits) Act 2021, paragraph 4 + href: https://www.legislation.gov.uk/ukpga/2021/32/pdfs/ukpgaen_20210032_en.pdf + - title: Government Actuary's Department, report on the Social Security Benefits Up-rating Order 2022 + href: https://assets.publishing.service.gov.uk/media/61e59a0a8fa8f5058bc04a51/E02705826_Un_Act_GAD_NIF_Uprating_Report_Jan22_Web_Accessible_-_UPLOADED_GOV_UK.pdf diff --git a/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/cpi_september.yaml b/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/cpi_september.yaml new file mode 100644 index 0000000000..9925e29396 --- /dev/null +++ b/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/cpi_september.yaml @@ -0,0 +1,34 @@ +description: CPI 12-month rate in September, the price input to the April uprating of the State Pension and most benefits the following year. +values: + # Outturn: ONS D7G7, as published. ONS does not amend published CPI + # (its last revisions were in 2000 and 2006), so these are the figures + # each review used. Store the published rate: rates derived from the + # 1dp index (D7BT) round differently in some years. + 2010-09-01: 0.031 + 2011-09-01: 0.052 + 2012-09-01: 0.022 + 2013-09-01: 0.027 + 2014-09-01: 0.012 + 2015-09-01: -0.001 + 2016-09-01: 0.010 + 2017-09-01: 0.030 + 2018-09-01: 0.024 + 2019-09-01: 0.017 + 2020-09-01: 0.005 + 2021-09-01: 0.031 + 2022-09-01: 0.101 + 2023-09-01: 0.067 + 2024-09-01: 0.017 + 2025-09-01: 0.038 + # Not yet published. From here the value is calendar-year CPI growth + # (yoy_growth.obr.consumer_price_index) plus forecast_gap.cpi_september. + 2026-09-01: null +metadata: + unit: /1 + label: September CPI inflation + preserve_calendar_dates: true + reference: + - title: ONS D7G7, CPI annual rate, all items + href: https://www.ons.gov.uk/economy/inflationandpriceindices/timeseries/d7g7/mm23 + - title: ONS, Revisions and correction of errors policies for consumer price inflation statistics (December 2025) + href: https://www.ons.gov.uk/economy/inflationandpriceindices/articles/revisionsandcorrectionoferrorspoliciesforconsumerpriceinflationstatistics/december2025 diff --git a/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/forecast_gap/awe_total_pay_may_july.yaml b/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/forecast_gap/awe_total_pay_may_july.yaml new file mode 100644 index 0000000000..dd1b9ca4dd --- /dev/null +++ b/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/forecast_gap/awe_total_pay_may_july.yaml @@ -0,0 +1,28 @@ +description: Forecast difference between May-July earnings growth and calendar-year earnings growth, added to calendar-year growth in years without a published May-July figure. +values: + # No gap applies to published years. + 2010-07-01: 0 + # OBR EFO March 2026: Q2 average earnings growth on a year earlier minus calendar-year growth (economy Table 1.6). + 2025-07-01: 0.00254 + 2026-07-01: 0.00316 + 2027-07-01: 0.00065 + 2028-07-01: -0.00015 + 2029-07-01: -0.00022 + 2030-07-01: -0.00017 + # After the EFO horizon: no gap, so the forecast is calendar-year growth. + 2031-07-01: 0 +metadata: + unit: /1 + label: May-July earnings forecast gap to calendar-year earnings + preserve_calendar_dates: true + documentation: | + The OBR does not forecast the ONS AWE series. Its average earnings + measure is wages and salaries per employee from the national accounts, + forecast by quarter; the gap uses Q2 (April to June), the quarter + nearest May to July. It is taken from the same Economic and Fiscal + Outlook as the calendar-year series and regenerated by + policyengine_uk/utils/import_obr_forecasts.py. It is zero after the EFO + horizon, where the May-July forecast falls back to calendar-year growth. + reference: + - title: OBR EFO March 2026 (detailed forecast tables, economy, Table 1.6) + href: https://obr.uk/efo/economic-and-fiscal-outlook-march-2026/ diff --git a/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/forecast_gap/cpi_september.yaml b/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/forecast_gap/cpi_september.yaml new file mode 100644 index 0000000000..a387dcf357 --- /dev/null +++ b/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/forecast_gap/cpi_september.yaml @@ -0,0 +1,31 @@ +description: Forecast difference between September CPI inflation and calendar-year CPI inflation, added to calendar-year growth in years without a published September figure. +values: + # No gap applies to published years. + 2010-09-01: 0 + # OBR EFO March 2026: September CPI (receipts Table 3.19) minus calendar-year CPI (economy Table 1.7). + 2025-09-01: 0.00438 + 2026-09-01: -0.00185 + 2027-09-01: 0.00103 + 2028-09-01: 0.00053 + 2029-09-01: 0.00028 + # OBR EFO March 2026: Q3 CPI 12-month rate minus calendar-year CPI inflation (economy Table 1.7). + 2030-09-01: 0 + # After the EFO horizon: no gap, so the forecast is calendar-year growth. + 2031-09-01: 0 +metadata: + unit: /1 + label: September CPI forecast gap to calendar-year CPI + preserve_calendar_dates: true + documentation: | + The OBR publishes September CPI forecasts in its receipts tables, as + the CPI used to uprate tax thresholds. Where that row stops before the + end of the forecast, the gap uses the quarter containing September (Q3 + CPI, economy Table 1.7). Both come from the same Economic and Fiscal + Outlook as the calendar-year series, and are regenerated by + policyengine_uk/utils/import_obr_forecasts.py. The gap is zero after the + EFO horizon, where the September forecast falls back to calendar-year CPI. + reference: + - title: OBR EFO March 2026 (detailed forecast tables, economy, Table 1.7) + href: https://obr.uk/efo/economic-and-fiscal-outlook-march-2026/ + - title: OBR EFO March 2026 (detailed forecast tables, receipts, Table 3.19) + href: https://obr.uk/efo/economic-and-fiscal-outlook-march-2026/ diff --git a/policyengine_uk/scenarios/no_economic_assumptions.py b/policyengine_uk/scenarios/no_economic_assumptions.py index 55a60abbe3..61623d7e43 100644 --- a/policyengine_uk/scenarios/no_economic_assumptions.py +++ b/policyengine_uk/scenarios/no_economic_assumptions.py @@ -9,16 +9,22 @@ def remove_economic_assumptions(simulation: Simulation): simulation.tax_benefit_system.reset_parameters() cutoff = f"{simulation.default_input_period}-01-01" - yoy_growth = ( - simulation.tax_benefit_system.parameters.gov.economic_assumptions.yoy_growth + economic_assumptions = ( + simulation.tax_benefit_system.parameters.gov.economic_assumptions ) - for parameter in yoy_growth.get_descendants(): - if not isinstance(parameter, Parameter): - continue - for value_at_instant in parameter.values_list: - if value_at_instant.instant_str >= cutoff: - value_at_instant.value = 0 + # Growth series and the statutory uprating inputs derived from growth + # (September CPI and May-July earnings, including their forecast gaps). + for node in ( + economic_assumptions.yoy_growth, + economic_assumptions.statutory_uprating_inputs, + ): + for parameter in node.get_descendants(): + if not isinstance(parameter, Parameter): + continue + for value_at_instant in parameter.values_list: + if value_at_instant.instant_str >= cutoff: + value_at_instant.value = 0 simulation.tax_benefit_system.process_parameters() diff --git a/policyengine_uk/tax_benefit_system.py b/policyengine_uk/tax_benefit_system.py index e9339b7453..d632be9dd9 100644 --- a/policyengine_uk/tax_benefit_system.py +++ b/policyengine_uk/tax_benefit_system.py @@ -24,6 +24,9 @@ from policyengine_uk.parameters.gov.economic_assumptions.create_economic_assumption_indices import ( create_economic_assumption_indices, ) +from policyengine_uk.parameters.gov.economic_assumptions.create_statutory_uprating_inputs import ( + add_statutory_uprating_inputs, +) from policyengine_uk.parameters.gov.economic_assumptions.lag_average_earnings import ( add_lagged_earnings, ) @@ -84,6 +87,7 @@ def process_parameters(self) -> None: Applies various parameter transformations including: - Private pension uprating factors - Lagged earnings and CPI indices + - Statutory uprating inputs (September CPI, May-July earnings) - Triple lock calculations for state pensions - Economic assumption indices - Parameter uprating and backdating @@ -94,6 +98,7 @@ def process_parameters(self) -> None: self.parameters = add_private_pension_uprating_factor(self.parameters) self.parameters = add_lagged_earnings(self.parameters) self.parameters = add_lagged_cpi(self.parameters) + self.parameters = add_statutory_uprating_inputs(self.parameters) self.parameters = add_triple_lock(self.parameters) self.parameters = create_economic_assumption_indices(self.parameters) self.parameters = add_lsr_deprecation_aliases(self.parameters) diff --git a/policyengine_uk/tests/test_import_obr_forecasts.py b/policyengine_uk/tests/test_import_obr_forecasts.py index 541e0d655d..2934cb5851 100644 --- a/policyengine_uk/tests/test_import_obr_forecasts.py +++ b/policyengine_uk/tests/test_import_obr_forecasts.py @@ -2,10 +2,14 @@ from zipfile import ZipFile from policyengine_uk.utils.import_obr_forecasts import ( + STATUTORY_GAP_SPECS, build_efo_href, + compute_statutory_forecast_gaps, extract_annual_series_from_xlsx, + extract_september_cpi_from_receipts, infer_forecast_start_year, infer_release, + update_forecast_gap_yaml, update_yoy_growth_yaml, ) @@ -61,6 +65,10 @@ def make_test_xlsx() -> bytes: make_inline_cell("B98", "2026"), make_number_cell("Q98", 3.33), ], + 99: [ + make_inline_cell("B99", "2026Q2"), + make_number_cell("Q99", 3.67), + ], } ) sheet_17 = make_sheet( @@ -94,6 +102,14 @@ def make_test_xlsx() -> bytes: make_number_cell("H99", 7.97), make_number_cell("I99", 3.34), ], + 100: [ + make_inline_cell("B100", "2026Q3"), + make_number_cell("C100", 3.2), + make_number_cell("E100", 2.08), + make_number_cell("F100", 2.3), + make_number_cell("H100", 8.0), + make_number_cell("I100", 3.3), + ], } ) sheet_116 = make_sheet( @@ -328,3 +344,110 @@ def test_update_yoy_growth_yaml_keeps_existing_values_when_obr_has_blank_years( assert "2025-01-01: 0.0952" in content assert "2026-01-01: 0.0797" in content assert "2027-01-01: 0.0553" in content + + +def make_receipts_xlsx() -> bytes: + workbook_xml = """ + + + + + + +""" + rels_xml = """ + + + + +""" + sheet_318 = make_sheet({2: [make_inline_cell("B2", "Other table")]}) + sheet_319 = make_sheet( + { + 4: [ + make_inline_cell("C4", "2026-27"), + make_inline_cell("D4", "2027-28"), + ], + 21: [ + make_inline_cell("B21", "Memo: CPI used to uprate thresholds"), + make_number_cell("C21", 3.81), + make_number_cell("D21", 2.12), + ], + } + ) + buffer = BytesIO() + with ZipFile(buffer, "w") as archive: + archive.writestr("xl/workbook.xml", workbook_xml) + archive.writestr("xl/_rels/workbook.xml.rels", rels_xml) + archive.writestr("xl/worksheets/sheet1.xml", sheet_318) + archive.writestr("xl/worksheets/sheet2.xml", sheet_319) + return buffer.getvalue() + + +def test_september_cpi_is_keyed_to_the_september_before_the_uprating_year(): + table, rates = extract_september_cpi_from_receipts(make_receipts_xlsx()) + + assert table == "3.19" + assert rates == {2025: 0.0381, 2026: 0.0212} + + +def test_statutory_forecast_gaps_use_the_statutory_quarter(): + gaps = compute_statutory_forecast_gaps(make_test_xlsx(), 2025, 2) + + # Q3 CPI 2.08% minus calendar-year 2.48%; no 2025 quarter in the fixture. + assert {y: g.value for y, g in gaps["cpi_september"].items()} == {2026: -0.004} + # Q2 earnings 3.67% minus calendar-year 3.33%. + assert {y: g.value for y, g in gaps["awe_total_pay_may_july"].items()} == { + 2026: 0.0034 + } + + +def test_statutory_forecast_gaps_prefer_published_september_cpi(): + gaps = compute_statutory_forecast_gaps( + make_test_xlsx(), 2025, 2, receipts_xlsx_bytes=make_receipts_xlsx() + ) + + # September 2025 (3.81%) minus calendar-year 2025 CPI (3.45%), and + # September 2026 (2.12%) minus calendar-year 2026 CPI (2.48%). + cpi = gaps["cpi_september"] + assert {y: g.value for y, g in cpi.items()} == {2025: 0.0036, 2026: -0.0036} + assert "receipts Table 3.19" in cpi[2026].source + + +def test_update_forecast_gap_yaml_rewrites_values_and_reference(tmp_path): + yaml_path = tmp_path / "cpi_september.yaml" + yaml_path.write_text("""description: Gap. +values: + 2010-09-01: 0 + 2025-09-01: 0.001 + 2031-09-01: 0 +metadata: + unit: /1 + documentation: | + Kept. + reference: + - title: OBR EFO November 2025 (detailed forecast tables, economy, Table 1.7) + href: https://obr.uk/efo/economic-and-fiscal-outlook-november-2025/ +""") + spec = next(s for s in STATUTORY_GAP_SPECS if s.key == "cpi_september") + gaps = compute_statutory_forecast_gaps( + make_test_xlsx(), 2025, 2, receipts_xlsx_bytes=make_receipts_xlsx() + )["cpi_september"] + + update_forecast_gap_yaml(yaml_path, spec, gaps, "March", 2026) + + content = yaml_path.read_text() + assert "2025-09-01: 0.0036" in content + assert "2026-09-01: -0.0036" in content + assert "2027-09-01: 0" in content + assert "0.001" not in content + assert "Kept." in content + assert "November 2025" not in content + assert ( + "OBR EFO March 2026 (detailed forecast tables, economy, Table 1.7)" in content + ) + assert ( + "OBR EFO March 2026 (detailed forecast tables, receipts, Table 3.19)" in content + ) + assert "https://obr.uk/efo/economic-and-fiscal-outlook-march-2026/" in content diff --git a/policyengine_uk/tests/test_triple_lock_outturn.py b/policyengine_uk/tests/test_triple_lock_outturn.py index cb20e43ac7..d6d167490b 100644 --- a/policyengine_uk/tests/test_triple_lock_outturn.py +++ b/policyengine_uk/tests/test_triple_lock_outturn.py @@ -1,24 +1,49 @@ -"""Verify the triple lock and downstream state pension amounts match published -DWP / OBR uprating outturn for years where the actual rate is known. +"""Differential test: the triple lock computed from the statutory inputs +against every published State Pension uprating from April 2011. -Issue #953: previously the triple lock was computed from OBR calendar-year -average earnings growth, which differs from the May-July AWE window DWP uses -for the actual triple lock. That left every uprating year ~0.5-1.2pp off. +The inputs are the September CPI rate and the May-July earnings growth +figure each review used (from the Explanatory Memorandum to the following +April's Up-rating Order). The rule reproduces every published rise from +April 2012 to April 2026, and every published weekly rate once rounded to +the nearest 5p. One year needs an override: in April 2011 the basic State +Pension rose by September 2010 RPI (4.6%), a one-off guarantee during the +switch from RPI to CPI, where the rule gives CPI (3.1%). The April 2022 +suspension of the earnings element under the Social Security (Up-rating of +Benefits) Act 2021 is ``include_earnings`` false, not an override. + +Issue #953: the rule previously used OBR calendar-year growth, which left +every uprating year 0.5-1.2pp off. """ +from decimal import ROUND_HALF_UP, Decimal + import pytest +from policyengine_uk.parameters.gov.dwp.state_pension.triple_lock.create_triple_lock import ( + read_uprating_years, + rule_rate, +) from policyengine_uk.system import system - -# (year, expected uprating rate). Sources are linked in -# parameters/gov/dwp/state_pension/triple_lock/outturn.yaml. -OUTTURN_RATES = [ - (2022, 0.031), - (2023, 0.101), - (2024, 0.085), - (2025, 0.041), - (2026, 0.048), +# (April, rise, element that set it, Up-rating Order). Rates from the +# Explanatory Memorandum to each Order. +PUBLISHED_UPRATINGS = [ + (2011, 0.046, "rpi", "SI 2011/821"), + (2012, 0.052, "cpi", "SI 2012/780"), + (2013, 0.025, "floor", "SI 2013/574"), + (2014, 0.027, "cpi", "SI 2014/516"), + (2015, 0.025, "floor", "SI 2015/457"), + (2016, 0.029, "earnings", "SI 2016/230"), + (2017, 0.025, "floor", "SI 2017/260"), + (2018, 0.030, "cpi", "SI 2018/281"), + (2019, 0.026, "earnings", "SI 2019/480"), + (2020, 0.039, "earnings", "SI 2020/234"), + (2021, 0.025, "floor", "SI 2021/162"), + (2022, 0.031, "cpi", "SI 2022/292"), + (2023, 0.101, "cpi", "SI 2023/316"), + (2024, 0.085, "earnings", "SI 2024/242"), + (2025, 0.041, "earnings", "SI 2025/295"), + (2026, 0.048, "earnings", "SI 2026/148"), ] # (year, expected weekly £). Cross-referenced against gov.uk benefit and @@ -35,28 +60,95 @@ (2026, 241.30), ] +parameters = system.parameters +state_pension = parameters.gov.dwp.state_pension +UPRATING_YEARS = read_uprating_years(parameters) + + +def uprating(year): + return parameters.gov.economic_assumptions.yoy_growth.triple_lock(f"{year}-01-01") -@pytest.mark.parametrize("year, expected", OUTTURN_RATES) -def test_triple_lock_yoy_matches_published_outturn(year, expected): - """The generated triple lock yoy parameter should equal the DWP-announced - rate for years where outturn is known.""" - yoy = system.parameters.gov.economic_assumptions.yoy_growth.triple_lock( - f"{year}-01-01" + +def nearest_5p(amount): + return float( + (Decimal(repr(amount)) / Decimal("0.05")).quantize(Decimal(1), ROUND_HALF_UP) + * Decimal("0.05") ) - assert yoy == pytest.approx(expected, abs=1e-3) + + +@pytest.mark.parametrize("year, rate, element, order", PUBLISHED_UPRATINGS) +def test_uprating_matches_published_rate(year, rate, element, order): + assert uprating(year) == pytest.approx(rate, abs=1e-9), order + + +@pytest.mark.parametrize( + "year, rate, element, order", + [row for row in PUBLISHED_UPRATINGS if row[2] != "rpi"], +) +def test_rule_reproduces_the_rate_from_the_statutory_inputs(year, rate, element, order): + """No override: the rule alone gives the published rate, set by the + published element.""" + inputs = UPRATING_YEARS[year] + assert inputs.outturn is None + assert rule_rate(inputs) == pytest.approx(rate, abs=1e-9) + value_by_element = { + "earnings": inputs.earnings if inputs.include_earnings else None, + "cpi": inputs.cpi if inputs.include_inflation else None, + "floor": inputs.minimum_rate, + } + assert value_by_element[element] == pytest.approx(rate, abs=1e-9) + + +def test_only_april_2011_needs_an_override(): + overridden = [year for year, inputs in UPRATING_YEARS.items() if inputs.outturn] + assert overridden == [2011] + april_2011 = UPRATING_YEARS[2011] + # The rule would pay September 2010 CPI; the basic State Pension rose by RPI. + assert rule_rate(april_2011) == pytest.approx(0.031) + assert april_2011.outturn == pytest.approx(0.046) + + +@pytest.mark.parametrize( + "amount, first_year", + [ + (state_pension.basic_state_pension.amount, 2011), + # The new State Pension started in April 2016; the triple lock has + # applied to it since April 2017. + (state_pension.new_state_pension.amount, 2017), + ], + ids=["basic", "new"], +) +def test_published_weekly_rates_are_the_uprated_rate_to_the_nearest_5p( + amount, first_year +): + for year in range(first_year, 2027): + previous = amount(f"{year - 1}-06-01") + published = amount(f"{year}-06-01") + assert nearest_5p(previous * (1 + uprating(year))) == pytest.approx( + published, abs=1e-9 + ), year + + +def test_triple_lock_is_never_below_the_statutory_minimum(): + """Social Security Administration Act 1992 s150A requires a rise of at + least earnings growth when earnings rise. For April 2022 the Social + Security (Up-rating of Benefits) Act 2021 required at least the higher of + CPI and 2.5% instead.""" + for year, inputs in UPRATING_YEARS.items(): + if year == 2022: + statutory_minimum = max(inputs.cpi, 0.025) + else: + statutory_minimum = max(inputs.earnings, 0.0) + assert uprating(year) >= statutory_minimum - 1e-12, year @pytest.mark.parametrize("year, expected", BASIC_STATE_PENSION_WEEKLY) def test_basic_state_pension_matches_published_rate(year, expected): - weekly = system.parameters.gov.dwp.state_pension.basic_state_pension.amount( - f"{year}-04-01" - ) + weekly = state_pension.basic_state_pension.amount(f"{year}-04-01") assert weekly == pytest.approx(expected, abs=0.01) @pytest.mark.parametrize("year, expected", NEW_STATE_PENSION_WEEKLY) def test_new_state_pension_matches_published_rate(year, expected): - weekly = system.parameters.gov.dwp.state_pension.new_state_pension.amount( - f"{year}-04-01" - ) + weekly = state_pension.new_state_pension.amount(f"{year}-04-01") assert weekly == pytest.approx(expected, abs=0.01) diff --git a/policyengine_uk/tests/test_triple_lock_properties.py b/policyengine_uk/tests/test_triple_lock_properties.py new file mode 100644 index 0000000000..1eb1fe838c --- /dev/null +++ b/policyengine_uk/tests/test_triple_lock_properties.py @@ -0,0 +1,254 @@ +"""Property tests for the State Pension uprating rule. + +Invariants, for every input path: + +1. The triple lock rate is at least each included element and the minimum + rate, and equals one of them. Without the triple lock, the rate is + earnings growth, or zero when earnings fall. +2. It never falls when an element rises. +3. Under the earnings-path guarantee, the pension level is never below the + earnings path, and each year's rise is at least the rule's own rate + (for one reading of the plan announced in September 2026, max(CPI, + 2.5%)), so the level + is never below the path compounded at those rates either. +4. The guarantee's top-up is the smallest 0.1 percentage point step that + meets the earnings path. +5. With the guarantee off, the path is the year-by-year rule. +6. The path agrees with an independent transcription of + PolicyEngine/uk-triple-lock's ``rules.rates_matrix`` (the reference + implementation of that reading of the September 2026 plan). +""" + +import math + +import pytest +from hypothesis import given, settings +from hypothesis import strategies as st + +from policyengine_uk.parameters.gov.dwp.state_pension.triple_lock.create_triple_lock import ( + UpratingYear, + round_to_published_precision, + round_up_to_published_precision, + rule_rate, + triple_lock_rate, + uprating_rates, +) + +# Published figures are to 0.1 percentage points; draw from the range seen +# since 1990 and beyond (deflation to double-digit inflation). +published_rate = st.integers(min_value=-60, max_value=150).map(lambda n: n / 1000) +minimum_rate = st.sampled_from([0.0, 0.02, 0.025, 0.03]) +LEVEL_TOLERANCE = 1e-9 + + +def uprating_year(earnings_path_guarantee=st.booleans()): + return st.builds( + UpratingYear, + earnings=published_rate, + cpi=published_rate, + minimum_rate=minimum_rate, + active=st.booleans(), + include_earnings=st.booleans(), + include_inflation=st.booleans(), + earnings_path_guarantee=earnings_path_guarantee, + ) + + +def paths(earnings_path_guarantee=st.booleans(), min_size=1, max_size=20): + return st.lists( + uprating_year(earnings_path_guarantee), min_size=min_size, max_size=max_size + ).map(lambda years: {2030 + i: year for i, year in enumerate(years)}) + + +@given( + earnings=published_rate, + cpi=published_rate, + floor=minimum_rate, + include_earnings=st.booleans(), + include_inflation=st.booleans(), +) +def test_triple_lock_is_the_highest_included_element( + earnings, cpi, floor, include_earnings, include_inflation +): + rate = triple_lock_rate(earnings, cpi, floor, include_earnings, include_inflation) + assert rate >= floor + if include_earnings: + assert rate >= earnings + if include_inflation: + assert rate >= cpi + included = [floor] + if include_earnings: + included.append(earnings) + if include_inflation: + included.append(cpi) + assert rate in included + + +@given(inputs=uprating_year()) +def test_rule_rate_is_the_triple_lock_or_the_statutory_earnings_link(inputs): + rate = rule_rate(inputs) + if inputs.active: + assert rate == triple_lock_rate( + inputs.earnings, + inputs.cpi, + inputs.minimum_rate, + inputs.include_earnings, + inputs.include_inflation, + ) + else: + assert rate == max(inputs.earnings, 0.0) + assert rate >= 0 + + +@given( + earnings=published_rate, + cpi=published_rate, + floor=minimum_rate, + rise=st.integers(min_value=1, max_value=50).map(lambda n: n / 1000), +) +def test_triple_lock_never_falls_when_an_element_rises(earnings, cpi, floor, rise): + rate = triple_lock_rate(earnings, cpi, floor) + assert triple_lock_rate(earnings + rise, cpi, floor) >= rate + assert triple_lock_rate(earnings, cpi + rise, floor) >= rate + assert triple_lock_rate(earnings, cpi, floor + rise) >= rate + + +@given(years=paths(earnings_path_guarantee=st.just(False))) +def test_without_guarantee_the_path_is_the_yearly_rule(years): + rates = uprating_rates(years) + assert rates == {year: rule_rate(inputs) for year, inputs in years.items()} + + +@settings(max_examples=300) +@given(years=paths()) +def test_guarantee_keeps_the_pension_on_or_above_its_earnings_path(years): + rates = uprating_rates(years) + level = 1.0 + earnings_path = None + rule_path = None + for year in sorted(years): + inputs = years[year] + # Each year's rise is at least the rule's own rate. + assert rates[year] >= rule_rate(inputs) + if inputs.earnings_path_guarantee: + if earnings_path is None: + earnings_path = rule_path = level + earnings_path *= 1 + inputs.earnings + rule_path *= 1 + rule_rate(inputs) + else: + earnings_path = rule_path = None + level *= 1 + rates[year] + if earnings_path is not None: + assert level >= earnings_path * (1 - LEVEL_TOLERANCE) + assert level >= rule_path * (1 - LEVEL_TOLERANCE) + + +@settings(max_examples=300) +@given(years=paths()) +def test_guarantee_top_up_is_the_smallest_step_that_reaches_the_path(years): + rates = uprating_rates(years) + level = 1.0 + earnings_path = None + for year in sorted(years): + inputs = years[year] + if inputs.earnings_path_guarantee: + if earnings_path is None: + earnings_path = level + earnings_path *= 1 + inputs.earnings + if rates[year] > rule_rate(inputs): + one_step_less = level * (1 + rates[year] - 0.001) + assert one_step_less < earnings_path * (1 + LEVEL_TOLERANCE) + else: + earnings_path = None + level *= 1 + rates[year] + + +@given(years=paths()) +def test_rates_stay_on_the_published_grid(years): + for rate in uprating_rates(years).values(): + assert rate * 1000 == pytest.approx(round(rate * 1000), abs=1e-6) + + +def reference_plan_rates(cpi, earnings, switch_index, floor=0.025, decimals=3): + """Transcription of uk-triple-lock ``rules.rates_matrix`` for the + ``burnham_2030`` policy, one draw, without numpy.""" + scale = 10**decimals + + def rnd(r): + return round(r, decimals) + + def rnd_up(r): + return math.ceil(round(r * scale, 9)) / scale + + rates, level, anchor = [], 1.0, 1.0 + for j, (c, e) in enumerate(zip(cpi, earnings)): + if j < switch_index: + rate = rnd(max(max(c, e), floor)) + level *= 1 + rate + anchor = level + else: + anchor *= 1 + e + floor_rate = rnd(max(max(c, floor), 0.0)) + rate = max(floor_rate, rnd_up(anchor / level - 1)) + level *= 1 + rate + rates.append(rate) + return rates + + +@settings(max_examples=300) +@given( + data=st.lists(st.tuples(published_rate, published_rate), min_size=1, max_size=15), + switch_index=st.integers(min_value=0, max_value=15), +) +def test_plan_matches_the_uk_triple_lock_reference(data, switch_index): + cpi = [c for c, _ in data] + earnings = [e for _, e in data] + years = { + 2020 + j: UpratingYear( + earnings=e, + cpi=c, + minimum_rate=0.025, + include_earnings=j < switch_index, + earnings_path_guarantee=j >= switch_index, + ) + for j, (c, e) in enumerate(data) + } + rates = uprating_rates(years) + expected = reference_plan_rates(cpi, earnings, switch_index) + assert [rates[year] for year in sorted(rates)] == pytest.approx(expected, abs=1e-12) + + +@given(rate=st.floats(min_value=-0.5, max_value=0.5, allow_nan=False)) +def test_rounding_to_published_precision(rate): + rounded = round_to_published_precision(rate) + assert abs(rounded - rate) <= 0.0005 + 1e-12 + assert round_to_published_precision(rounded) == rounded + rounded_up = round_up_to_published_precision(rate) + assert rate - 1e-9 <= rounded_up < rate + 0.001 + 1e-12 + + +def test_rounding_takes_halves_away_from_zero(): + assert round_to_published_precision(0.0255) == 0.026 + assert round_to_published_precision(-0.0095) == -0.010 + assert round_up_to_published_precision(0.025000000000000355) == 0.025 + assert round_up_to_published_precision(0.02500001) == 0.026 + + +def test_plan_example_by_hand(): + """Two years under the plan: CPI 2%, earnings 4%, then CPI 2%, earnings 1%. + + Year 1 anchors the earnings path at 1 and moves it to 1.04; the floor + pays 2.5%, so the top-up to 4.0% binds. Year 2: the path is + 1.04 x 1.01 = 1.0504 against a level of 1.04, a 1.0% rise, below the + 2.5% floor, so the floor pays. + """ + plan = dict( + minimum_rate=0.025, include_earnings=False, earnings_path_guarantee=True + ) + rates = uprating_rates( + { + 2030: UpratingYear(earnings=0.04, cpi=0.02, **plan), + 2031: UpratingYear(earnings=0.01, cpi=0.02, **plan), + } + ) + assert rates == {2030: 0.04, 2031: 0.025} diff --git a/policyengine_uk/tests/test_triple_lock_statutory_inputs.py b/policyengine_uk/tests/test_triple_lock_statutory_inputs.py new file mode 100644 index 0000000000..ebfee9cadc --- /dev/null +++ b/policyengine_uk/tests/test_triple_lock_statutory_inputs.py @@ -0,0 +1,339 @@ +"""The statutory uprating inputs, their forecasts, and how scenarios move them. + +The April uprating of the State Pension uses September CPI and May-July +earnings growth from the previous year. These tests check that forecasts of +those inputs follow the OBR where it publishes them, fall back to +calendar-year growth after, respond to macro scenarios applied before the +data load, and can be set directly; and that the earnings-path guarantee +models one reading of the plan announced in September 2026 as a +parameter reform. +""" + +import pandas as pd +import pytest + +from policyengine_uk import Simulation +from policyengine_uk.data.dataset_schema import UKSingleYearDataset +from policyengine_uk.model_api import Scenario +from policyengine_uk.scenarios import no_economic_assumptions +from policyengine_uk.system import system + +INPUTS = "gov.economic_assumptions.statutory_uprating_inputs" +OBR = "gov.economic_assumptions.yoy_growth.obr" +TRIPLE_LOCK = "gov.dwp.state_pension.triple_lock" +WEEKS_IN_YEAR = 52 + +PENSIONER = { + "people": {"person": {"age": {2026: 70}}}, + "benunits": {"benunit": {"members": ["person"]}}, + "households": {"household": {"members": ["person"]}}, +} + + +def parameters_under(changes): + simulation = Simulation( + situation=PENSIONER, + scenario=Scenario(parameter_changes=changes, applied_before_data_load=True), + ) + return simulation.tax_benefit_system.parameters + + +def uprating(parameters, year): + return parameters.gov.economic_assumptions.yoy_growth.triple_lock(f"{year}-01-01") + + +def new_state_pension_weekly(parameters, year): + return parameters.gov.dwp.state_pension.new_state_pension.amount(f"{year}-06-01") + + +def statutory_inputs(parameters): + return parameters.gov.economic_assumptions.statutory_uprating_inputs + + +def test_april_2027_uses_published_may_july_2026_earnings(): + """May-July 2026 total pay growth was 3.9%, above any CPI forecast, so the + new State Pension rises from £241.30 to £250.71 (before rounding to 5p).""" + parameters = system.parameters + assert uprating(parameters, 2027) == pytest.approx(0.039) + assert new_state_pension_weekly(parameters, 2027) == pytest.approx( + 241.30 * 1.039, abs=0.005 + ) + + +def test_every_published_year_has_its_own_value(): + """A missing year would silently carry the previous year's figure.""" + inputs = statutory_inputs(system.parameters) + for name, month_day, last_published in [ + ("cpi_september", "09-01", 2025), + ("awe_total_pay_may_july", "07-01", 2026), + ]: + parameter = getattr(inputs, name) + explicit = {value.instant_str for value in parameter.values_list} + for year in range(2010, last_published + 1): + assert f"{year}-{month_day}" in explicit, (name, year) + + +@pytest.mark.parametrize( + "name, series, month_day, first_forecast_year", + [ + ("cpi_september", "consumer_price_index", "09-01", 2026), + ("awe_total_pay_may_july", "average_earnings", "07-01", 2027), + ], +) +def test_forecasts_are_calendar_growth_plus_the_obr_gap( + name, series, month_day, first_forecast_year +): + parameters = system.parameters + statutory = getattr(statutory_inputs(parameters), name) + gap = getattr(statutory_inputs(parameters).forecast_gap, name) + calendar = getattr(parameters.gov.economic_assumptions.yoy_growth.obr, series) + last_year = max(int(v.instant_str[:4]) for v in calendar.values_list) + assert last_year >= 2073 + for year in range(first_forecast_year, last_year + 1): + observed = f"{year}-{month_day}" + assert statutory(observed) == pytest.approx( + calendar(f"{year}-01-01") + gap(observed), abs=1e-12 + ), year + if year > 2030: + # After the EFO horizon the forecast is calendar-year growth. + assert gap(observed) == 0 + assert statutory(f"{last_year + 1}-{month_day}") is None + + +# OBR March 2026 EFO: September CPI from receipts Table 3.19 ("CPI used to +# uprate thresholds"), Q3 CPI from economy Table 1.7 for 2030, and Q2 average +# earnings growth on a year earlier from economy Table 1.6. +OBR_SEPTEMBER_CPI = { + 2026: 0.021157, + 2027: 0.020719, + 2028: 0.020299, + 2029: 0.020480, + 2030: 0.019996, +} +OBR_Q2_EARNINGS = {2027: 0.024266, 2028: 0.020795, 2029: 0.021852, 2030: 0.023890} + + +def test_baseline_forecasts_track_the_obr_statutory_forecasts(): + """Calendar-year growth is held to 0.1pp in yoy_growth.yaml, so the + forecasts match the OBR's to within that rounding.""" + inputs = statutory_inputs(system.parameters) + for year, value in OBR_SEPTEMBER_CPI.items(): + assert inputs.cpi_september(f"{year}-09-01") == pytest.approx(value, abs=0.0005) + for year, value in OBR_Q2_EARNINGS.items(): + assert inputs.awe_total_pay_may_july(f"{year}-07-01") == pytest.approx( + value, abs=0.0005 + ) + + +def test_rule_reproduces_the_obr_triple_lock_forecast_from_its_own_inputs(): + """OBR Long-term economic determinants (March 2026 EFO), 'Triple Lock' + row: 3.7% in April 2027 and 2.5% in April 2028-2031. With the published + May-July 2026 figure removed, so earnings follow the OBR forecast, the + rule gives the same rates.""" + + def drop_may_july_2026(simulation): + system_ = simulation.tax_benefit_system + system_.reset_parameters() + statutory_inputs(system_.parameters).awe_total_pay_may_july.update( + period="year:2026-07-01:1", value=None + ) + system_.process_parameters() + + simulation = Simulation( + situation=PENSIONER, + scenario=Scenario( + simulation_modifier=drop_may_july_2026, applied_before_data_load=True + ), + ) + parameters = simulation.tax_benefit_system.parameters + assert statutory_inputs(parameters).awe_total_pay_may_july( + "2026-07-01" + ) == pytest.approx(0.034 + 0.00316) + assert [uprating(parameters, year) for year in range(2027, 2032)] == ( + pytest.approx([0.037, 0.025, 0.025, 0.025, 0.025]) + ) + + +def test_horizon_runs_to_the_end_of_the_economic_assumptions(): + triple_lock = system.parameters.gov.economic_assumptions.yoy_growth.triple_lock + years = sorted(int(value.instant_str[:4]) for value in triple_lock.values_list) + assert years[0] == 2011 + assert years[-1] == 2074 + assert years == list(range(2011, 2075)) + # 2035 onwards follows the long-run earnings path, not a value carried + # from the last year of the old horizon (2034). + assert uprating(system.parameters, 2040) == pytest.approx(0.038) + + +def test_macro_scenario_on_calendar_growth_moves_the_uprating(): + """Raising 2027 earnings growth from 2.4% to 5% moves May-July 2027 + earnings with it (plus the OBR gap) and so the April 2028 rise.""" + parameters = parameters_under( + {f"{OBR}.average_earnings": {"year:2027-01-01:1": 0.05}} + ) + assert statutory_inputs(parameters).awe_total_pay_may_july( + "2027-07-01" + ) == pytest.approx(0.05 + 0.00065) + assert uprating(parameters, 2028) == pytest.approx(0.051) + assert uprating(system.parameters, 2028) == pytest.approx(0.025) + assert new_state_pension_weekly(parameters, 2028) > new_state_pension_weekly( + system.parameters, 2028 + ) + # Only that year moves. + assert uprating(parameters, 2029) == uprating(system.parameters, 2029) + + +def test_macro_scenario_moves_the_state_pension_in_a_microsimulation(): + """The rate reaches the benefit: a full-rate new State Pension recipient + in 2026 data is paid the uprated full rate in 2028.""" + person = pd.DataFrame( + { + "person_id": [1], + "person_benunit_id": [1], + "person_household_id": [1], + "age": [70], + "state_pension_reported": [241.30 * WEEKS_IN_YEAR], + } + ) + dataset = UKSingleYearDataset( + person=person, + benunit=pd.DataFrame({"benunit_id": [1]}), + household=pd.DataFrame( + { + "household_id": [1], + "region": ["LONDON"], + "tenure_type": ["OWNED_OUTRIGHT"], + "council_tax": [2_000.0], + "rent": [0.0], + } + ), + fiscal_year=2026, + ) + scenario = Scenario( + parameter_changes={f"{OBR}.average_earnings": {"year:2027-01-01:1": 0.05}}, + applied_before_data_load=True, + ) + baseline = Simulation(dataset=dataset) + reformed = Simulation(dataset=dataset, scenario=scenario) + + baseline_pension = float(baseline.calculate("new_state_pension", 2028)[0]) + reformed_pension = float(reformed.calculate("new_state_pension", 2028)[0]) + full_rate_2027 = 241.30 * 1.039 + # Uprating indices are stored to 5 decimal places. + assert baseline_pension == pytest.approx( + full_rate_2027 * 1.025 * WEEKS_IN_YEAR, rel=1e-5 + ) + assert reformed_pension == pytest.approx( + full_rate_2027 * 1.051 * WEEKS_IN_YEAR, rel=1e-5 + ) + + +def test_setting_a_statutory_input_directly_overrides_its_forecast(): + """A scenario can supply September CPI and May-July earnings itself, for + example from a model of the monthly series. The value replaces the + forecast for that year only, whatever calendar-year growth says.""" + parameters = parameters_under( + { + f"{INPUTS}.awe_total_pay_may_july": {"2027": 0.07}, + f"{INPUTS}.cpi_september": {"2028": 0.061}, + f"{OBR}.average_earnings": {"year:2027-01-01:1": 0.01}, + } + ) + assert uprating(parameters, 2028) == pytest.approx(0.07) + assert uprating(parameters, 2029) == pytest.approx(0.061) + assert uprating(parameters, 2030) == uprating(system.parameters, 2030) + inputs = statutory_inputs(parameters) + assert inputs.awe_total_pay_may_july("2028-07-01") == pytest.approx( + statutory_inputs(system.parameters).awe_total_pay_may_july("2028-07-01") + ) + + +PLAN = { + f"{TRIPLE_LOCK}.include_earnings": {"year:2030-01-01:100": False}, + f"{TRIPLE_LOCK}.earnings_path_guarantee": {"year:2030-01-01:100": True}, +} + + +def test_plan_is_a_parameter_reform(): + """From April 2030: rises of at least max(CPI, 2.5%), and the pension + never below an earnings link from its 2029-30 level. + + Earnings inputs of 1.0% (May-July 2029 and 2030) then 6.0% (2031) and + 3.0% (2032), CPI 2.0% throughout: + + - April 2030 and 2031: the floor, 2.5%, is above earnings (the triple + lock would also pay 2.5%). + - April 2032: the earnings path has grown 1.01 x 1.01 x 1.06 = 1.0813 + from the 2029-30 level against 1.025 x 1.025 = 1.0506 for the pension, + so it rises by 1.0813 / 1.0506 - 1 = 2.92%, rounded up to 3.0%, where + the triple lock pays 6.0%. + - April 2033: the path grows 3.0% to 1.1137 and the pension, at + 1.0821, again needs 2.92%, rounded up to 3.0%, as the triple lock pays. + """ + growth = { + f"{INPUTS}.awe_total_pay_may_july": { + "2029": 0.01, + "2030": 0.01, + "2031": 0.06, + "2032": 0.03, + }, + f"{INPUTS}.cpi_september": {str(year): 0.02 for year in range(2029, 2033)}, + } + triple_lock = parameters_under(growth) + plan = parameters_under({**growth, **PLAN}) + + assert [uprating(triple_lock, year) for year in range(2030, 2034)] == ( + pytest.approx([0.025, 0.025, 0.06, 0.03]) + ) + assert [uprating(plan, year) for year in range(2030, 2034)] == ( + pytest.approx([0.025, 0.025, 0.030, 0.030]) + ) + # Unchanged before the switch. + for year in range(2027, 2030): + assert uprating(plan, year) == uprating(triple_lock, year) + + base = new_state_pension_weekly(plan, 2029) + assert base == new_state_pension_weekly(triple_lock, 2029) + earnings_path = base + for year in range(2030, 2034): + earnings_path *= 1 + [0.01, 0.01, 0.06, 0.03][year - 2030] + # Uprating indices are stored to 5 decimal places. + assert new_state_pension_weekly(plan, year) >= earnings_path * (1 - 1e-5) + assert new_state_pension_weekly(plan, year) <= new_state_pension_weekly( + triple_lock, year + ) + + +def test_plan_is_off_under_current_law(): + triple_lock = system.parameters.gov.dwp.state_pension.triple_lock + for year in range(2011, 2075): + assert not triple_lock.earnings_path_guarantee(f"{year}-04-30") + + +def test_without_the_triple_lock_the_pension_follows_earnings(): + """With the triple lock off, the statutory review (SSAA 1992 s150A) + requires a rise of at least earnings growth, and none when earnings + fall: 1.2% and then 0%, where the triple lock pays its 2.5% floor.""" + growth = { + f"{INPUTS}.awe_total_pay_may_july": {"2027": 0.012, "2028": -0.004}, + f"{INPUTS}.cpi_september": {"2027": 0.02, "2028": 0.02}, + } + triple_lock = parameters_under(growth) + no_triple_lock = parameters_under( + {**growth, f"{TRIPLE_LOCK}.active": {"year:2028-01-01:100": False}} + ) + assert [uprating(triple_lock, year) for year in (2028, 2029)] == ( + pytest.approx([0.025, 0.025]) + ) + assert [uprating(no_triple_lock, year) for year in (2028, 2029)] == ( + pytest.approx([0.012, 0.0]) + ) + assert uprating(no_triple_lock, 2027) == uprating(triple_lock, 2027) + + +def test_no_economic_assumptions_leaves_only_the_floor(): + simulation = Simulation(situation=PENSIONER, scenario=no_economic_assumptions) + parameters = simulation.tax_benefit_system.parameters + cutoff_year = int(simulation.default_input_period) + for year in range(cutoff_year + 2, 2075): + assert uprating(parameters, year) == pytest.approx(0.025), year diff --git a/policyengine_uk/utils/create_triple_lock.py b/policyengine_uk/utils/create_triple_lock.py deleted file mode 100644 index b4af9b8c66..0000000000 --- a/policyengine_uk/utils/create_triple_lock.py +++ /dev/null @@ -1,22 +0,0 @@ -from policyengine_uk.system import system - -# Run this script to generate the triple lock parameter for updated CPI and average earnings forecasts from the OBR. - -cpi = system.parameters.gov.economic_assumptions.indices.obr.consumer_price_index -average_earnings = ( - system.parameters.gov.economic_assumptions.indices.obr.average_earnings -) - -START_YEAR = 2021 - -triple_lock = system.parameters.gov.dwp.state_pension.triple_lock -lock_value = triple_lock(START_YEAR - 1) - -for year in range(START_YEAR, 2029): - earnings_increase = average_earnings(year - 1) / average_earnings(year - 2) - cpi_increase = cpi(year - 1) / cpi(year - 2) - triple_lock_increase = max(earnings_increase, cpi_increase, 1.025) - lock_value *= triple_lock_increase - print( - f" {year}-01-01: {lock_value:.3f} # Earnings increase FY{year - 1}/{year - 2} = {earnings_increase - 1:.1%}, CPI increase FY{year - 1}/{year - 2} = {cpi_increase - 1:.1%}" - ) diff --git a/policyengine_uk/utils/import_obr_forecasts.py b/policyengine_uk/utils/import_obr_forecasts.py index 3d7e6c8eed..0bcaa3cf0b 100644 --- a/policyengine_uk/utils/import_obr_forecasts.py +++ b/policyengine_uk/utils/import_obr_forecasts.py @@ -92,6 +92,45 @@ class SeriesSpec: ) +QUARTER_RE = re.compile(r"^(\d{4})\s*Q([1-4])$") + + +@dataclass(frozen=True) +class StatutoryGapSpec: + """Forecast gap between a statutory uprating input and calendar-year growth. + + The OBR does not forecast the statutory measures themselves, so the gap is + the quarter containing (or nearest) the statutory period minus the + calendar-year figure, both from the same workbook column. + """ + + key: str + series_key: str + quarter: int + month_day: str + description: str + + +STATUTORY_GAP_SPECS = ( + StatutoryGapSpec( + key="cpi_september", + series_key="consumer_price_index", + quarter=3, + month_day="09-01", + description="Q3 CPI 12-month rate minus calendar-year CPI inflation", + ), + StatutoryGapSpec( + key="awe_total_pay_may_july", + series_key="average_earnings", + quarter=2, + month_day="07-01", + description=( + "Q2 average earnings growth on a year earlier minus calendar-year growth" + ), + ), +) + + def get_repo_root() -> Path: current = Path(__file__).resolve() while current != current.parent: @@ -108,6 +147,14 @@ def get_yoy_growth_path() -> Path: ) +def get_forecast_gap_dir() -> Path: + return ( + get_repo_root() + / "policyengine_uk/parameters/gov/economic_assumptions" + / "statutory_uprating_inputs/forecast_gap" + ) + + def normalise_label(value: str | None) -> str: if not value: return "" @@ -270,6 +317,199 @@ def extract_annual_series_from_xlsx( return result +def extract_series_by_period_from_xlsx( + xlsx_bytes: bytes, series_keys: set[str] +) -> dict[str, dict[str, float]]: + """Full-precision values (as fractions) keyed by the row label in column B, + e.g. "2026" or "2026Q3".""" + specs = [spec for spec in SERIES_SPECS if spec.key in series_keys] + rows_by_sheet = { + sheet: read_sheet_rows(xlsx_bytes, sheet) for sheet in {s.sheet for s in specs} + } + result: dict[str, dict[str, float]] = {} + for spec in specs: + rows = rows_by_sheet[spec.sheet] + column = find_series_column(rows, spec) + values: dict[str, float] = {} + for row in rows: + label = str(row.get("B") or "").strip() + raw_value = row.get(column) + if not label or raw_value in (None, ""): + continue + if YEAR_RE.match(label) or QUARTER_RE.match(label): + values[label.replace(" ", "")] = float(raw_value) / 100 + result[spec.key] = values + return result + + +SEPTEMBER_CPI_LABEL = "cpi used to uprate thresholds" +FISCAL_YEAR_RE = re.compile(r"^(\d{4})-(\d{2})$") + + +@dataclass(frozen=True) +class ForecastGap: + value: float + source: str + reference_title: str + + +def extract_september_cpi_from_receipts( + xlsx_bytes: bytes, +) -> tuple[str, dict[int, float]]: + """September CPI 12-month rates from the receipts tables. + + The OBR publishes them as the memo row "CPI used to uprate thresholds", + one column per fiscal year of uprating: 2027-28 holds September 2026. + Returns the table number and the rates keyed by September's year. + """ + with ZipFile(BytesIO(xlsx_bytes)) as archive: + sheet_names = list(_sheet_paths(archive)) + for sheet in sheet_names: + rows = read_sheet_rows(xlsx_bytes, sheet) + for index, row in enumerate(rows): + if SEPTEMBER_CPI_LABEL not in normalise_label(row.get("B")): + continue + columns = {} + for header in reversed(rows[:index]): + columns = { + column: int(match.group(1)) + for column, value in header.items() + if value and (match := FISCAL_YEAR_RE.match(str(value).strip())) + } + if columns: + break + rates = { + fiscal_year - 1: float(row[column]) / 100 + for column, fiscal_year in columns.items() + if row.get(column) not in (None, "") + } + if rates: + return sheet, rates + raise ValueError("Could not find September CPI in the receipts workbook") + + +def compute_statutory_forecast_gaps( + xlsx_bytes: bytes, + forecast_start_year: int, + forecast_years: int, + receipts_xlsx_bytes: bytes | None = None, +) -> dict[str, dict[int, ForecastGap]]: + """Gap for each statutory input and forecast year, rounded to 1e-5. + + September CPI comes from the receipts tables where they are supplied and + cover the year; otherwise each input uses its quarter from the economy + tables. + """ + series = extract_series_by_period_from_xlsx( + xlsx_bytes, {spec.series_key for spec in STATUTORY_GAP_SPECS} + ) + september_table, september_cpi = ( + extract_september_cpi_from_receipts(receipts_xlsx_bytes) + if receipts_xlsx_bytes is not None + else (None, {}) + ) + gaps: dict[str, dict[int, ForecastGap]] = {} + for spec in STATUTORY_GAP_SPECS: + values = series[spec.series_key] + table = next(s.table for s in SERIES_SPECS if s.key == spec.series_key) + gaps[spec.key] = {} + for year in range(forecast_start_year, forecast_start_year + forecast_years): + annual = values.get(str(year)) + if annual is None: + continue + if spec.key == "cpi_september" and year in september_cpi: + statutory = september_cpi[year] + source = ( + f"September CPI (receipts Table {september_table}) minus " + f"calendar-year CPI (economy Table {table})" + ) + reference = ( + f"detailed forecast tables, receipts, Table {september_table}" + ) + else: + statutory = values.get(f"{year}Q{spec.quarter}") + if statutory is None: + continue + source = f"{spec.description} (economy Table {table})" + reference = f"detailed forecast tables, economy, Table {table}" + gaps[spec.key][year] = ForecastGap( + value=round(statutory - annual, 5) + 0.0, + source=source, + reference_title=reference, + ) + return gaps + + +def render_forecast_gap_values( + spec: StatutoryGapSpec, gaps: dict[int, ForecastGap], month: str, year: int +) -> str: + """The values block of a forecast_gap YAML file.""" + if not gaps: + raise ValueError(f"No forecast gaps for {spec.key}") + lines = [ + "values:", + " # No gap applies to published years.", + f" 2010-{spec.month_day}: 0", + ] + source = None + for gap_year in sorted(gaps): + if gaps[gap_year].source != source: + source = gaps[gap_year].source + lines.append(f" # OBR EFO {month} {year}: {source}.") + lines.append(f" {gap_year}-{spec.month_day}: {gaps[gap_year].value:g}") + lines += [ + " # After the EFO horizon: no gap, so the forecast is calendar-year growth.", + f" {max(gaps) + 1}-{spec.month_day}: 0", + ] + return "\n".join(lines) + "\n" + + +def render_forecast_gap_references( + spec: StatutoryGapSpec, gaps: dict[int, ForecastGap], month: str, year: int +) -> str: + table = next(s.table for s in SERIES_SPECS if s.key == spec.series_key) + titles = [f"detailed forecast tables, economy, Table {table}"] + for gap in gaps.values(): + if gap.reference_title not in titles: + titles.append(gap.reference_title) + lines = [" reference:"] + for title in titles: + lines += [ + f" - title: OBR EFO {month} {year} ({title})", + f" href: {build_efo_href(month, year)}", + ] + return "\n".join(lines) + "\n" + + +def update_forecast_gap_yaml( + yaml_path: Path, + spec: StatutoryGapSpec, + gaps: dict[int, ForecastGap], + month: str, + year: int, +) -> None: + """Rewrite the values block and the reference list (the last metadata key).""" + content = yaml_path.read_text() + values_pattern = re.compile( + r"^values:\n.*?(?=^metadata:)", re.MULTILINE | re.DOTALL + ) + content, values_count = values_pattern.subn( + lambda _: render_forecast_gap_values(spec, gaps, month, year), + content, + count=1, + ) + reference_pattern = re.compile(r"^ reference:\n.*\Z", re.MULTILINE | re.DOTALL) + content, reference_count = reference_pattern.subn( + lambda _: render_forecast_gap_references(spec, gaps, month, year), + content, + count=1, + ) + if not (values_count and reference_count): + raise ValueError(f"Could not find the values and reference in {yaml_path}") + yaml.safe_load(content) + yaml_path.write_text(content) + + def infer_release(source_name: str) -> tuple[str, int]: match = MONTH_RE.search(source_name) if not match: @@ -439,6 +679,22 @@ def build_arg_parser() -> argparse.ArgumentParser: default=6, help="Number of forecast years to update (default: 6)", ) + parser.add_argument( + "--receipts-url", + help=( + "OBR receipts detailed forecast tables (XLSX), for September CPI; " + "without it the September CPI gap uses Q3 CPI" + ), + ) + parser.add_argument("--receipts-file", help="Local receipts XLSX file path") + parser.add_argument( + "--skip-statutory-gaps", + action="store_true", + help=( + "Do not update the statutory uprating input forecast gaps " + "(September CPI, May-July earnings)" + ), + ) parser.add_argument( "--dry-run", action="store_true", @@ -468,6 +724,15 @@ def main(argv: list[str] | None = None) -> int: month, year ) print_summary(series_values, forecast_start_year, args.forecast_years) + receipts_bytes = None + if args.receipts_url or args.receipts_file: + _, receipts_bytes = load_source_bytes(args.receipts_url, args.receipts_file) + statutory_gaps = compute_statutory_forecast_gaps( + workbook_bytes, forecast_start_year, args.forecast_years, receipts_bytes + ) + for key, gaps in statutory_gaps.items(): + window = ", ".join(f"{y}: {g.value:g}" for y, g in sorted(gaps.items())) + print(f"- forecast gap {key}: {window}") if args.dry_run: return 0 @@ -485,6 +750,14 @@ def main(argv: list[str] | None = None) -> int: forecast_years=args.forecast_years, ) print(f"Updated {target_yaml_path}") + + if not args.skip_statutory_gaps and args.output is None: + for spec in STATUTORY_GAP_SPECS: + gap_path = get_forecast_gap_dir() / f"{spec.key}.yaml" + update_forecast_gap_yaml( + gap_path, spec, statutory_gaps[spec.key], month, year + ) + print(f"Updated {gap_path}") return 0 From 8e2437ac99db6df71e8747e8fb844c5cb63e6a5e Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Tue, 29 Sep 2026 13:47:28 -0400 Subject: [PATCH 2/6] Address review: exact OBR gaps, safe importer, precise docs - Measure each forecast gap from the calendar-year growth stored in yoy_growth.yaml, so the baseline inputs equal the OBR's statutory-basis figures (May-July 2027 earnings was 2.5% after rounding, OBR 2.4%). - Importer: render and check every file before writing any; write gaps in fixed point (3e-05 loaded as a string); put gaps next to --yaml-path; require the receipts tables unless gaps are skipped; add --gaps-only. - Accept numpy scalars in the rounding helpers. - Docs: a double lock is the floor at 0; active=false ignores the floor, the include flags and the one-year 2021 and 2022 changes to s150A; calendar-year edits need year:YYYY-01-01:1 keys; levers work through Scenario, not reform dictionaries; the baseline has no triple-lock ratchet premium (OBR: 0.56pp); earnings are as used, not first published. Co-Authored-By: Claude Opus 5.5 --- .../triple-lock-statutory-inputs.changed.md | 2 +- .../book/assumptions/nowcasting-comparison.md | 2 +- docs/book/programs/gov/dwp/state-pension.md | 45 ++-- .../dwp/state_pension/triple_lock/active.yaml | 1 + .../triple_lock/create_triple_lock.py | 20 +- .../triple_lock/earnings_path_guarantee.yaml | 4 +- .../triple_lock/include_earnings.yaml | 1 + .../triple_lock/include_inflation.yaml | 1 + .../triple_lock/minimum_rate.yaml | 2 + .../gov/economic_assumptions/README.md | 6 +- .../create_statutory_uprating_inputs.py | 22 +- .../forecast_gap/awe_total_pay_may_july.yaml | 28 ++- .../forecast_gap/cpi_september.yaml | 28 +-- .../tests/test_import_obr_forecasts.py | 192 +++++++++++++++- .../tests/test_triple_lock_properties.py | 18 +- .../test_triple_lock_statutory_inputs.py | 51 +++-- policyengine_uk/utils/import_obr_forecasts.py | 206 +++++++++++++----- 17 files changed, 492 insertions(+), 137 deletions(-) diff --git a/changelog.d/triple-lock-statutory-inputs.changed.md b/changelog.d/triple-lock-statutory-inputs.changed.md index 9cbc76543b..5ebfaa37db 100644 --- a/changelog.d/triple-lock-statutory-inputs.changed.md +++ b/changelog.d/triple-lock-statutory-inputs.changed.md @@ -1 +1 @@ -Compute the State Pension triple lock from its statutory inputs, September CPI and May-July average weekly earnings total pay growth, instead of calendar-year OBR growth. The inputs hold ONS outturn and OBR forecasts on the statutory basis, and macro scenarios that edit calendar-year growth move them. April 2027 now rises by 3.9% (published May-July 2026 earnings growth) rather than 3.4%, and the rates run to the end of the economic-assumption series instead of 2034. +Compute the State Pension triple lock from its statutory inputs, September CPI and May-July average weekly earnings total pay growth, instead of calendar-year OBR growth. The inputs hold the published figures each uprating review used, then forecasts: the OBR's September CPI forecast and its Q2 earnings growth (a proxy for May-July AWE) to the end of the EFO, and calendar-year growth after. Macro scenarios that edit calendar-year growth move them. April 2027 now rises by 3.9% (published May-July 2026 earnings) rather than 3.4%, and the rates run to the end of the economic-assumption series instead of 2034. The `active` toggle, which had no effect, now leaves the statutory minimum of earnings growth. diff --git a/docs/book/assumptions/nowcasting-comparison.md b/docs/book/assumptions/nowcasting-comparison.md index 15d4fa795b..f86d67c466 100644 --- a/docs/book/assumptions/nowcasting-comparison.md +++ b/docs/book/assumptions/nowcasting-comparison.md @@ -48,7 +48,7 @@ earnings index. | Aspect | Resolution Foundation | PolicyEngine UK | |-------|------------------------|-----------------| | Working-age benefits | Uses statutory uprating with explicit overrides for announced policy (e.g. CoL Payments, benefit freezes). | Same approach. Parameters under `gov/dwp/` and `gov/hmrc/child_benefit/` track legislated rates; ad hoc payments live under `gov/treasury/cost_of_living_support`. | -| State Pension | Models the triple lock explicitly, using its own internal earnings/CPI forecasts. | Models the triple lock from its statutory inputs (September CPI, May-July AWE total pay growth, 2.5% floor) in `gov/economic_assumptions/statutory_uprating_inputs/` and `gov/dwp/state_pension/triple_lock/`, with OBR forecasts of those inputs; see the State Pension page. | +| State Pension | Models the triple lock explicitly, using its own internal earnings/CPI forecasts. | Models the triple lock from its statutory inputs (September CPI, May-July AWE total pay growth, 2.5% floor) in `gov/economic_assumptions/statutory_uprating_inputs/` and `gov/dwp/state_pension/triple_lock/`, forecast from the OBR's September CPI and Q2 earnings growth to the end of the EFO and calendar-year growth after; see the State Pension page. | ### Take-up diff --git a/docs/book/programs/gov/dwp/state-pension.md b/docs/book/programs/gov/dwp/state-pension.md index 5e9fc65a5b..e144b6809b 100644 --- a/docs/book/programs/gov/dwp/state-pension.md +++ b/docs/book/programs/gov/dwp/state-pension.md @@ -74,8 +74,8 @@ Each holds published figures and then a null. From the null onwards, `create_statutory_uprating_inputs.py` fills in a forecast: calendar-year growth in the matching OBR series (`yoy_growth.obr.consumer_price_index` or `yoy_growth.obr.average_earnings`) plus `forecast_gap`. The gap is the OBR's -forecast of the statutory measure minus its calendar-year forecast, from the -same Economic and Fiscal Outlook: +statutory-basis forecast minus the calendar-year growth stored in +`yoy_growth.yaml`, so that in the baseline the input equals the OBR's figure: - September CPI: the OBR's September CPI forecast (receipts Table 3.19, the CPI used to uprate tax thresholds), or Q3 CPI (economy Table 1.7) in years @@ -85,14 +85,26 @@ same Economic and Fiscal Outlook: salaries per employee, and Q2 is the quarter nearest May to July. After the EFO horizon the gap is zero, so the inputs follow calendar-year -growth. `policyengine_uk/utils/import_obr_forecasts.py` regenerates the gaps -with the calendar-year series (pass `--receipts-file` or `--receipts-url` -for September CPI). +growth. Smooth forecasts pay the higher of earnings, CPI and 2.5% each year, +so the baseline has none of the extra cost the triple lock builds up when +September CPI and May-July earnings take turns to spike; the OBR's long-run +projections add 0.56 percentage points a year over earnings for it (Fiscal +risks and sustainability, July 2026). To capture it, supply simulated paths +of the statutory inputs, as below. + +`policyengine_uk/utils/import_obr_forecasts.py` regenerates the gaps with the +calendar-year series, from the EFO economy and receipts tables +(`--receipts-file` or `--receipts-url`). After editing `yoy_growth.yaml` by +hand, run it with `--gaps-only`. ### Scenarios A macro scenario applied before the data load that edits calendar-year -growth moves the forecast inputs one for one, and so the triple lock: +growth moves the forecast inputs one for one, and so the triple lock. +Calendar-year series are keyed to 1 January, so key the change +`year:YYYY-01-01:1`: a bare year names the fiscal year from 6 April, which +lands in the next calendar-year value and so moves the following year's +inputs. ```python from policyengine_uk.model_api import Scenario @@ -127,20 +139,25 @@ Scenario( ) ``` -These changes must go through `Scenario(parameter_changes=...)`. The rates -are built when parameters are processed, and a `reform=` dictionary edits -parameters after that, so it changes the parameter but not the uprating. +These changes, and the reform levers below, must go through +`Scenario(parameter_changes=...)`. The rates are built when parameters are +processed, and a `reform=` dictionary edits parameters after that, so it +changes the parameter but not the uprating. ### Reform levers +- `triple_lock/minimum_rate.yaml` sets the floor. A double lock, the higher + of earnings and CPI, is the floor set to 0 (or below 0 to allow cash + cuts). +- `triple_lock/include_earnings.yaml` and + `triple_lock/include_inflation.yaml` drop an element but keep the floor. + A CPI link is `include_earnings` false with the floor at or below 0. - `triple_lock/active.yaml` switches the triple lock off. The pension then rises by the statutory minimum from the review under the Social Security Administration Act 1992, section 150A: earnings growth, and nothing when - earnings fall. -- `triple_lock/include_earnings.yaml` and - `triple_lock/include_inflation.yaml` drop an element, e.g. for a double - lock. -- `triple_lock/minimum_rate.yaml` sets the floor. + earnings fall. The floor and the include flags no longer apply, and + neither do the one-year changes to section 150A for April 2021 and April + 2022. - `triple_lock/earnings_path_guarantee.yaml` (off under current law) keeps the pension on or above an earnings path started from its level in the year before the guarantee first applies. With `include_earnings` false, diff --git a/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/active.yaml b/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/active.yaml index 00413c770f..cdf75f7d8f 100644 --- a/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/active.yaml +++ b/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/active.yaml @@ -2,6 +2,7 @@ description: Whether the triple lock applies to the April uprating of the State values: 2010-01-01: true metadata: + documentation: Changes take effect through Scenario(parameter_changes=...), which rebuilds the uprating rates; a reform dictionary changes this parameter but not the rates. unit: bool label: Triple lock reference: diff --git a/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/create_triple_lock.py b/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/create_triple_lock.py index 260715799e..b977aca217 100644 --- a/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/create_triple_lock.py +++ b/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/create_triple_lock.py @@ -13,8 +13,11 @@ publishes, as the uprating review uses the published figure. With ``active`` false there is no triple lock, only the statutory minimum -from the review under Social Security Administration Act 1992 s150A: the -pension rises by earnings growth, and not at all when earnings fall. +from the review under Social Security Administration Act 1992 s150A as it +stands: the pension rises by earnings growth, and not at all when earnings +fall. The one-year modifications of s150A for April 2021 and April 2022 are +not applied, and ``include_earnings``, ``include_inflation`` and +``minimum_rate`` are ignored. Years with a published rate that the rule cannot reproduce are overridden in ``triple_lock/outturn.yaml``. Only April 2011, when the basic State Pension @@ -55,21 +58,24 @@ def uprating_instant(year: int) -> str: """Date at which policy parameters are read for the April ``year`` uprating. - 30 April is the date fiscal-year conversion samples, so a reform keyed to - the bare year (the fiscal year from 6 April), to ``year:YYYY-01-01:1`` or - to 1 January all take effect in the uprating for that April. + 30 April is the date fiscal-year conversion samples. A YAML value from + 1 January, or a parameter change keyed to the bare year (the fiscal year + from 6 April) or to ``year:YYYY-01-01:N``, takes effect in the uprating + for that April. A change keyed to a single day covers only that day. """ return f"{year}-04-30" def round_to_published_precision(rate: float) -> float: """Round a growth rate to 0.1 percentage points, halves away from zero.""" - return float(Decimal(repr(rate)).quantize(PUBLISHED_PRECISION, ROUND_HALF_UP)) + return float( + Decimal(repr(float(rate))).quantize(PUBLISHED_PRECISION, ROUND_HALF_UP) + ) def round_up_to_published_precision(rate: float) -> float: """Round a guaranteed minimum rate up to the next 0.1 percentage points.""" - cleaned = Decimal(repr(rate)).quantize(RATIO_PRECISION, ROUND_HALF_UP) + cleaned = Decimal(repr(float(rate))).quantize(RATIO_PRECISION, ROUND_HALF_UP) return float(cleaned.quantize(PUBLISHED_PRECISION, ROUND_CEILING)) diff --git a/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/earnings_path_guarantee.yaml b/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/earnings_path_guarantee.yaml index 0ab5e1c258..ce8546216c 100644 --- a/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/earnings_path_guarantee.yaml +++ b/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/earnings_path_guarantee.yaml @@ -15,7 +15,9 @@ metadata: include_earnings false and this guarantee true from 2030. Each guaranteed year, the rise is the higher of the triple lock elements still included and the rise needed to reach the earnings path, - rounded up to 0.1 percentage points. + rounded up to 0.1 percentage points. Changes take effect through + Scenario(parameter_changes=...), which rebuilds the uprating rates; a + reform dictionary changes this parameter but not the rates. reference: - title: BBC News live, Labour conference speech, 29 September 2026 href: https://www.bbc.co.uk/news/live/c6x2zrv774gvt diff --git a/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/include_earnings.yaml b/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/include_earnings.yaml index 88592646d7..b66a2fe614 100644 --- a/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/include_earnings.yaml +++ b/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/include_earnings.yaml @@ -7,6 +7,7 @@ values: 2022-01-01: false 2023-01-01: true metadata: + documentation: Changes take effect through Scenario(parameter_changes=...), which rebuilds the uprating rates; a reform dictionary changes this parameter but not the rates. unit: bool label: Include average earnings in triple lock reference: diff --git a/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/include_inflation.yaml b/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/include_inflation.yaml index cd825015f6..ed5bd0b566 100644 --- a/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/include_inflation.yaml +++ b/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/include_inflation.yaml @@ -2,5 +2,6 @@ description: Whether the triple lock should include inflation. values: 2011-01-01: true metadata: + documentation: Changes take effect through Scenario(parameter_changes=...), which rebuilds the uprating rates; a reform dictionary changes this parameter but not the rates. unit: bool label: Include inflation in triple lock \ No newline at end of file diff --git a/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/minimum_rate.yaml b/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/minimum_rate.yaml index e9e4db4ffa..b89790d9ef 100644 --- a/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/minimum_rate.yaml +++ b/policyengine_uk/parameters/gov/dwp/state_pension/triple_lock/minimum_rate.yaml @@ -12,3 +12,5 @@ metadata: The triple lock is a policy commitment introduced by the Coalition Government in 2010, not enshrined in primary legislation. The 2.5% minimum is one of three components ensuring pensions increase by the highest of earnings, inflation, or 2.5%. + Changes take effect through Scenario(parameter_changes=...), which rebuilds the + uprating rates; a reform dictionary changes this parameter but not the rates. diff --git a/policyengine_uk/parameters/gov/economic_assumptions/README.md b/policyengine_uk/parameters/gov/economic_assumptions/README.md index 7effffc4be..74a070a776 100644 --- a/policyengine_uk/parameters/gov/economic_assumptions/README.md +++ b/policyengine_uk/parameters/gov/economic_assumptions/README.md @@ -69,7 +69,8 @@ growth (ONS KAC3). They are keyed to their observation month, not to Each series holds published figures and then a null. From there, [`create_statutory_uprating_inputs.py`](./create_statutory_uprating_inputs.py) fills in calendar-year growth from `yoy_growth.obr` plus `forecast_gap`, the -OBR's forecast of the statutory measure minus its calendar-year forecast. +OBR's statutory-basis forecast minus the calendar-year growth stored here, so +the baseline input equals the OBR's figure. The gap is zero after the EFO horizon, so the forecasts fall back to calendar-year growth. A scenario that edits calendar-year growth therefore moves them, and a scenario can also set them directly. @@ -93,6 +94,7 @@ which the review uses: the April 2025 and April 2026 rises used October's 5. Regenerate `statutory_uprating_inputs/forecast_gap/` from the same EFO. `policyengine_uk/utils/import_obr_forecasts.py` does this when it updates `yoy_growth.yaml`; pass the receipts tables with `--receipts-file` or - `--receipts-url` so September CPI uses the OBR's September forecast. + `--receipts-url` so September CPI uses the OBR's September forecast. If + you edit `yoy_growth.yaml` by hand instead, rerun it with `--gaps-only`. [rpi-cpi]: https://obr.uk/box/the-long-run-difference-between-rpi-and-cpi-inflation/ diff --git a/policyengine_uk/parameters/gov/economic_assumptions/create_statutory_uprating_inputs.py b/policyengine_uk/parameters/gov/economic_assumptions/create_statutory_uprating_inputs.py index d542f09f30..fe1fdac8c7 100644 --- a/policyengine_uk/parameters/gov/economic_assumptions/create_statutory_uprating_inputs.py +++ b/policyengine_uk/parameters/gov/economic_assumptions/create_statutory_uprating_inputs.py @@ -4,8 +4,9 @@ - ``cpi_september``: the CPI 12-month rate in September (ONS D7G7). - ``awe_total_pay_may_july``: average weekly earnings, total pay, whole - economy, the three months May to July on a year earlier, as first published - (ONS KAC3). + economy, the three months May to July on a year earlier (ONS KAC3), as + used in each year's uprating review: the October labour market release, + with the latest year holding September's first estimate until then. Each is keyed to its observation month (1 September, 1 July) and carries ``preserve_calendar_dates`` so fiscal-year conversion leaves the dates alone. @@ -14,12 +15,13 @@ For a year without a published figure, the forecast is calendar-year growth in the matching series of ``gov.economic_assumptions.yoy_growth.obr`` plus -``forecast_gap``: the OBR's forecast of the statutory-basis quarter minus its -calendar-year forecast, both from the same Economic and Fiscal Outlook. The -gap is zero after the EFO horizon, so the forecast falls back to -calendar-year growth. A macro scenario that edits calendar-year growth moves -the inputs one for one; a scenario that sets an input directly replaces the -forecast for the years it covers. +``forecast_gap``: the OBR's statutory-basis forecast (its September CPI, or +the quarter nearest the statutory period) minus the stored calendar-year +growth, so the baseline reproduces the OBR's figure. The gap is zero after +the EFO horizon, so the forecast falls back to calendar-year growth. A macro +scenario that edits calendar-year growth moves the inputs one for one; a +scenario that sets an input directly replaces the forecast for the years it +covers. """ from policyengine_core.parameters import Parameter, ParameterNode @@ -71,6 +73,8 @@ def add_statutory_uprating_inputs(parameters: ParameterNode) -> ParameterNode: observed = f"{year}-{month_day}" if parameter(observed) is not None: continue - forecast = calendar_growth(f"{year}-01-01") + (gap(observed) or 0) + forecast = float(calendar_growth(f"{year}-01-01")) + float( + gap(observed) or 0 + ) parameter.update(period=f"year:{observed}:1", value=forecast) return parameters diff --git a/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/forecast_gap/awe_total_pay_may_july.yaml b/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/forecast_gap/awe_total_pay_may_july.yaml index dd1b9ca4dd..126d1ffcaf 100644 --- a/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/forecast_gap/awe_total_pay_may_july.yaml +++ b/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/forecast_gap/awe_total_pay_may_july.yaml @@ -2,13 +2,13 @@ description: Forecast difference between May-July earnings growth and calendar-y values: # No gap applies to published years. 2010-07-01: 0 - # OBR EFO March 2026: Q2 average earnings growth on a year earlier minus calendar-year growth (economy Table 1.6). - 2025-07-01: 0.00254 - 2026-07-01: 0.00316 - 2027-07-01: 0.00065 - 2028-07-01: -0.00015 - 2029-07-01: -0.00022 - 2030-07-01: -0.00017 + # OBR EFO March 2026: Q2 average earnings growth on a year earlier (economy Table 1.6) minus calendar-year growth in yoy_growth.yaml. + 2025-07-01: 0.00259 + 2026-07-01: 0.00274 + 2027-07-01: 0.00027 + 2028-07-01: -0.00021 + 2029-07-01: -0.00015 + 2030-07-01: -0.00011 # After the EFO horizon: no gap, so the forecast is calendar-year growth. 2031-07-01: 0 metadata: @@ -18,11 +18,15 @@ metadata: documentation: | The OBR does not forecast the ONS AWE series. Its average earnings measure is wages and salaries per employee from the national accounts, - forecast by quarter; the gap uses Q2 (April to June), the quarter - nearest May to July. It is taken from the same Economic and Fiscal - Outlook as the calendar-year series and regenerated by - policyengine_uk/utils/import_obr_forecasts.py. It is zero after the EFO - horizon, where the May-July forecast falls back to calendar-year growth. + forecast by quarter; the gap is its Q2 (April to June) growth, the + quarter nearest May to July, minus the calendar-year growth stored in + yoy_growth.yaml, so that stored growth plus the gap is the OBR's Q2 + figure. In the EFOs from November 2023 to March 2026, every OBR + triple-lock forecast equals the highest of this Q2 growth, Q3 CPI and + 2.5%. policyengine_uk/utils/import_obr_forecasts.py regenerates the gaps + with each EFO (or with --gaps-only after editing yoy_growth.yaml). The + gap is zero after the EFO horizon, where the May-July forecast falls back + to calendar-year growth. reference: - title: OBR EFO March 2026 (detailed forecast tables, economy, Table 1.6) href: https://obr.uk/efo/economic-and-fiscal-outlook-march-2026/ diff --git a/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/forecast_gap/cpi_september.yaml b/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/forecast_gap/cpi_september.yaml index a387dcf357..37f30d0198 100644 --- a/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/forecast_gap/cpi_september.yaml +++ b/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/forecast_gap/cpi_september.yaml @@ -2,13 +2,13 @@ description: Forecast difference between September CPI inflation and calendar-ye values: # No gap applies to published years. 2010-09-01: 0 - # OBR EFO March 2026: September CPI (receipts Table 3.19) minus calendar-year CPI (economy Table 1.7). - 2025-09-01: 0.00438 - 2026-09-01: -0.00185 - 2027-09-01: 0.00103 - 2028-09-01: 0.00053 - 2029-09-01: 0.00028 - # OBR EFO March 2026: Q3 CPI 12-month rate minus calendar-year CPI inflation (economy Table 1.7). + # OBR EFO March 2026: September CPI (receipts Table 3.19) minus calendar-year CPI growth in yoy_growth.yaml. + 2025-09-01: 0.00411 + 2026-09-01: -0.00184 + 2027-09-01: 0.00072 + 2028-09-01: 0.0003 + 2029-09-01: 0.00048 + # OBR EFO March 2026: Q3 CPI 12-month rate (economy Table 1.7) minus calendar-year growth in yoy_growth.yaml. 2030-09-01: 0 # After the EFO horizon: no gap, so the forecast is calendar-year growth. 2031-09-01: 0 @@ -17,13 +17,15 @@ metadata: label: September CPI forecast gap to calendar-year CPI preserve_calendar_dates: true documentation: | - The OBR publishes September CPI forecasts in its receipts tables, as - the CPI used to uprate tax thresholds. Where that row stops before the + The gap is the OBR's September CPI forecast minus the calendar-year CPI + growth stored in yoy_growth.yaml, so that stored growth plus the gap is + the OBR's figure. The OBR publishes September CPI in its receipts tables, + as the CPI used to uprate tax thresholds. Where that row stops before the end of the forecast, the gap uses the quarter containing September (Q3 - CPI, economy Table 1.7). Both come from the same Economic and Fiscal - Outlook as the calendar-year series, and are regenerated by - policyengine_uk/utils/import_obr_forecasts.py. The gap is zero after the - EFO horizon, where the September forecast falls back to calendar-year CPI. + CPI, economy Table 1.7). policyengine_uk/utils/import_obr_forecasts.py + regenerates the gaps with each EFO (or with --gaps-only after editing + yoy_growth.yaml). The gap is zero after the EFO horizon, where the + September forecast falls back to calendar-year CPI. reference: - title: OBR EFO March 2026 (detailed forecast tables, economy, Table 1.7) href: https://obr.uk/efo/economic-and-fiscal-outlook-march-2026/ diff --git a/policyengine_uk/tests/test_import_obr_forecasts.py b/policyengine_uk/tests/test_import_obr_forecasts.py index 2934cb5851..e219ab20e8 100644 --- a/policyengine_uk/tests/test_import_obr_forecasts.py +++ b/policyengine_uk/tests/test_import_obr_forecasts.py @@ -1,8 +1,17 @@ +from datetime import date from io import BytesIO from zipfile import ZipFile +import pytest +import yaml + from policyengine_uk.utils.import_obr_forecasts import ( STATUTORY_GAP_SPECS, + ForecastGap, + format_gap, + main, + read_calendar_values, + render_forecast_gap_yaml, build_efo_href, compute_statutory_forecast_gaps, extract_annual_series_from_xlsx, @@ -35,7 +44,7 @@ def make_sheet(rows: dict[int, list[str]]) -> bytes: return xml.encode() -def make_test_xlsx() -> bytes: +def make_test_xlsx(earnings_quarter: bool = True) -> bytes: workbook_xml = """ @@ -65,10 +74,16 @@ def make_test_xlsx() -> bytes: make_inline_cell("B98", "2026"), make_number_cell("Q98", 3.33), ], - 99: [ - make_inline_cell("B99", "2026Q2"), - make_number_cell("Q99", 3.67), - ], + **( + { + 99: [ + make_inline_cell("B99", "2026Q2"), + make_number_cell("Q99", 3.67), + ] + } + if earnings_quarter + else {} + ), } ) sheet_17 = make_sheet( @@ -451,3 +466,170 @@ def test_update_forecast_gap_yaml_rewrites_values_and_reference(tmp_path): "OBR EFO March 2026 (detailed forecast tables, receipts, Table 3.19)" in content ) assert "https://obr.uk/efo/economic-and-fiscal-outlook-march-2026/" in content + + +def test_gaps_are_measured_from_the_stored_calendar_growth(): + """Stored growth plus the gap must equal the OBR figure, so the gap is + taken from whatever yoy_growth.yaml holds, not the unrounded workbook.""" + gaps = compute_statutory_forecast_gaps( + make_test_xlsx(), + 2026, + 1, + calendar_values={ + "consumer_price_index": {2026: 0.025}, + "average_earnings": {2026: 0.033}, + }, + ) + assert gaps["cpi_september"][2026].value == pytest.approx(0.0208 - 0.025) + assert gaps["awe_total_pay_may_july"][2026].value == pytest.approx(0.0367 - 0.033) + + +def test_read_calendar_values(tmp_path): + path = tmp_path / "yoy_growth.yaml" + path.write_text(YOY_GROWTH_FIXTURE) + values = read_calendar_values(path) + assert values["consumer_price_index"] == { + 2024: 0.02, + 2025: 0.0, + 2026: 0.0, + 2031: 0.02, + } + + +@pytest.mark.parametrize( + "value, text", + [ + (3e-05, "0.00003"), + (-4e-05, "-0.00004"), + (0.0003, "0.0003"), + (0.0, "0"), + (-0.00184, "-0.00184"), + ], +) +def test_small_gaps_are_written_in_fixed_point(value, text): + """YAML 1.1 reads 3e-05 as a string, which breaks parameter loading.""" + assert format_gap(value) == text + spec = next(s for s in STATUTORY_GAP_SPECS if s.key == "cpi_september") + content = render_forecast_gap_yaml( + GAP_FIXTURE, spec, {2027: ForecastGap(value, "test", "test")}, "March", 2026 + ) + assert isinstance(yaml.safe_load(content)["values"][date(2027, 9, 1)], (int, float)) + + +SERIES_KEYS = [ + "rpi", + "average_earnings", + "consumer_price_index", + "cpih", + "house_prices", + "mortgage_interest", + "rent", +] +YOY_GROWTH_FIXTURE = "obr:\n" + "".join( + f""" {key}: + values: + 2024-01-01: 0.0200 + 2025-01-01: 0.0000 + 2026-01-01: 0.0000 + 2031-01-01: 0.0200 + metadata: + reference: + - title: Old + href: https://example.com/old +""" + for key in SERIES_KEYS +) +GAP_FIXTURE = """description: Gap. +values: + 2010-09-01: 0 +metadata: + unit: /1 + reference: + - title: Old + href: https://example.com/old +""" + + +def write_tree(tmp_path): + """A yoy_growth.yaml with the forecast_gap files next to it.""" + yoy = tmp_path / "economic_assumptions" / "yoy_growth.yaml" + gap_dir = yoy.parent / "statutory_uprating_inputs" / "forecast_gap" + gap_dir.mkdir(parents=True) + yoy.write_text(YOY_GROWTH_FIXTURE) + for spec in STATUTORY_GAP_SPECS: + (gap_dir / f"{spec.key}.yaml").write_text( + GAP_FIXTURE.replace("09-01", spec.month_day) + ) + return yoy, gap_dir + + +def run_main(tmp_path, workbook, *extra): + economy = tmp_path / "economy_march_2026.xlsx" + economy.write_bytes(workbook) + receipts = tmp_path / "receipts.xlsx" + receipts.write_bytes(make_receipts_xlsx()) + return main( + [ + "--file", + str(economy), + "--forecast-start-year", + "2025", + "--forecast-years", + "2", + "--release-month", + "March", + "--release-year", + "2026", + "--receipts-file", + str(receipts), + *extra, + ] + ) + + +def test_main_writes_growth_and_gaps_next_to_the_yaml_path(tmp_path): + yoy, gap_dir = write_tree(tmp_path) + assert run_main(tmp_path, make_test_xlsx(), "--yaml-path", str(yoy)) == 0 + + assert "2026-01-01: 0.0248" in yoy.read_text() + cpi = yaml.safe_load((gap_dir / "cpi_september.yaml").read_text())["values"] + # September 2026 (2.12%) minus the 2026 CPI just written (2.48%). + assert cpi[date(2026, 9, 1)] == pytest.approx(-0.0036) + awe = yaml.safe_load((gap_dir / "awe_total_pay_may_july.yaml").read_text()) + assert awe["values"][date(2026, 7, 1)] == pytest.approx(0.0034) + + +def test_main_writes_nothing_when_a_gap_cannot_be_built(tmp_path): + """Without the Q2 earnings row there is no earnings gap: the run fails + before any file changes, so growth and gaps never mix two forecasts.""" + yoy, gap_dir = write_tree(tmp_path) + before = {path: path.read_text() for path in [yoy, *gap_dir.iterdir()]} + + with pytest.raises(ValueError, match="No forecast gaps"): + run_main( + tmp_path, make_test_xlsx(earnings_quarter=False), "--yaml-path", str(yoy) + ) + + assert {path: path.read_text() for path in before} == before + + +def test_main_requires_the_receipts_tables_for_the_gaps(tmp_path): + yoy, _ = write_tree(tmp_path) + economy = tmp_path / "economy_march_2026.xlsx" + economy.write_bytes(make_test_xlsx()) + with pytest.raises(ValueError, match="receipts"): + main(["--file", str(economy), "--yaml-path", str(yoy)]) + + +def test_gaps_only_leaves_growth_alone(tmp_path): + yoy, gap_dir = write_tree(tmp_path) + growth = yoy.read_text() + assert ( + run_main(tmp_path, make_test_xlsx(), "--yaml-path", str(yoy), "--gaps-only") + == 0 + ) + + assert yoy.read_text() == growth + cpi = yaml.safe_load((gap_dir / "cpi_september.yaml").read_text())["values"] + # September 2026 (2.12%) minus the stored 2026 CPI (0%). + assert cpi[date(2026, 9, 1)] == pytest.approx(0.0212) diff --git a/policyengine_uk/tests/test_triple_lock_properties.py b/policyengine_uk/tests/test_triple_lock_properties.py index 1eb1fe838c..2de3a187bc 100644 --- a/policyengine_uk/tests/test_triple_lock_properties.py +++ b/policyengine_uk/tests/test_triple_lock_properties.py @@ -22,7 +22,7 @@ import math import pytest -from hypothesis import given, settings +from hypothesis import HealthCheck, given, settings from hypothesis import strategies as st from policyengine_uk.parameters.gov.dwp.state_pension.triple_lock.create_triple_lock import ( @@ -39,6 +39,10 @@ published_rate = st.integers(min_value=-60, max_value=150).map(lambda n: n / 1000) minimum_rate = st.sampled_from([0.0, 0.02, 0.025, 0.03]) LEVEL_TOLERANCE = 1e-9 +# Generation is cheap, but a loaded CI runner can trip the speed check. +PROPERTIES = settings( + max_examples=300, deadline=None, suppress_health_check=[HealthCheck.too_slow] +) def uprating_year(earnings_path_guarantee=st.booleans()): @@ -60,6 +64,7 @@ def paths(earnings_path_guarantee=st.booleans(), min_size=1, max_size=20): ).map(lambda years: {2030 + i: year for i, year in enumerate(years)}) +@PROPERTIES @given( earnings=published_rate, cpi=published_rate, @@ -84,6 +89,7 @@ def test_triple_lock_is_the_highest_included_element( assert rate in included +@PROPERTIES @given(inputs=uprating_year()) def test_rule_rate_is_the_triple_lock_or_the_statutory_earnings_link(inputs): rate = rule_rate(inputs) @@ -100,6 +106,7 @@ def test_rule_rate_is_the_triple_lock_or_the_statutory_earnings_link(inputs): assert rate >= 0 +@PROPERTIES @given( earnings=published_rate, cpi=published_rate, @@ -113,13 +120,14 @@ def test_triple_lock_never_falls_when_an_element_rises(earnings, cpi, floor, ris assert triple_lock_rate(earnings, cpi, floor + rise) >= rate +@PROPERTIES @given(years=paths(earnings_path_guarantee=st.just(False))) def test_without_guarantee_the_path_is_the_yearly_rule(years): rates = uprating_rates(years) assert rates == {year: rule_rate(inputs) for year, inputs in years.items()} -@settings(max_examples=300) +@PROPERTIES @given(years=paths()) def test_guarantee_keeps_the_pension_on_or_above_its_earnings_path(years): rates = uprating_rates(years) @@ -143,7 +151,7 @@ def test_guarantee_keeps_the_pension_on_or_above_its_earnings_path(years): assert level >= rule_path * (1 - LEVEL_TOLERANCE) -@settings(max_examples=300) +@PROPERTIES @given(years=paths()) def test_guarantee_top_up_is_the_smallest_step_that_reaches_the_path(years): rates = uprating_rates(years) @@ -163,6 +171,7 @@ def test_guarantee_top_up_is_the_smallest_step_that_reaches_the_path(years): level *= 1 + rates[year] +@PROPERTIES @given(years=paths()) def test_rates_stay_on_the_published_grid(years): for rate in uprating_rates(years).values(): @@ -195,7 +204,7 @@ def rnd_up(r): return rates -@settings(max_examples=300) +@PROPERTIES @given( data=st.lists(st.tuples(published_rate, published_rate), min_size=1, max_size=15), switch_index=st.integers(min_value=0, max_value=15), @@ -218,6 +227,7 @@ def test_plan_matches_the_uk_triple_lock_reference(data, switch_index): assert [rates[year] for year in sorted(rates)] == pytest.approx(expected, abs=1e-12) +@PROPERTIES @given(rate=st.floats(min_value=-0.5, max_value=0.5, allow_nan=False)) def test_rounding_to_published_precision(rate): rounded = round_to_published_precision(rate) diff --git a/policyengine_uk/tests/test_triple_lock_statutory_inputs.py b/policyengine_uk/tests/test_triple_lock_statutory_inputs.py index ebfee9cadc..1f459e018e 100644 --- a/policyengine_uk/tests/test_triple_lock_statutory_inputs.py +++ b/policyengine_uk/tests/test_triple_lock_statutory_inputs.py @@ -9,6 +9,7 @@ parameter reform. """ +import numpy as np import pandas as pd import pytest @@ -51,8 +52,10 @@ def statutory_inputs(parameters): def test_april_2027_uses_published_may_july_2026_earnings(): - """May-July 2026 total pay growth was 3.9%, above any CPI forecast, so the - new State Pension rises from £241.30 to £250.71 (before rounding to 5p).""" + """May-July 2026 total pay growth was 3.9% (first estimate), above the + model's September 2026 CPI forecast (the OBR's 2.1%; August 2026 CPI was + 3.1%). Unless September CPI exceeds 3.9%, the new State Pension rises from + £241.30 to £250.71 (before rounding to 5p).""" parameters = system.parameters assert uprating(parameters, 2027) == pytest.approx(0.039) assert new_state_pension_weekly(parameters, 2027) == pytest.approx( @@ -113,16 +116,18 @@ def test_forecasts_are_calendar_growth_plus_the_obr_gap( OBR_Q2_EARNINGS = {2027: 0.024266, 2028: 0.020795, 2029: 0.021852, 2030: 0.023890} -def test_baseline_forecasts_track_the_obr_statutory_forecasts(): - """Calendar-year growth is held to 0.1pp in yoy_growth.yaml, so the - forecasts match the OBR's to within that rounding.""" +def test_baseline_forecasts_reproduce_the_obr_statutory_forecasts(): + """The gaps are measured from the calendar-year growth stored in + yoy_growth.yaml, so stored growth plus the gap is the OBR's figure, to + the 1e-5 the gaps are stored to. This fails if yoy_growth.yaml is + refreshed without regenerating the gaps.""" inputs = statutory_inputs(system.parameters) for year, value in OBR_SEPTEMBER_CPI.items(): - assert inputs.cpi_september(f"{year}-09-01") == pytest.approx(value, abs=0.0005) + observed = inputs.cpi_september(f"{year}-09-01") + assert observed == pytest.approx(value, abs=1e-5), year for year, value in OBR_Q2_EARNINGS.items(): - assert inputs.awe_total_pay_may_july(f"{year}-07-01") == pytest.approx( - value, abs=0.0005 - ) + observed = inputs.awe_total_pay_may_july(f"{year}-07-01") + assert observed == pytest.approx(value, abs=1e-5), year def test_rule_reproduces_the_obr_triple_lock_forecast_from_its_own_inputs(): @@ -148,7 +153,7 @@ def drop_may_july_2026(simulation): parameters = simulation.tax_benefit_system.parameters assert statutory_inputs(parameters).awe_total_pay_may_july( "2026-07-01" - ) == pytest.approx(0.034 + 0.00316) + ) == pytest.approx(0.036736, abs=1e-5) assert [uprating(parameters, year) for year in range(2027, 2032)] == ( pytest.approx([0.037, 0.025, 0.025, 0.025, 0.025]) ) @@ -167,14 +172,18 @@ def test_horizon_runs_to_the_end_of_the_economic_assumptions(): def test_macro_scenario_on_calendar_growth_moves_the_uprating(): """Raising 2027 earnings growth from 2.4% to 5% moves May-July 2027 - earnings with it (plus the OBR gap) and so the April 2028 rise.""" + earnings with it (plus the OBR gap, 0.027pp) and so the April 2028 rise. + + Calendar-year series are keyed to 1 January, so the change is keyed + year:2027-01-01:1; a bare "2027" names the fiscal year from 6 April 2027 + and would change the 2028 calendar-year value instead.""" parameters = parameters_under( {f"{OBR}.average_earnings": {"year:2027-01-01:1": 0.05}} ) assert statutory_inputs(parameters).awe_total_pay_may_july( "2027-07-01" - ) == pytest.approx(0.05 + 0.00065) - assert uprating(parameters, 2028) == pytest.approx(0.051) + ) == pytest.approx(0.05 + 0.00027) + assert uprating(parameters, 2028) == pytest.approx(0.05) assert uprating(system.parameters, 2028) == pytest.approx(0.025) assert new_state_pension_weekly(parameters, 2028) > new_state_pension_weekly( system.parameters, 2028 @@ -224,7 +233,7 @@ def test_macro_scenario_moves_the_state_pension_in_a_microsimulation(): full_rate_2027 * 1.025 * WEEKS_IN_YEAR, rel=1e-5 ) assert reformed_pension == pytest.approx( - full_rate_2027 * 1.051 * WEEKS_IN_YEAR, rel=1e-5 + full_rate_2027 * 1.05 * WEEKS_IN_YEAR, rel=1e-5 ) @@ -337,3 +346,17 @@ def test_no_economic_assumptions_leaves_only_the_floor(): cutoff_year = int(simulation.default_input_period) for year in range(cutoff_year + 2, 2075): assert uprating(parameters, year) == pytest.approx(0.025), year + + +def test_numpy_inputs_are_accepted(): + """Draws from a time-series model often arrive as numpy scalars.""" + simulation = Simulation(situation=PENSIONER) + simulation.apply_parameter_changes( + { + f"{INPUTS}.cpi_september": {"2027": np.float64(0.045)}, + f"{OBR}.average_earnings": {"year:2028-01-01:1": np.float64(0.031)}, + } + ) + parameters = simulation.tax_benefit_system.parameters + assert uprating(parameters, 2028) == pytest.approx(0.045) + assert uprating(parameters, 2029) == pytest.approx(0.031) diff --git a/policyengine_uk/utils/import_obr_forecasts.py b/policyengine_uk/utils/import_obr_forecasts.py index 0bcaa3cf0b..47701204df 100644 --- a/policyengine_uk/utils/import_obr_forecasts.py +++ b/policyengine_uk/utils/import_obr_forecasts.py @@ -99,9 +99,11 @@ class SeriesSpec: class StatutoryGapSpec: """Forecast gap between a statutory uprating input and calendar-year growth. - The OBR does not forecast the statutory measures themselves, so the gap is - the quarter containing (or nearest) the statutory period minus the - calendar-year figure, both from the same workbook column. + The OBR publishes September CPI only as the receipts tables' memo row of + the CPI used to uprate tax thresholds, and does not forecast May-July AWE + at all. Where the September row is not supplied or does not cover a year, + the statutory figure is the quarter containing (or nearest) the statutory + period, from the same economy-table column as the calendar-year figure. """ key: str @@ -117,16 +119,14 @@ class StatutoryGapSpec: series_key="consumer_price_index", quarter=3, month_day="09-01", - description="Q3 CPI 12-month rate minus calendar-year CPI inflation", + description="Q3 CPI 12-month rate", ), StatutoryGapSpec( key="awe_total_pay_may_july", series_key="average_earnings", quarter=2, month_day="07-01", - description=( - "Q2 average earnings growth on a year earlier minus calendar-year growth" - ), + description="Q2 average earnings growth on a year earlier", ), ) @@ -147,12 +147,10 @@ def get_yoy_growth_path() -> Path: ) -def get_forecast_gap_dir() -> Path: - return ( - get_repo_root() - / "policyengine_uk/parameters/gov/economic_assumptions" - / "statutory_uprating_inputs/forecast_gap" - ) +def get_forecast_gap_dir(yoy_growth_path: Path | None = None) -> Path: + """The forecast_gap directory next to a yoy_growth.yaml.""" + yoy_growth_path = yoy_growth_path or get_yoy_growth_path() + return yoy_growth_path.parent / "statutory_uprating_inputs" / "forecast_gap" def normalise_label(value: str | None) -> str: @@ -393,16 +391,22 @@ def compute_statutory_forecast_gaps( forecast_start_year: int, forecast_years: int, receipts_xlsx_bytes: bytes | None = None, + calendar_values: dict[str, dict[int, float]] | None = None, ) -> dict[str, dict[int, ForecastGap]]: """Gap for each statutory input and forecast year, rounded to 1e-5. - September CPI comes from the receipts tables where they are supplied and - cover the year; otherwise each input uses its quarter from the economy - tables. + The gap is the OBR's statutory-basis figure minus the calendar-year growth + PolicyEngine stores, so that stored growth plus the gap reproduces the + OBR figure. ``calendar_values`` defaults to the values this importer + writes to yoy_growth.yaml. September CPI comes from the receipts tables + where they are supplied and cover the year; otherwise each input uses its + quarter from the economy tables. """ series = extract_series_by_period_from_xlsx( xlsx_bytes, {spec.series_key for spec in STATUTORY_GAP_SPECS} ) + if calendar_values is None: + calendar_values = extract_annual_series_from_xlsx(xlsx_bytes) september_table, september_cpi = ( extract_september_cpi_from_receipts(receipts_xlsx_bytes) if receipts_xlsx_bytes is not None @@ -411,17 +415,18 @@ def compute_statutory_forecast_gaps( gaps: dict[str, dict[int, ForecastGap]] = {} for spec in STATUTORY_GAP_SPECS: values = series[spec.series_key] + calendar = calendar_values.get(spec.series_key, {}) table = next(s.table for s in SERIES_SPECS if s.key == spec.series_key) gaps[spec.key] = {} for year in range(forecast_start_year, forecast_start_year + forecast_years): - annual = values.get(str(year)) + annual = calendar.get(year) if annual is None: continue if spec.key == "cpi_september" and year in september_cpi: statutory = september_cpi[year] source = ( f"September CPI (receipts Table {september_table}) minus " - f"calendar-year CPI (economy Table {table})" + "calendar-year CPI growth in yoy_growth.yaml" ) reference = ( f"detailed forecast tables, receipts, Table {september_table}" @@ -430,7 +435,10 @@ def compute_statutory_forecast_gaps( statutory = values.get(f"{year}Q{spec.quarter}") if statutory is None: continue - source = f"{spec.description} (economy Table {table})" + source = ( + f"{spec.description} (economy Table {table}) minus " + "calendar-year growth in yoy_growth.yaml" + ) reference = f"detailed forecast tables, economy, Table {table}" gaps[spec.key][year] = ForecastGap( value=round(statutory - annual, 5) + 0.0, @@ -440,6 +448,28 @@ def compute_statutory_forecast_gaps( return gaps +def read_calendar_values(yaml_path: Path) -> dict[str, dict[int, float]]: + """Calendar-year growth by year for each series in a yoy_growth.yaml.""" + obr = yaml.safe_load(yaml_path.read_text())["obr"] + return { + key: { + int(str(instant)[:4]): float(value) + for instant, value in series["values"].items() + } + for key, series in obr.items() + if isinstance(series, dict) and "values" in series + } + + +def format_gap(value: float) -> str: + """Fixed-point text: YAML 1.1 reads exponent notation such as 3e-05 as a + string.""" + if value == 0: + return "0" + text = f"{value:.5f}".rstrip("0") + return text + "0" if text.endswith(".") else text + + def render_forecast_gap_values( spec: StatutoryGapSpec, gaps: dict[int, ForecastGap], month: str, year: int ) -> str: @@ -456,7 +486,9 @@ def render_forecast_gap_values( if gaps[gap_year].source != source: source = gaps[gap_year].source lines.append(f" # OBR EFO {month} {year}: {source}.") - lines.append(f" {gap_year}-{spec.month_day}: {gaps[gap_year].value:g}") + lines.append( + f" {gap_year}-{spec.month_day}: {format_gap(gaps[gap_year].value)}" + ) lines += [ " # After the EFO horizon: no gap, so the forecast is calendar-year growth.", f" {max(gaps) + 1}-{spec.month_day}: 0", @@ -481,15 +513,15 @@ def render_forecast_gap_references( return "\n".join(lines) + "\n" -def update_forecast_gap_yaml( - yaml_path: Path, +def render_forecast_gap_yaml( + content: str, spec: StatutoryGapSpec, gaps: dict[int, ForecastGap], month: str, year: int, -) -> None: - """Rewrite the values block and the reference list (the last metadata key).""" - content = yaml_path.read_text() +) -> str: + """Rewrite the values block and the reference list (the last metadata key), + and check every value loads as a number.""" values_pattern = re.compile( r"^values:\n.*?(?=^metadata:)", re.MULTILINE | re.DOTALL ) @@ -505,9 +537,24 @@ def update_forecast_gap_yaml( count=1, ) if not (values_count and reference_count): - raise ValueError(f"Could not find the values and reference in {yaml_path}") - yaml.safe_load(content) - yaml_path.write_text(content) + raise ValueError(f"Could not find the values and reference for {spec.key}") + loaded = yaml.safe_load(content)["values"] + bad = {k: v for k, v in loaded.items() if not isinstance(v, (int, float))} + if bad: + raise ValueError(f"Non-numeric forecast gaps for {spec.key}: {bad}") + return content + + +def update_forecast_gap_yaml( + yaml_path: Path, + spec: StatutoryGapSpec, + gaps: dict[int, ForecastGap], + month: str, + year: int, +) -> None: + yaml_path.write_text( + render_forecast_gap_yaml(yaml_path.read_text(), spec, gaps, month, year) + ) def infer_release(source_name: str) -> tuple[str, int]: @@ -581,7 +628,26 @@ def update_yoy_growth_yaml( forecast_start_year: int, forecast_years: int, ) -> None: - content = yaml_path.read_text() + yaml_path.write_text( + render_yoy_growth_yaml( + yaml_path.read_text(), + series_values, + month, + year, + forecast_start_year, + forecast_years, + ) + ) + + +def render_yoy_growth_yaml( + content: str, + series_values: dict[str, dict[int, float]], + month: str, + year: int, + forecast_start_year: int, + forecast_years: int, +) -> str: yaml.safe_load(content) forecast_end_year = forecast_start_year + forecast_years - 1 href = build_efo_href(month, year) @@ -618,7 +684,8 @@ def update_yoy_growth_yaml( ) content = replace_series_section(content, spec.key, section) - yaml_path.write_text(content) + yaml.safe_load(content) + return content def print_summary( @@ -687,6 +754,14 @@ def build_arg_parser() -> argparse.ArgumentParser: ), ) parser.add_argument("--receipts-file", help="Local receipts XLSX file path") + parser.add_argument( + "--gaps-only", + action="store_true", + help=( + "Leave yoy_growth.yaml as it is and regenerate the forecast gaps " + "against the calendar-year growth it holds" + ), + ) parser.add_argument( "--skip-statutory-gaps", action="store_true", @@ -723,41 +798,64 @@ def main(argv: list[str] | None = None) -> int: forecast_start_year = args.forecast_start_year or infer_forecast_start_year( month, year ) + source_yaml_path = args.yaml_path or get_yoy_growth_path() + write_gaps = not args.skip_statutory_gaps and args.output is None + if args.gaps_only and not write_gaps: + raise ValueError("--gaps-only cannot be combined with --output") + if write_gaps and not (args.receipts_url or args.receipts_file): + message = ( + "Pass --receipts-file or --receipts-url so September CPI uses the " + "OBR's September forecast, or --skip-statutory-gaps" + ) + if not args.dry_run: + raise ValueError(message) + print(f"Warning: {message}") + print_summary(series_values, forecast_start_year, args.forecast_years) receipts_bytes = None if args.receipts_url or args.receipts_file: _, receipts_bytes = load_source_bytes(args.receipts_url, args.receipts_file) + calendar_values = ( + read_calendar_values(source_yaml_path) if args.gaps_only else series_values + ) statutory_gaps = compute_statutory_forecast_gaps( - workbook_bytes, forecast_start_year, args.forecast_years, receipts_bytes + workbook_bytes, + forecast_start_year, + args.forecast_years, + receipts_bytes, + calendar_values, ) for key, gaps in statutory_gaps.items(): - window = ", ".join(f"{y}: {g.value:g}" for y, g in sorted(gaps.items())) + window = ", ".join( + f"{y}: {format_gap(g.value)}" for y, g in sorted(gaps.items()) + ) print(f"- forecast gap {key}: {window}") - if args.dry_run: - return 0 - - source_yaml_path = args.yaml_path or get_yoy_growth_path() - target_yaml_path = args.output or source_yaml_path - if target_yaml_path != source_yaml_path: - target_yaml_path.write_text(source_yaml_path.read_text()) - update_yoy_growth_yaml( - yaml_path=target_yaml_path, - series_values=series_values, - month=month, - year=year, - forecast_start_year=forecast_start_year, - forecast_years=args.forecast_years, - ) - print(f"Updated {target_yaml_path}") - - if not args.skip_statutory_gaps and args.output is None: + # Render and check every file before writing any, so a failure cannot + # leave growth and gaps from different forecasts. + outputs: dict[Path, str] = {} + if not args.gaps_only: + outputs[args.output or source_yaml_path] = render_yoy_growth_yaml( + source_yaml_path.read_text(), + series_values, + month, + year, + forecast_start_year, + args.forecast_years, + ) + if write_gaps: + gap_dir = get_forecast_gap_dir(source_yaml_path) for spec in STATUTORY_GAP_SPECS: - gap_path = get_forecast_gap_dir() / f"{spec.key}.yaml" - update_forecast_gap_yaml( - gap_path, spec, statutory_gaps[spec.key], month, year + gap_path = gap_dir / f"{spec.key}.yaml" + outputs[gap_path] = render_forecast_gap_yaml( + gap_path.read_text(), spec, statutory_gaps[spec.key], month, year ) - print(f"Updated {gap_path}") + + if args.dry_run: + return 0 + for path, content in outputs.items(): + path.write_text(content) + print(f"Updated {path}") return 0 From f5629fd8dbc8ea6835e27407173c084443d639f9 Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Wed, 30 Sep 2026 11:34:12 -0400 Subject: [PATCH 3/6] Close mutation-testing gaps in the triple lock tests Mutation testing of the first commit left 18 mutants alive. New tests kill them: an independent table of the inputs each review used (typos in non-binding years), exact OBR gaps, zero inputs and zero overrides kept as values, an override under the earnings-path guarantee, levers read on 30 April of the year they name, input rounding, the horizon following the shorter calendar series, the missing-input guard, no_economic_assumptions zeroing the gaps, half-up ties, and the receipts header row. Co-Authored-By: Claude Opus 5.5 --- .../awe_total_pay_may_july.yaml | 12 +-- .../tests/test_import_obr_forecasts.py | 5 ++ .../tests/test_triple_lock_outturn.py | 29 ++++++ .../tests/test_triple_lock_properties.py | 26 ++++++ .../test_triple_lock_statutory_inputs.py | 90 +++++++++++++++++++ 5 files changed, 157 insertions(+), 5 deletions(-) diff --git a/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/awe_total_pay_may_july.yaml b/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/awe_total_pay_may_july.yaml index e7db53d4a0..61090446d6 100644 --- a/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/awe_total_pay_may_july.yaml +++ b/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/awe_total_pay_may_july.yaml @@ -3,11 +3,13 @@ values: # Outturn: the figure each year's review used, as stated in the # Explanatory Memorandum to the following April's Social Security Benefits # Up-rating Order (for 2021, the Government Actuary's report on the 2022 - # Order). Reviews use the October labour market release, which the - # explanatory notes to the 2021 Up-rating of Benefits Act call the final - # figure. It can differ from the September first estimate (May-July 2024: - # 4.0% in September, 4.1% in October and used) and from the latest ONS - # vintage of KAC3 (4.4% for May-July 2024). + # Order). Reviews use the latest estimate before the autumn uprating + # statement: the October labour market release's figure, unchanged in + # November in every year checked, which the explanatory notes to the 2021 + # Up-rating of Benefits Act call the final figure. It can differ from the + # September first estimate (May-July 2024: 4.0% in September, 4.1% in + # October and used) and from the latest ONS vintage of KAC3 (4.4% for + # May-July 2024). 2010-07-01: 0.013 # SI 2011/821 EM, footnote 7 2011-07-01: 0.028 # SI 2012/780 EM 2012-07-01: 0.016 # SI 2013/574 EM diff --git a/policyengine_uk/tests/test_import_obr_forecasts.py b/policyengine_uk/tests/test_import_obr_forecasts.py index e219ab20e8..99a625fb65 100644 --- a/policyengine_uk/tests/test_import_obr_forecasts.py +++ b/policyengine_uk/tests/test_import_obr_forecasts.py @@ -380,6 +380,11 @@ def make_receipts_xlsx() -> bytes: sheet_318 = make_sheet({2: [make_inline_cell("B2", "Other table")]}) sheet_319 = make_sheet( { + # An earlier table on the same sheet, with other years. + 2: [ + make_inline_cell("C2", "2020-21"), + make_inline_cell("D2", "2021-22"), + ], 4: [ make_inline_cell("C4", "2026-27"), make_inline_cell("D4", "2027-28"), diff --git a/policyengine_uk/tests/test_triple_lock_outturn.py b/policyengine_uk/tests/test_triple_lock_outturn.py index d6d167490b..588157b62e 100644 --- a/policyengine_uk/tests/test_triple_lock_outturn.py +++ b/policyengine_uk/tests/test_triple_lock_outturn.py @@ -46,6 +46,25 @@ (2026, 0.048, "earnings", "SI 2026/148"), ] +# The inputs each review used, transcribed independently of the parameter +# files, so a typo in a year where the input did not set the rate is still +# caught. September CPI: ONS D7G7. May-July earnings: DWP, Abstract of DWP +# benefit rate statistics 2025, table 5 ("Earnings (KAC3)"), except 2017, +# where the table shows a later revision (2.3%) and the Explanatory +# Memorandum to SI 2018/281 gives the 2.2% the review used. +SEPTEMBER_CPI_USED = { + 2010: 0.031, 2011: 0.052, 2012: 0.022, 2013: 0.027, 2014: 0.012, + 2015: -0.001, 2016: 0.010, 2017: 0.030, 2018: 0.024, 2019: 0.017, + 2020: 0.005, 2021: 0.031, 2022: 0.101, 2023: 0.067, 2024: 0.017, + 2025: 0.038, +} # fmt: skip +MAY_JULY_EARNINGS_USED = { + 2010: 0.013, 2011: 0.028, 2012: 0.016, 2013: 0.012, 2014: 0.006, + 2015: 0.029, 2016: 0.024, 2017: 0.022, 2018: 0.026, 2019: 0.039, + 2020: -0.010, 2021: 0.083, 2022: 0.055, 2023: 0.085, 2024: 0.041, + 2025: 0.048, +} # fmt: skip + # (year, expected weekly £). Cross-referenced against gov.uk benefit and # pension rates publications. BASIC_STATE_PENSION_WEEKLY = [ @@ -76,6 +95,16 @@ def nearest_5p(amount): ) +def test_inputs_are_the_figures_each_review_used(): + inputs = parameters.gov.economic_assumptions.statutory_uprating_inputs + for year, value in SEPTEMBER_CPI_USED.items(): + assert inputs.cpi_september(f"{year}-09-01") == pytest.approx(value), year + for year, value in MAY_JULY_EARNINGS_USED.items(): + assert inputs.awe_total_pay_may_july(f"{year}-07-01") == pytest.approx(value), ( + year + ) + + @pytest.mark.parametrize("year, rate, element, order", PUBLISHED_UPRATINGS) def test_uprating_matches_published_rate(year, rate, element, order): assert uprating(year) == pytest.approx(rate, abs=1e-9), order diff --git a/policyengine_uk/tests/test_triple_lock_properties.py b/policyengine_uk/tests/test_triple_lock_properties.py index 2de3a187bc..1abe452122 100644 --- a/policyengine_uk/tests/test_triple_lock_properties.py +++ b/policyengine_uk/tests/test_triple_lock_properties.py @@ -239,6 +239,9 @@ def test_rounding_to_published_precision(rate): def test_rounding_takes_halves_away_from_zero(): assert round_to_published_precision(0.0255) == 0.026 + # Round-half-even would give 0.024 and -0.024. + assert round_to_published_precision(0.0245) == 0.025 + assert round_to_published_precision(-0.0245) == -0.025 assert round_to_published_precision(-0.0095) == -0.010 assert round_up_to_published_precision(0.025000000000000355) == 0.025 assert round_up_to_published_precision(0.02500001) == 0.026 @@ -262,3 +265,26 @@ def test_plan_example_by_hand(): } ) assert rates == {2030: 0.04, 2031: 0.025} + + +def test_overrides_are_paid_as_published_even_when_zero(): + rates = uprating_rates( + {2030: UpratingYear(earnings=0.03, cpi=0.02, minimum_rate=0.025, outturn=0.0)} + ) + assert rates == {2030: 0.0} + + +def test_an_override_under_the_guarantee_is_paid_as_is_and_the_path_carries_on(): + """An override sets the rate with no top-up, but the earnings path still + grows that year: 1.05 against a level of 1.01, so the next year needs + 1.05 / 1.01 - 1 = 3.96%, rounded up to 4.0%.""" + plan = dict( + minimum_rate=0.025, include_earnings=False, earnings_path_guarantee=True + ) + rates = uprating_rates( + { + 2030: UpratingYear(earnings=0.05, cpi=0.0, outturn=0.01, **plan), + 2031: UpratingYear(earnings=0.0, cpi=0.0, **plan), + } + ) + assert rates == {2030: 0.01, 2031: 0.04} diff --git a/policyengine_uk/tests/test_triple_lock_statutory_inputs.py b/policyengine_uk/tests/test_triple_lock_statutory_inputs.py index 1f459e018e..a3101905d0 100644 --- a/policyengine_uk/tests/test_triple_lock_statutory_inputs.py +++ b/policyengine_uk/tests/test_triple_lock_statutory_inputs.py @@ -360,3 +360,93 @@ def test_numpy_inputs_are_accepted(): parameters = simulation.tax_benefit_system.parameters assert uprating(parameters, 2028) == pytest.approx(0.045) assert uprating(parameters, 2029) == pytest.approx(0.031) + + +def test_a_zero_input_is_kept_not_forecast(): + """Zero is a value, not a missing figure: with the triple lock off, the + rise is the 0% earnings input, not the 2.4% forecast.""" + parameters = parameters_under( + { + f"{INPUTS}.awe_total_pay_may_july": {"2027": 0.0}, + f"{INPUTS}.cpi_september": {"2027": 0.0}, + f"{TRIPLE_LOCK}.active": {"year:2028-01-01:1": False}, + } + ) + inputs = statutory_inputs(parameters) + assert inputs.awe_total_pay_may_july("2027-07-01") == 0 + assert inputs.cpi_september("2027-09-01") == 0 + assert uprating(parameters, 2028) == 0 + + +def test_no_economic_assumptions_zeroes_the_forecast_inputs(): + simulation = Simulation(situation=PENSIONER, scenario=no_economic_assumptions) + inputs = statutory_inputs(simulation.tax_benefit_system.parameters) + cutoff_year = int(simulation.default_input_period) + for year in range(max(cutoff_year, 2027), 2074): + assert inputs.cpi_september(f"{year}-09-01") == 0, year + assert inputs.awe_total_pay_may_july(f"{year}-07-01") == 0, year + assert inputs.forecast_gap.cpi_september(f"{year}-09-01") == 0, year + + +def test_policy_levers_apply_from_the_april_of_the_year_they_name(): + """A floor of 3% keyed to the bare year 2028 (the fiscal year from + 6 April 2028) sets the April 2028 rise, read on 30 April, and no other + year's; so do the include flags.""" + low_growth = { + f"{INPUTS}.awe_total_pay_may_july": {"2027": 0.01, "2028": 0.01}, + f"{INPUTS}.cpi_september": {"2027": 0.05, "2028": 0.01}, + } + floor = parameters_under( + {**low_growth, f"{TRIPLE_LOCK}.minimum_rate": {"2028": 0.03}} + ) + assert [uprating(floor, year) for year in (2028, 2029)] == pytest.approx( + [0.05, 0.025] + ) + floor_binds = parameters_under( + { + **low_growth, + f"{INPUTS}.cpi_september": {"2027": 0.01, "2028": 0.01}, + f"{TRIPLE_LOCK}.minimum_rate": {"2028": 0.03}, + } + ) + assert [uprating(floor_binds, year) for year in (2027, 2028, 2029)] == ( + pytest.approx([0.039, 0.03, 0.025]) + ) + no_cpi = parameters_under( + {**low_growth, f"{TRIPLE_LOCK}.include_inflation": {"2028": False}} + ) + assert uprating(no_cpi, 2028) == pytest.approx(0.025) + + +def test_inputs_are_rounded_to_published_precision(): + """The review uses the published one-decimal figure: 3.04% CPI pays + 3.0%.""" + parameters = parameters_under( + { + f"{INPUTS}.cpi_september": {"2027": 0.0304}, + f"{INPUTS}.awe_total_pay_may_july": {"2027": 0.0296}, + } + ) + assert uprating(parameters, 2028) == pytest.approx(0.030) + + +def test_horizon_ends_with_the_shorter_calendar_series(): + parameters = parameters_under( + {f"{OBR}.consumer_price_index": {"year:2075-01-01:1": 0.02}} + ) + triple_lock = parameters.gov.economic_assumptions.yoy_growth.triple_lock + assert max(int(v.instant_str[:4]) for v in triple_lock.values_list) == 2074 + + +def test_the_rule_needs_the_forecasts_filled_in_first(): + """Read before the forecasts are filled in, September 2026 CPI is still + null, and the rule refuses rather than guessing.""" + from policyengine_uk.parameters.gov.dwp.state_pension.triple_lock.create_triple_lock import ( + read_uprating_years, + ) + from policyengine_uk.tax_benefit_system import CountryTaxBenefitSystem + + raw = CountryTaxBenefitSystem() + raw.reset_parameters() + with pytest.raises(ValueError, match="September 2026 CPI"): + read_uprating_years(raw.parameters) From 64fca7f5bc26f2ffaaef0ca6f6522945ffe1bfb1 Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Thu, 1 Oct 2026 09:46:44 -0400 Subject: [PATCH 4/6] Make the OBR importer's file writes all-or-nothing Independent review (Astra gate, round 1) found that an OS error while writing the second or third file left yoy_growth.yaml and one gap file on the new forecast and the other on the old. Each file is now staged beside its target and moved into place; any failure restores the files already moved and removes the staged copies. Tests fail the move at each of the three files, and fail the staging step. Co-Authored-By: Claude Opus 5.5 --- .../tests/test_import_obr_forecasts.py | 51 +++++++++++++++++++ policyengine_uk/utils/import_obr_forecasts.py | 35 ++++++++++++- 2 files changed, 84 insertions(+), 2 deletions(-) diff --git a/policyengine_uk/tests/test_import_obr_forecasts.py b/policyengine_uk/tests/test_import_obr_forecasts.py index 99a625fb65..39ab79e4d9 100644 --- a/policyengine_uk/tests/test_import_obr_forecasts.py +++ b/policyengine_uk/tests/test_import_obr_forecasts.py @@ -1,10 +1,14 @@ +import os from datetime import date +from pathlib import Path from io import BytesIO from zipfile import ZipFile import pytest import yaml +from policyengine_uk.utils import import_obr_forecasts + from policyengine_uk.utils.import_obr_forecasts import ( STATUTORY_GAP_SPECS, ForecastGap, @@ -638,3 +642,50 @@ def test_gaps_only_leaves_growth_alone(tmp_path): cpi = yaml.safe_load((gap_dir / "cpi_september.yaml").read_text())["values"] # September 2026 (2.12%) minus the stored 2026 CPI (0%). assert cpi[date(2026, 9, 1)] == pytest.approx(0.0212) + + +@pytest.mark.parametrize("failing_call", [1, 2, 3]) +def test_main_restores_every_file_when_a_write_fails( + tmp_path, monkeypatch, failing_call +): + """A failure while moving the second or third file into place (or the + first) leaves growth and both gap files exactly as they were.""" + yoy, gap_dir = write_tree(tmp_path) + before = {path: path.read_text() for path in [yoy, *gap_dir.iterdir()]} + real_replace = os.replace + calls = [] + + def flaky_replace(source, target): + calls.append(target) + if len(calls) == failing_call: + raise OSError("disk full") + real_replace(source, target) + + monkeypatch.setattr(import_obr_forecasts.os, "replace", flaky_replace) + with pytest.raises(OSError, match="disk full"): + run_main(tmp_path, make_test_xlsx(), "--yaml-path", str(yoy)) + + assert {path: path.read_text() for path in before} == before + leftovers = [p for p in tmp_path.rglob("*.staged")] + assert leftovers == [] + + +def test_main_leaves_files_alone_when_staging_fails(tmp_path, monkeypatch): + yoy, gap_dir = write_tree(tmp_path) + before = {path: path.read_text() for path in [yoy, *gap_dir.iterdir()]} + real_write_text = Path.write_text + staged = [] + + def flaky_write_text(self, *args, **kwargs): + if self.name.endswith(".staged"): + staged.append(self) + if len(staged) == 2: + raise OSError("disk full") + return real_write_text(self, *args, **kwargs) + + monkeypatch.setattr(Path, "write_text", flaky_write_text) + with pytest.raises(OSError, match="disk full"): + run_main(tmp_path, make_test_xlsx(), "--yaml-path", str(yoy)) + + assert {path: path.read_text() for path in before} == before + assert list(tmp_path.rglob("*.staged")) == [] diff --git a/policyengine_uk/utils/import_obr_forecasts.py b/policyengine_uk/utils/import_obr_forecasts.py index 47701204df..6a2686dcf7 100644 --- a/policyengine_uk/utils/import_obr_forecasts.py +++ b/policyengine_uk/utils/import_obr_forecasts.py @@ -1,6 +1,7 @@ from __future__ import annotations import argparse +import os import re from dataclasses import dataclass from io import BytesIO @@ -778,6 +779,36 @@ def build_arg_parser() -> argparse.ArgumentParser: return parser +def write_all_or_none(outputs: dict[Path, str]) -> None: + """Write every file or leave all of them as they were. + + Each file is staged beside its target, then moved into place. If staging + or a move fails, files already moved get their original text back and + the staged copies are removed, so growth and gaps never mix forecasts. + """ + originals = {path: path.read_text() if path.exists() else None for path in outputs} + staged: dict[Path, Path] = {} + replaced: list[Path] = [] + try: + for path, content in outputs.items(): + staging = path.with_name(f".{path.name}.staged") + staging.write_text(content) + staged[path] = staging + for path, staging in staged.items(): + os.replace(staging, path) + replaced.append(path) + except BaseException: + for path in replaced: + if originals[path] is None: + path.unlink() + else: + path.write_text(originals[path]) + raise + finally: + for staging in staged.values(): + staging.unlink(missing_ok=True) + + def main(argv: list[str] | None = None) -> int: args = build_arg_parser().parse_args(argv) @@ -853,8 +884,8 @@ def main(argv: list[str] | None = None) -> int: if args.dry_run: return 0 - for path, content in outputs.items(): - path.write_text(content) + write_all_or_none(outputs) + for path in outputs: print(f"Updated {path}") return 0 From a9186e1d18fa0c0e06fe15afb5b0be764e43ff0b Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Thu, 1 Oct 2026 10:12:33 -0400 Subject: [PATCH 5/6] Roll back importer writes from on-disk backups Independent review (gate round 2) found that restoring an original with write_text truncates the target first, so a persistent I/O fault during rollback could leave a file empty with no copy of the original. Each existing target is now copied to a backup beside it before anything moves, and rollback restores it with an atomic move. If a restore fails, its backup is kept and named in the error. A new test fails a move and the first restore and checks no file is emptied and the original survives. Co-Authored-By: Claude Opus 5.5 --- .../tests/test_import_obr_forecasts.py | 34 +++++++++++++- policyengine_uk/utils/import_obr_forecasts.py | 44 +++++++++++++++---- 2 files changed, 68 insertions(+), 10 deletions(-) diff --git a/policyengine_uk/tests/test_import_obr_forecasts.py b/policyengine_uk/tests/test_import_obr_forecasts.py index 39ab79e4d9..9e14ed94b9 100644 --- a/policyengine_uk/tests/test_import_obr_forecasts.py +++ b/policyengine_uk/tests/test_import_obr_forecasts.py @@ -666,7 +666,7 @@ def flaky_replace(source, target): run_main(tmp_path, make_test_xlsx(), "--yaml-path", str(yoy)) assert {path: path.read_text() for path in before} == before - leftovers = [p for p in tmp_path.rglob("*.staged")] + leftovers = [*tmp_path.rglob("*.staged"), *tmp_path.rglob("*.backup")] assert leftovers == [] @@ -688,4 +688,36 @@ def flaky_write_text(self, *args, **kwargs): run_main(tmp_path, make_test_xlsx(), "--yaml-path", str(yoy)) assert {path: path.read_text() for path in before} == before + assert [*tmp_path.rglob("*.staged"), *tmp_path.rglob("*.backup")] == [] + + +def test_a_failed_restore_keeps_the_original_and_never_empties_a_file( + tmp_path, monkeypatch +): + """The third move fails and so does the first restore. No file may be + left empty: the unrestored file keeps the new text, its original is kept + in a backup named in the error, and the other file is restored.""" + yoy, gap_dir = write_tree(tmp_path) + before = {path: path.read_text() for path in [yoy, *gap_dir.iterdir()]} + real_replace = os.replace + calls = [] + + def failing_replace(source, target): + calls.append(target) + if len(calls) in (3, 4): + raise OSError("I/O error") + real_replace(source, target) + + monkeypatch.setattr(import_obr_forecasts.os, "replace", failing_replace) + with pytest.raises(OSError, match="original text is kept in") as error: + run_main(tmp_path, make_test_xlsx(), "--yaml-path", str(yoy)) + + unrestored = Path(calls[3]) + backup = unrestored.with_name(f".{unrestored.name}.backup") + assert str(backup) in str(error.value) + assert backup.read_text() == before[unrestored] + for path, original in before.items(): + assert path.read_text(), path + if path not in (unrestored, Path(calls[2])): + assert path.read_text() == original, path assert list(tmp_path.rglob("*.staged")) == [] diff --git a/policyengine_uk/utils/import_obr_forecasts.py b/policyengine_uk/utils/import_obr_forecasts.py index 6a2686dcf7..cba8bce113 100644 --- a/policyengine_uk/utils/import_obr_forecasts.py +++ b/policyengine_uk/utils/import_obr_forecasts.py @@ -3,6 +3,7 @@ import argparse import os import re +import shutil from dataclasses import dataclass from io import BytesIO from pathlib import Path @@ -782,31 +783,56 @@ def build_arg_parser() -> argparse.ArgumentParser: def write_all_or_none(outputs: dict[Path, str]) -> None: """Write every file or leave all of them as they were. - Each file is staged beside its target, then moved into place. If staging - or a move fails, files already moved get their original text back and - the staged copies are removed, so growth and gaps never mix forecasts. + Each new file is staged beside its target and each existing target is + copied to a backup beside it before anything moves. Targets are then + replaced by atomic moves. On any failure, each replaced target is put + back by an atomic move from its backup, which never truncates it, so a + second failure cannot leave a file empty. If a restore fails, its backup + is kept and named in the error. Growth and gaps never mix forecasts + silently. """ - originals = {path: path.read_text() if path.exists() else None for path in outputs} staged: dict[Path, Path] = {} + backups: dict[Path, Path] = {} replaced: list[Path] = [] + keep: set[Path] = set() try: for path, content in outputs.items(): staging = path.with_name(f".{path.name}.staged") staging.write_text(content) staged[path] = staging + for path in outputs: + if path.exists(): + backup = path.with_name(f".{path.name}.backup") + shutil.copy2(path, backup) + backups[path] = backup for path, staging in staged.items(): os.replace(staging, path) replaced.append(path) - except BaseException: + except BaseException as error: + unrestored = [] for path in replaced: - if originals[path] is None: - path.unlink() - else: - path.write_text(originals[path]) + try: + if path in backups: + os.replace(backups[path], path) + else: + path.unlink() + except OSError: + unrestored.append(path) + if unrestored: + keep = {backups[path] for path in unrestored if path in backups} + raise OSError( + "Could not restore " + + ", ".join(str(path) for path in unrestored) + + "; the original text is kept in " + + ", ".join(str(backup) for backup in sorted(keep)) + ) from error raise finally: for staging in staged.values(): staging.unlink(missing_ok=True) + for backup in backups.values(): + if backup not in keep: + backup.unlink(missing_ok=True) def main(argv: list[str] | None = None) -> int: From c45f23f2ab992c5279513231c94348b87faf17cb Mon Sep 17 00:00:00 2001 From: Max Ghenis Date: Fri, 2 Oct 2026 10:09:50 -0400 Subject: [PATCH 6/6] Document triple-lock review qualifications and downstream changes Label the September 2026 earnings estimate and derived pension rates as provisional, describe the exact compatibility changes, and document that projected weekly State Pension amounts are not rounded to 5p. Co-Authored-By: Claude Opus 5.5 --- .../triple-lock-statutory-inputs-compatibility.changed.md | 1 + changelog.d/triple-lock-statutory-inputs.changed.md | 2 +- .../gov/dwp/state_pension/basic_state_pension/amount.yaml | 3 ++- .../gov/dwp/state_pension/new_state_pension/amount.yaml | 3 ++- .../statutory_uprating_inputs/awe_total_pay_may_july.yaml | 5 +++-- 5 files changed, 9 insertions(+), 5 deletions(-) create mode 100644 changelog.d/triple-lock-statutory-inputs-compatibility.changed.md diff --git a/changelog.d/triple-lock-statutory-inputs-compatibility.changed.md b/changelog.d/triple-lock-statutory-inputs-compatibility.changed.md new file mode 100644 index 0000000000..8bca1a60f4 --- /dev/null +++ b/changelog.d/triple-lock-statutory-inputs-compatibility.changed.md @@ -0,0 +1 @@ +Changed `yoy_growth.triple_lock` to use the previous year's September CPI and May-July AWE inputs rounded to 0.1 percentage points before taking the maximum of the included elements and configured floor, including CPI forecast gaps for observation years 2026–2030 and earnings forecast gaps for 2027–2030, with April 2027 determined by the provisional 3.9% earnings input; `triple_lock.outturn` is null from 2012, the generated uprating series runs through April 2074 and follows rounded lagged calendar-year earnings from April 2035 under the stored baseline, and basic and new State Pension levels are about 0.48% higher in 2027–2034 than under the previous baseline, with a growing gap thereafter and the 2027-onward changes provisional until the October 2026 labour market release. diff --git a/changelog.d/triple-lock-statutory-inputs.changed.md b/changelog.d/triple-lock-statutory-inputs.changed.md index 5ebfaa37db..c3849ce615 100644 --- a/changelog.d/triple-lock-statutory-inputs.changed.md +++ b/changelog.d/triple-lock-statutory-inputs.changed.md @@ -1 +1 @@ -Compute the State Pension triple lock from its statutory inputs, September CPI and May-July average weekly earnings total pay growth, instead of calendar-year OBR growth. The inputs hold the published figures each uprating review used, then forecasts: the OBR's September CPI forecast and its Q2 earnings growth (a proxy for May-July AWE) to the end of the EFO, and calendar-year growth after. Macro scenarios that edit calendar-year growth move them. April 2027 now rises by 3.9% (published May-July 2026 earnings) rather than 3.4%, and the rates run to the end of the economic-assumption series instead of 2034. The `active` toggle, which had no effect, now leaves the statutory minimum of earnings growth. +Computed the State Pension triple lock from September CPI and May-July AWE total pay growth used in each uprating review, followed by OBR September CPI and Q2 earnings forecasts and then calendar-year growth, with macro scenarios moving those forecasts and `active: false` leaving non-negative earnings growth; April 2027's 3.9% rise and projected £250.71 new State Pension weekly rate use the 15 September 2026 first earnings estimate and stay provisional until the October labour market release, the vintage the review uses. diff --git a/policyengine_uk/parameters/gov/dwp/state_pension/basic_state_pension/amount.yaml b/policyengine_uk/parameters/gov/dwp/state_pension/basic_state_pension/amount.yaml index 5e88ff3f61..2fe6e3b7b1 100644 --- a/policyengine_uk/parameters/gov/dwp/state_pension/basic_state_pension/amount.yaml +++ b/policyengine_uk/parameters/gov/dwp/state_pension/basic_state_pension/amount.yaml @@ -29,6 +29,7 @@ metadata: unit: currency-GBP label: Basic State Pension amount uprating: gov.economic_assumptions.indices.triple_lock + documentation: Projected weekly State Pension rates are not rounded to 5p; SSAA 1992 s.150A(4) permits discretionary rounding up or down (https://www.legislation.gov.uk/ukpga/1992/5/section/150A/4). reference: - title: House of Commons Library - href: https://researchbriefings.files.parliament.uk/documents/SN05649/SN05649.pdf \ No newline at end of file + href: https://researchbriefings.files.parliament.uk/documents/SN05649/SN05649.pdf diff --git a/policyengine_uk/parameters/gov/dwp/state_pension/new_state_pension/amount.yaml b/policyengine_uk/parameters/gov/dwp/state_pension/new_state_pension/amount.yaml index 3dab192729..6271ec4868 100644 --- a/policyengine_uk/parameters/gov/dwp/state_pension/new_state_pension/amount.yaml +++ b/policyengine_uk/parameters/gov/dwp/state_pension/new_state_pension/amount.yaml @@ -15,6 +15,7 @@ metadata: unit: currency-GBP label: New State Pension amount uprating: gov.economic_assumptions.indices.triple_lock + documentation: Projected weekly State Pension rates are not rounded to 5p; SSAA 1992 s.150A(4) permits discretionary rounding up or down (https://www.legislation.gov.uk/ukpga/1992/5/section/150A/4). reference: - title: House of Commons Library - href: https://researchbriefings.files.parliament.uk/documents/SN05649/SN05649.pdf \ No newline at end of file + href: https://researchbriefings.files.parliament.uk/documents/SN05649/SN05649.pdf diff --git a/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/awe_total_pay_may_july.yaml b/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/awe_total_pay_may_july.yaml index 61090446d6..8da101a37e 100644 --- a/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/awe_total_pay_may_july.yaml +++ b/policyengine_uk/parameters/gov/economic_assumptions/statutory_uprating_inputs/awe_total_pay_may_july.yaml @@ -26,8 +26,9 @@ values: 2023-07-01: 0.085 # SI 2024/242 EM 2024-07-01: 0.041 # SI 2025/295 EM 2025-07-01: 0.048 # SI 2026/148 EM - # First estimate (ONS labour market release, 15 September 2026). Replace - # with the October release, due 20 October 2026, once published. + # Provisional: 3.9% is the 15 September 2026 first estimate and stays + # provisional until the October labour market release (due 20 October), + # the vintage the uprating review uses; replace it once published. 2026-07-01: 0.039 # Not yet published. From here the value is calendar-year earnings growth # (yoy_growth.obr.average_earnings) plus forecast_gap.awe_total_pay_may_july.