Derive economic assumption index horizon from the growth series

changelog.d/derive-index-horizons.fixed.md

@@ -0,0 +1 @@
+Economic assumption indices now end where the growth series end (2074) instead of at 2039, so the State Pension, tax thresholds, benefit rates and every input grown by an index keep moving after 2039 rather than freezing at their 2039 level.

docs/book/assumptions/nowcasting-comparison.md

@@ -36,7 +36,7 @@ totals over the next two-to-three fiscal years.
 | Wage floor | Models the National Living Wage / NMW explicitly each year, lifting anyone below their age-appropriate minimum to the floor. | Earnings are uprated by the OBR average-earnings index without an explicit floor; minimum wage rates are held in `gov/hmrc/minimum_wage/` and projected after the last rate set by the OBR's national living wage forecast, then average earnings; they enter through the rules that read them (the Universal Credit minimum income floor and benefit cap earnings exception, the childcare earnings tests, ESA permitted work and the Carer's Allowance earnings limit), not through earnings. |
 | Spillovers | Includes a spillover effect for workers just above the wage floor (compression of the lower wage distribution). | No automatic spillover; earnings shift uniformly with the OBR index. |
 | NLW extensions | Models the 2024-25 extension of the NLW to 21- and 22-year-olds, and provisionally the 2029-30 extension to 18-20-year-olds. | NLW age bands are encoded in parameters (`gov/hmrc/minimum_wage`). A change to a band, such as extending the NLW to younger workers, reaches outputs through the rules that read a person's age-band rate (the Universal Credit minimum income floor and the childcare earnings tests); earnings are not re-anchored to the floor. |
-| Beyond 2025-26 | Assumes the wage floor grows in line with average earnings. | Minimum wage rates after the last one set grow by uprating rather than a wage-floor rule: with the OBR's national living wage forecast to the end of the EFO, then with average earnings. Like the model's other indices, the projection ends in 2039 and holds flat from 2040. |
+| Beyond 2025-26 | Assumes the wage floor grows in line with average earnings. | Minimum wage rates after the last one set grow by uprating rather than a wage-floor rule: with the OBR's national living wage forecast to the end of the EFO, then with average earnings. Like the model's other indices, the projection ends with the growth series, in 2074. |
 
 **Practical implication.** Low-paid workers' incomes can grow faster under
 the RF nowcast than under PolicyEngine because the wage floor binds and

docs/engineering/excise-duties.md

@@ -193,7 +193,7 @@ year's RPI growth from the OBR forecast in
 hundredth of a penny. The script writes the steps to 1 April 2031, and a test
 fails when they fall out of date with the RPI forecast. After a new EFO, set
 the script's July rate and RPI growth to its forecast and rerun it. Later steps come from the parameter's uprating metadata.
-The RPI index ends in 2039, so the last step is 1 April 2040.
+The RPI index ends with the growth series, in 2074, so the last step is 1 April 2075.
 
 An annual reform names a model year and replaces the rate for all of it. For
 model years to 2030 it leaves other years unchanged. `Scenario.parameter_changes`

policyengine_uk/parameters/gov/economic_assumptions/README.md

@@ -120,13 +120,14 @@ benefit-unit capital without overwriting programme-specific countable capital.
 Changing that formula to carry every direct input would alter situation
 semantics.
 
-Source coverage and processed parameter dates are distinct: fiscal-year
-conversion materialises annual values through 2040, while cumulative economic
-indices compound only through 2039. A later parameter lookup retains the final
-scalar; for a raw growth series that repeats the growth rate, but for a
-cumulative index it freezes the level. Extending the dataset to the raw
-series' final year would therefore be both costly and inconsistent with Core
-uprating after the cumulative-index horizon.
+Source coverage and processed parameter dates are distinct. Cumulative
+economic indices compound to the last year any growth series covers, which is
+one year past `yoy_growth.yaml` because the lagged series run a year past
+their source. A series that ends sooner holds its final growth rate to that
+year: that is an extrapolation, and it matters more the further a series
+falls short of the horizon. Past the horizon a parameter lookup retains the
+final scalar; for a raw growth series that repeats the growth rate, but for a
+cumulative index it freezes the level.
 
 From 2031 onward, dataset projection also grows stored columns whose variables
 have a class-level `uprating` index but are absent from `uprating_indices.yaml`.
@@ -179,9 +180,9 @@ The additional indices have their own source limits:
 
 | Stored input | Variable index | Source coverage and terminal behavior |
 |---|---|---|
-| Salary-sacrifice pension contributions | `indices.obr.average_earnings` | Earnings growth has source values through 2073; its cumulative index compounds through 2039. |
-| Water and sewerage charges | `indices.ofwat.water_bills` | Raw growth ends in 2029 at 5.0%; that rate carries into later years when the cumulative index is generated, through 2039. |
-| Housing service charges | `indices.obr.lagged_average_earnings` | Lagged growth is generated through 2074 from the preceding year's earnings; its cumulative index compounds through 2039. |
+| Salary-sacrifice pension contributions | `indices.obr.average_earnings` | Earnings growth has source values through 2073; its cumulative index compounds through 2074. |
+| Water and sewerage charges | `indices.ofwat.water_bills` | Raw growth ends in 2029 at 5.0%; that rate carries into later years when the cumulative index is generated, through 2074. |
+| Housing service charges | `indices.obr.lagged_average_earnings` | Lagged growth is generated through 2074 from the preceding year's earnings; its cumulative index compounds through 2074. |
 | Rail usage | `gov.dft.rail.ridership_index` | The source cumulative index ends in 2029 at 1.044. Its terminal level stays flat, so the projection ratio is one in 2031 and 2032. |
 
 The `indices.*` paths in this table are relative to

policyengine_uk/parameters/gov/economic_assumptions/create_earnings_uprating.py

@@ -13,7 +13,7 @@
 ``create_economic_assumption_indices`` compounds it into
 ``gov.economic_assumptions.indices.earnings_uprating``. The yearly series
 starts from a 2010 base and runs to one year past the statutory inputs; the
-index, like the others built there, stops in 2039. Published
+index, like the others built there, ends with the growth series. Published
 benefit amounts remain explicit; the index projects later years.
 """
 

policyengine_uk/parameters/gov/economic_assumptions/create_economic_assumption_indices.py

@@ -5,11 +5,37 @@
 )
 
 
+def last_year(parameter: Parameter) -> int:
+    """The first year whose 1 January reads the parameter's final value."""
+    final_instant = max(value.instant_str for value in parameter.values_list)
+    year = int(final_instant[:4])
+    return year if final_instant.endswith("-01-01") else year + 1
+
+
+def growth_horizon(yoy_growth: ParameterNode) -> int:
+    """The last year any growth series covers, where the indices end.
+
+    The indices read each series on 1 January, so a value dated later in a
+    year first reaches them the year after. Pass the series as the builder
+    sees them: fiscal conversion moves a value dated before May to that year's
+    January, so a converted series can report a year less.
+    """
+    return max(
+        (
+            last_year(parameter)
+            for parameter in yoy_growth.get_descendants()
+            if isinstance(parameter, Parameter)
+        ),
+        default=0,
+    )
+
+
 def create_economic_assumption_indices(
     parameters: ParameterNode,
 ) -> ParameterNode:
     econ_assumptions: ParameterNode = parameters.gov.economic_assumptions
     yoy_growth: ParameterNode = econ_assumptions.yoy_growth
+    horizon = growth_horizon(yoy_growth)
     # Index levels dated 1 January, one per calendar year, so a bare-year
     # reform key names the calendar year (policyengine_uk.utils.dated_reforms).
     indices = ParameterNode(
@@ -40,7 +66,7 @@ def create_economic_assumption_indices(
             start_year = int(descendant.values_list[-1].instant_str[:4])
             values = {start_year: 1.0}
 
-            for year in range(start_year + 1, 2040):
+            for year in range(start_year + 1, horizon + 1):
                 yoy_growth = descendant(year)
                 indices_value = values[year - 1] * (1 + yoy_growth)
                 # Rounding an earnings index down can put the projected

policyengine_uk/parameters/gov/hmrc/business_rates/statistics.yaml

@@ -31,8 +31,8 @@ metadata:
 # "Business rates" line, Northern Ireland with its "Northern Ireland business
 # rates" line (gov.economic_assumptions.yoy_growth.obr.business_rates), which
 # appears to cover the regional rate only (see NORTHERN_IRELAND below). The
-# economic-assumption indices run to 2039, so every series is flat in cash
-# terms from 2039-40.
+# economic-assumption indices end where the growth series end, so every series
+# is flat in cash terms after that year.
 revenue:
   description: Total revenue raised by business rates.
   ENGLAND:

policyengine_uk/parameters/gov/hmrc/minimum_wage/non_apprentice.yaml

@@ -40,7 +40,7 @@ description: Minimum wage rate for non-apprentices by age.
 # The index is dated 1 January, so before annualisation a projected rate
 # starts on 1 January. Each year's figure is read at 30 April, the rate for
 # the fiscal year that starts that April. Like the model's other indices, this
-# one ends in 2039, so projected rates hold flat from 2040.
+# one ends with the growth series, and projected rates hold flat after it.
 #
 # The younger bands' projection is an assumption: each grows from its own
 # last rate by the same index, so it keeps its ratio to the national living

policyengine_uk/tax_benefit_system.py

@@ -118,7 +118,7 @@ def _install_parameters(system, parameters: ParameterNode) -> None:
 DEFAULT_DATASET_ENV_VAR = "POLICYENGINE_UK_DEFAULT_DATASET"
 
 # Cache for fully-processed parameter tree, so convert_to_fiscal_year_parameters
-# (22,538 param.update() calls) only runs once per process.
+# (about 66,000 param.update() calls) only runs once per process.
 _processed_parameters_cache = None
 # The parameters as loaded from the YAML files, before processing: each
 # parameter's statutory values at their legal dates. Reforms are written

policyengine_uk/tests/test_corporate_receipts_statistics.py

@@ -13,7 +13,11 @@
 
 from policyengine_uk import Microsimulation, Simulation, parameters
 from policyengine_uk.data import UKSingleYearDataset
+from policyengine_uk.parameters.gov.economic_assumptions.create_economic_assumption_indices import (
+    growth_horizon,
+)
 from policyengine_uk.scenarios import no_economic_assumptions
+from policyengine_uk.system import system
 
 SDLT = parameters.gov.hmrc.stamp_duty.statistics
 BUSINESS_RATES = parameters.gov.hmrc.business_rates.statistics.revenue
@@ -207,8 +211,11 @@ def test_projection_tracks_obr_receipts(name):
         assert statistic(INSTANT(year)) == pytest.approx(expected, rel=2e-5)
 
 
-# The economic-assumption indices run to 2039, so the projections do too.
-LAST_INDEXED_YEAR = 2039
+# The economic-assumption indices end where the growth series end, so the
+# projections do too.
+LAST_INDEXED_YEAR = growth_horizon(
+    system.parameters.gov.economic_assumptions.yoy_growth
+)
 
 
 @settings(max_examples=200, deadline=None, derandomize=True)
@@ -244,7 +251,7 @@ def test_projection_is_flat_after_the_last_indexed_year(name):
     statistic = _statistic(name)
     last = statistic(INSTANT(LAST_INDEXED_YEAR))
     assert statistic(INSTANT(LAST_INDEXED_YEAR - 1)) < last
-    for year in range(LAST_INDEXED_YEAR + 1, 2050):
+    for year in range(LAST_INDEXED_YEAR + 1, LAST_INDEXED_YEAR + 10):
         assert statistic(INSTANT(year)) == last