Route UK dated reforms through policyengine-uk's statutory pipeline

changelog.d/572.fixed.md

@@ -0,0 +1 @@
+Align UK household and population reform routes with policy effective dates and inclusive date intervals, reject ambiguous bare-year keys, and pass parameter reforms at country-model construction so dated changes are reprocessed when supported by policyengine-uk. A UK policy or dynamic carrying both parameter values and a simulation modifier (as a `Policy` or `Dynamic` sum does) now applies its values and then its modifier, where the modifier used to replace the values.

docs/households.md

@@ -150,7 +150,7 @@ Scale parameters use bracket indexing:
 reform = {"gov.irs.credits.ctc.amount.base[0].amount": 3_000}
 ```
 
-Time-varying reforms use a nested dict of `YYYY-MM-DD → value`:
+Time-varying reforms use a nested dict of date keys to values:
 
 ```python
 reform = {
@@ -161,6 +161,10 @@ reform = {
 }
 ```
 
+For UK households, a `YYYY-MM-DD` key is effective from that date until the next value for the same parameter, or December 31, 2100 if no later value follows. A `YYYY-MM-DD.YYYY-MM-DD` key applies over that inclusive interval. Scalar values default to January 1 of `year`.
+
+UK households reject bare years such as `"2027"` or `2027`, because the underlying packages assign different meanings to them. Use `"2027-04-06.2028-04-05"` for one UK fiscal year or `"2027-01-01"` for an effective date. See [Reforms](reforms.md) for the shared `Simulation(policy=...)` and `Simulation(dynamic=...)` date contract. The US household route retains its country package's period-key handling.
+
 Structural reforms (new variables, formula swaps) require the `Simulation` path — see [Reforms](reforms.md).
 
 ## Year

docs/reforms.md

@@ -29,7 +29,19 @@ reform = {
 }
 ```
 
-Dates that haven't passed become "from this date onward." Earlier dates replace the baseline schedule.
+A `YYYY-MM-DD` key is an effective date: its value applies from that date until the next date in the same parameter's reform schedule. If no later value follows, the wrapper emits an interval ending on December 31, 2100.
+
+Use `YYYY-MM-DD.YYYY-MM-DD` for an inclusive interval. For example, this UK reform applies for fiscal year 2027 only:
+
+```python
+reform = {
+    "gov.hmrc.income_tax.rates.uk[0].rate": {
+        "2027-04-06.2028-04-05": 0.21,
+    },
+}
+```
+
+`Simulation(policy=...)`, `Simulation(dynamic=...)`, and UK `calculate_household(reform=...)` accept these two date formats and reject an interval whose end precedes its start. They reject bare years such as `"2027"` or `2027`, because policyengine-uk reads them as fiscal year 2027 only, while policyengine-core's `Reform.from_dict` reads them as from 2027 onward. Use `"2027-04-06.2028-04-05"` for one UK fiscal year or `"2027-01-01"` for an effective date. The US household route retains its country package's period-key handling.
 
 ### Multiple changes
 

src/policyengine/tax_benefit_models/common/reform.py

@@ -1,4 +1,4 @@
-"""Compile a simple reform dict into the format policyengine_core expects.
+"""Compile a simple reform dict into country-model inputs or parameter values.
 
 Accepted shapes for the agent-facing API:
 
@@ -10,6 +10,13 @@
     # With explicit effective date(s).
     reform = {"gov.irs.deductions.salt.cap": {"2026-01-01": 0}}
 
+    # With an inclusive start and end date.
+    reform = {
+        "gov.hmrc.income_tax.rates.uk[0].rate": {
+            "2027-04-06.2028-04-05": 0.21,
+        },
+    }
+
     # Multiple parameters.
     reform = {
         "gov.irs.deductions.salt.cap": 0,
@@ -25,9 +32,17 @@
 dict uses the bracket-head path instead of ``[0].amount`` the
 ``ValueError`` will list the close match.
 
-The compiled form is ``{param_path: {period: value}}`` — exactly what
-``policyengine_us.Simulation(reform=...)`` /
-``policyengine_uk.Simulation(reform=...)`` accept at construction.
+The parameter-value compiler accepts ``YYYY-MM-DD`` as an effective date
+with no end date, or ``YYYY-MM-DD.YYYY-MM-DD`` as an inclusive interval.
+Months and days may be unpadded (``2027-4-6``), as ``strptime`` reads them.
+Other keys, including bare years, are rejected: policyengine-uk interprets
+a bare year as one fiscal year, while policyengine-core interprets it as
+an effective year continuing onward. Use ``2027-04-06.2028-04-05`` for one
+UK fiscal year, or ``2027-01-01`` for an effective date.
+
+The low-level :func:`compile_reform` returns ``{param_path: {period: value}}``
+without interpreting period keys. Parameter-value routes convert values
+to explicit intervals before passing them to the country model.
 
 Scalar reforms default to ``{year}-01-01`` so a caller running
 mid-year does not accidentally get a blended partial-year result.
@@ -103,8 +118,8 @@ def _reform_dict_to_parameter_values(
     """Compile a flat reform dict into a list of ``ParameterValue`` objects.
 
     Uses :func:`compile_reform` for path validation and effective-date
-    defaulting, then materialises each ``{path: {date: value}}`` pair
-    as an open-ended ``ParameterValue`` bound to a
+    defaulting, then materialises each effective date or inclusive interval
+    as a ``ParameterValue`` bound to a
     ``Parameter(name=path, tax_benefit_model_version=model_version)``.
     """
     from policyengine.core.parameter import Parameter
@@ -117,6 +132,32 @@ def _reform_dict_to_parameter_values(
     parameter_values: list[ParameterValue] = []
     for path, date_to_value in compiled.items():
         for effective_date, value in date_to_value.items():
+            try:
+                parts = str(effective_date).split(".")
+                if len(parts) > 2:
+                    raise ValueError("Too many dates")
+                start_date = datetime.datetime.strptime(parts[0], "%Y-%m-%d")
+                end_date = (
+                    datetime.datetime.strptime(parts[1], "%Y-%m-%d")
+                    if len(parts) == 2
+                    else None
+                )
+            except ValueError as exc:
+                raise ValueError(
+                    f"Invalid reform date key {effective_date!r}. Accepted formats are "
+                    "'YYYY-MM-DD' (effective date) and "
+                    "'YYYY-MM-DD.YYYY-MM-DD' (inclusive interval); months and "
+                    "days may be unpadded. Bare years are "
+                    "ambiguous: policyengine-uk's reform dict reads '2027' as fiscal "
+                    "year 2027 only, while policyengine-core's Reform.from_dict "
+                    "reads it as from 2027 on. Use '2027-04-06.2028-04-05' for one "
+                    "UK fiscal year, or '2027-01-01' for an effective date."
+                ) from exc
+            if end_date is not None and end_date < start_date:
+                raise ValueError(
+                    f"Reform interval {effective_date!r} has an end date before "
+                    "its start date."
+                )
             data_type = type(value) if isinstance(value, (int, float, bool)) else float
             parameter_values.append(
                 ParameterValue(
@@ -125,8 +166,8 @@ def _reform_dict_to_parameter_values(
                         tax_benefit_model_version=model_version,
                         data_type=data_type,
                     ),
-                    start_date=datetime.datetime.strptime(effective_date, "%Y-%m-%d"),
-                    end_date=None,
+                    start_date=start_date,
+                    end_date=end_date,
                     value=value,
                 )
             )
@@ -160,7 +201,7 @@ def compile_reform_to_policy(
     """Compile a flat reform dict into a fully-assembled ``Policy``.
 
     Accepts the same ``{param.path: value}`` /
-    ``{param.path: {date: value}}`` shape as :func:`compile_reform`,
+    ``{param.path: {date_or_interval: value}}`` shape as :func:`compile_reform`,
     but returns a ready-to-use ``Policy`` with ``ParameterValue``
     objects so ``Simulation(policy={...})`` works without hand-building
     ``Parameter`` / ``ParameterValue``.

src/policyengine/tax_benefit_models/uk/household.py

@@ -26,13 +26,16 @@
 from policyengine.tax_benefit_models.common import (
     EntityResult,
     HouseholdResult,
-    compile_reform,
     dispatch_extra_variables,
     normalize_axes,
     validate_annual_household_inputs,
     values_for_entity,
 )
+from policyengine.tax_benefit_models.common.reform import (
+    _reform_dict_to_parameter_values,
+)
 from policyengine.utils.household_validation import validate_household_input
+from policyengine.utils.parametric_reforms import reform_dict_from_parameter_values
 
 from .model import uk_latest
 
@@ -129,9 +132,11 @@ def calculate_household(
             Must be non-empty.
         benunit, household: Optional per-entity overrides.
         year: Calendar year. Defaults to 2026.
-        reform: Optional reform dict. Scalar values default to
-            ``{year}-01-01``; invalid parameter paths raise with a
-            close-match suggestion.
+        reform: Optional reform dict. Scalars start at ``{year}-01-01``.
+            ``YYYY-MM-DD`` keys are effective dates; a value lasts until
+            the next effective date or the end of the model horizon.
+            ``YYYY-MM-DD.YYYY-MM-DD`` keys specify inclusive intervals.
+            Invalid parameter paths raise with a close-match suggestion.
         extra_variables: Flat list of extra UK variables to compute;
             the library dispatches each to its entity.
         axes: Optional household-calculator axes. Pass either the lower-level
@@ -181,7 +186,11 @@ def calculate_household(
         names=extra_variables or [],
     )
     output_columns = _default_output_columns(extra_by_entity)
-    reform_dict = compile_reform(reform, year=year, model_version=uk_latest)
+    reform_dict = reform_dict_from_parameter_values(
+        _reform_dict_to_parameter_values(
+            reform or {}, year=year, model_version=uk_latest
+        )
+    )
     normalized_axes = normalize_axes(axes=axes, year=year, model_version=uk_latest)
     axes_active = normalized_axes is not None
 

src/policyengine/tax_benefit_models/uk/model.py

@@ -167,6 +167,8 @@ def run(self, simulation: "Simulation") -> "Simulation":
         from policyengine_uk import Microsimulation
 
         from policyengine.utils.parametric_reforms import (
+            build_reform_dict,
+            merge_reform_dicts,
             simulation_modifier_from_parameter_values,
         )
 
@@ -208,23 +210,42 @@ def run(self, simulation: "Simulation") -> "Simulation":
                 data_year_data=dataset.data_year_data,
             )
 
-        microsim = Microsimulation(dataset=_policyengine_uk_input(dataset))
+        policy = simulation.policy
+        dynamic = simulation.dynamic
+        policy_modifier = policy.simulation_modifier if policy else None
+        dynamic_modifier = dynamic.simulation_modifier if dynamic else None
+        # Each object's parameter values come before its modifier, and the
+        # policy before the dynamic: policy values, policy modifier, dynamic
+        # values, dynamic modifier. A modifier never replaces its object's
+        # parameter values (a Policy or Dynamic sum can carry both).
+        # Parameter values reach policyengine-uk as a reform dict at
+        # construction, which writes them onto the statutory timeline and
+        # reprocesses the parameters. The dynamic's values join that reform
+        # (merge_reform_dicts puts them last, so they win) unless a policy
+        # modifier must run before them; then they are applied after that
+        # modifier, to the annualised parameters, as before.
+        reform_dict = build_reform_dict(policy)
+        late_dynamic_values = None
+        if policy_modifier is None:
+            reform_dict = merge_reform_dicts(reform_dict, build_reform_dict(dynamic))
+        elif dynamic is not None and dynamic.parameter_values:
+            late_dynamic_values = dynamic.parameter_values
+        # policyengine-uk also builds an internal baseline from the same
+        # input, which its formulas read through simulation.baseline.
+        # _policyengine_uk_input gives this run fresh tables, so enum
+        # encoding cannot mutate the caller's dataset.
+        microsim = Microsimulation(
+            dataset=_policyengine_uk_input(dataset), reform=reform_dict
+        )
 
-        if simulation.policy and simulation.policy.simulation_modifier is not None:
-            simulation.policy.simulation_modifier(microsim)
-        elif simulation.policy:
-            modifier = simulation_modifier_from_parameter_values(
-                simulation.policy.parameter_values
-            )
-            modifier(microsim)
+        if policy_modifier is not None:
+            policy_modifier(microsim)
 
-        if simulation.dynamic and simulation.dynamic.simulation_modifier is not None:
-            simulation.dynamic.simulation_modifier(microsim)
-        elif simulation.dynamic:
-            modifier = simulation_modifier_from_parameter_values(
-                simulation.dynamic.parameter_values
-            )
-            modifier(microsim)
+        if late_dynamic_values is not None:
+            simulation_modifier_from_parameter_values(late_dynamic_values)(microsim)
+
+        if dynamic_modifier is not None:
+            dynamic_modifier(microsim)
 
         data = {
             "person": pd.DataFrame(),

src/policyengine/utils/parametric_reforms.py

@@ -131,7 +131,15 @@ def merge_reform_dicts(
     """Merge two reform dicts, with override values taking precedence.
 
     Either or both dicts can be None. When both have entries for the same
-    parameter, period-level values from override replace those in base.
+    parameter, an override value replaces the base value with the same
+    period key, and every override period comes after the base's periods.
+
+    Where an override period overlaps a base period with a different key,
+    the result depends on the country model. policyengine-uk applies a
+    parameter's periods in the order given, so the override wins.
+    policyengine-core's ``Reform.from_dict``, which the US route uses,
+    applies them in order of start date, so the later-starting period wins
+    whichever dict it came from.
 
     Args:
         base: The base reform dict (e.g., from policy).
@@ -147,7 +155,13 @@ def merge_reform_dicts(
 
     merged = {k: dict(v) for k, v in base.items()}
     for param_name, period_values in override.items():
-        if param_name not in merged:
-            merged[param_name] = {}
-        merged[param_name].update(period_values)
+        # Override values go after the base's, so a country model that
+        # applies a parameter's periods in order lets them win where they
+        # overlap a base period with a different key.
+        kept = {
+            period: value
+            for period, value in merged.get(param_name, {}).items()
+            if period not in period_values
+        }
+        merged[param_name] = {**kept, **period_values}
     return merged