diff --git a/changelog.d/disable-simulated-benefits-scoped-awards.fixed.md b/changelog.d/disable-simulated-benefits-scoped-awards.fixed.md new file mode 100644 index 0000000000..16a0ad9cb1 --- /dev/null +++ b/changelog.d/disable-simulated-benefits-scoped-awards.fixed.md @@ -0,0 +1,4 @@ +- Fix the `disable_simulated_benefits` reform (`gov.contrib.policyengine.disable_simulated_benefits`), which raised an error before setting any benefit. The simulation now links itself to its tax-benefit system before applying structural reforms, so a reform that sets inputs can reach it (the public service budget adjustments failed on the same missing link); a clone or branch that applies a reform links the system to itself while it does, so the reform never sets its parent's inputs. A reform applied after the simulation is built (one passed as `reform=`, or applied to a built simulation, clone or branch) now keeps through `reset_calculations` and `derivative` every input it stores and leaves in place, however it stores it, including one that replaces an input set since the build; an input set since the build that the reform did not store is still reset, as before. The reform reads the dataset year whether it is stored as text or a number. It no longer lists Attendance Allowance, DLA and PIP, which have no reported amount: the data gives their receipt as a rate category and the model pays that category's rate. +- Under `disable_simulated_benefits`, set the claimant's and partner's income-related ESA and income-based JSA (`claimant_or_partner_esa_income`, `claimant_or_partner_jsa_income`) for each year from the dataset-year reports of whoever is the claimant or partner that year, and set Income Support from their reports alone. The reform holds each benefit at its dataset-year amount while the reported amounts are uprated, so in a later year the stored award matched neither reading of the reports and another member's award was taken to be the claimant's. Another member's own Income Support report is left out of household income, as it is with the reform off, where Income Support is calculated only for the claimant's family. +- Neutralizing `esa_income` or `jsa_income` also neutralizes the claimant's and partner's award, so an award entered for them, or set by the reform, no longer outlives the benefit it is part of. +- Read the claimant's and partner's income-related ESA and income-based JSA in `income_support_eligible` from `claimant_or_partner_esa_income` and `claimant_or_partner_jsa_income`, as the means tests and passports do, so an award entered for them directly, or set by the reform, is read the same way everywhere. The two variables share one formula, which now treats an award within half a penny of zero as no award, as `income_support_eligible` did, so the float residual left by reports that exactly match the tariff income is no longer the claimant's or partner's award, and no passport or status that reads their award counts it. A member outside the couple is still on their own award as before: it is read from their own report, without that rule. When `esa_income` or `jsa_income` holds the plain total of the reported awards, Income Support now reads the plain total of the claimant's and partner's reports, as the two variables already did; it used to read their award after the capital test. diff --git a/policyengine_uk/reforms/policyengine/disable_simulated_benefits.py b/policyengine_uk/reforms/policyengine/disable_simulated_benefits.py index 54338e3e92..2a33083fa3 100644 --- a/policyengine_uk/reforms/policyengine/disable_simulated_benefits.py +++ b/policyengine_uk/reforms/policyengine/disable_simulated_benefits.py @@ -1,58 +1,131 @@ from policyengine_core.model_api import * +# Benefits whose simulated amount the reform replaces with the amount reported +# in the dataset year. Each needs a _reported variable. Attendance +# Allowance, DLA and PIP are not listed: the data records their receipt as a +# rate category (aa_category, dla_m_category, dla_sc_category, pip_dl_category, +# pip_m_category), and the model pays that category's rate. The model has no +# reported amount for them to copy. +BENEFITS = [ + "afcs", + "bsp", + "carers_allowance", + "child_benefit", + "child_tax_credit", + "council_tax_benefit", + "esa_contrib", + "esa_income", + "housing_benefit", + "iidb", + "incapacity_benefit", + "income_support", + "jsa_contrib", + "jsa_income", + "pension_credit", + "sda", + "ssmg", + "state_pension", + "universal_credit", + "winter_fuel_allowance", + "working_tax_credit", +] + +# Tax credits are not paid below a minimum award, and each has a variable +# before that minimum, which takes the same reported amount. +PRE_MINIMUM = { + "child_tax_credit": "child_tax_credit_pre_minimum", + "working_tax_credit": "working_tax_credit_pre_minimum", +} + +# Benefit-unit awards that cover only the claimant and partner, set from their +# own reports. The reported amounts of anyone else in the benefit unit (such +# as a non-dependent adult, who claims in their own right) stay in esa_income +# and jsa_income, which count in household income. They never enter the +# claimant's or partner's award. +CLAIMANT_OR_PARTNER_AWARDS = { + "esa_income": "claimant_or_partner_esa_income", + "jsa_income": "claimant_or_partner_jsa_income", +} + +# Income Support is the claimant's family's award, so it is set from the +# claimant's and partner's reports only. That is all the model ever pays as +# income_support: with the reform off, would_claim_IS and +# income_support_eligible read only their reports too. Another member's own +# Income Support report (a non-dependent adult claiming in their own right) is +# in household income neither with the reform nor without it. ESA and JSA +# differ: esa_income and jsa_income count every member's award either way. +CLAIMANT_OR_PARTNER_ONLY = ["income_support"] + +YEARS_IN_FUTURE = 10 + def disable_simulated_benefits(parameters, period): if parameters(period).gov.contrib.policyengine.disable_simulated_benefits: class DisableSimulatedBenefits(Reform): + """Use the dataset year's reported benefit amounts in every year. + + Each listed benefit is set, in the dataset year and the nine years + after it, to its amount reported in the dataset year. The amount + is not uprated, while most of the *_reported variables are. + + The claimant-or-partner awards are set the same way, from the + dataset-year reports of whoever is the claimant or partner in + each year (is_claimant_or_partner); Income Support is set from + those reports alone. So every amount is the same in every year, + except that these three change when who is the claimant or + partner changes. Readers of those awards, including the + Income Support gate, cannot tell an award set here from one + entered directly. In a later year, esa_income and jsa_income match + neither the award on that year's uprated reports nor their total. + Without the claimant-or-partner awards set here, the readers would + take the whole of esa_income or jsa_income, including another + member's award, to be the claimant's or partner's. + + As the amounts are inputs, a reform that changes a benefit's + parameters does not change them. A reform that neutralizes a + benefit removes it, and neutralizing esa_income or jsa_income + also neutralizes its claimant-or-partner award + (CountryTaxBenefitSystem.variable_parts). A neutralized variable + ignores what the reform sets for it. + """ + def apply(self): simulation = self.simulation + system = simulation.tax_benefit_system + time_period = int(str(simulation.dataset.time_period)[:4]) + + def reported(variable, members=None): + """A benefit's reported amount in the dataset year, for + the given members, summed over the benefit's entity.""" + amounts = simulation.calculate(variable + "_reported", time_period) + if members is not None: + amounts = amounts * members + entity = system.variables[variable].entity.key + population = simulation.populations[entity] + if population.entity.is_person: + return amounts + return population.sum(amounts) - BENEFITS = [ - "afcs", - "attendance_allowance", - "bsp", - "carers_allowance", - "child_benefit", - "child_tax_credit", - "council_tax_benefit", - "dla_m", - "dla_sc", - "esa_contrib", - "esa_income", - "housing_benefit", - "iidb", - "incapacity_benefit", - "income_support", - "jsa_contrib", - "jsa_income", - "pension_credit", - "pip_dl", - "pip_m", - "sda", - "ssmg", - "state_pension", - "universal_credit", - "winter_fuel_allowance", - "working_tax_credit", - ] - time_period = simulation.dataset.time_period - YEARS_IN_FUTURE = 10 - for variable in BENEFITS: - entity = simulation.tax_benefit_system.variables[ - variable - ].entity.key - reported_value = simulation.calculate( - variable + "_reported", time_period, map_to=entity + for year in range(time_period, time_period + YEARS_IN_FUTURE): + claimant_or_partner = simulation.calculate( + "is_claimant_or_partner", year ) - for year in range(time_period, time_period + YEARS_IN_FUTURE): - simulation.set_input(variable, year, reported_value) - - if variable in ["child_tax_credit", "working_tax_credit"]: - # CTC and WTC have their own pre_minimum variables because tax credits aren't paid if - # below a threshold. - variable = variable + "_pre_minimum" - for year in range(time_period, time_period + YEARS_IN_FUTURE): - simulation.set_input(variable, year, reported_value) + values = {} + for variable in BENEFITS: + members = ( + claimant_or_partner + if variable in CLAIMANT_OR_PARTNER_ONLY + else None + ) + values[variable] = reported(variable, members) + if variable in PRE_MINIMUM: + values[PRE_MINIMUM[variable]] = values[variable] + if variable in CLAIMANT_OR_PARTNER_AWARDS: + values[CLAIMANT_OR_PARTNER_AWARDS[variable]] = reported( + variable, claimant_or_partner + ) + for target, value in values.items(): + simulation.set_input(target, year, value) return DisableSimulatedBenefits diff --git a/policyengine_uk/simulation.py b/policyengine_uk/simulation.py index 26bbbe18f7..446ceae557 100644 --- a/policyengine_uk/simulation.py +++ b/policyengine_uk/simulation.py @@ -50,6 +50,32 @@ from microdf import MicroDataFrame + +class _InputKeyLog(set): + """A simulation's record of its inputs while Simulation.apply_reform runs. + + policyengine-core records each input in the simulation's + ``_user_input_keys`` as (variable, branch, period). ``Holder._set`` adds + the key once it has stored the value, again when it replaces one, and on + every route: ``Simulation.set_input``, ``Holder.set_input``, a + ``set_input`` helper, another thread. A value core does not store adds + no key: one ``Simulation.set_input`` gets for a period after the + variable's end, one for a neutralized variable, or one for a sub-period + that a ``set_input`` helper skips because it already holds an input. + This set also adds each key to every set in ``logs``: one for each + reform being applied to the simulation, the innermost last. + """ + + def __init__(self, keys, logs: list): + super().__init__(keys) + self.logs = logs + + def add(self, key) -> None: + super().add(key) + for log in self.logs: + log.add(key) + + # Cache for fully-loaded, uprated, enum-pre-encoded multi-year datasets keyed # by URL. Avoids repeating HDF5 reading (0.84s), uprating (0.69s) and enum # encoding (0.67s) on every Simulation() call after the first. @@ -201,7 +227,12 @@ def __init__( self.tax_benefit_system.reset_parameter_caches() - # Apply structural reforms based on parameters + # Apply structural reforms based on parameters. Core applies a reform + # to the tax-benefit system, so reforms that set inputs (such as + # disable_simulated_benefits and adjust_budgets) reach the simulation + # through it. Core's own constructor sets this link; this one does not + # call it, so set it here (apply_reform also sets it for each reform). + self.tax_benefit_system.simulation = self structural_reform = create_structural_reforms_from_parameters( self.tax_benefit_system.parameters, period_(self.default_input_period), @@ -239,6 +270,53 @@ def __init__( if scenario.parameter_changes is not None: self.apply_parameter_changes(scenario.parameter_changes) + def apply_reform(self, reform) -> None: + # Reforms that set inputs (disable_simulated_benefits, adjust_budgets) + # reach the simulation through its tax-benefit system's link. A clone + # or branch may share its parent's system, so link the system to this + # simulation while the reform is applied, then restore the link. + system = self.tax_benefit_system + linked = getattr(system, "simulation", None) + system.simulation = self + # Log the inputs the reform stores (see _InputKeyLog). Within another + # reform applied to this simulation (a tuple's parts, or a reform + # that applies one), log them for that one too. + keys = getattr(self, "_user_input_keys", None) + enclosing = keys.logs if isinstance(keys, _InputKeyLog) else [] + stored = set() + self._user_input_keys = _InputKeyLog(keys or (), enclosing + [stored]) + try: + super().apply_reform(reform) + finally: + # Core rebuilds the record as a plain set once the reform is + # applied, but not if it raised. Resume the enclosing reform's log. + keys = set(self._user_input_keys) + self._user_input_keys = _InputKeyLog(keys, enclosing) if enclosing else keys + if linked is not None: + system.simulation = linked + # input_variables is taken when the simulation is built. + # reset_calculations and derivative delete the values of every + # variable it does not name, and the marginal rate formulas those of + # every such variable that has a formula. A reform applied afterwards + # (a Reform class passed as reform=, or one applied to a clone or + # branch) can store inputs: new ones, or ones that replace an input + # set since the build. Name every variable with an input the reform + # stored that this simulation still holds, under a branch name it + # reads and a reset deletes, or a reset would delete what the reform + # stored and recalculate the formula. Assign a new list: a clone or + # branch shares its parent's. + input_variables = getattr(self, "input_variables", None) + if input_variables is not None: + branches = set(self._get_visible_branch_names()) + supplied = { + name + for name, branch, _ in stored & self._user_input_keys + if branch in branches + } + added = sorted(supplied - set(input_variables)) + if added: + self.input_variables = list(input_variables) + added + def delete_arrays(self, variable: str, period: Period = None) -> None: super().delete_arrays(variable, period) # From policyengine-core 3.32.27 (policyengine-core#561) the deletion diff --git a/policyengine_uk/tax_benefit_system.py b/policyengine_uk/tax_benefit_system.py index 06acdeb86c..da8a86da31 100644 --- a/policyengine_uk/tax_benefit_system.py +++ b/policyengine_uk/tax_benefit_system.py @@ -1,7 +1,7 @@ # Standard library imports import copy from pathlib import Path -from typing import Any, Dict, List +from typing import Any, Dict, List, Tuple # PolicyEngine core imports from policyengine_core.parameters.operations.propagate_parameter_metadata import ( @@ -76,6 +76,20 @@ class CountryTaxBenefitSystem(TaxBenefitSystem): variables: Dict[str, Variable] + # Variables that hold part of another variable's value. Neutralizing a + # variable neutralizes its parts, so a value entered or set for a part + # (as disable_simulated_benefits sets the claimant's and partner's awards + # for each year) cannot outlive the award it is part of. + variable_parts: Dict[str, Tuple[str, ...]] = { + "esa_income": ("claimant_or_partner_esa_income",), + "jsa_income": ("claimant_or_partner_jsa_income",), + } + + def neutralize_variable(self, variable_name: str) -> None: + super().neutralize_variable(variable_name) + for part in self.variable_parts.get(variable_name, ()): + super().neutralize_variable(part) + def reset_parameter_caches(self): """Reset all caches in the tax-benefit system.""" self._parameters_at_instant_cache = {} diff --git a/policyengine_uk/tests/test_disable_simulated_benefits.py b/policyengine_uk/tests/test_disable_simulated_benefits.py new file mode 100644 index 0000000000..5dd0539fb9 --- /dev/null +++ b/policyengine_uk/tests/test_disable_simulated_benefits.py @@ -0,0 +1,741 @@ +"""disable_simulated_benefits pays each family the benefits it reported. + +The reform (gov.contrib.policyengine.disable_simulated_benefits) sets each +listed benefit, in the dataset year and the nine years after it, to the +amount reported in the dataset year. esa_income_reported and +jsa_income_reported are uprated, so in a later year a stored esa_income or +jsa_income equals neither the award on that year's reports nor their plain +total. Read by value alone, it would then +be taken to be wholly the claimant's or partner's, another member's award +included. So the reform also sets the claimant's and partner's awards +(claimant_or_partner_esa_income, claimant_or_partner_jsa_income) for each +year, from the dataset-year reports of whoever is the claimant or partner +that year, and Income Support from theirs alone. Both of Income Support's income-related limbs (SSCBA 1992 +s.124(1)(f) and (h)) and the council tax reduction passport read those +awards. + +Each family below is its own benefit unit and household. Reported amounts are +entered for the dataset year only, so later years uprate them. Everything +else is entered for every year. +""" + +from types import SimpleNamespace + +import numpy as np +import pandas as pd +import pytest +from hypothesis import HealthCheck, given, settings +from hypothesis import strategies as st + +from policyengine_core.periods import period + +from policyengine_uk import CountryTaxBenefitSystem, Simulation +from policyengine_uk.data import UKMultiYearDataset, UKSingleYearDataset +from policyengine_uk.model_api import Reform +from policyengine_uk.reforms.policyengine.disable_simulated_benefits import ( + BENEFITS, + CLAIMANT_OR_PARTNER_AWARDS, + CLAIMANT_OR_PARTNER_ONLY, + PRE_MINIMUM, + YEARS_IN_FUTURE, + disable_simulated_benefits, +) +from policyengine_uk.utils.benefit_unit import claimant_or_partner_award +from policyengine_uk.utils.scenario import Scenario +from policyengine_uk.variables.gov.dwp.esa_income import income_related_esa_award + +DATASET_YEAR = 2025 +# The dataset year and two later years. +YEARS = [DATASET_YEAR, DATASET_YEAR + 1, DATASET_YEAR + 2] + +REFORM = Scenario( + applied_before_data_load=True, + parameter_changes={"gov.contrib.policyengine.disable_simulated_benefits": True}, +) + + +def every_year(value): + return {year: value for year in YEARS} + + +def adult(age, claimant_or_partner, carer=False, **reported): + person = { + "age": every_year(age), + "is_claimant_or_partner": every_year(claimant_or_partner), + "receives_carer_benefit": every_year(carer), + "current_education": every_year("NOT_IN_EDUCATION"), + } + for name, amount in reported.items(): + person[name] = {DATASET_YEAR: amount} + return person + + +# A carer who reports Income Support satisfies every other condition of +# income_support_eligible, so in these families only the income-related +# limbs can bar the claim. +CARER = dict(age=40, claimant_or_partner=True, carer=True) +NOT_IN_COUPLE = dict(age=30, claimant_or_partner=False) + +# family: (members, capital) +FAMILIES = { + # An adult outside the couple, such as a non-dependent, claims ESA and + # JSA in their own right. + "excluded_adult": ( + [ + adult(**CARER, income_support_reported=1_000), + adult( + **NOT_IN_COUPLE, esa_income_reported=3_000, jsa_income_reported=2_000 + ), + ], + 0, + ), + # The same, with no Income Support award to passport the claimant. + "excluded_adult_no_award": ( + [ + adult(age=40, claimant_or_partner=True), + adult( + **NOT_IN_COUPLE, esa_income_reported=3_000, jsa_income_reported=2_000 + ), + ], + 0, + ), + # The claimant's own income-related ESA. + "own_esa": ( + [adult(**CARER, income_support_reported=1_000, esa_income_reported=3_000)], + 0, + ), + # The claimant's own income-based JSA. + "own_jsa": ( + [adult(**CARER, income_support_reported=1_000, jsa_income_reported=2_000)], + 0, + ), + # The claimant's own £200 of ESA with £10,000 of capital: tariff income of + # £832 a year leaves no award on the formula's reading, but the reform + # pays the reported amount, before any screen. + "own_esa_over_tariff": ( + [adult(**CARER, income_support_reported=1_000, esa_income_reported=200)], + 10_000, + ), + # Only an adult outside the couple reports Income Support. + "excluded_income_support": ( + [ + adult(age=40, claimant_or_partner=True), + adult(**NOT_IN_COUPLE, income_support_reported=1_000), + ], + 0, + ), +} +NAMES = list(FAMILIES) + + +def situation(): + people, benunits, households = {}, {}, {} + for family, (members, capital) in FAMILIES.items(): + names = [] + for i, person in enumerate(members): + name = f"{family}_{i}" + people[name] = person + names.append(name) + benunits[family] = { + "members": names, + "income_support_assessable_capital": every_year(capital), + "esa_income_assessable_capital": every_year(capital), + "jsa_income_assessable_capital": every_year(capital), + } + households[family] = {"members": names} + return {"people": people, "benunits": benunits, "households": households} + + +def by_family(simulation, variable, year): + return dict(zip(NAMES, simulation.calculate(variable, year).tolist())) + + +def by_person(simulation, variable, year): + names = [ + f"{family}_{i}" + for family, (members, _) in FAMILIES.items() + for i in range(len(members)) + ] + return dict(zip(names, simulation.calculate(variable, year).tolist())) + + +@pytest.fixture(scope="module") +def reformed(): + return Simulation(situation=situation(), scenario=REFORM) + + +def test_every_listed_benefit_has_what_the_reform_reads_and_sets(): + # The reform used to list Attendance Allowance, DLA and PIP, which have + # no reported amount, and so raised before setting anything. + variables = CountryTaxBenefitSystem().variables + for benefit in BENEFITS: + assert benefit in variables, benefit + assert f"{benefit}_reported" in variables, benefit + for name in [ + *PRE_MINIMUM.values(), + *CLAIMANT_OR_PARTNER_AWARDS.values(), + *CLAIMANT_OR_PARTNER_ONLY, + ]: + assert name in variables, name + assert set(PRE_MINIMUM) <= set(BENEFITS) + assert set(CLAIMANT_OR_PARTNER_AWARDS) <= set(BENEFITS) + assert set(CLAIMANT_OR_PARTNER_ONLY) <= set(BENEFITS) + + +def test_the_reform_is_off_by_default(): + simulation = Simulation(situation=situation()) + for variable in ["esa_income", "claimant_or_partner_esa_income"]: + assert simulation.get_holder(variable).get_known_periods() == [] + + +def test_the_reform_sets_the_awards_for_ten_years(reformed): + expected = list(range(DATASET_YEAR, DATASET_YEAR + YEARS_IN_FUTURE)) + for variable in [ + "esa_income", + "jsa_income", + "income_support", + "claimant_or_partner_esa_income", + "claimant_or_partner_jsa_income", + ]: + periods = reformed.get_holder(variable).get_known_periods() + assert sorted(int(str(p)[:4]) for p in periods) == expected, variable + + +@pytest.mark.parametrize("year", YEARS) +def test_the_stored_awards_are_the_dataset_years_reports(reformed, year): + # Every member's report, at its dataset-year amount in every year. + assert by_family(reformed, "esa_income", year) == { + "excluded_adult": 3_000, + "excluded_adult_no_award": 3_000, + "own_esa": 3_000, + "own_jsa": 0, + "own_esa_over_tariff": 200, + "excluded_income_support": 0, + } + assert by_family(reformed, "jsa_income", year) == { + "excluded_adult": 2_000, + "excluded_adult_no_award": 2_000, + "own_esa": 0, + "own_jsa": 2_000, + "own_esa_over_tariff": 0, + "excluded_income_support": 0, + } + + +@pytest.mark.parametrize("year", YEARS[1:]) +def test_later_years_need_the_awards_set_explicitly(reformed, year): + # In a later year the reports have been uprated, so the stored £3,000 + # equals neither their plain total nor the award on them. Read by value + # alone, it would be the claimant's or partner's: the excluded adult's + # ESA would become the couple's. + reported_total = reformed.calculate("esa_income_reported", year, map_to="benunit") + excluded = NAMES.index("excluded_adult") + assert reported_total[excluded] > 3_000 + by_value_alone = claimant_or_partner_award( + reformed.populations["benunit"], + period(year), + "esa_income", + "esa_income_reported", + income_related_esa_award, + ) + assert by_value_alone[excluded] == 3_000 + + +@pytest.mark.parametrize("year", YEARS) +def test_the_claimant_or_partner_awards_are_their_own_reports(reformed, year): + assert by_family(reformed, "claimant_or_partner_esa_income", year) == { + "excluded_adult": 0, + "excluded_adult_no_award": 0, + "own_esa": 3_000, + "own_jsa": 0, + "own_esa_over_tariff": 200, + "excluded_income_support": 0, + } + assert by_family(reformed, "claimant_or_partner_jsa_income", year) == { + "excluded_adult": 0, + "excluded_adult_no_award": 0, + "own_esa": 0, + "own_jsa": 2_000, + "own_esa_over_tariff": 0, + "excluded_income_support": 0, + } + # Income Support is the claimant's family's award, from their own reports. + assert by_family(reformed, "income_support", year) == { + "excluded_adult": 1_000, + "excluded_adult_no_award": 0, + "own_esa": 1_000, + "own_jsa": 1_000, + "own_esa_over_tariff": 1_000, + "excluded_income_support": 0, + } + + +@pytest.mark.parametrize("year", YEARS) +def test_both_income_support_limbs_read_the_claimant_or_partner_awards(reformed, year): + # The carer with an Income Support award is eligible beside an adult + # outside the couple who reports ESA and JSA: neither limb bars the claim. + # The same carer's own ESA bars it under s.124(1)(h), and their own JSA + # under s.124(1)(f), every year. The reform pays the £200 of ESA the + # tariff income would extinguish, so that bars it too. + assert by_family(reformed, "income_support_eligible", year) == { + "excluded_adult": True, + "excluded_adult_no_award": False, + "own_esa": False, + "own_jsa": False, + "own_esa_over_tariff": False, + "excluded_income_support": False, + } + + +@pytest.mark.parametrize("year", YEARS) +def test_the_council_tax_reduction_passport(reformed, year): + # The working-age schemes passport an applicant or partner on Income + # Support, income-based JSA or income-related ESA (SI 2012/2886 Sch, + # para 33(2)). Another member's award passports nobody else. + assert by_family( + reformed, "council_tax_reduction_relevant_income_based_benefit", year + ) == { + "excluded_adult": True, + "excluded_adult_no_award": False, + "own_esa": True, + "own_jsa": True, + "own_esa_over_tariff": True, + "excluded_income_support": False, + } + + +@pytest.mark.parametrize("year", YEARS) +def test_each_person_is_on_their_own_award(reformed, year): + on_esa = by_person(reformed, "is_on_income_related_esa", year) + on_jsa = by_person(reformed, "is_on_income_based_jsa", year) + assert not on_esa["excluded_adult_no_award_0"] + assert on_esa["excluded_adult_no_award_1"] + assert not on_jsa["excluded_adult_no_award_0"] + assert on_jsa["excluded_adult_no_award_1"] + assert on_esa["own_esa_0"] and not on_jsa["own_esa_0"] + assert on_jsa["own_jsa_0"] and not on_esa["own_jsa_0"] + + +def test_a_multi_year_dataset_uses_its_first_year(): + # A situation's dataset stores its year as text and a multi-year dataset + # as a number; both start the reform's ten years. Here the second year's + # reports differ, but the reform reads the first year's. The second + # member is 18 and the first 40, so after the data's two years the + # model's presumption reads them as the claimant's child, as the data + # does, and the roles are the same in every year. + def year(fiscal_year, esa): + person = pd.DataFrame( + { + "person_id": [1, 2, 3], + "person_benunit_id": [1, 1, 2], + "person_household_id": [1, 1, 2], + "age": [40, 18, 40], + "is_claimant_or_partner": [True, False, True], + "esa_income_reported": esa, + } + ) + return UKSingleYearDataset( + person=person, + benunit=pd.DataFrame({"benunit_id": [1, 2]}), + household=pd.DataFrame({"household_id": [1, 2]}), + fiscal_year=fiscal_year, + ) + + dataset = UKMultiYearDataset( + datasets=[ + year(DATASET_YEAR, [0.0, 3_000.0, 3_000.0]), + year(DATASET_YEAR + 1, [0.0, 3_100.0, 3_100.0]), + ] + ) + simulation = Simulation(dataset=dataset, scenario=REFORM) + for year_ in YEARS: + roles = simulation.calculate("is_claimant_or_partner", year_) + assert roles.tolist() == [True, False, True] + assert simulation.calculate("esa_income", year_).tolist() == [3_000, 3_000] + assert simulation.calculate( + "claimant_or_partner_esa_income", year_ + ).tolist() == [0, 3_000] + + +REPORTED = ["esa_income_reported", "jsa_income_reported", "income_support_reported"] + + +@st.composite +def reporting_families(draw): + """One to three adults, the first one or two the claimant and partner, + each reporting ESA, JSA and Income Support in the dataset year.""" + amounts = st.sampled_from([0, 0, 200, 3_000, 65_536.01]) + n_couple = draw(st.integers(1, 2)) + n_others = draw(st.integers(0, 2)) + members = [] + for i in range(n_couple + n_others): + members.append( + { + "age": every_year(40 - i), + "is_claimant_or_partner": every_year(i < n_couple), + "receives_carer_benefit": every_year(draw(st.booleans())), + "current_education": every_year("NOT_IN_EDUCATION"), + **{name: {DATASET_YEAR: draw(amounts)} for name in REPORTED}, + } + ) + return members, draw(st.sampled_from([0, 10_000, 20_000])) + + +@settings( + max_examples=10, + deadline=None, + derandomize=True, + suppress_health_check=[HealthCheck.too_slow, HealthCheck.data_too_large], +) +@given(st.lists(reporting_families(), min_size=1, max_size=8)) +def test_the_reform_splits_every_familys_reports_between_the_awards(drawn): + people, benunits, households = {}, {}, {} + for i, (members, capital) in enumerate(drawn): + names = [f"p{i}_{j}" for j in range(len(members))] + people.update(zip(names, members)) + benunits[f"b{i}"] = { + "members": names, + "income_support_assessable_capital": every_year(capital), + "esa_income_assessable_capital": every_year(capital), + "jsa_income_assessable_capital": every_year(capital), + } + households[f"h{i}"] = {"members": names} + simulation = Simulation( + situation={"people": people, "benunits": benunits, "households": households}, + scenario=REFORM, + ) + + def total(name, couple_only): + return np.array( + [ + sum( + member[name][DATASET_YEAR] + for member in members + if member["is_claimant_or_partner"][DATASET_YEAR] or not couple_only + ) + for members, _ in drawn + ], + dtype=np.float32, + ) + + for year in YEARS: + for award, scoped in CLAIMANT_OR_PARTNER_AWARDS.items(): + stored = simulation.calculate(award, year) + couple = simulation.calculate(scoped, year) + # Conservation: the stored award is everyone's dataset-year + # reports, and the couple's part is their own. + np.testing.assert_allclose( + stored, total(f"{award}_reported", False), rtol=1e-6 + ) + np.testing.assert_allclose( + couple, total(f"{award}_reported", True), rtol=1e-6 + ) + assert ((couple >= 0) & (couple <= stored)).all() + np.testing.assert_allclose( + simulation.calculate("income_support", year), + total("income_support_reported", True), + rtol=1e-6, + ) + # Either income-related award of the couple bars Income Support, and + # the passport follows the couple's awards alone. + esa = simulation.calculate("claimant_or_partner_esa_income", year) > 0 + jsa = simulation.calculate("claimant_or_partner_jsa_income", year) > 0 + income_support = simulation.calculate("income_support", year) > 0 + eligible = simulation.calculate("income_support_eligible", year) + assert not (eligible & (esa | jsa)).any() + np.testing.assert_array_equal( + simulation.calculate( + "council_tax_reduction_relevant_income_based_benefit", year + ), + esa | jsa | income_support, + ) + + +# The reform class itself, as create_structural_reforms_from_parameters builds +# it when the parameter is on. +ON = SimpleNamespace( + gov=SimpleNamespace( + contrib=SimpleNamespace( + policyengine=SimpleNamespace(disable_simulated_benefits=True) + ) + ) +) +DisableSimulatedBenefits = disable_simulated_benefits(lambda _: ON, None) +SET_BY_THE_REFORM = [ + "esa_income", + "claimant_or_partner_esa_income", + "claimant_or_partner_jsa_income", + "income_support", +] + + +def known_years(simulation, variable): + holder = simulation.get_holder(variable) + return sorted(int(str(p)[:4]) for p in holder.get_known_periods()) + + +def test_a_clone_or_branch_applies_the_reform_to_itself_alone(): + # A clone's copy of the tax-benefit system links to the clone, and a + # branch shares its parent's system. The reform reaches the simulation + # through that link, so it must reach the clone or branch applying it, + # never the parent. + parent = Simulation(situation=situation()) + clone = parent.clone() + assert clone.tax_benefit_system.simulation is clone + clone.apply_reform(DisableSimulatedBenefits) + branch = parent.get_branch("reformed") + assert branch.tax_benefit_system is parent.tax_benefit_system + branch.apply_reform(DisableSimulatedBenefits) + assert parent.tax_benefit_system.simulation is parent + ten_years = list(range(DATASET_YEAR, DATASET_YEAR + YEARS_IN_FUTURE)) + for variable in SET_BY_THE_REFORM: + assert known_years(parent, variable) == [], variable + assert known_years(clone, variable) == ten_years, variable + excluded = NAMES.index("excluded_adult") + for year in YEARS: + for simulation in (clone, branch): + assert simulation.calculate("esa_income", year)[excluded] == 3_000 + assert ( + simulation.calculate("claimant_or_partner_esa_income", year)[excluded] + == 0 + ) + # The parent still simulates ESA: the excluded adult's uprated report. + assert parent.calculate("esa_income", year)[excluded] >= 3_000 + + +# Applied after the simulation is built: passed as reform=, or applied to a +# clone or a branch. +LATER = ["reform=", "clone", "branch"] + + +@pytest.mark.parametrize("route", LATER) +def test_a_reform_applied_later_keeps_what_it_sets_through_a_reset(route): + # input_variables is taken when a simulation is built, and + # reset_calculations and derivative delete every other variable's values. + # A reform applied later names what it sets, or a reset deletes the + # claimant-or-partner awards and recalculates them. esa_income is entered + # here too, so a reset kept it while deleting its claimant-or-partner + # part, and the formula then read the excluded adult's ESA as the couple's. + entered = situation() + for benunit in entered["benunits"].values(): + benunit["esa_income"] = {DATASET_YEAR: 3_000} + if route == "reform=": + parent = None + simulation = Simulation(situation=entered, reform=DisableSimulatedBenefits) + else: + parent = Simulation(situation=entered) + names = parent.input_variables + recorded = list(names) + simulation = ( + parent.clone() if route == "clone" else parent.get_branch("reformed") + ) + simulation.apply_reform(DisableSimulatedBenefits) + # A clone or branch shares its parent's list until it names more. + assert parent.input_variables is names + assert parent.input_variables == recorded + assert "claimant_or_partner_esa_income" not in recorded + set_by_the_reform = [ + *BENEFITS, + *PRE_MINIMUM.values(), + *CLAIMANT_OR_PARTNER_AWARDS.values(), + ] + for variable in set_by_the_reform: + assert variable in simulation.input_variables, variable + excluded = NAMES.index("excluded_adult") + year = YEARS[1] + before = { + variable: simulation.calculate(variable, year) + for variable in [*set_by_the_reform, "income_support_eligible"] + } + assert before["claimant_or_partner_esa_income"][excluded] == 0 + assert before["income_support_eligible"][excluded] + # derivative recalculates on a clone after the same deletions. + assert not simulation.derivative( + "claimant_or_partner_esa_income", "employment_income", year + ).any() + simulation.reset_calculations() + for variable, value in before.items(): + np.testing.assert_array_equal( + simulation.calculate(variable, year), value, err_msg=variable + ) + + +@pytest.mark.parametrize("route", ["itself", "clone", "branch"]) +def test_a_reform_that_replaces_inputs_set_after_the_build_keeps_them(route): + # Inputs set after the build are recorded as inputs, but input_variables + # does not name them. A reform applied to the same simulation, or to a + # clone, that replaces one adds no new record, so the reform's inputs are + # named by what it sets. A branch stores its inputs under its own name, + # so there every record is new. + entered = situation() + for benunit in entered["benunits"].values(): + benunit["esa_income"] = {DATASET_YEAR: 3_000} + parent = Simulation(situation=entered) + for year in range(DATASET_YEAR, DATASET_YEAR + YEARS_IN_FUTURE): + parent.set_input( + "claimant_or_partner_esa_income", year, np.full(len(NAMES), 100) + ) + # Set after the build and not by the reform, so the reform does not name it. + parent.set_input( + "housing_benefit_applicable_income", YEARS[1], np.zeros(len(NAMES)) + ) + names = parent.input_variables + recorded = list(names) + assert "claimant_or_partner_esa_income" not in recorded + if route == "itself": + simulation = parent + elif route == "clone": + simulation = parent.clone() + else: + simulation = parent.get_branch("reformed") + simulation.apply_reform(DisableSimulatedBenefits) + if route != "itself": + assert parent.input_variables is names + assert parent.input_variables == recorded + for variable in SET_BY_THE_REFORM: + assert variable in simulation.input_variables, variable + assert "housing_benefit_applicable_income" not in simulation.input_variables + excluded = NAMES.index("excluded_adult") + year = YEARS[1] + before = { + variable: simulation.calculate(variable, year) + for variable in [*SET_BY_THE_REFORM, "income_support_eligible"] + } + assert before["claimant_or_partner_esa_income"][excluded] == 0 + assert before["income_support_eligible"][excluded] + assert not simulation.derivative( + "claimant_or_partner_esa_income", "employment_income", year + ).any() + simulation.reset_calculations() + for variable, value in before.items(): + np.testing.assert_array_equal( + simulation.calculate(variable, year), value, err_msg=variable + ) + + +class RemoveIncomeRelatedAwards(Reform): + def apply(self): + self.neutralize_variable("esa_income") + self.neutralize_variable("jsa_income") + + +def test_neutralizing_an_award_removes_the_claimant_or_partners_award(): + # The claimant's and partner's award is part of esa_income or + # jsa_income: a reform that removes the award removes their part too, + # including the part the reform set as an input. + simulation = Simulation(situation=situation(), scenario=REFORM) + simulation.apply_reform(RemoveIncomeRelatedAwards) + for year in YEARS: + for variable in [ + "esa_income", + "jsa_income", + "claimant_or_partner_esa_income", + "claimant_or_partner_jsa_income", + ]: + assert simulation.calculate(variable, year).tolist() == [0] * len(NAMES), ( + variable, + year, + ) + # Without the income-related awards nothing bars Income Support, and + # only Income Support passports council tax reduction. + assert by_family(simulation, "income_support_eligible", year) == { + "excluded_adult": True, + "excluded_adult_no_award": False, + "own_esa": True, + "own_jsa": True, + "own_esa_over_tariff": True, + "excluded_income_support": False, + } + assert by_family( + simulation, "council_tax_reduction_relevant_income_based_benefit", year + ) == dict( + zip(NAMES, (simulation.calculate("income_support", year) > 0).tolist()) + ) + assert not simulation.calculate("is_on_income_related_esa", year).any() + assert not simulation.calculate("is_on_income_based_jsa", year).any() + + +def test_an_award_neutralized_first_is_not_set(): + # Neutralized before the reform runs: neither the award nor the + # claimant's and partner's part of it takes what the reform sets. + def remove(simulation): + simulation.tax_benefit_system.neutralize_variable("esa_income") + + scenario = Scenario( + applied_before_data_load=True, + parameter_changes=REFORM.parameter_changes, + simulation_modifier=remove, + ) + simulation = Simulation(situation=situation(), scenario=scenario) + assert known_years(simulation, "esa_income") == [] + assert known_years(simulation, "claimant_or_partner_esa_income") == [] + assert len(known_years(simulation, "claimant_or_partner_jsa_income")) == ( + YEARS_IN_FUTURE + ) + for year in YEARS: + assert not simulation.calculate("claimant_or_partner_esa_income", year).any() + assert ( + by_family(simulation, "claimant_or_partner_jsa_income", year)["own_jsa"] + == 2_000 + ) + + +def test_neutralizing_an_award_removes_an_entered_claimant_or_partner_award(): + # The same without the reform, for an award entered directly. + simulation = Simulation(situation=situation()) + simulation.set_input( + "claimant_or_partner_esa_income", DATASET_YEAR, np.full(len(NAMES), 3_000) + ) + assert simulation.calculate("claimant_or_partner_esa_income", DATASET_YEAR).all() + simulation.apply_reform(RemoveIncomeRelatedAwards) + assert not simulation.calculate( + "claimant_or_partner_esa_income", DATASET_YEAR + ).any() + + +@pytest.mark.parametrize("year", YEARS) +def test_another_members_income_support_is_never_the_familys(reformed, year): + # The model pays Income Support only to the claimant's family, so + # another member's own report is left out of income_support (and so of + # household income) with the reform off, as the reform leaves it out. + for simulation in (Simulation(situation=situation()), reformed): + income_support = by_family(simulation, "income_support", year) + assert income_support["excluded_income_support"] == 0 + + +def test_the_claimant_and_partner_are_taken_in_each_year(): + # The amounts are held at the dataset year, but who the claimant and + # partner are is read in each year, as every reader of the awards reads + # it. Here the claimant in the dataset year is someone else's + # non-dependant afterwards, and the other adult claims instead. + first, later = DATASET_YEAR, YEARS[1:] + + def member(age, claimant_or_partner_in_first_year, **reported): + person = adult(age, claimant_or_partner_in_first_year, **reported) + person["is_claimant_or_partner"] = { + first: claimant_or_partner_in_first_year, + **{year: not claimant_or_partner_in_first_year for year in later}, + } + return person + + people = { + "a": member(40, True, esa_income_reported=3_000, income_support_reported=1_000), + "b": member(35, False, esa_income_reported=1_000), + } + simulation = Simulation( + situation={ + "people": people, + "benunits": {"family": {"members": ["a", "b"]}}, + "households": {"home": {"members": ["a", "b"]}}, + }, + scenario=REFORM, + ) + expected = {first: (3_000, 1_000)} + expected.update({year: (1_000, 0) for year in later}) + for year, (claimant_or_partner_esa, income_support) in expected.items(): + assert simulation.calculate("esa_income", year)[0] == 4_000 + assert ( + simulation.calculate("claimant_or_partner_esa_income", year)[0] + == claimant_or_partner_esa + ) + assert simulation.calculate("income_support", year)[0] == income_support diff --git a/policyengine_uk/tests/test_entered_directly_lifecycle.py b/policyengine_uk/tests/test_entered_directly_lifecycle.py index 8da15defc3..ebf572ed41 100644 --- a/policyengine_uk/tests/test_entered_directly_lifecycle.py +++ b/policyengine_uk/tests/test_entered_directly_lifecycle.py @@ -2,6 +2,7 @@ income_support_eligible reads income-related ESA and income-based JSA from the claimant's and partner's reported awards (SSCBA 1992 s.124(1)(f), (h)), +through claimant_or_partner_esa_income and claimant_or_partner_jsa_income, unless esa_income or jsa_income holds a different award from the one all the reported amounts give, which is then taken to be the couple's. The rule depends only on the value the simulation reads, so it holds through @@ -54,7 +55,15 @@ def family(award): def eligible(simulation): - simulation.delete_arrays("income_support_eligible") + # The gate reads the claimant-or-partner awards, which are calculated and + # cached like any other variable. Recalculate them too after changing an + # input they depend on. + for variable in ( + "income_support_eligible", + "claimant_or_partner_esa_income", + "claimant_or_partner_jsa_income", + ): + simulation.delete_arrays(variable) return bool(simulation.calculate("income_support_eligible", YEAR)[0]) diff --git a/policyengine_uk/tests/test_income_support_direct_inputs.py b/policyengine_uk/tests/test_income_support_direct_inputs.py index 6f519f99c0..597c6de878 100644 --- a/policyengine_uk/tests/test_income_support_direct_inputs.py +++ b/policyengine_uk/tests/test_income_support_direct_inputs.py @@ -1,9 +1,10 @@ """A jsa_income that differs from the reported award is the couple's award. income_support_eligible reads income-based JSA (SSCBA 1992 s.124(1)(f)) from -the claimant's and partner's reported awards after the jsa_income capital -screen. Where jsa_income holds a different award from the one all the -reported amounts give, that award is used instead and taken to be the +claimant_or_partner_jsa_income: the claimant's and partner's reported awards +after the jsa_income capital screen, or the raw total of their reports where +jsa_income holds the raw total of everyone's. Where jsa_income holds a +different award from either, that award is used instead and taken to be the claimant's or partner's. That covers an award entered directly (when the simulation is built or later, or on a branch) and a reform that changes or removes jsa_income. An award entered for one year says nothing about another. @@ -53,7 +54,15 @@ def carer_family(benunit_inputs=None, other_adult=None): def eligible(simulation, year=YEAR): - simulation.delete_arrays("income_support_eligible") + # The gate reads the claimant-or-partner awards, which are calculated and + # cached like any other variable. Recalculate them too after changing an + # input they depend on. + for variable in ( + "income_support_eligible", + "claimant_or_partner_esa_income", + "claimant_or_partner_jsa_income", + ): + simulation.delete_arrays(variable) return bool(simulation.calculate("income_support_eligible", year)[0]) @@ -115,13 +124,16 @@ def test_a_reform_that_removes_income_based_jsa_lifts_the_bar(): def test_the_raw_reported_total_is_read_through_the_reports(): - """disable_simulated_benefits sets jsa_income to the raw total of every - member's report. Here the partner reports £200 and an adult outside the - couple £3,000, with capital of £10,000. Tariff income of - ceil(4,000 / 250) x £1 x 52 = £832 a year extinguishes the partner's - award, so the claimant and partner have no income-based JSA. The raw - total, £3,200, differs from the screened award, £3,200 - £832 = £2,368, - but is still what the reports give, so the reports decide whose it is.""" + """A jsa_income equal to the raw total of every member's report, before + the capital test, is read as reported amounts paid in full (as + disable_simulated_benefits pays them): the claimant's and partner's + award is the raw total of their own reports. Here the partner reports + £200 and an adult outside the couple £3,000, with capital of £10,000. + Tariff income of ceil(4,000 / 250) x £1 x 52 = £832 a year extinguishes + the partner's award on the formula's reading, so the formula's £2,368 + (£3,200 - £832) does not bar the claim. The raw total, £3,200, pays the + partner's £200 unscreened, so it does. The excluded adult's £3,000 is + theirs on either reading.""" members = ["carer", "partner", "other_adult"] simulation = Simulation( situation={ @@ -156,7 +168,8 @@ def test_the_raw_reported_total_is_read_through_the_reports(): assert simulation.calculate("jsa_income", YEAR)[0] == 2_368 assert eligible(simulation) simulation.set_input("jsa_income", YEAR, [3_200]) - assert eligible(simulation) + assert not eligible(simulation) + assert simulation.calculate("claimant_or_partner_jsa_income", YEAR)[0] == 200 # A different award entered directly is the couple's. simulation.set_input("jsa_income", YEAR, [4_000]) assert not eligible(simulation) @@ -220,10 +233,12 @@ def test_an_excluded_adults_report_can_explain_a_direct_award(): def test_a_claimants_report_can_explain_a_direct_award(): - # Intended: a direct £200 with £10,000 of capital and no reports is the - # claimant's, and bars the claim. If the claimant reports exactly £200, - # the raw total explains it, and the claimant's £200 is extinguished by - # tariff income of £832 a year, so it no longer bars the claim. + # Intended: with £10,000 of capital, the formula's reading of the + # claimant's own £200 report leaves nothing after tariff income of £832 a + # year, so it does not bar the claim. A direct £200 with no report is the + # claimant's, and bars it. A direct £200 equal to the claimant's report is + # the raw total of the reports, read as the claimant's £200 paid in full + # (as disable_simulated_benefits pays it), so it bars the claim too. claimant_reports = { "age": {YEAR: 40}, "is_claimant_or_partner": {YEAR: True}, @@ -231,25 +246,26 @@ def test_a_claimants_report_can_explain_a_direct_award(): "income_support_reported": {YEAR: 1_000}, } - def single(report): + def single(report, entered=None): person = {**claimant_reports, "jsa_income_reported": {YEAR: report}} + benunit = { + "members": ["carer"], + "income_support_assessable_capital": {YEAR: 10_000}, + "jsa_income_assessable_capital": {YEAR: 10_000}, + } + if entered is not None: + benunit["jsa_income"] = {YEAR: entered} return Simulation( situation={ "people": {"carer": person}, - "benunits": { - "family": { - "members": ["carer"], - "income_support_assessable_capital": {YEAR: 10_000}, - "jsa_income_assessable_capital": {YEAR: 10_000}, - "jsa_income": {YEAR: 200}, - } - }, + "benunits": {"family": benunit}, "households": {"home": {"members": ["carer"]}}, } ) - assert not eligible(single(0)) assert eligible(single(200)) + assert not eligible(single(0, entered=200)) + assert not eligible(single(200, entered=200)) def test_the_half_penny_tolerance_endpoints(): diff --git a/policyengine_uk/tests/test_income_support_esa_entered_directly.py b/policyengine_uk/tests/test_income_support_esa_entered_directly.py index bf19537350..ce9b42862f 100644 --- a/policyengine_uk/tests/test_income_support_esa_entered_directly.py +++ b/policyengine_uk/tests/test_income_support_esa_entered_directly.py @@ -1,10 +1,12 @@ """Income-related ESA that is not the award on the reported amounts. -income_support_eligible reads the claimant's and partner's reported -income-related ESA, after the esa_income capital test. When esa_income holds -anything else (an award entered directly, or a reform that replaces or -removes it), the reported amounts do not say whose award it is, so the gate -takes esa_income itself to be the claimant's or partner's. The test is on the +income_support_eligible reads the claimant's and partner's income-related ESA +from claimant_or_partner_esa_income: their reported awards after the +esa_income capital test, or the plain total of their reports where esa_income +holds the plain total of everyone's. When esa_income holds anything else (an +award entered directly, or a reform that replaces or removes it), the +reported amounts do not say whose award it is, so the gate takes esa_income +itself to be the claimant's or partner's. The test is on the value the simulation reads, so it holds however that value got there: set before or after the simulation is built, on a clone or a branch, for this year or another, deleted and recalculated. @@ -60,7 +62,15 @@ def family(benunit=None): def eligible(sim): - sim.delete_arrays("income_support_eligible") + # The gate reads the claimant-or-partner awards, which are calculated and + # cached like any other variable. Recalculate them too after changing an + # input they depend on. + for variable in ( + "income_support_eligible", + "claimant_or_partner_esa_income", + "claimant_or_partner_jsa_income", + ): + sim.delete_arrays(variable) return bool(sim.calculate("income_support_eligible", YEAR)[0]) @@ -143,11 +153,14 @@ def test_abolishing_income_related_esa_removes_the_bar(): def test_the_plain_reported_total_is_read_through_the_reports(): - # disable_simulated_benefits sets esa_income to the plain total of the - # reported awards, before the capital test. With £10,000 of capital the - # tariff income (£832 a year) extinguishes the partner's own £200, so the - # claimant and partner have no income-related ESA; the £3,000 is the - # excluded adult's. + # An esa_income equal to the plain total of the reported awards, before + # the capital test, is read as reported amounts paid in full (as + # disable_simulated_benefits pays them): the claimant's and partner's + # award is the plain total of their own reports. With £10,000 of capital + # the tariff income (£832 a year) would extinguish the partner's own + # £200 on the formula's reading, so the formula's award does not bar the + # claim. The plain total pays the partner's £200 unscreened, so it does. + # Either way the excluded adult's £3,000 is theirs, not the couple's. partner = { "age": {YEAR: 42}, "is_claimant_or_partner": {YEAR: True}, @@ -157,8 +170,18 @@ def test_the_plain_reported_total_is_read_through_the_reports(): {"carer": CARER, "partner": partner, "other_adult": EXCLUDED_ADULT}, {"esa_income_assessable_capital": {YEAR: 10_000}}, ) - sim.set_input("esa_income", YEAR, np.array([3_200.0])) + assert sim.calculate("esa_income", YEAR)[0] == 2_368 assert eligible(sim) + sim.set_input("esa_income", YEAR, np.array([3_200.0])) + assert not eligible(sim) + assert sim.calculate("claimant_or_partner_esa_income", YEAR)[0] == 200 + # Without the partner's report, the plain total is the excluded adult's. + alone = simulation( + {"carer": CARER, "other_adult": EXCLUDED_ADULT}, + {"esa_income_assessable_capital": {YEAR: 10_000}}, + ) + alone.set_input("esa_income", YEAR, np.array([3_000.0])) + assert eligible(alone) def test_an_award_equal_to_the_reported_amounts_is_read_through_them(): diff --git a/policyengine_uk/tests/test_income_support_gate_reads_claimant_or_partner_awards.py b/policyengine_uk/tests/test_income_support_gate_reads_claimant_or_partner_awards.py new file mode 100644 index 0000000000..b14da3de7b --- /dev/null +++ b/policyengine_uk/tests/test_income_support_gate_reads_claimant_or_partner_awards.py @@ -0,0 +1,565 @@ +"""income_support_eligible reads the claimant-or-partner awards. + +Its income-related ESA and income-based JSA limbs (SSCBA 1992 s.124(1)(h) +and (f)) used to work out the claimant's and partner's award inline, by the +value rule that claimant_or_partner_esa_income and +claimant_or_partner_jsa_income also apply. They now read those variables, so +every reader shares one award, including one entered directly or set by the +disable_simulated_benefits reform. This file checks the switch against the +code it replaced (policyengine-uk main at e89b8abcf, with #2013, #2025 +and #2027 merged; comments removed). + +The two agree in formula mode, for awards entered directly, set later, set on +a branch, and recalculated after deletion. They differ, by design, in one +case: a stored award equal to the plain total of everyone's reports but not +to the formula's award. The old limbs then read the claimant's and partner's +award after the benefit's screen. The variables, and now the limbs, read the +plain total of their reports, the reports paid in full, as the reform that +stores plain totals pays them. That case is checked against its own +expectation. + +The replaced code also ignored a claimant-or-partner award entered directly +(review r3c on #2027). It worked the couple's award out from esa_income and +everyone's reports, so an adult outside the couple could decide whether the +entered award barred the claim. The last tests enter the claimant's and +partner's award and check that it alone decides its limb. +""" + +import numpy as np +import pytest +from hypothesis import event, given +from hypothesis import strategies as st +from policyengine_core.periods import period as as_period + +from policyengine_uk import Simulation +from policyengine_uk.model_api import add +from policyengine_uk.tests.test_income_support_eligibility_properties import ( + DIFFERENTIAL_SETTINGS, + FAMILIES, + input_settings, + situation as property_situation, +) +from policyengine_uk.variables.gov.dwp.esa_income import income_related_esa_award +from policyengine_uk.variables.gov.dwp.jsa_income import income_related_jsa_award + +YEAR = 2025 +HALF_PENNY = 0.005 +LIMBS = { + "esa_income": ( + "esa_income_reported", + income_related_esa_award, + "claimant_or_partner_esa_income", + ), + "jsa_income": ( + "jsa_income_reported", + income_related_jsa_award, + "claimant_or_partner_jsa_income", + ), +} + + +def pre_switch_income_support_eligible(benunit, period, parameters): + IS = parameters(period).gov.dwp.income_support + person = benunit.members + claimant_or_partner = person("is_claimant_or_partner", period) + has_award = claimant_or_partner & (person("income_support_reported", period) > 0) + youngest_child_5_or_under = ( + benunit("youngest_child_age_for_legacy_benefits", period) + <= IS.eligibility.lone_parent_youngest_child_age_limit + ) + lone_parent_with_young_child = ( + benunit("is_lone_parent", period) & youngest_child_5_or_under + ) + placed_child = ( + person("is_looked_after_by_local_authority", period) + & person("is_child_for_child_benefit", period) + & ~claimant_or_partner + ) + single_with_placed_child = benunit("is_single", period) & benunit.any(placed_child) + prescribed_category = person("is_carer_for_benefits", period) | benunit.project( + lone_parent_with_young_child | single_with_placed_child + ) + under_qualifying_age = ~person( + "has_attained_state_pension_credit_qualifying_age", period + ) + WORK = IS.eligibility.remunerative_work + hours = person("income_support_remunerative_work_hours", period) + not_treated_as_working = person("is_carer_for_benefits", period) + works_as_claimant = ~not_treated_as_working & (hours >= WORK.claimant_hours) + works_as_partner = ( + claimant_or_partner & ~not_treated_as_working & (hours >= WORK.partner_hours) + ) + other_member_works = ( + benunit.project(benunit.sum(works_as_partner)) - works_as_partner + ) > 0 + no_contributory_jsa = person("jsa_contrib", period) <= 0 + no_contributory_esa = person("esa_contrib", period) <= 0 + claimant = ( + has_award + & prescribed_category + & under_qualifying_age + & no_contributory_esa + & no_contributory_jsa + & ~works_as_claimant + & ~other_member_works + ) + esa_income = benunit("esa_income", period) + reported_total = add(benunit, period, ["esa_income_reported"]) + award_on_all_reports = income_related_esa_award(benunit, period, reported_total) + award_on_claimant_or_partner_reports = income_related_esa_award( + benunit, + period, + benunit.sum(person("esa_income_reported", period) * claimant_or_partner), + ) + stored = esa_income.dtype + half_penny = 0.005 + as_reported = np.isclose( + esa_income, award_on_all_reports.astype(stored), rtol=0, atol=half_penny + ) | np.isclose(esa_income, reported_total.astype(stored), rtol=0, atol=half_penny) + income_related_esa = (esa_income > half_penny) & ( + ~as_reported | (award_on_claimant_or_partner_reports > half_penny) + ) + jsa_income = benunit("jsa_income", period) + jsa_reported_total = add(benunit, period, ["jsa_income_reported"]) + jsa_award_on_all_reports = income_related_jsa_award( + benunit, period, jsa_reported_total + ) + jsa_award_on_claimant_or_partner_reports = income_related_jsa_award( + benunit, + period, + benunit.sum(person("jsa_income_reported", period) * claimant_or_partner), + ) + jsa_as_reported = np.isclose( + jsa_income, + jsa_award_on_all_reports.astype(jsa_income.dtype), + rtol=0, + atol=half_penny, + ) | np.isclose( + jsa_income, + jsa_reported_total.astype(jsa_income.dtype), + rtol=0, + atol=half_penny, + ) + income_based_jsa = (jsa_income > half_penny) & ( + ~jsa_as_reported | (jsa_award_on_claimant_or_partner_reports > half_penny) + ) + on_pension_credit = benunit( + "meets_pension_credit_age_conditions", period + ) & benunit("would_claim_pc", period) + capital = benunit("income_support_assessable_capital", period) + return ( + benunit.any(claimant) + & ~on_pension_credit + & ~income_related_esa + & ~income_based_jsa + & (capital <= IS.means_test.capital.limit) + ) + + +def pre_switch_limb(benunit, period, award): + """One of the replaced inline limbs. The ESA and JSA blocks were the same + code with the names swapped; returns the limb and whether the stored award + equals only the plain total (the case that differs by design).""" + reported, award_on, _ = LIMBS[award] + person = benunit.members + claimant_or_partner = person("is_claimant_or_partner", period) + stored = benunit(award, period) + reported_total = add(benunit, period, [reported]) + on_all_reports = award_on(benunit, period, reported_total) + claimant_or_partner_reported = benunit.sum( + person(reported, period) * claimant_or_partner + ) + on_claimant_or_partner_reports = award_on( + benunit, period, claimant_or_partner_reported + ) + as_formula = np.isclose( + stored, on_all_reports.astype(stored.dtype), rtol=0, atol=HALF_PENNY + ) + as_reported_total = np.isclose( + stored, reported_total.astype(stored.dtype), rtol=0, atol=HALF_PENNY + ) + limb = (stored > HALF_PENNY) & ( + ~(as_formula | as_reported_total) + | (on_claimant_or_partner_reports > HALF_PENNY) + ) + # The intended reading in the plain-total case: the claimant's and + # partner's reports paid in full. + plain_total_only = as_reported_total & ~as_formula + intended = np.where( + plain_total_only, + (stored > HALF_PENNY) & (claimant_or_partner_reported > HALF_PENNY), + limb, + ) + return limb, intended, plain_total_only + + +def fresh(simulation, variables): + for variable in variables: + simulation.delete_arrays(variable) + + +def compare(simulation, year=YEAR): + """Assert the switched gate and limbs against the replaced code.""" + period = as_period(year) + benunit = simulation.populations["benunit"] + fresh( + simulation, + ["income_support_eligible", *[limb[2] for limb in LIMBS.values()]], + ) + differs = np.zeros(benunit.count, dtype=bool) + for award, (_, _, variable) in LIMBS.items(): + old, intended, plain_total_only = pre_switch_limb(benunit, period, award) + new = simulation.calculate(variable, year) > 0 + np.testing.assert_array_equal(new, intended, err_msg=award) + np.testing.assert_array_equal( + new[~plain_total_only], old[~plain_total_only], err_msg=award + ) + differs |= plain_total_only & (new != old) + old_gate = pre_switch_income_support_eligible( + benunit, period, simulation.tax_benefit_system.parameters + ) + new_gate = simulation.calculate("income_support_eligible", year) + np.testing.assert_array_equal(new_gate[~differs], old_gate[~differs]) + return differs + + +CARER = { + "age": 40, + "is_claimant_or_partner": True, + "receives_carer_benefit": True, + "income_support_reported": 1_000, +} +PARTNER = {"age": 42, "is_claimant_or_partner": True} +OTHER_ADULT = { + "age": 30, + "is_claimant_or_partner": False, + "current_education": "NOT_IN_EDUCATION", +} + +# Each family is (members, capital). The carer satisfies every other +# condition, so only the income-related limbs decide eligibility. +EXAMPLES = [ + ([CARER], 0), + ([CARER, {**OTHER_ADULT, "esa_income_reported": 3_000}], 0), + ([CARER, {**OTHER_ADULT, "jsa_income_reported": 3_000}], 0), + ([CARER, {**PARTNER, "esa_income_reported": 3_000}], 0), + ([CARER, {**PARTNER, "jsa_income_reported": 3_000}], 0), + # £10,000 of capital: tariff income of £832 a year extinguishes the + # partner's own £200 on the formula's reading. + ( + [ + CARER, + {**PARTNER, "esa_income_reported": 200, "jsa_income_reported": 200}, + { + **OTHER_ADULT, + "esa_income_reported": 3_000, + "jsa_income_reported": 3_000, + }, + ], + 10_000, + ), +] +OVER_TARIFF = len(EXAMPLES) - 1 + + +def simulation(families, entered=None): + people, benunits, households = {}, {}, {} + for i, (members, capital) in enumerate(families): + names = [] + for j, inputs in enumerate(members): + name = f"p{i}_{j}" + people[name] = {k: {YEAR: v} for k, v in inputs.items()} + names.append(name) + benunits[f"b{i}"] = { + "members": names, + "income_support_assessable_capital": {YEAR: capital}, + "esa_income_assessable_capital": {YEAR: capital}, + "jsa_income_assessable_capital": {YEAR: capital}, + } + for award, values in (entered or {}).items(): + benunits[f"b{i}"][award] = {YEAR: values[i]} + households[f"h{i}"] = {"members": names} + return Simulation( + situation={"people": people, "benunits": benunits, "households": households} + ) + + +def gate(simulation): + fresh( + simulation, + ["income_support_eligible", *[limb[2] for limb in LIMBS.values()]], + ) + return simulation.calculate("income_support_eligible", YEAR).tolist() + + +ENTERED = [4_000, 0, 4_000, 0, 3_000, 4_000] + + +def test_formula_mode_agrees(): + sim = simulation(EXAMPLES) + assert not compare(sim).any() + # Only the claimant's and partner's own awards bar the claim. + assert gate(sim) == [True, True, True, False, False, True] + + +def test_awards_entered_in_the_situation_agree(): + for award in LIMBS: + assert not compare(simulation(EXAMPLES, {award: ENTERED})).any(), award + + +def test_awards_set_after_calculating_agree(): + sim = simulation(EXAMPLES) + compare(sim) + for award in LIMBS: + sim.set_input(award, YEAR, np.array(ENTERED, dtype=float)) + assert not compare(sim).any(), award + + +def test_awards_set_on_a_branch_agree_there_and_leave_the_parent_alone(): + sim = simulation(EXAMPLES) + before = gate(sim) + branch = sim.get_branch("entered", clone_system=False) + for award in LIMBS: + branch.set_input(award, YEAR, np.array(ENTERED, dtype=float)) + assert not compare(branch).any() + assert not compare(sim).any() + assert gate(sim) == before + nested = branch.get_branch("zero", clone_system=False) + nested.set_input("esa_income", YEAR, np.zeros(len(EXAMPLES))) + assert not compare(nested).any() + + +def test_a_deleted_award_is_recalculated_and_agrees(): + sim = simulation(EXAMPLES) + before = gate(sim) + for award in LIMBS: + sim.set_input(award, YEAR, np.array(ENTERED, dtype=float)) + assert not compare(sim).any() + for award in LIMBS: + sim.delete_arrays(award) + assert not compare(sim).any() + assert gate(sim) == before + + +def test_a_plain_total_reads_the_claimant_and_partner_reports_in_full(): + # Intended difference. The stored award is the plain total of everyone's + # reports. In the last family the partner's £200 is extinguished by + # tariff income on the formula's reading; the old limb read that, and did + # not bar the claim. The plain total pays the £200 in full, so it does. + for award, (reported, _, variable) in LIMBS.items(): + sim = simulation(EXAMPLES) + plain_total = sim.calculate(reported, YEAR, map_to="benunit") + sim.set_input(award, YEAR, plain_total) + differs = compare(sim) + assert differs.tolist() == [i == OVER_TARIFF for i in range(len(EXAMPLES))] + assert sim.calculate(variable, YEAR)[OVER_TARIFF] == 200 + old_gate = pre_switch_income_support_eligible( + sim.populations["benunit"], + as_period(YEAR), + sim.tax_benefit_system.parameters, + ) + assert old_gate[OVER_TARIFF] + assert not gate(sim)[OVER_TARIFF] + + +# Stored values at and around both readings, half a penny either side. +OFFSETS = [0, 0, 0.004, -0.004, 0.006, -0.006] + + +@st.composite +def stored_award_families(draw): + """A claimant, an optional partner and an optional adult outside the + couple, each reporting ESA and JSA, and how each award is stored.""" + amounts = st.sampled_from([0, 0, 0.004, 200, 3_000, 65_536.01]) + members = [ + {**CARER, "esa_income_reported": draw(amounts), "jsa_income_reported": 0} + ] + members[0]["jsa_income_reported"] = draw(amounts) + if draw(st.booleans()): + members.append( + { + **PARTNER, + "esa_income_reported": draw(amounts), + "jsa_income_reported": draw(amounts), + } + ) + if draw(st.booleans()): + members.append( + { + **OTHER_ADULT, + "esa_income_reported": draw(amounts), + "jsa_income_reported": draw(amounts), + } + ) + capital = draw(st.sampled_from([0, 6_250, 10_000, 20_000])) + storage = { + award: ( + draw(st.sampled_from(["formula", "plain_total", "entered", "zero"])), + draw(st.sampled_from(OFFSETS)), + ) + for award in LIMBS + } + return (members, capital), storage + + +@DIFFERENTIAL_SETTINGS +@given(st.lists(stored_award_families(), min_size=1, max_size=10)) +def test_the_limbs_read_every_stored_value_as_intended(drawn): + families = [family for family, _ in drawn] + sim = simulation(families) + for award, (reported, _, _) in LIMBS.items(): + formula = sim.calculate(award, YEAR).astype(float) + plain_total = sim.calculate(reported, YEAR, map_to="benunit") + stored = [] + for i, (_, storage) in enumerate(drawn): + mode, offset = storage[award] + base = { + "formula": formula[i], + "plain_total": plain_total[i], + "entered": 4_000, + "zero": 0, + }[mode] + stored.append(max(0, base + offset)) + event(f"{award} stored as {mode}") + sim.set_input(award, YEAR, np.array(stored)) + if compare(sim).any(): + event("the plain-total reading changes a family's eligibility") + # The claimant's and partner's award is never negative and never more + # than the stored award it is part of (to the half penny of the rule). + for award, (_, _, variable) in LIMBS.items(): + scoped = sim.calculate(variable, YEAR) + assert (scoped >= 0).all(), award + assert (scoped <= sim.calculate(award, YEAR) + 0.005).all(), award + + +@DIFFERENTIAL_SETTINGS +@given(FAMILIES, st.data()) +def test_the_switched_gate_matches_the_replaced_code(drawn, data): + # Families from the eligibility properties, with adults outside the + # couple who report awards, and awards calculated or entered directly + # (£0 or £4,000, never a plain total). The gates agree exactly. + capital_as_savings, esa_income, jsa_income = data.draw(input_settings(len(drawn))) + units = [(*family, extra) for family, extra in drawn] + sim = Simulation( + situation=property_situation(units, capital_as_savings, esa_income, jsa_income) + ) + assert not compare(sim).any() + + +# What the adult outside the couple reports in the entered-award cases. With +# £3,000 of the award stored for the benefit unit, a report of £3,000 is what +# the reports give, so the replaced code read the stored award through them +# and took it to be the outside adult's; £2,000 and £5,000 are not, so it +# took the stored award to be the couple's. Neither reading looked at the +# claimant-or-partner award entered beside it. +OUTSIDE_REPORTS = [2_000, 3_000, 5_000] + + +def entered_directly(award, scoped): + """A carer otherwise eligible for Income Support and an adult outside the + couple who reports each of OUTSIDE_REPORTS of the award, one family each. + £3,000 of the award is entered for the benefit unit and `scoped` as the + claimant's and partner's.""" + reported, _, variable = LIMBS[award] + families = [ + ([CARER, {**OTHER_ADULT, reported: other}], 0) for other in OUTSIDE_REPORTS + ] + n = len(families) + sim = simulation(families, {award: [3_000] * n, variable: [scoped] * n}) + # Income-based JSA closed on 1 April 2026; the cases need a year in which + # it still pays awards. + assert sim.tax_benefit_system.parameters(YEAR).gov.dwp.JSA.income.active + assert sim.calculate(award, YEAR).tolist() == [3_000] * n + assert sim.calculate(variable, YEAR).tolist() == [scoped] * n + return sim + + +@pytest.mark.parametrize("award", LIMBS) +def test_an_entered_claimant_or_partner_award_bars_the_claim(award): + # The couple are on the income-related award (s.124(1)(h) for ESA, (f) + # for JSA), so Income Support is barred whatever the outside adult + # reports. The replaced code let the claim through where that report was + # £3,000; #2013's gate, which had no JSA limb, let every JSA case through. + sim = entered_directly(award, 3_000) + assert sim.calculate("income_support_eligible", YEAR).tolist() == [False] * 3 + assert sim.calculate("income_support", YEAR).tolist() == [0] * 3 + + +@pytest.mark.parametrize("award", LIMBS) +def test_an_entered_zero_claimant_or_partner_award_never_bars_the_claim(award): + # The £3,000 stored for the benefit unit is all the outside adult's, so it + # never bars the claim. The replaced code barred it where that adult's + # report was not £3,000. + sim = entered_directly(award, 0) + assert sim.calculate("income_support_eligible", YEAR).tolist() == [True] * 3 + assert (sim.calculate("income_support", YEAR) > 0).all() + + +@st.composite +def entered_awards(draw, n): + """What is entered for each of n families. esa_income and jsa_income are + calculated, or entered for every family as £0, £3,000 (what an outside + adult's report can give) or £4,000. The claimant-or-partner awards are + always entered, for every family.""" + entries = {} + for award, (_, _, variable) in LIMBS.items(): + if draw(st.booleans()): + entries[award] = draw( + st.lists(st.sampled_from([0, 3_000, 4_000]), min_size=n, max_size=n) + ) + entries[variable] = draw( + st.lists(st.sampled_from([0, 0, 3_000, 4_000]), min_size=n, max_size=n) + ) + return entries + + +@DIFFERENTIAL_SETTINGS +@given(FAMILIES, st.data()) +def test_an_entered_claimant_or_partner_award_alone_decides_its_limb(drawn, data): + # Families from the eligibility properties, each with an adult outside + # the couple, in three copies: + # - as drawn, with the claimant-or-partner awards entered; + # - the same, with the outside adult's ESA and JSA reports redrawn; + # - as drawn, with the claimant-or-partner awards entered as zero. + # The outside adult's reports never change the gate, and an entered award + # bars the claim exactly when it is positive (s.124(1)(f), (h)). + k = len(drawn) + capital_as_savings = data.draw(st.booleans()) + entries = data.draw(entered_awards(k)) + outside = st.sampled_from([0, 2_000, 3_000, 5_000]) + redrawn = data.draw(st.lists(st.tuples(outside, outside), min_size=k, max_size=k)) + as_drawn = [(*family, extra) for family, extra in drawn] + moved = [ + (*family, {**extra, "esa_income_reported": esa, "jsa_income_reported": jsa}) + for (family, extra), (esa, jsa) in zip(drawn, redrawn) + ] + situation = property_situation( + as_drawn + moved + as_drawn, + capital_as_savings, + entries.get("esa_income"), + entries.get("jsa_income"), + ) + variables = [variable for _, _, variable in LIMBS.values()] + for i in range(3 * k): + for variable in variables: + value = entries[variable][i % k] if i < 2 * k else 0 + situation["benunits"][f"b{i}"][variable] = {YEAR: value} + sim = Simulation(situation=situation) + for variable in variables: + expected = entries[variable] * 2 + [0] * k + assert sim.calculate(variable, YEAR).tolist() == expected, variable + eligible = sim.calculate("income_support_eligible", YEAR) + entered, redrawn_eligible, unbarred = ( + eligible[:k], + eligible[k : 2 * k], + eligible[2 * k :], + ) + np.testing.assert_array_equal(redrawn_eligible, entered) + barred = np.zeros(k, dtype=bool) + for variable in variables: + barred |= np.array(entries[variable]) > 0 + np.testing.assert_array_equal(entered, unbarred & ~barred) + if (unbarred & barred).any(): + event("an entered award bars an otherwise eligible family") + if (unbarred & ~barred).any(): + event("an entered zero leaves a family eligible") diff --git a/policyengine_uk/tests/test_reform_inputs_survive_reset.py b/policyengine_uk/tests/test_reform_inputs_survive_reset.py new file mode 100644 index 0000000000..d6495331d3 --- /dev/null +++ b/policyengine_uk/tests/test_reform_inputs_survive_reset.py @@ -0,0 +1,352 @@ +"""A reform applied to a built simulation keeps the inputs it stores. + +input_variables is taken when a simulation is built, and reset_calculations +and derivative delete the values of every variable it does not name. So +Simulation.apply_reform names each variable the reform stored an input for +that the simulation still holds afterwards, however the reform stored it, and +no other: an input set since the build that the reform did not store is still +deleted by a reset, as before. See _InputKeyLog in policyengine_uk/simulation.py. + +Each simulation is one adult, the claimant, with £3,000 of income-related ESA +entered for 2026, so their claimant-or-partner award that year is £3,000 +unless an input replaces it. +""" + +import threading +from functools import cache + +import numpy as np +import pytest +from hypothesis import HealthCheck, given, settings +from hypothesis import strategies as st + +from policyengine_core.periods import period + +from policyengine_uk import Simulation +from policyengine_uk.model_api import ( + MONTH, + YEAR, + BenUnit, + Reform, + Variable, + set_input_dispatch_by_period, +) + +SCOPED = "claimant_or_partner_esa_income" +AWARD_YEAR = 2026 +SITUATION = { + "people": { + "claimant": { + "age": {2025: 40, 2026: 40, 2027: 40}, + "is_claimant_or_partner": {2025: True, 2026: True, 2027: True}, + } + }, + "benunits": { + "benunit": {"members": ["claimant"], "esa_income": {AWARD_YEAR: 3_000}} + }, + "households": {"household": {"members": ["claimant"]}}, +} + + +def as_reform(apply): + return type("TestReform", (Reform,), {"apply": apply}) + + +def built(): + # An input set after the build: recorded as an input, not named. + simulation = Simulation(situation=SITUATION) + simulation.set_input(SCOPED, AWARD_YEAR, [100]) + assert SCOPED not in simulation.input_variables + return simulation + + +def through_the_simulation(reform): + reform.simulation.set_input(SCOPED, AWARD_YEAR, [0]) + + +def through_the_holder(reform): + simulation = reform.simulation + holder = simulation.get_holder(SCOPED) + holder.set_input(period(AWARD_YEAR), [0], simulation.branch_name) + + +def from_another_thread(reform): + worker = threading.Thread( + target=reform.simulation.set_input, args=(SCOPED, AWARD_YEAR, [0]) + ) + worker.start() + worker.join() + + +def while_another_simulation_applies_a_reform(reform): + simulation = reform.simulation + other = simulation.clone() + other.apply_reform( + as_reform(lambda _: simulation.set_input(SCOPED, AWARD_YEAR, [0])) + ) + + +def after_applying_another_reform_to_itself(reform): + simulation = reform.simulation + simulation.apply_reform(as_reform(lambda _: None)) + simulation.set_input(SCOPED, AWARD_YEAR, [0]) + + +STORING = { + "the simulation's set_input": as_reform(through_the_simulation), + "a holder's set_input": as_reform(through_the_holder), + "another thread": as_reform(from_another_thread), + "while another simulation applies a reform": as_reform( + while_another_simulation_applies_a_reform + ), + "after applying another reform to itself": as_reform( + after_applying_another_reform_to_itself + ), + "a tuple's second reform": ( + as_reform(lambda _: None), + as_reform(through_the_simulation), + ), +} + + +@pytest.mark.parametrize("route", STORING) +def test_a_reform_keeps_what_it_stores_through_a_reset(route): + simulation = built() + names = simulation.input_variables + recorded = list(names) + simulation.apply_reform(STORING[route]) + assert simulation.input_variables == recorded + [SCOPED] + assert names == recorded + assert type(simulation._user_input_keys) is set + assert simulation.tax_benefit_system.simulation is simulation + assert simulation.calculate(SCOPED, AWARD_YEAR).tolist() == [0] + assert not simulation.derivative(SCOPED, "employment_income", AWARD_YEAR).any() + simulation.reset_calculations() + assert simulation.calculate(SCOPED, AWARD_YEAR).tolist() == [0] + + +class ended_award(Variable): + value_type = float + entity = BenUnit + label = "An award that ends in 2026" + definition_period = YEAR + end = "2026-12-31" + + def formula(benunit, period, parameters): + return 42 + + +def stores_then_deletes_another_year(simulation): + def apply(reform): + reform.simulation.set_input(SCOPED, AWARD_YEAR + 1, [0]) + reform.simulation.delete_arrays(SCOPED, AWARD_YEAR + 1) + + simulation.apply_reform(as_reform(apply)) + return SCOPED, AWARD_YEAR, 3_000 + + +def sets_a_year_after_the_variables_end(simulation): + simulation.tax_benefit_system.add_variable(ended_award) + simulation.set_input("ended_award", AWARD_YEAR, [100]) + simulation.apply_reform( + as_reform( + lambda reform: reform.simulation.set_input( + "ended_award", AWARD_YEAR + 1, [0] + ) + ) + ) + return "ended_award", AWARD_YEAR, 42 + + +class monthly_flag(Variable): + value_type = bool + entity = BenUnit + label = "A monthly flag a yearly input sets in each month" + definition_period = MONTH + set_input = set_input_dispatch_by_period + + def formula(benunit, period, parameters): + return False + + +def sets_a_year_whose_months_a_helper_keeps(simulation): + # The helper sets a yearly input in each month, skipping a month that + # already holds an input. + simulation.tax_benefit_system.add_variable(monthly_flag) + for month in range(1, 13): + simulation.set_input("monthly_flag", f"{AWARD_YEAR}-{month:02}", [True]) + simulation.apply_reform( + as_reform( + lambda reform: reform.simulation.set_input( + "monthly_flag", AWARD_YEAR, [False] + ) + ) + ) + january = f"{AWARD_YEAR}-01" + assert simulation.calculate("monthly_flag", january).tolist() == [True] + return "monthly_flag", january, False + + +NOT_STORING = { + "stores and deletes another year": stores_then_deletes_another_year, + "sets a year after the variable's end": sets_a_year_after_the_variables_end, + "sets a year whose months a helper keeps": sets_a_year_whose_months_a_helper_keeps, +} + + +@pytest.mark.parametrize("case", NOT_STORING) +def test_a_reform_names_no_input_it_did_not_store(case): + # The reform tries to set the variable but leaves no input of its own, + # while one set since the build survives. That one is not the reform's, + # so it is not named, and a reset deletes it as it would have anyway. + simulation = built() + recorded = list(simulation.input_variables) + variable, year, calculated = NOT_STORING[case](simulation) + assert simulation.input_variables == recorded + simulation.reset_calculations() + assert simulation.calculate(variable, year).tolist() == [calculated] + + +class added_award(Variable): + value_type = float + entity = BenUnit + label = "An award added after the build" + definition_period = YEAR + + def formula(benunit, period, parameters): + return 42 + + +def test_a_reform_names_no_input_of_a_variable_it_neutralizes(): + # Core stores nothing for a variable neutralized before the simulation + # makes its holder. (The build makes a holder for every variable it has + # then, and a holder keeps the definition it was made with, so this + # variable is added after the build.) + def apply(reform): + reform.neutralize_variable("added_award") + reform.simulation.set_input("added_award", AWARD_YEAR, [0]) + + simulation = built() + simulation.tax_benefit_system.add_variable(added_award) + recorded = list(simulation.input_variables) + with pytest.warns(Warning, match="neutralized"): + simulation.apply_reform(as_reform(apply)) + assert simulation.input_variables == recorded + assert simulation.calculate("added_award", AWARD_YEAR).tolist() == [0] + + +@pytest.mark.parametrize("route", ["clone", "branch"]) +def test_a_reform_names_nothing_it_stores_on_a_clone_or_branch(route): + def apply(reform): + simulation = reform.simulation + other = ( + simulation.clone() if route == "clone" else simulation.get_branch("other") + ) + other.set_input(SCOPED, AWARD_YEAR, [0]) + + simulation = built() + names = simulation.input_variables + recorded = list(names) + simulation.apply_reform(as_reform(apply)) + assert simulation.input_variables is names + assert names == recorded + assert simulation.calculate(SCOPED, AWARD_YEAR).tolist() == [100] + + +def test_a_reform_that_raises_leaves_the_record_as_it_was(): + def apply(reform): + reform.simulation.set_input(SCOPED, AWARD_YEAR, [0]) + raise ValueError("raised by the reform") + + simulation = built() + recorded = list(simulation.input_variables) + with pytest.raises(ValueError, match="raised by the reform"): + simulation.apply_reform(as_reform(apply)) + # A failed reform names nothing, and logs nothing afterwards. + assert simulation.input_variables == recorded + assert type(simulation._user_input_keys) is set + assert simulation.tax_benefit_system.simulation is simulation + simulation.set_input(SCOPED, AWARD_YEAR, [5]) + assert simulation.calculate(SCOPED, AWARD_YEAR).tolist() == [5] + + +# Property: drawn inputs set since the build, then a reform that sets and +# deletes inputs by any route. The reform names exactly the variables it +# leaves an input of its own for, a reset keeps each of those inputs, and +# every variable it does not name recalculates as if nothing had been set. +VARIABLES = [SCOPED, "claimant_or_partner_jsa_income"] +YEARS = [AWARD_YEAR - 1, AWARD_YEAR, AWARD_YEAR + 1] +ROUTES = ["simulation", "holder", "thread", "delete"] +EARLIER = 7 + + +@cache +def base(): + return Simulation(situation=SITUATION) + + +def write(simulation, route, variable, year, value): + if route == "simulation": + simulation.set_input(variable, year, [value]) + elif route == "holder": + holder = simulation.get_holder(variable) + holder.set_input(period(year), [value], simulation.branch_name) + elif route == "thread": + worker = threading.Thread( + target=simulation.set_input, args=(variable, year, [value]) + ) + worker.start() + worker.join() + else: + simulation.delete_arrays(variable, year) + + +cells = st.tuples(st.sampled_from(VARIABLES), st.sampled_from(YEARS)) + + +@settings( + max_examples=20, + deadline=None, + derandomize=True, + suppress_health_check=[HealthCheck.too_slow], +) +@given( + st.sets(cells), + st.lists(st.tuples(st.sampled_from(ROUTES), cells), max_size=6), +) +def test_a_reform_names_exactly_the_variables_it_leaves_inputs_for(earlier, operations): + parent = base() + names = parent.input_variables + recorded = list(names) + simulation = parent.clone() + for variable, year in earlier: + simulation.set_input(variable, year, [EARLIER]) + # What the reform leaves: the value of its last write to each variable + # and year, unless it deleted that year afterwards. + left = {} + for value, (route, (variable, year)) in enumerate(operations, start=100): + if route == "delete": + left.pop((variable, year), None) + else: + left[(variable, year)] = value + + def apply(reform): + for value, (route, (variable, year)) in enumerate(operations, start=100): + write(reform.simulation, route, variable, year, value) + + simulation.apply_reform(as_reform(apply)) + named = {variable for variable, _ in left} + assert simulation.input_variables == recorded + sorted(named) + assert parent.input_variables is names + assert names == recorded + simulation.reset_calculations() + for (variable, year), value in left.items(): + assert simulation.calculate(variable, year).tolist() == [value] + pristine = parent.clone() + for variable in set(VARIABLES) - named: + for year in YEARS: + np.testing.assert_array_equal( + simulation.calculate(variable, year), + pristine.calculate(variable, year), + err_msg=f"{variable} {year}", + ) diff --git a/policyengine_uk/utils/benefit_unit.py b/policyengine_uk/utils/benefit_unit.py index 80e936c34b..437198f123 100644 --- a/policyengine_uk/utils/benefit_unit.py +++ b/policyengine_uk/utils/benefit_unit.py @@ -1,5 +1,11 @@ """Benefit-unit aggregation helpers for means tests.""" +import numpy as np + +# The tolerance for reading a stored award: values within half a penny of +# each other are the same, and an award within half a penny of zero is none. +HALF_PENNY = 0.005 + def add_for_members(benunit, period, variables, members): """Sum variables over the benefit-unit members a means test counts. @@ -22,3 +28,59 @@ def add_for_members(benunit, period, variables, members): else: total = total + benunit(name, period) return total + + +def claimant_or_partner_award(benunit, period, award, reported, award_on_reports): + """The claimant's and partner's part of a stored benefit-unit award. + + ``award`` names the stored award (esa_income or jsa_income), which covers + every member of the benefit unit, and ``reported`` the person-level + reported amount it is built from. ``award_on_reports(benunit, period, + amount)`` is the award paid on a reported amount: the benefit's screen + and tariff income (income_related_esa_award, income_related_jsa_award). + + Which awards the stored value holds is decided by its value, compared in + the precision it is stored in, to within half a penny: + + - equal to the award on everyone's reports (its formula's result), the + claimant's and partner's part is the award on their own reports; + - otherwise, equal to the plain total of everyone's reports, the + claimant's and partner's part is the plain total of their own reports; + - anything else (an award entered directly, or a reform that replaces or + scales it) is taken to be wholly the claimant's or partner's. + + An award within half a penny of zero is no award, whether stored or + worked out here: a stored zero gives zero, and so does the float residual + of about £0.000002 left by reports that exactly match the tariff income. + + The claimant_or_partner_* variables use this as their formula, and every + reader of the claimant's or partner's award, the Income Support gate + included, reads those variables. So an award set on them directly (as + disable_simulated_benefits does for each year) is read the same way + everywhere. + """ + person = benunit.members + stored = benunit(award, period) + amounts = person(reported, period) + reported_total = benunit.sum(amounts) + claimant_or_partner_reported = benunit.sum( + amounts * person("is_claimant_or_partner", period) + ) + # Compare in the stored precision (float32): the formula's own award must + # match the award recomputed here in float64. + precision = stored.dtype + as_formula = np.isclose( + stored, + award_on_reports(benunit, period, reported_total).astype(precision), + rtol=0, + atol=HALF_PENNY, + ) + as_reported_total = np.isclose( + stored, reported_total.astype(precision), rtol=0, atol=HALF_PENNY + ) + scoped = np.where( + as_formula, + award_on_reports(benunit, period, claimant_or_partner_reported), + np.where(as_reported_total, claimant_or_partner_reported, stored), + ) + return np.where((stored > HALF_PENNY) & (scoped > HALF_PENNY), scoped, 0) diff --git a/policyengine_uk/variables/gov/dwp/claimant_or_partner_esa_income.py b/policyengine_uk/variables/gov/dwp/claimant_or_partner_esa_income.py index ae5dabed94..bda90a24b7 100644 --- a/policyengine_uk/variables/gov/dwp/claimant_or_partner_esa_income.py +++ b/policyengine_uk/variables/gov/dwp/claimant_or_partner_esa_income.py @@ -1,4 +1,5 @@ from policyengine_uk.model_api import * +from policyengine_uk.utils.benefit_unit import claimant_or_partner_award from policyengine_uk.variables.gov.dwp.esa_income import income_related_esa_award @@ -12,20 +13,24 @@ class claimant_or_partner_esa_income(Variable): "Anyone else in the benefit unit claims in their own right, so their " "award is left out here: a non-dependent adult, or a young person " "with an award of their own, which is payable to them. It still " - "counts in esa_income and so in household income. Which awards " - "esa_income holds is decided by value: " + "counts in esa_income and so in household income. Income Support's " + "gate (income_support_eligible) reads this too (s.124(1)(h)). " + "Which awards esa_income holds is decided by value: " "when it holds the award its formula gives on all reported awards, " "this is the same award on the claimant's and partner's reports; " - "when it holds the plain total of the reports (the " - "disable_simulated_benefits reform), this is the plain total of the " - "claimant's and partner's reports; when it holds anything else (an " - "award entered directly, or a reform that replaces it), that value " - "is taken to be the claimant's or partner's. Values are compared to " - "within half a penny after rounding to the precision esa_income is stored " - "in. A stored zero is zero. " - "An entered esa_income equal to either reported amount is read through " - "the reports, so to enter the claimant's or partner's award whatever " - "other members report, enter this variable directly." + "when it holds the plain total of the reports, this is the plain " + "total of the claimant's and partner's reports; when it holds " + "anything else (an award entered directly, or a reform that replaces " + "it), that value is taken to be the claimant's or partner's. Values " + "are compared to within half a penny after rounding to the precision " + "esa_income is stored in, and an award within half a penny of zero, " + "stored or worked out, is no award: a stored zero is zero, and so is " + "the float residual left by reports that exactly match the tariff " + "income. An entered esa_income equal to either reported amount is read " + "through the reports, so to enter the claimant's or partner's award " + "whatever other members report, enter this variable directly. The " + "disable_simulated_benefits reform does that for each year, from the " + "claimant's and partner's reports in the dataset year." ) definition_period = YEAR unit = GBP @@ -36,29 +41,10 @@ class claimant_or_partner_esa_income(Variable): ) def formula(benunit, period, parameters): - person = benunit.members - esa_income = benunit("esa_income", period) - reported = person("esa_income_reported", period) - reported_total = benunit.sum(reported) - claimant_or_partner_reported = benunit.sum( - reported * person("is_claimant_or_partner", period) + return claimant_or_partner_award( + benunit, + period, + "esa_income", + "esa_income_reported", + income_related_esa_award, ) - award_on_all_reports = income_related_esa_award(benunit, period, reported_total) - award_on_claimant_or_partner_reports = income_related_esa_award( - benunit, period, claimant_or_partner_reported - ) - # Compare in the precision esa_income is stored in (float32), so the - # formula's own award always matches the award recomputed here. - stored = esa_income.dtype - as_formula = np.isclose( - esa_income, award_on_all_reports.astype(stored), rtol=0, atol=0.005 - ) - as_reported_total = np.isclose( - esa_income, reported_total.astype(stored), rtol=0, atol=0.005 - ) - scoped = where( - as_formula, - award_on_claimant_or_partner_reports, - where(as_reported_total, claimant_or_partner_reported, esa_income), - ) - return where(esa_income > 0, scoped, 0) diff --git a/policyengine_uk/variables/gov/dwp/claimant_or_partner_jsa_income.py b/policyengine_uk/variables/gov/dwp/claimant_or_partner_jsa_income.py index b528c41a52..7571840388 100644 --- a/policyengine_uk/variables/gov/dwp/claimant_or_partner_jsa_income.py +++ b/policyengine_uk/variables/gov/dwp/claimant_or_partner_jsa_income.py @@ -1,4 +1,5 @@ from policyengine_uk.model_api import * +from policyengine_uk.utils.benefit_unit import claimant_or_partner_award from policyengine_uk.variables.gov.dwp.jsa_income import income_related_jsa_award @@ -12,20 +13,24 @@ class claimant_or_partner_jsa_income(Variable): "Anyone else in the benefit unit claims in their own right, so their " "award is left out here: a non-dependent adult, or a young person " "with an award of their own, which is payable to them. It still " - "counts in jsa_income and so in household income. Which awards " - "jsa_income holds is decided by value: " + "counts in jsa_income and so in household income. Income Support's " + "gate (income_support_eligible) reads this too (s.124(1)(f)). " + "Which awards jsa_income holds is decided by value: " "when it holds the award its formula gives on all reported awards, " "this is the same award on the claimant's and partner's reports; " - "when it holds the plain total of the reports (the " - "disable_simulated_benefits reform), this is the plain total of the " - "claimant's and partner's reports; when it holds anything else (an " - "award entered directly, or a reform that replaces it), that value " - "is taken to be the claimant's or partner's. Values are compared to " - "within half a penny after rounding to the precision jsa_income is stored " - "in. A stored zero is zero. " - "An entered jsa_income equal to either reported amount is read through " - "the reports, so to enter the claimant's or partner's award whatever " - "other members report, enter this variable directly." + "when it holds the plain total of the reports, this is the plain " + "total of the claimant's and partner's reports; when it holds " + "anything else (an award entered directly, or a reform that replaces " + "it), that value is taken to be the claimant's or partner's. Values " + "are compared to within half a penny after rounding to the precision " + "jsa_income is stored in, and an award within half a penny of zero, " + "stored or worked out, is no award: a stored zero is zero, and so is " + "the float residual left by reports that exactly match the tariff " + "income. An entered jsa_income equal to either reported amount is read " + "through the reports, so to enter the claimant's or partner's award " + "whatever other members report, enter this variable directly. The " + "disable_simulated_benefits reform does that for each year, from the " + "claimant's and partner's reports in the dataset year." ) definition_period = YEAR unit = GBP @@ -36,29 +41,10 @@ class claimant_or_partner_jsa_income(Variable): ) def formula(benunit, period, parameters): - person = benunit.members - jsa_income = benunit("jsa_income", period) - reported = person("jsa_income_reported", period) - reported_total = benunit.sum(reported) - claimant_or_partner_reported = benunit.sum( - reported * person("is_claimant_or_partner", period) + return claimant_or_partner_award( + benunit, + period, + "jsa_income", + "jsa_income_reported", + income_related_jsa_award, ) - award_on_all_reports = income_related_jsa_award(benunit, period, reported_total) - award_on_claimant_or_partner_reports = income_related_jsa_award( - benunit, period, claimant_or_partner_reported - ) - # Compare in the precision jsa_income is stored in (float32), so the - # formula's own award always matches the award recomputed here. - stored = jsa_income.dtype - as_formula = np.isclose( - jsa_income, award_on_all_reports.astype(stored), rtol=0, atol=0.005 - ) - as_reported_total = np.isclose( - jsa_income, reported_total.astype(stored), rtol=0, atol=0.005 - ) - scoped = where( - as_formula, - award_on_claimant_or_partner_reports, - where(as_reported_total, claimant_or_partner_reported, jsa_income), - ) - return where(jsa_income > 0, scoped, 0) diff --git a/policyengine_uk/variables/gov/dwp/income_support_eligible.py b/policyengine_uk/variables/gov/dwp/income_support_eligible.py index be1f79c4e4..ec746c2b1e 100644 --- a/policyengine_uk/variables/gov/dwp/income_support_eligible.py +++ b/policyengine_uk/variables/gov/dwp/income_support_eligible.py @@ -1,6 +1,4 @@ from policyengine_uk.model_api import * -from policyengine_uk.variables.gov.dwp.esa_income import income_related_esa_award -from policyengine_uk.variables.gov.dwp.jsa_income import income_related_jsa_award class income_support_eligible(Variable): @@ -26,21 +24,26 @@ class income_support_eligible(Variable): "Jobseeker's Allowance. The partner must not be engaged in " "remunerative work (24 hours a week or more, unless a carer). Neither " "the claimant nor the partner may be entitled to income-related ESA " - "or income-based JSA, and the other member of a couple may not be " - "entitled to State Pension Credit, which the model takes to be so " - "where the couple meets the Pension Credit age conditions and would " - "claim it. An adult in the benefit unit who is neither the " + "or income-based JSA: the model reads their award from " + "claimant_or_partner_esa_income and claimant_or_partner_jsa_income, " + "as the means tests and passports do. The other member of a couple " + "may not be entitled to State Pension Credit, which the model takes " + "to be so where the couple meets the Pension Credit age conditions " + "and would claim it. An adult in the benefit unit who is neither the " "claimant nor the partner (such as a non-dependent adult) does not " "affect eligibility, with a declared exception for a stored " - "esa_income or jsa_income that is not the formula's own result. " - "Whether entered directly or replaced by a reform (including one that " - "scales it), such a value is read through the reported awards when it " - "equals what they give (the award after the benefit's screen, or " - "their plain total, to within half a penny after rounding to the " - "precision it is stored in), and is otherwise taken to be the " - "claimant's or partner's. So another member's report can change how " - "such a value is read. A stored award within half a penny of zero " - "never bars the claim." + "esa_income or jsa_income that is not the formula's own result and " + "for which no claimant-or-partner award is entered. Whether entered " + "directly or replaced by a reform (including one that scales it), " + "such a value is read through the reported awards when it equals " + "what they give (the award after the benefit's screen, or their " + "plain total, to within half a penny after rounding to the precision " + "it is stored in), and is otherwise taken to be the claimant's or " + "partner's. So another member's report can change how such a value " + "is read. A claimant-or-partner award entered directly, or set by the " + "disable_simulated_benefits reform, is read as it stands. Worked out " + "from a stored award, an award within half a penny of zero never " + "bars the claim." ) definition_period = YEAR reference = ( @@ -144,76 +147,24 @@ def formula(benunit, period, parameters): ) # ... and the other member of a couple is not entitled to an # income-related allowance. An income-related allowance covers the - # couple, so it bars Income Support whichever of them has it: the - # award on the claimant's and partner's reported amounts, after the - # same screen as esa_income (esa_income_eligible). When esa_income - # holds what the reported amounts give, either after that screen (the - # formula) or as their plain total (the disable_simulated_benefits - # reform), the reports say whose award it is. When it holds anything - # else (an award entered directly, or a reform that replaces, scales - # or removes it), they do not, and it is taken to be the claimant's - # or partner's. A stored value equal to either amount, to within half - # a penny after rounding to its stored precision, is read through the - # reports. Either way, no income-related ESA is paid when esa_income - # is zero, so a zero never bars the claim. An amount within half a - # penny of zero counts as zero, the same tolerance as the comparisons: - # reports that exactly match the tariff income can leave a float - # residual of about £0.000002. - esa_income = benunit("esa_income", period) - reported_total = add(benunit, period, ["esa_income_reported"]) - award_on_all_reports = income_related_esa_award(benunit, period, reported_total) - award_on_claimant_or_partner_reports = income_related_esa_award( - benunit, - period, - benunit.sum(person("esa_income_reported", period) * claimant_or_partner), - ) - # Compare in the precision esa_income is stored in (float32), so the - # formula's own award always matches the award recomputed here. - stored = esa_income.dtype - half_penny = 0.005 - as_reported = np.isclose( - esa_income, award_on_all_reports.astype(stored), rtol=0, atol=half_penny - ) | np.isclose( - esa_income, reported_total.astype(stored), rtol=0, atol=half_penny - ) - income_related_esa = (esa_income > half_penny) & ( - ~as_reported | (award_on_claimant_or_partner_reports > half_penny) - ) + # couple, so it bars Income Support whichever of them has it. The + # claimant's and partner's award is claimant_or_partner_esa_income, + # which every reader of it shares. On the formula's award it is the + # award on their own reports after esa_income's screen + # (esa_income_eligible); on the plain total of the reports it is the + # plain total of theirs; on any other stored esa_income it is that + # value. An award within half a penny of zero, stored or worked out, + # is no award (claimant_or_partner_award): a stored zero never bars + # the claim, and nor does the float residual of about £0.000002 left + # by reports that exactly match the tariff income. A value entered for + # it directly, or set for each year by disable_simulated_benefits, is + # read as it stands. + income_related_esa = benunit("claimant_or_partner_esa_income", period) > 0 # s.124(1)(f): neither the claimant nor the other member of a couple # is, and the couple are not, entitled to an income-based jobseeker's - # allowance, read the same way: the award on the claimant's and - # partner's reported amounts after the screen jsa_income applies - # (jsa_income_eligible), unless jsa_income holds something the - # reported amounts do not give. - jsa_income = benunit("jsa_income", period) - jsa_reported_total = add(benunit, period, ["jsa_income_reported"]) - jsa_award_on_all_reports = income_related_jsa_award( - benunit, period, jsa_reported_total - ) - jsa_award_on_claimant_or_partner_reports = income_related_jsa_award( - benunit, - period, - benunit.sum(person("jsa_income_reported", period) * claimant_or_partner), - ) - # Compare in the stored award's precision: the holder keeps it as - # float32, while the sums are float64. - jsa_as_reported = np.isclose( - jsa_income, - jsa_award_on_all_reports.astype(jsa_income.dtype), - rtol=0, - atol=half_penny, - ) | np.isclose( - jsa_income, - jsa_reported_total.astype(jsa_income.dtype), - rtol=0, - atol=half_penny, - ) - # As for ESA, an amount within half a penny of zero is no award: a - # stored zero never bars the claim, and nor does the float residual - # left by reports that exactly match the tariff income. - income_based_jsa = (jsa_income > half_penny) & ( - ~jsa_as_reported | (jsa_award_on_claimant_or_partner_reports > half_penny) - ) + # allowance, read the same way through claimant_or_partner_jsa_income + # (the screen is jsa_income_eligible). + income_based_jsa = benunit("claimant_or_partner_jsa_income", period) > 0 # s.124(1)(g): the other member of a couple is not entitled to State # Pension Credit. Entitlement needs a claim (SSAA 1992 s.1). Reading the # Pension Credit amount here would be circular (Pension Credit income