Skip to content

Property test claims an added non-dependant never changes the legacy award readers; the severe disability premium is a counterexample #2293

Closed

Description

@MaxGhenis

What happens

policyengine_uk/tests/test_legacy_award_readers_properties.py::test_other_members_awards_never_change_the_readers asserts that adding a benefit-unit member who is not the claimant, the partner or a dependant never changes any variable in FAMILY_READERS. That is false for at least one case. The test passes on main only because its 15 derandomised examples do not draw it.

Counterexample

Reproduced on main at 9475c95ab (10 October 2026):

  • The claimant is 18, not in education, with receives_carer_benefit true, is_disabled_for_benefits true, pip_dl 2000, income_support_reported 200, in England, with no savings.
  • The test's added member is an 18-year-old not in education with no awards.
Variable Without the added member With the added member
severe_disability_premium 4,729.28 0.00
income_support 13,771.55 9,042.27
council_tax_reduction_applicable_income (in FAMILY_READERS) 25,074.71 20,345.43

council_tax_reduction_applicable_income is the only FAMILY_READERS entry that differs in this case.

Why

The added adult is a non-dependant, so the severe disability premium stops. Income support falls by the premium, and the Council Tax Reduction income reader includes income support. The model's behaviour looks right. The test's claim is too strong: other members' awards should not change the readers, but an added non-dependant can change the family's own award.

Why it matters now

Hypothesis derives its derandomised examples from the test function's source. Any edit to that function's body reshuffles the examples and can surface this failure in an unrelated PR.

Fix

Narrow the invariant to what is true, for example one of:

  • compare the readers with the family's own awards held fixed, so a premium that depends on household composition is excluded;
  • have the generator add a member who cannot be a non-dependant for the severe disability premium;
  • exclude readers that flow through severe_disability_premium from this particular property, and add a separate property that states the premium rule.

Then add this counterexample as an explicit @example.

Found by the session working on #2279. Its repro script is repro_legacy_readers_sdp_counterexample.py; the UK hub re-ran it on main.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions