Reconcile UC deduction and benefit cap components under the protected floor

[vahid-ahmadi]

Five confirmed bugs in the merged UC deductions / protected floor code, plus the doc clarifications that follow from the first fix.

1. Deductions did not reconcile when the protected floor bound

universal_credit applied the floor to the sum of benefit_cap_reduction and uc_deductions at the point of computing the award. Neither component was rewritten, so under a floor reform both reported their unfloored values.

Reproduction (floor 0.85, benefit_cap_reduction £1,000, latent rate 0.30, standard allowance £4,801.70):

	before	after
uc_deductions	£960.34	£720.25
benefit cap reduction applied to UC	£1,000.00 (benefit_cap_reduction)	£0.00 (uc_benefit_cap_reduction)
components sum	£1,960.34	£720.25
reduction actually applied	£720.25	£720.25

Fix. The floor now lives in the components rather than in the award formula:

• New uc_maximum_reduction holds the allowance the floor leaves — max((1 − floor) × standard allowance, 0), infinite when the floor is zero.
• uc_deductions is limited to that allowance, so deductions bend only where they alone exceed it.
• New uc_benefit_cap_reduction takes the headroom deductions leave: min(benefit_cap_reduction, max(allowance − uc_deductions, 0)).
• universal_credit subtracts the two components separately, so reconciliation holds by construction.

Ordering: the benefit cap reduction absorbs the floor first. Under the Fair Repayment Rate the 15% deductions cap equals the allowance an 85% floor leaves, so cappable deductions alone essentially never breach the floor — only benefit cap reductions push past it. JRF's own worked example (standard allowance £92, deduction £14, benefit cap £59, floor £78) preserves the £14 deduction and eliminates the £59 cap reduction, which is exactly this ordering. Test test_the_benefit_cap_reduction_absorbs_the_floor_first pins it.

benefit_cap_reduction stays gross. It also drives Housing Benefit, which the UC floor does not protect, so flooring it would leak a UC reform into HB. Keeping it gross is also what avoids a dependency cycle: uc_deductions and uc_has_deduction both read benefit_cap_reduction for the one-penny award test, so flooring it in place would have made benefit_cap_reduction → uc_deductions → benefit_cap_reduction. The new chain is acyclic — universal_credit → uc_benefit_cap_reduction → {benefit_cap_reduction, uc_maximum_reduction, uc_deductions} — and the full suites run clean, which is what would surface a cycle.

Residual question, deliberately not changed. The above-cap "last resort" excess (last resort and child maintenance deductions), which current law exempts from the 15% deductions cap, is still subject to the floor where deductions alone exceed the allowance. JRF's briefing does not say whether their floor exempts those categories — its worked example involves only cappable deductions and the benefit cap — so this remains a modelling choice, documented in protected_floor.yaml, in uc_deductions.py, in the validation page, and pinned by test_protected_floor_binds_on_the_above_cap_excess.

2. A floor above 1 manufactured money

(1 − floor) × standard allowance goes negative above a floor of 1, and subtracting a negative reduction added to the award: protected_floor = 1.2 on a £6,000 entitlement returned £6,960.34. The allowance now clamps at zero, where the floor protects the whole standard allowance from any reduction. Test: test_a_floor_above_the_standard_allowance_adds_nothing.

3. uc_deductions applied without a UC claim

uc_deductions lacked defined_for = "would_claim_uc". With would_claim_uc = False and an imputed uc_latent_deduction_rate, universal_credit was £0 but uc_deductions was £720.25. Harmless today because the fallback uc_has_deduction formula gates on the claim, but it breaks the documented migration path where datasets impute the latent rate directly and the fallback retires. Guard added to uc_deductions and to the new uc_benefit_cap_reduction. Test: test_no_deductions_without_a_universal_credit_claim.

4. Wrong comment in test_protected_floor_binds_on_the_above_cap_excess

It read "Latent demand of 30% is 15% cappable plus a 15% last resort excess" — 15 + 15 = 30, but the test asserts a 20% rate. The real split is 25% cappable (the calibration cap), cut to 15% by the deductions cap, plus a 5% excess. Comment only; assertions were already right.

5. Parameter provenance

• deductions/cap.yaml: the 25% cap moves from 2021-04-01 to 2021-04-30, the actual change date and the convention the file already uses for the Fair Repayment Rate.
• deductions/minimum_payable.yaml and uc_deductions.py: the one-penny rule is Schedule 6 paragraph 3(1)(a) of SI 2013/380, not Schedule 6 generally.

Validated statistics are unaffected

Microsimulation tests are gated on the dataset, so this is reasoned rather than run:

• Current law leaves every value bit-identical. protected_floor is 0, so uc_maximum_reduction is infinite, uc_deductions passes through min(D, ∞) = D, and uc_benefit_cap_reduction is min(B, ∞) = B. universal_credit becomes max(pre − B − D, 0), which is what max(pre − min(B + D, ∞), 0) already evaluated to. Confirmed numerically: the unreformed case above returns deductions £960.34, cap reduction £1,000, UC £4,039.66 both before and after.
• The defined_for guard changes nothing in the microdata. uc_deductions was already zero without a claim, because uc_deduction_rate → uc_latent_deduction_rate returns zero unless uc_has_deduction, which requires would_claim_uc. The guard only matters once datasets impute the latent rate directly.
• The cap date change has zero numeric effect. convert_to_fiscal_year_parameters samples every parameter at 30 April of each year, and cap.yaml carries no fiscal_year_blend, so 2021 reads 0.25 under either date. The validated years are 2024 and 2025 in any case.
• Nothing else moved. The latent distribution, cap values, calibration cap, region factors and uc_deduction_rate are untouched. uc_has_deduction, uc_deduction_rate and the deduction draws are unchanged, so incidence, at-cap shares and mean amounts are unchanged.

The one doc-side change is a label clarification: the validation table's at-cap and above-cap rows are shares of all UC households, which is what the model computes (w[on_uc & …].sum() / uc_weight in test_uc_deductions_aggregates.py) and what DWP publishes. Figures are unchanged.

Tests

• policyengine_uk/tests/test_uc_deductions.py: 17 passed (4 new).
• policyengine_uk/tests/ -m "not microsimulation" (excluding tests/policy): 153 passed, 15 skipped.
• policyengine-core test policyengine_uk/tests/policy: 1118 passed.

🤖 Generated with Claude Code

[vahid-ahmadi]

Verified the five fixes independently against the branch — all confirmed:

• Current law bit-identical: unreformed repro returns uc_deductions=960.34, universal_credit=4,039.66, unchanged.
• Reconciliation holds: floor 0.85 with gross cap reduction £1,000 gives 720.25 + 0.00 = 720.25 applied.
• Floor > 1 clamped: returns £6,000, not £6,960.34.
• No-claim guard: uc_deductions = 0.00 (was £720.25).
• JRF worked example reproduces: SA £92.34/wk against their £92, allowance £13.85 against their £14, deduction survives whole, cap reduction absorbed to zero. That is a good independent check of the absorption ordering.

One residual worth documenting (not blocking)

uc_deductions still derives its available award from the gross benefit_cap_reduction, so where the gross reduction nearly exhausts the award the one-penny constraint bites before the floor allowance does, and the split moves:

floor 0.85, allowance = 15% of SA = 720.25, entitlement 6,000

  gross bcr    deductions   cap applied      UC     total reduction
      1,000        720.25          0.00   5,279.75         720.25
      4,000        720.25          0.00   5,279.75         720.25
      5,500        499.88        220.37   5,279.75         720.25
      5,900         99.88        620.37   5,279.75         720.25

The award is correct in every row and the components reconcile — this is strictly an attribution issue between the two components, not a UC error, so it is much smaller than the bug this PR fixes. But under the absorption ordering the deduction should survive whole at 720.25 throughout, and in the last two rows it does not.

It bites only when entitlement − gross benefit cap reduction < (1 − floor) × standard allowance, which is rare — but it is exactly the heavily-capped population that JRF-style floor analysis targets, and it would understate reported deductions there. A fix would compute the award in uc_deductions from the floor-limited cap reduction, though that risks reintroducing the cycle this PR carefully avoided.

Suggest either fixing it or noting it in the docs alongside the existing last-resort caveat.

[vahid-ahmadi]

@MaxGhenis review request — this fixes bugs in the deductions work merged in #1815.

The judgement call is the absorption ordering. The floor limits combined reductions; something has to give first. This PR makes the benefit cap reduction absorb the floor while the deduction survives, on the reasoning that under the FRR the deductions cap (15%) equals a typical floor allowance, so cappable deductions alone essentially never breach it. JRF's worked example supports this — their floor allowance is GBP 14/week and their deduction is GBP 14, so it survives whole and the GBP 59 cap reduction goes. The model reproduces that exactly (SA GBP 92.34/wk vs their GBP 92, allowance GBP 13.85 vs their GBP 14).

If you read their design differently — proportional abatement, say — that changes the split though not the award.

Three things worth your eye:

1. benefit_cap_reduction is deliberately left gross. Flooring it in place would leak a UC reform into Housing Benefit, which the floor does not protect, and would create a cycle through the one-penny test. The new uc_benefit_cap_reduction is UC-only.
2. A residual I found and documented (comment above): where the gross cap reduction nearly exhausts the award, the penny constraint bites before the floor allowance and the split moves, so a deduction that should survive whole does not. The award is right in every case and components reconcile — it is attribution only — but it affects exactly the heavily-capped population JRF-style analysis targets. Fix-or-document, your call.
3. Whether the above-cap last-resort excess should be exempt from the floor. Unchanged behaviour here, documented in the parameter, the variable and the docs. JRF's briefing does not settle it.

Current law is bit-identical (verified numerically), and the gated Test job passes, so the validated statistics in #1815 are intact.

[MaxGhenis]

Heads-up from #2089 (each scheme its own benefit cap), stacked on #2084. #2089 adds policyengine_uk/variables/gov/dwp/universal_credit/benefit_cap/uc_benefit_cap_reduction.py, the UC Regs 2013 reg 81 reduction: the excess less the childcare costs element. universal_credit, uc_deductions and uc_has_deduction read it.

This PR adds a variable of the same name (the cap reduction left after the protected floor), so whichever lands second should rename one. Suggestion: keep #2089's as the reg 81 amount, and call this PR's uc_benefit_cap_reduction_after_floor, reading uc_benefit_cap_reduction.

[juaristi22]

Review of #1819 — Reconcile UC deduction and benefit cap components under the protected floor

Base repository: PolicyEngine/policyengine-uk
PR number: 1819
Reviewed head SHA: d287b18
Merge base SHA: 3d12d26
Mode: full
Scope: changed behavior and affected dependencies
Source evidence: Official sources are linked below; local source copies and validation artifacts were retained.
Review status: PARTIAL

Source Documents

• current-policy-source-s1.pdf.
• current-policy-source-s2.pdf.
• current-policy-source-s5.pdf.
• current-policy-source-s9.html.
• current-policy-source-s10.html.
• current-policy-source-s11.html.
• current-policy-source-s13.html.
• current-policy-source-s26.html.
• current-policy-source-s27.html.
• current-policy-source-s28.pdf.

Critical

None.

Should Address

A1 — Give the changelog fragment a configured type (OPEN)

Severity: SHOULD ADDRESS. Trigger: building the release notes with the repository's configured towncrier. Expected: this PR's five changes appear under its configured fixed/added types. Observed: new changelog.d/uc-deductions-floor-reconciliation.md:1 has no type segment and is omitted entirely from an executed python -m towncrier build --draft --version 0.0; draft output (current-pr-1819-towncrier-draft.log; retained locally) contains no reconciliation/new UC cap component entry. pyproject.toml configures breaking/added/changed/fixed/removed types. Name the fragment with a recognized segment, such as .fixed.md. This is introduced by the new file and affects release documentation, not household calculations.

Validation: one batched run of changed test_uc_deductions.py, cap scenario regression and directly relevant UC/HB YAML directories: 179 passed, pytest14.03s, bounded CLI17.46s (log (current-pr-1819-tests.log; retained locally)). Imports resolved to the exact snapshot; no dependencies installed. Code traces show YEAR benunit quantities, vectorized headroom/floors, would_claim_uc guards and no new cycle. The gross cap continues to feed HB independently.

Actual-input diagnostics use a couple aged35/33, four children aged13/11/9/7, London HA rent£30,000, zero earnings/assets, and an observed latent deduction demand0.30/ADVANCE_ONLY; UC eligibility and cap are computed. At floor0/.85/1.2, final UC is £19,769.96/£47,502.06/£48,632.64. Deduction plus applied UC cap component equals the actual lost entitlement in all three cases, and household net income changes consistently. Gross benefit cap reduction remains£27,355.24; the >1 floor adds no award above pre-cap entitlement. Diagnostic log (current-pr-1819-household-diagnostics.log; retained locally).

An actual legacy HB claimant variant supplies reported HB receipt and a no-UC-claim take-up choice; its HB eligibility is computed. The .85 UC floor leaves HB£21,277.40 and net income£25,148.45 exactly unchanged, with both UC components0. HB diagnostic (current-pr-1819-hb-diagnostic.log; retained locally). The claim choice is an input status, not forced eligibility.

Material gap: the separately requested end-to-end case with a reported UC award below the protected floor was not completed before the shared17-minute substantive deadline. The low-award/minimum-payable unit/YAML tests pass, but they do not establish that additional reported-award scenario under an active floor. No incorrect result is inferred from this missing case. Source dates/historical JRF quotation are owned by the policy role.

Snapshot clean; only RUN_ROOT scripts/logs/artifacts written. No posting.

Suggestions

None.

Evidence Gaps

• End-to-end reported UC award below the protected floor was not completed before the shared substantive deadline.
• Exact 30 April 2021 operational-cap commencement day is not corroborated by the acquired Budget 2021, which states April 2021.
• The JRF URL now serves an updated/corrected 2026 briefing; the original October 2025 worked amounts and exact debt-before-cap priority remain a source-version gap.
• Private microsimulation validation totals were not independently rerun.
• Role checks incomplete: current-pr-1819-review-code.md; see recovered report.
• Role checks incomplete: current-pr-1819-review-policy.md; see recovered report.

Notes

• Real demographic capped UC and legacy HB paths executed; cap/source-date interpretation owned by policy role.
• Current-law deductions ceiling versus operational cap separated; protected floor is a proposal.

Validation Summary

179 passed, 14.03s pytest /17.46s CLI; actual-family floor0/.85/1.2 component reconciliation and final UC/net income pass; legacy HB final award/net income unaffected; draft towncrier omits untyped fragment

Original statutory/government/JRF sources read with SHA-bound extracts. current-pr-1819-review-sources.json; current-pr-1819-review-source-cache-check.json. No duplicate calculation diagnostics; original PDF extraction via bundled pypdf. Canonical source-manifest validator exited 0.

CI at d287b18: Install + smoke-import (py3.9)=SUCCESS, Test documentation builds=SUCCESS, Install + smoke-import (py3.10)=SUCCESS, Install + smoke-import (py3.11)=SUCCESS, Install + smoke-import (py3.12)=SUCCESS, Install + smoke-import (py3.13)=SUCCESS, Install + smoke-import (py3.14)=SUCCESS, Lint=SUCCESS, Test=SUCCESS.

Detailed code, policy, source and reproduction artifacts were retained locally.

Timing

elapsed seconds: 1151.70s; setup seconds: 272.06s

Review Severity

COMMENT. Open findings: 0 critical, 1 should address, 0 suggestions.

[MaxGhenis]

This PR's fragment changelog.d/uc-deductions-floor-reconciliation.md has no towncrier type, so towncrier would leave it out of CHANGELOG.md and never delete it (#2314). Please rename it to uc-deductions-floor-reconciliation.<type>.md with one of breaking, added, changed, fixed or removed. Once #2324 merges, its Changelog fragments check fails this PR until the fragment has a type.

🤖 Generated with Claude Code