Skip to content

Land ten reviewed pull requests together (rollup A) - #2364

Merged
MaxGhenis merged 42 commits into
mainfrom
hub/rollup-2026-10-11-a
Oct 12, 2026
Merged

Land ten reviewed pull requests together (rollup A)#2364
MaxGhenis merged 42 commits into
mainfrom
hub/rollup-2026-10-11-a

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 12, 2026

Copy link
Copy Markdown
Collaborator

Lands ten reviewed pull requests with one CI run. Each member's head commit is merged here unchanged, so merging this PR with a merge commit marks every member as merged.

Why a rollup

The PolicyEngine org is at GitHub's limit of 60 concurrent Actions jobs, with about 360 queued, and each PE-UK pull request costs seven jobs and about 70 runner-minutes. Testing the ten together costs one run, not ten. It also tests the combination: on 2026-10-11 main went red because #2251 and #2270 each passed CI alone and failed together.

Members

PR Change Independent review of the member's diff
#2303 Assert the 2023-24 Class 4 lower profits limit instead of expecting failure Opus 5.5, APPROVE at 7f7fc5d19; the head moved only by a merge of main (patch-identical)
#2269 Split the CI Test job into shards and cut its runtime GPT-6.1 Sol, APPROVE at 841d2a889; patch-identical after a merge of main
#2312 Apply the Carer's Allowance earnings limit GPT-6.1 Sol, four rounds, APPROVE at 17d034a47
#2323 Round the High Income Child Benefit Charge as s. 681C(3) requires GPT-6.1 Sol, three rounds, APPROVE at eb99a944a
#2327 Relieve a miscellaneous loss only against miscellaneous income GPT-6.1 Sol, APPROVE at 72ec8db6d
#2348 Set the Personal Savings Allowance on rest-of-UK bands for Scottish taxpayers GPT-6.1 Sol, APPROVE at 60c6054a4
#2352 Band savings and dividends above property income GPT-6.1 Sol, APPROVE at c1067e325
#1846 Warn instead of silently mislabelling a dataset's fiscal year @juaristi22, APPROVE at 00c07d6a8
#2186 Put road-fuel litre divisors on DESNZ pump prices @juaristi22, APPROVE at 583b3d20f
#2194 Pick up new documentation pages by folder @juaristi22, APPROVE at c53eb9058

Each member's own description carries its mechanism, invariants, tests and real-run impact. This PR adds no change of its own.

Left out, because they conflict with a member

Their owners merge main once this lands.

Checks on the combined tree before pushing

  • uvx ruff format --check .: clean.
  • uvx ruff check --select F811,F821 policyengine_uk: clean, so no method is defined twice across members.
  • Every merge was automatic; no conflict was resolved by hand.

Merge rule

Merge with a merge commit only, at the head CI tested. If a member's branch gains a commit after this point, that commit is not included and the member stays open.

🤖 Generated with Claude Code

vahid-ahmadi and others added 30 commits October 7, 2026 14:12
@vahid-ahmadi @claude
Warn instead of silently mislabelling a dataset's fiscal year
6727494 
`UKSingleYearDataset` took `fiscal_year=2025` as a default but used it only
when building from DataFrames: with a file path the period is read from the
file and the argument was ignored without comment. Both halves failed
quietly. Passing a conflicting year alongside a file had no effect, and
omitting it when assembling frames by hand labelled the data 2025 — already
a year ahead of the newest release, and wrong differently every year.

The default now warns on use and the conflicting-argument case warns too,
so neither can mislabel a dataset unnoticed. Removing the default outright
would break callers, so it is deprecated rather than dropped.

Closes #1845

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@vahid-ahmadi @claude
Stop silencing warnings process-wide on import
00c07d6 
Three LHA variable modules called warnings.filterwarnings("ignore") at
import, left over from a 2023 module that time-shifted BRMA rent tables
with pandas, and bi_household_phaseout did the same inside its formula.
Importing policyengine_uk therefore hid every warning in the process,
including the two fiscal_year warnings this PR adds.

- Remove the four blanket filters. The guarded divisions they were
  hiding (basic, new and additional State Pension, and the basic income
  household phase-out) now use np.divide(..., where=), so no 0/0
  RuntimeWarning reaches callers.
- Fix the three parameter files whose unknown keys now warn at import
  (notes -> documentation, top-level label -> metadata.label).
- Raise the missing-fiscal_year deprecation as FutureWarning, which
  Python shows to end users by default.
- Add a fresh-interpreter test under default filters for both
  ambiguous paths; pytest.warns overrides filters and could not catch
  this.
- Give the changelog fragment its .fixed type.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vahid-ahmadi @claude
Put road-fuel litre divisors on DESNZ pump prices and re-derive the l…
583b3d2 
…itre proxies (#2169 part A)

petrol_litres and diesel_litres divided spending by RAC prices dated
1 January 2024, held flat, while Microcosm UK prices household fuel
spending on DESNZ calendar-year averages. The divisors now hold the
DESNZ QEP table 4.1.2 ULSP and ULSD annual averages for 2010-2025, a
provisional January-September 2026 average from table 4.1.1, and are
flat in cash terms afterwards.

The petrol and diesel spending litre proxies are re-derived from the
new prices with the existing formula, (1 + volume) x price_t /
price_t-1 / (1 + population) - 1, so weighted litres still follow the
HMRC/OBR road-fuel volume path. A new test fails if the proxy and the
prices drift apart.

On the certified Microcosm UK 2024-25 file, 2025 litres are 23.699bn
(the joint sensitivity in the issue), year-on-year litres growth is
unchanged, and household fuel duty rises by about 0.25bn a year.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vahid-ahmadi @claude
Pick up new documentation pages by folder
c53eb90 
Add pattern entries to the myst.yml table of contents for the assumptions,
validation and policy folders and a Programmes section for
docs/book/programs/gov/<agency>/*.md, so a new Markdown page appears without
editing myst.yml. Explicit entries now carry the .md extension so the patterns
don't add them twice, and the Policy section's two book pages come from the
pattern (same order). Notebooks are not pattern-matched.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Split the Test job into a YAML job and three shards of the Python tests
77dae43 
The Test job took 59 minutes (run 38075384588): 2.6 minutes of YAML tests,
then 55.8 minutes of pytest on four balanced xdist workers, 220
worker-minutes in all. No test dominates: 51 property-test files take 74%
of it and the slowest test takes 3.5%. So the job is split rather than one
test fixed.

- `--shard K/N` (tests/conftest.py) keeps the tests whose node id hashes to
  shard K of N. The shard is a function of the node id alone, so every
  collected test is in exactly one shard, including tests added later.
- The workflows run the YAML tests in one job and the Python tests as a
  matrix of three shards, N taken from the size of the matrix.
- A final `Test` job runs .github/check_test_shards.py over the shards'
  JUnit files and an unsharded `pytest --collect-only -q` listing. It fails
  unless every collected test ran in exactly one shard and none failed.
- Coverage is measured with sys.monitoring (COVERAGE_CORE=sysmon) instead
  of the C trace function. On one property-test file the two report the
  same 12,262 of 18,401 lines, and sys.monitoring adds 6% to the CPU time
  where the trace function adds 33%.

Refs #2099

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Run the final Test job on the shards' runners and report failed jobs …
841d2a8 
…first

On 10 October the 5-second final Test job waited 32 minutes for an
ubuntu-latest runner after the shards finished (run 38085920460). It now
runs on ubuntu-24.04-arm like the shards. The job-result check runs
first, and the shard check still runs after a failure so its report
names the failed tests. Record the measured job times by the timeouts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Assert the 2023-24 lower profits limit alignment instead of expecting…
a310fa2 
… it to fail

#1968 (the model's 2023-24 Class 4 LPL was £11,908, not the £12,570 NICA
2022 s2 sets from 6 April 2023) was fixed by #2058 on 10 October. The
non-strict xfail kept for it now passes silently as an XPASS, so the
alignment test protected nothing for that year. Remove the marker so a
regression to £11,908 fails the suite.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Drop the #1968 exception from the NI threshold freeze test's invarian…
7f7fc5d 
…ts too

The sibling module repeated the 2023-24 lower profits limit as an open
defect (#1968); #2058 fixed it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Apply the Carer's Allowance earnings limit (SSCBA 1992 s. 70(1)(b), S…
d639a03 
…I 1976/409 reg. 8)

Fixes #2309. Carer's Allowance was paid whatever the carer earned. A person
whose net weekly earnings exceed the limit (£204 from 6 April 2026) is
gainfully employed and not entitled. Net earnings are pay and trading profit
less the income tax and NI deducted from them and half of pension
contributions (Computation of Earnings Regulations 1996 regs. 10(4), 13).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Round the High Income Child Benefit Charge as ITEPA 2003 s. 681C(3) r…
9b589b3 
…equires

Fixes #2321. The charge used the unrounded share of the phase-out band. The
percentage is rounded down to a whole number, and the Child Benefit total and
the charge down to whole pounds. Three tests that encoded unrounded charges now
expect the statutory ones; the softer-taper opt-out case moves its income so
the rounded charge stays positive.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Relieve a miscellaneous loss only against miscellaneous income (ITA 2…
d5e4076 
…007 s. 152(4))

Fixes #2326. taxable_miscellaneous_income added miscellaneous_income as it
stood, so a loss reduced total and adjusted net income and the tax on pay,
pensions, savings and dividends. A net loss now counts as nil; carrying it
forward is not modelled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Test that the Child Benefit total is rounded down before the percenta…
ba13d61 
…ge applies

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Update the opt-out Python tests to the statutory rounding
ae1869c 
The cliff case now expects the Child Benefit rounded down to pounds, and the
dataset softer-taper case moves its taper end so the rounded charge stays
positive (2%, £46).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Follow regs 13-14 and reg 5 in the Carer's Allowance earnings test; k…
0d15fc7 
…eep UC carer status

Review changes: trading profit takes reg. 14's notional basic-rate tax and
main-rate Class 4 rather than progressive PAYE, with one set of personal
reliefs; a fraction of a penny of weekly earnings is disregarded (reg. 5);
a reported award the person would claim still marks them as a carer for
Universal Credit when the earnings limit stops payment (UC reg. 30 has no
earnings test); the PAYE approximation is described as one. A property
test that assumes an entitled carer now keeps the carer's pay under the
limit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Keep a miscellaneous loss out of total income and UC earnings too
7145d5b 
Review changes: total_income (ITA 2007 s. 23 Step 1) and Universal Credit
earned income now read taxable_miscellaneous_income, so a loss neither
reduces total income (the Winter Fuel test, non-dependant deductions) nor
widens the UC earnings error in #1961. The docstring states s. 152(5)'s
same-provision limit and the input's wider scope. Tests add the personal
allowance taper, total income and UC earnings; the property test covers
total income, with 25 examples and a taper-region example.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Return total income from the miscellaneous-loss property test helper
9e2192d 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Round the charge exactly in float32 and in the household-based reform
14f3f88 
Review fixes for #2323:
- Work out the appropriate percentage in float64, multiplying before
  dividing, so every exact £200 step gives exactly that many points when
  incomes are held as float32 (50 of 100 steps lost a point before).
- Take an amount to the nearest penny before rounding down to pounds, so
  four children's £4,199 stored as 4,198.9995 stays £4,199.
- Apply the s. 681C(3) rounding in the household-based charge reform too.
- Describe the opt-out test as the continuous charge share, as distinct
  from the rounded statutory charge.
- Regression tests: float32 steps, four children, single-parent reform
  equivalence and the couple's combined charge.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Give trading profit the full personal allowance; read the Scottish ba…
c4aa96c 
…sic rate robustly

Second-round review fixes for #2312:
- Reg. 14(1) deducts the personal reliefs the claimant is entitled to, with
  no reduction for allowance the pay uses, so trading profit now takes the
  full personal allowance. Sharing one allowance deducted too much tax and
  could make a carer over the limit eligible (Case 16).
- Read the Scottish basic rate from the parameter tree's second bracket. The
  scale at an instant drops the null starter bracket of 2016-17 and 2017-18,
  so index 1 was the 40% higher rate in those years.
- Case 12 carries £5,000 of capital allowances, which reg. 13(6)(a) does not
  deduct; Case 15 now expects the full allowance.
- Wording: the UC carer status description names the reported-award
  evidence and calls would_claim a modelling convention; the property-test
  docstring drops Class 2; the 2020 limit cites the 2020-04-06 version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Set the Personal Savings Allowance on rest-of-UK bands for Scottish t…
9a69394 
…axpayers

ITA 2007 s. 12B(8)(a)(iv) and (b)(iv) count a Scottish or Welsh taxpayer's
higher- and additional-rate income as what would be charged at those rates
if they were not a Scottish or Welsh taxpayer. savings_allowance read
tax_band, which follows the Scottish bands, so a Scottish taxpayer with
taxable income from £62,430 to £125,140 got no allowance (law: £500) and one
from £31,092 to £37,700 got £500 (law: £1,000). It now works out the band on
the rest-of-UK thresholds itself; tax_band keeps the Scottish bands for the
Marriage Allowance conditions. This also corrects 2016-17, where tax_band put
the higher band at £150,000.

Fixes #2347

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Band savings and dividends above property income
bdca314 
ITA 2007 s. 16 treats savings and dividend income as the highest part of
total income, so property income, which is non-savings income, fills the
bands below them. Since #1408, earned_taxable_income leaves property income
out (property_income_tax taxes it separately), and the savings bands, the
savings starting-rate taper and dividend_income_tax all stacked on
earned_taxable_income, so they were banded as if property income were not
there. £40,000 of property income and £20,000 of dividends paid £2,096.25 of
dividend tax instead of £4,528.75.

New non_savings_taxable_income (earned_taxable_income plus property income
after allowances, from the new taxable_property_income_after_allowances,
which property_income_tax now reads too) is the base for those variables.

Fixes #2351

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Correct the property integration test's savings tax, which encoded th…
1a80b21 
…e bug

The case expected nil savings tax for two people with property income and
savings interest. With savings above the property income (ITA 2007 s. 16),
person 1 is a higher-rate taxpayer: (2,900 - 500) x 40% = 960; person 2 a
basic-rate one: (1,525 - 1,000) x 20% = 105.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Drop the rounding slack: exact steps are exact without it
a0516c0 
Second-round review fix for #2323. With whole-pound band edges, an income on
an exact step gives an integer-valued quotient when multiplied before dividing
in float64, which IEEE division returns exactly, so the 1e-6 slack was not
needed. Under a wide taper it rounded a penny income just below a step up to
it: £79,399.99 under a £60,000-£2,000,000 band (as a reform or test can set)
charged £14 instead of £0. round_down_to_pounds now floors whole pence by
integer division.

Regressions: every tested step and the float32 income just below it under the
statutory, £2,000,000 and £10,000,000 band ends, and the £79,399.99 case. The
property docstring states the whole-pound and whole-pence domain.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Give the higher allowance at exactly a rate limit; test the boundaries
49fad84 
Review fixes for #2348:
- ITA 2007 s. 10(2)-(3) charges income up to a limit at the lower rate and
  only income above it at the higher one, so taxable income exactly at the
  basic rate limit has no higher-rate income (£1,000) and exactly at the
  higher rate limit no additional-rate income (£500). The test is now ">".
- YAML Cases 8-11 at and one pound over each limit; Case 7 for period 2017,
  which the old tax_band formula also covered; the property tests always run
  the boundary rows and the statute oracle uses ">".
- Changelog and comment wording.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Use the full band base for the annual allowance charge; tidy the base…
2de2210 
… variables

Review follow-ups for #2352:
- taxed_income, which sets the annual allowance charge's marginal rate,
  stacked on earned_taxable_income too and so left property income out;
  it now adds non_savings_taxable_income (Case 6: £10,000 of excess
  contributions with £60,000 of property income pay 40%, not 20%).
- One non_savings_income_after_allowances variable feeds both
  taxable_property_income_after_allowances and property_income_tax.
- Cite ss. 16A and 25(3A) for 2027-28; say the identity holds without
  Marriage Allowance and pin that case (#1947); point the starting-rate
  comment at #1799; replace the monotonic check, which main also passed,
  with a higher-rate check main fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Deduct Class 2 from trading earnings before April 2024
b934055 
Third-round review fix for #2312. Until it was omitted from 6 April 2024,
reg. 14(2)(a) of the Computation of Earnings Regulations deducted the Class 2
contributions payable on the chargeable profit. The deduction now uses the
engine's Class 2 rules (class_2_liable, the year's contribution weeks) on the
profit itself, not ni_class_2's base after capital allowances. The flat rate
is nil from 2024-25, so 2026 is unchanged.

Case 17 (2018): £7,000 of profit and £1,400 of personal pension
contributions give £118.20 a week, under the £120 limit; without Class 2 it
was £121.15, over it. The year-by-country oracle deducts Class 2 for
2016-2018.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
Merge branch 'main' into ci-test-runtime-2099
205fb64
@MaxGhenis
Merge branch 'main' into remove-stale-xfail-1968
7eeb3d2
@MaxGhenis
Merge remote-tracking branch 'origin/main' into hicbc-rounding
eb99a94
@MaxGhenis
Merge remote-tracking branch 'origin/main' into misc-loss-relief
72ec8db
@MaxGhenis
Merge remote-tracking branch 'origin/main' into scottish-savings-allo…
60c6054 
…wance
MaxGhenis and others added 12 commits October 11, 2026 17:50
@MaxGhenis
Merge remote-tracking branch 'origin/main' into property-income-band-…
c1067e3 
…base
@MaxGhenis
Merge remote-tracking branch 'origin/main' into carers-allowance-earn…
17d034a 
…ings-limit
@MaxGhenis @claude
Merge #2303: Assert the 2023-24 Class 4 lower profits limit instead o…
b4a78d0 
…f expecting it to fail

Rollup member; reviewed head 7eeb3d2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Merge #2269: Split the CI Test job into shards and cut its runtime
ff51368 
Rollup member; reviewed head 205fb64.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Merge #2312: Apply the Carer's Allowance earnings limit
2e8c882 
Rollup member; reviewed head 17d034a.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Merge #2323: Round the High Income Child Benefit Charge as s. 681C(3)…
0820203 
… requires

Rollup member; reviewed head eb99a94.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Merge #2327: Relieve a miscellaneous loss only against miscellaneous …
5133c71 
…income

Rollup member; reviewed head 72ec8db.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Merge #2348: Set the Personal Savings Allowance on rest-of-UK bands f…
87cb78c 
…or Scottish taxpayers

Rollup member; reviewed head 60c6054.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Merge #2352: Band savings and dividends above property income
c36829d 
Rollup member; reviewed head c1067e3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Merge #1846: Warn instead of silently mislabelling a dataset's fiscal…
ef72059 
… year

Rollup member; reviewed head 00c07d6.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Merge #2186: Put road-fuel litre divisors on DESNZ pump prices
9013856 
Rollup member; reviewed head 583b3d2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis @claude
Merge #2194: Pick up new documentation pages by folder
a8e4532 
Rollup member; reviewed head c53eb90.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis

MaxGhenis commented Oct 12, 2026

Copy link
Copy Markdown
Collaborator Author

UK hub landing audit for rollup #2364, head a8e453203d786202707942ac203dcc07671e47ef.

@MaxGhenis
MaxGhenis merged commit f061b21 into main Oct 12, 2026
11 checks passed
@MaxGhenis
MaxGhenis deleted the hub/rollup-2026-10-11-a branch October 12, 2026 03:07
This was referenced Oct 12, 2026
Closed
Closed
Merged
Merged
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@MaxGhenis @vahid-ahmadi