Add a stratified UK sample and audit outputs for round 2 of the policyengine-uk audit

[MaxGhenis]

Round 2 of the internal policyengine-uk audit (round 1: #212). Nothing here is published; the UK board stays internal pending d1254. The record, with the per-cell audit and every policyengine-uk issue and fix, is in ~/reviews/policybench-uk-round2/README.md.

What this adds

• policybench/uk_audit.py: a seeded, stratified sample of 200 households (seed 20261011) drawn from the pinned public transfer dataset.

  • Twelve strata:
    • Scottish taxpayers, and low-income Scottish families;
    • pensioners, including a low-income group;
    • carers;
    • working families paying for childcare;
    • the High Income Child Benefit Charge range;
    • the personal allowance taper;
    • self-employment losses;
    • private renters, including a low-income group;
    • an unstratified control.
  • Selection: each stratum selects households by structure. The low-income strata select on capital: savings plus corporate financial wealth plus property other than the home. A first draw on savings alone let in pensioners with large corporate wealth.
  • Stated facts: the sample then states the facts the data lacks: State Pension, Attendance Allowance, caring hours, childcare costs, social tenancies, a council and its Council Tax liability, and a rental area. Each comes from a documented, seeded distribution. These values are constructed, not observed.
  • Dates of birth:
    • A date of birth on or after 6 April of the fiscal year moves back 182 days, clamped to 7 October of the previous year through 5 April. That keeps the age policyengine-uk derives (the age on 6 October) equal to the stated one.
    • A baby moved this way stays aged 0. A test runs the moved baby through the engine.

• policybench/uk_audit_councils.csv: 47 councils, each with its 2026-27 Band D and the source it was read from (a URL, or for Birmingham, Glasgow and Aberdeen the document title). Councils whose figure could not be read from an official page are left out.

• uk_audit output spec: the seven UK headline outputs, plus seven new ones:

  • the High Income Child Benefit Charge;
  • Carer's Allowance;
  • Attendance Allowance;
  • Housing Benefit;
  • Council Tax Reduction;
  • Tax-Free Childcare;
  • Scottish Child Payment.

  The UK headline set is unchanged.

• Prompts and situations:

  • The new facts are stated in words, and State Pension becomes a promptable input; the transfer path states it once.
  • The Council Tax liability is stated after the single person discount, because policyengine-uk reads council_tax as the liability before Council Tax Reduction.
  • UK negative amounts print as −£.
  • The adults of a household with children are stated as its parents (is_parent), which Tax-Free Childcare reads. Without it, 4 Tax-Free Childcare references were wrong; they were rebuilt before the audit.

• CLI and export:

  • policybench uk-audit-sample;
  • policybench run --program-set;
  • policybench export-full-run --program-set.

  Without the flags, worker commands, workload fingerprints and exports are unchanged. export-full-run refuses to write a non-headline output set to the public app data unless --skip-app-data is given.

• Pin: policyengine-uk 2.125.1 → 2.126.7. That includes round 1's #2248, #2258 and #2259. 2.126.8 came out the same day, and all 2,800 round 2 references are identical on it. US pins are unchanged.

Changes to the headline UK board, and known limitations

Headline UK prompts change in wording only:

• tenure values are now words ("renting privately", "owned outright");
• negative amounts print as −£.

US prompts are unchanged.

PROMPT_CONTRACT_VERSION is not bumped. It namespaces the response cache for every country, so a bump would invalidate the US cache for a UK-only wording change. The UK headline board is internal and has not been run since #212. The next UK headline run must start from a fresh run directory, because the version is also in the resume fingerprint.

is_parent changes no headline UK reference: 0 of 700 cells move when it is dropped (round 1's 100 households, headline outputs). In round 2 it moved 4 Tax-Free Childcare references, which were rebuilt before the audit.

Known limitations of the round 2 sample (the references follow the stated facts, so the audit cells stay valid; the sampler should change in round 3):

• 18-19-year-olds: the pinned transfer data carries no qualifying-young-person flag (is_child_or_QYP left policyengine-uk in #1655). So an 18- or 19-year-old living with a parent counts as an adult, on both the headline path and this sampler. 5 of the 200 households state such a pair as a couple: 075 and 079 (carers stratum), 177, 179 and 196.
• Attendance Allowance in Scotland: one Scottish adult (scenario 069) is stated as getting Attendance Allowance, which Pension Age Disability Payment replaces in Scotland. policyengine-uk pays Attendance Allowance from aa_category whatever the country.
• Overlap with round 1: 3 of the 200 households were also in round 1's 100.

Invariants (tested, including property-based tests)

• The sample is deterministic for a seed. Strata draw without replacement across strata, and changing a later stratum leaves earlier draws unchanged.
• Every household is in Great Britain, in a council in its region. Its liability is Band D × the statutory band ratio, less 25% for a single adult. Band I appears only in Wales.
• Every adult aged 67 or over has a State Pension, except in the control stratum. A drawn State Pension lies within the bounds of the scheme the date of birth implies.
• No one in the sample is born during the fiscal year. A moved date of birth keeps a baby aged 0 on 6 October, and a moved baby passes policyengine-uk's age check.
• Childcare costs appear only for children aged 11 or under, and only in the childcare stratum. Gift Aid appears only in the control stratum.
• Every uk_audit output maps to a policyengine-uk variable, and every council and rental area is a policyengine-uk enum member.
• Zero-weight households are never candidates, a missing capital value counts as nil, and an internal output set cannot be exported to the app data.
• Regenerating with seed 20261011 reproduces all 200 frozen round 2 scenarios exactly.

tests/test_uk_audit.py and tests/test_export_full_run.py: 61 passed locally. CI runs the full suite (test, lint, app, paper); this PR merges only once all four pass on its head.

What the run found (internal)

Six models, 200 households, $36.67. The reference audit (blind judge on ChatGPT-subscription Codex lanes, then my adjudication against the engine code and the law) found new policyengine-uk defects. Each is filed with a fix PR or an issue; the record lists them and the cells they explain. Prompt-side findings for round 3 are listed in ~/reviews/policybench-uk-round2/policybench_followups.md; they are not in this PR, because round 2's frozen prompts must stay as run.

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
policybench-site	Ready	Preview	Oct 11, 2026 6:25pm UTC

[MaxGhenis]

Merged by the session that opened it (PolicyBench UK round 2), on the standing merge gates:

• Head merged: 076ef83580044285e4319f35d1890576e2dfaf5a, squashed to 60f618590f7987a7c3fe86dd59d06d0305872c81.
• Checks at that head: app, lint, paper and test all passed (gh pr checks exit 0).
• Independent review: GPT-6.1 Sol, two rounds. Round 1 asked for changes (the export guard, NaN amounts, zero weights, the State Pension guard, the numpy floor); round 2 approved this exact head, conditional on CI. Both are in ~/reviews/policybench-uk-round2/reviews/ (pb-215-review.md, pb-215-review-r2.md).
• Nothing published: the uk_audit output set is refused by export-full-run unless app data is skipped, and round 2's results stay under results/local.

Round 2's nits that were left for round 3 (record program_set in exports, an income assertion in the low-income strata test) are in ~/reviews/policybench-uk-round2/policybench_followups.md.