Charge BADR, residential property and carried interest gains at their own CGT schedules

[juaristi22]

Fixes #1858

capital_gains_tax now charges four schedules instead of one: gains qualifying for Business Asset Disposal Relief or Investors' Relief (capital_gains_badr), residential property gains (capital_gains_residential_property), carried interest (capital_gains_carried_interest) and the rest of capital_gains at the main rates. The three inputs default to zero, and at zero the liability is bit for bit what it was.

Modelling choices and assumptions (please check each)

Each item gives the choice, its basis, the alternative rejected, and where to check it.

1. The inputs are subsets of capital_gains, not additions to it. Each is a GBP component of the person's pre-response total; the remainder is charged at the main rates. A component entered without a matching total is ignored (tax 0). Inputs that together exceed the total are clipped in the fixed order BADR, residential, carried interest, and the main remainder is then 0. This matches the microcosm export, where the residential column is the whole gain of a flagged person (UK CGT distribution: age/region conditioning, banded and residential targets, asset types (#725, #467) microcosm#939). Check: the "exceed capital_gains", "without capital_gains" and "net loss" cases in capital_gains_tax_components.yaml; the clipping block in capital_gains_tax.py.
2. BADR and Investors' Relief share one input and the BADR limit. The lifetime limit is applied to the year's qualifying gains as if none had been used before; the excess is charged at main rates. The Investors' Relief limit of £10m before 30 October 2024 is ignored. The limit's 11 March 2020 commencement is sampled at 30 April like every fiscal-year parameter, so 2019-20 reads £10m for the whole year; no engine consequence. Basis: TCGA 1992 s. 169N(3)-(4), FA 2020 Sch. 3, FA 2025 ss. 8 and 10. Check: the £1.5m (2025) and £1.01m (2024) BADR cases; the badr/lifetime_limit.yaml description.
3. A realisation response is shared across schedules in proportion to pre-response shares. The components are scaled by realised over pre-response gains before clipping. Consequence: marginal_tax_rate_on_capital_gains, which adds £1,000 to capital_gains, becomes a share-weighted marginal rate, so a residential-only taxpayer now responds to a residential-rate reform. Rejected alternative: fixed components with the response absorbed by the main remainder, which discards the response for anyone whose gains are all on one of the new schedules and blinds the elasticity to reforms on those schedules. Check: TestResponsePlumbing in test_capital_gains_tax_components.py; YAML tests set capital_gains directly, so their ratio is exactly 1 (the two residential 2024 cases agree).
4. The annual exempt amount goes to the schedule with the highest top rate first: carried interest, residential, main, BADR under current law, ties in that order. Basis: TCGA 1992 s. 1K(5), "in whatever way is most beneficial". This is exactly optimal in every year of current law (checked year by year on the marginal saving of a pound of AEA on each schedule); under a reform it can miss the optimum by at most the AEA times the largest gap between two schedules' marginal rates. Rejected: a per-person optimisation. Check: the "BADR gains take the basic rate band" case, whose comment shows the AEA on BADR would have cost £120 more.
5. Basic rate band: relief gains take it first by statute; the rest goes where it saves most. Basis: TCGA 1992 s. 1I(4)-(6) (no taxpayer choice for BADR gains) and s. 1I(7)-(9) (the taxpayer's allocation, implemented as widest higher-minus-basic gap first, ties in declaration order). Carried interest from 6 April 2025 has a zero gap and comes last, which reproduces its exclusion from s. 1I by FA 2025 s. 12 without a special case. Check: the 2025 case where £7,000 of residential gains pays 24% although a free allocation would have given 18%; the 2024 mixed case, where the band goes to carried interest, then main, then residential.
6. The higher/additional split has no basis in CGT law and is kept exactly as it was. Gains above the basic rate band are split at higher_rate_limit − basic_rate_limit as the old formula did, including the pre-existing quirk that gains receive the whole higher band whatever the taxpayer's income. Relief gains consume that band first ("the lowest part of the gains"), then the others by additional-minus-higher gap, zero in law. Kept for bit-identity and because additional_rate is a live reform lever; the quirk is listed under follow-ups. Cost of the relief-first ordering under a reform with additional_rate above higher_rate: at most the notional band width times the rate gap, £87,440 × 5pp = £4,372 per affected person at 2025 thresholds (the band runs from £37,700 to £125,140 of taxable income). Measured on this head under 0.20/0.40/0.45 with £1m of relief gains and £100,000 of main gains and no income: £183,650, against £179,278 if the main gains took the higher band. In law the two rates are equal and the ordering costs nothing. Check: the identity test; higher_rate_band in the formula.
7. The new schedules carry an additional_rate equal to their higher rate, and a rate-only reform no longer reaches them. A reform that sets only gov.hmrc.cgt.{basic,higher,additional}_rate leaves residential property, carried interest and relief gains at current law. Measured on this head for 2025, income £100,000, gains £100,000 on one schedule, under 0.20/0.40/0.45 on the main rates only: main-rate gains go from £23,280 to £39,278; residential (£23,280), relief (£13,580) and carried-interest (£31,040) gains do not move. Under the old single schedule all four went to £39,278. Today no published dataset carries the inputs, so nothing changes for a consumer such as PolicyEngine/uk-equalising-cgt; once microcosm Replace if hasattr(person.simulation, "dataset"): with if person.simulation.dataset is not None: #939 puts residential gains on the spine (about £12bn of the base) a rate-only reform understates the yield of equalisation. To tax every gain at income tax rates a reform must also set residential_property.{basic,higher,additional}_rate and carried_interest.{basic,higher,additional}_rate, and set badr.lifetime_limit to zero, because badr.rate is flat and cannot express three bands; with that dict all four schedules pay £39,278. The changelog fragment says the same. Check: TestEqualisationReforms (both behaviours pinned); the additional_rate parameter descriptions.
8. No fiscal-year blend on any new parameter. Every new rate changes on 6 April (BADR 2025 and 2026, residential 2024, carried interest 2025), so the 30 April sample is exact. The 2024-25 main rates keep the calendar day-weight (207/365); HMRC's reported forestalling ahead of the Budget is not modelled. Check: TestScheduleParameters asserts exactly 0.24 for residential in 2024 and no fiscal_year_blend flag.
9. Carried interest stays at 32% from 6 April 2026 although the law moves it into income tax; the flat 32% stands in for that treatment and the parameter descriptions say so.
10. Parameter history starts at 2015-01-01 like the existing CGT files. Residential and carried-interest rates before 6 April 2016 equal the then-general 18/28 rates, so they are right by construction; the pre-existing 2015-16 main-rate values are untouched (Roadmap: complete, interaction-aware UK tax–benefit coverage (2000-onward) #1775 item B2).
11. Bit-identity when the inputs are zero. The 17 existing YAML cases are untouched and pass. TestZeroInputsAreBitIdentical runs the old formula, copied operation for operation, against the new variable on 150 single-person households per year for eight years (2018, 2020, 2022, 2023, 2024, 2025, 2026, 2030) with gift aid and pension variants, and under an equalised-rates reform and a fractional AEA, comparing the float32 bit patterns. The only theoretical divergence is the sign of a zero under a negative rate parameter.
12. What the data can feed today. Only capital_gains_residential_property exists on the microcosm spine (UK CGT distribution: age/region conditioning, banded and residential targets, asset types (#725, #467) microcosm#939); BADR and carried interest read as zero until HMRC Tables 4 and 9 are imputed (Add HMRC CGT Tables 7–9 source packages chronicle#273 has Table 9). This PR does not close the +11.4% liability gap on its own; the residential input alone moves 2024 liability up by roughly £0.3bn.
13. Unchanged behaviour, not new: Scottish taxpayers use the UK basic rate limit (correct for CGT); gift aid and personal pension contributions extend the band as before; losses still produce zero tax with no carry-forward.

Also in this PR: dataset files are opened read-only

The first CI run failed on 29 dataset-backed tests, none of them CGT-specific, with PermissionError: file ... exists but it can not be written from PyTables. huggingface_hub 1.32.0 (released 17 September 2026, the only cache-relevant version change between the green main run on 16 September and the red runs on 18 September; #1860's run failed the same way) stores cached downloads as read-only blobs in its new cross-repo blob store. UKSingleYearDataset and UKMultiYearDataset opened the file with pd.HDFStore's default append mode, which PyTables checks for write access. Reading a dataset needs no write access, so both loaders now open with mode="r"; three tests in test_simulation_dataset_sources.py save a small dataset, make the file read-only and load it through both classes and the Simulation constructor (all three fail without the fix). Reproduced locally by loading a read-only copy of enhanced_frs_2024_25.h5. The same two-line change already exists as #1847 (opened 3 September, CI green, no review yet), with its own tests and fragment; that is the standalone fix to merge first so #1860 and the other open PRs go green. Once #1847 is on main this branch rebases and drops its loader commit, tests and fragment.

Found in passing, not fixed here: UKMultiYearDataset.validate_file_path never recognises a multi-year file (it looks for person/2025-style keys in h5py's top-level listing), so a multi-year file given by path falls through to the single-year loader. Separate issue to follow.

Verification

• uv run policyengine-core test policyengine_uk/tests/policy/baseline/gov/hmrc/capital_gains_tax -c policyengine_uk: 17 existing cases and 16 new cases pass.
• uv run pytest policyengine_uk/tests/test_capital_gains_tax_components.py policyengine_uk/tests/test_capital_gains_responses.py policyengine_uk/tests/test_fiscal_year_parameters.py policyengine_uk/tests/code_health policyengine_uk/tests/test_parameter_descriptions.py: pass (the dataset identity test skips locally without a token; CI runs it).
• Full YAML policy suite (policyengine-core test policyengine_uk/tests/policy): 1,192 passed. Full pytest suite (pytest policyengine_uk/tests/): 257 passed, 45 skipped (the microsimulation-marked tests skip without a dataset token; CI runs them).
• Offline microsimulation on the cached enhanced_frs_2024_25.h5 (113,617 persons; the dataset carries none of the three inputs): capital_gains_tax is bit-identical to the legacy formula for 2024, 2025, 2026 and 2027, with weighted totals of £10.86bn, £12.77bn, £13.21bn and £13.70bn on both sides.
• uv run pytest policyengine_uk/tests/test_capital_gains_tax_components.py: 40 passed, 1 skipped locally (the dataset identity test needs the token; CI runs it), including the two TestEqualisationReforms cases.
• uv run pytest policyengine_uk/tests/test_simulation_dataset_sources.py: 10 passed (3 new read-only cases; 3 failed with the loader fix reverted).
• ruff format --check . clean; ruff check clean on the touched files.
• Docs: docs/book/policy/model-baseline.md now dates the main-rate rise to disposals from 30 October 2024 with the 2024-25 day-weighting (it said 2025-26), and gains the BADR, residential and carried-interest notes.
• Not run: make documentation (jupyter-book is not installed in this environment; the docs changes are prose edits to docs/book/policy/model-baseline.md and docs/book/assumptions/growthfactors.md). No populace-725 staging artifact was available locally, so the residential-only effect on the microcosm dataset is not measured here.

Follow-ups

• Engine: a reformable 2024-25 forestalling weight instead of the 207/365 calendar blend (utils/parameters.py::fiscal_year_average); the higher-band quirk in item 6; per-schedule marginal rates; a "lifetime limit already used" input; carried interest into income tax from 2026-27.
• Microcosm: impute BADR gains from HMRC Table 4 conditioned on capital_gains_asset_type, and carried interest from Table 9 (Add HMRC CGT Tables 7–9 source packages chronicle#273). Model income shifting between capital gains and income #1805 (income shifting) reuses the same inputs.

🤖 Generated with Claude Code

[vahid-ahmadi]

Review at b1e53053 (Claude Code, high effort; worktree checkout of the four commits on main 709a9b61; from the worktree against the engine venv: the CGT YAML directory 33 passed, the components, responses, fiscal-year, descriptions and dataset-source suites 80 passed with the token-gated identity test skipped, the whole tests/policy suite 1,192 passed; ruff check and ruff format --check clean on the eight touched files; CI green including the docs build. Draft, so a comment rather than a review.)

The formula is right, and I checked it against the law and by hand rather than against its own tests. Every new parameter matches what I read on legislation.gov.uk: BADR 10% → 14% from 6 April 2025 → 18% from 6 April 2026 under FA 2025 s.8, the £1m lifetime limit from 11 March 2020 under FA 2020 Sch 3, residential 18/28 → 18/24 from 6 April 2024 under F(No.2)A 2024 s.6, carried interest 18/28 → flat 32% from 6 April 2025 under FA 2025 s.12 which also inserts s.1I(A1) to take carry out of the band rules, and the 2024-25 main-rate blend at 207 of 365 days. Three hand cases reproduce the engine to the penny: 2025 with £30,000 income and £100,000 all residential gives £22,063.80; 2025 with £60,000 income and £2m of which £1.5m BADR gives £379,280 with the BADR clipped to £1m and the excess on the main schedule; and the 2024 blend-year mixed case with £20,000 carried, £20,000 residential and £10,000 main gives £9,010.10 with the AEA taken off carry and the basic-rate band going carry → main → residential by gap, exactly as item 5 says. The clipping order and the "component without a total is ignored" rule are what the code does, since every min_ is bounded by max_(0, capital_gains). The response scaling is divide-by-zero safe and, because the constructor moves capital_gains into capital_gains_before_response, the MTR's extra £1,000 is share-weighted as claimed: half carry and half main gives 0.28, all residential 0.24, no gains 0.0. On the AEA claim I brute-forced the allocation on a grid of incomes including the band straddles, four schedules at six amounts each, £100 AEA steps, for 2024, 2025 and 2026 and found no case where the greedy order is beaten; the residential-versus-main tie is symmetric and the straddle does not break it because AEA taken off one schedule inside the band frees band for the next. The bit-identity test really is the old formula operation for operation, comparing float32 bit patterns, and the seventeen pre-existing YAML cases are untouched. The loader fix is necessary — the three new tests fail with it reverted — and the attribution holds: CI installs with uv pip install -e .[dev] and runs bare pytest, the green run had huggingface-hub 1.31.0 and the red one 1.32.0, and the failing path is the new sharded blob store, which is also why the lock's 0.33.4 never mattered.

1. Should-fix — the body does not say what this does to the standard equalisation reform, and it matters for a live consumer. With three schedules carrying their own rates, a reform that sets only gov.hmrc.cgt.{basic,higher,additional}_rate no longer touches them. Measured on this head for 2025, income £100,000, gains £100,000: all-main gains go from £23,280 to £39,278 under 0.20/0.40/0.45, while all-residential, all-BADR and all-carried gains stay at £23,280, £13,580 and £31,040. Under the old single schedule every one of those went to £39,278. Today that changes nothing, because no published dataset carries the inputs, but PolicyEngine/uk-equalising-cgt applies exactly that reform dict, and once microcosm #939 lands residential gains on the spine (about £12bn of the base) a rate-only reform leaves them at 18/24 and understates the yield by something of the order of £2bn. A full equalisation now has to set the residential and carried-interest schedules' three rates too, and for BADR set badr.lifetime_limit to zero, because badr.rate is band-independent and cannot express 20/40/45. Say this in the body and the changelog fragment, since the fragment is what a consumer reads on upgrade.

2. Question — item 6's higher-band quirk has a reform cost worth a number. Relief gains consume the notional higher band first, so under a 0.40/0.45 reform a BADR-plus-main taxpayer can have up to the full £74,870 band of main gains pushed into the additional band, about £3,700 per affected person at 2025 thresholds. The body lists the quirk under follow-ups; a sentence with the magnitude would let a reader decide whether the follow-up is urgent.

Nits. The body says the bit-identity test runs 2018–2030; it runs eight specific years. The lifetime limit dated 11 March 2020 is the correct commencement, but the 30 April sample gives £10m for all of 2019-20, with no engine consequence. The residential and carried-interest files cite FA 2016 s.83 for the 2015 value, which is right by construction as the body says. docs/book/policy/model-baseline.md:36 still says the main rates rose in 2025-26, which the repo's own effective-dates fragment corrected to 30 October 2024, and this PR appends to that sentence without fixing it.

Could not verify. The offline microsimulation totals and the "+£0.3bn from residential" figure, which need a spine that carries the input, and optimality under arbitrary reforms beyond current-law rates.

---

Approve once 1 is written down. Nothing else needs a code change. On the split-out question: the loader fix is one line and stands on its own; I would split it so #1860 and the rest go green today.

[juaristi22]

Thanks for checking it against the law and by hand. Addressed at 896ea98, on top of b1e5305; the body is updated to match.

1. Should-fix — written down. The changelog fragment now says that a reform changing only gov.hmrc.cgt.{basic,higher,additional}_rate no longer reaches residential property, carried interest or relief gains, and what a full equalisation has to set: both other schedules' three rates, and badr.lifetime_limit to zero, because badr.rate is flat and cannot express three bands. Body item 7 carries your numbers, which I reproduced on this head (main-rate gains 23,280 → 39,278; residential 23,280, relief 13,580 and carried interest 31,040 unchanged under the main rates alone; all four 39,278 under the full dict), and the uk-equalising-cgt consequence once #939 puts residential gains on the spine. TestEqualisationReforms pins both behaviours at 2025 values.

2. Higher-band ordering — the number. At most the notional band width times the rate gap. The band the formula uses is thresholds[2] − thresholds[1], £87,440 in taxable-income terms (37,700 to 125,140), so up to £4,372 per affected person at 2025 thresholds; £74,870 subtracts the gross higher-rate threshold, and the formula works after allowances. Measured: £1m of relief gains plus £100,000 of main gains and no income, under 0.20/0.40/0.45, gives £183,650 against £179,278 had the main gains taken the band. Nothing in law, where the two rates are equal. Now in body item 6, next to the follow-up.

Nits. Item 11 names the eight years. Item 2 notes that the 11 March 2020 commencement is sampled at 30 April, so 2019-20 reads £10m all year, with no engine consequence. model-baseline.md now dates the main-rate rise to disposals from 30 October 2024 with the 2024-25 day-weighting, instead of 2025-26.

Split-out. The standalone fix already exists: #1847 (Max, 3 September, CI green, no review yet) makes the same two-line change with its own tests and fragment. Merging that first is what gets #1860 and the other open PRs green; once it is on main I rebase this branch and drop b1e5305 with its tests and fragment. Until then the commit stays here so this PR's CI runs the dataset-backed tests, including the token-gated identity test, which passed on b1e5305.

[vahid-ahmadi]

Second pass at 896ea98e (Claude Code, high effort; worktree checkout, the response commit read against the dispositions — the fragment, the baseline doc and 53 lines of tests; from the worktree against the engine venv: the components suite 40 passed with the token-gated identity test skipped, the whole tests/policy suite 1,192 passed; CI seven of seven green).

Everything closed as described.

1. Verified. The changelog fragment now says a reform changing only the three main rates no longer reaches residential property, carried interest or relief gains, and states what a full equalisation must set: both other schedules' three rates and badr.lifetime_limit to zero because the relief rate is flat. Body item 7 carries the numbers and the uk-equalising-cgt consequence. TestEqualisationReforms pins both behaviours — the main-rate reform leaving the other schedules at current law, and the full dict taxing every gain alike — at 2025 values, alongside the fiscal-year values and the no-blend check.

2. Verified, and the number is better than mine. The band the formula uses is the taxable-income width, £87,440, not the £74,870 gross figure I quoted, so the ceiling is £4,372 per affected person at 2025 thresholds, and your measured case (£1m relief plus £100,000 main, no income, £183,650 against £179,278) shows it. Now in item 6 beside the follow-up.

Nits verified. Item 11 names the eight years; item 2 notes the 30 April sample of the 11 March 2020 commencement; model-baseline.md dates the main-rate rise to disposals from 30 October 2024 with the 2024-25 day-weighting.

On the split-out. Agreed on the sequence: #1847 carries the same two-line loader fix with its own tests and fragment, so merging that first unblocks #1860 and the rest, and this branch drops b1e5305 on rebase. Keeping the commit here until then so the dataset-backed tests run is the right call.

---

Approving. The consequence for rate-only reforms is now where a consumer will read it, and pinned so it cannot drift.