Skip to content

test_built_total_gains fails on reduced (TESTING=1) builds of main #535

Description

@MaxGhenis

policyengine_uk_data/tests/test_cgt_band_donors.py::test_built_total_gains fails on reduced (TESTING=1, 32-epoch) builds of main, so PR CI can go red on PRs that have nothing to do with capital gains.

Measured locally on main b45c373 (TESTING=1 build): £242.6bn of above-AEA gains against HMRC's £65.9bn, a relative error of 268%. The reduced-build tolerance is 0.5 × 5.0 = 250%.

Seen in PR CI:

Full builds pass. Production-setting builds of main and #498 (512 epochs) pass all eight test_cgt_band_donors.py tests at the strict bounds.

So the reduced-build check sits at its threshold. It catches calibration noise rather than the "order-of-magnitude pathologies" its comment describes. Options:

  • loosen the reduced-build tolerance with evidence of its spread across seeds;
  • run more epochs for this check;
  • skip it under TESTING, as test_built_cgt_taxpayer_count already does.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions