Fill CBO total AGI for 2015-2018 with IRS SOI actuals

[MaxGhenis]

Problem

calibration.gov.cbo.income_by_source.adjusted_gross_income had a 2015 value of $9.034T, cited to a Tax Foundation page whose Table 1 is tax-year 2013 AGI for all returns excluding dependent filers ($9,033,840 million; 138.3 million returns). It had no values for 2016-2018, so the parameter held that figure through 2018. IRS SOI's all-returns figure for tax year 2013 is $9.094T, and for tax year 2015 it is $10.210T.

Its derived sibling adjusted_gross_income_per_capita (total ÷ calibration.gov.census.populations.total, built by tools/per_capita_uprating.py) is the default uprating index for 234 dollar-input variables. It drifted down over 2015-2018 and then jumped 31.7% into 2019.

Change

2015-2018 now hold IRS SOI Publication 1304 Table 1.1 values: all returns, adjusted gross income less deficit. Each value cites its own tax year's table. CBO values from 2019 are unchanged.

Year	Old total	New total (SOI)	Old per capita	New per capita	Old growth	New growth
2015	$9.034T	$10.210T	27,705	31,312
2016	(held)	$10.226T	27,461	31,085	−0.9%	−0.7%
2017	(held)	$11.010T	27,275	33,241	−0.7%	+6.9%
2018	(held)	$11.643T	27,086	34,910	−0.7%	+5.0%
2019	$11.962T (CBO)	unchanged	35,672	35,672	+31.7%	+2.2%

The SOI-to-CBO splice is close to level. SOI's tax-year 2019 total ($11.967T) is within 0.04% of the series' 2019 value ($11.962T).

The node description now says that total AGI uses SOI actuals for 2015-2018.

A new guard, test_default_series_has_a_value_for_every_year in tests/core/test_uprating_series.py, checks that the default series has a Jan-1 value for every year from its first to its last. It fails on main (2016-2018 missing) and passes here.

Impact

Parameters. I dumped every leaf parameter's full values_list after system init (105,634 parameters, so uprating, extensions and per-capita derivation are all included) before and after the change. Only the AGI total and its per-capita sibling differ. policyengine-core parameter uprating only appends values after a parameter's last value or start_instant. Every parameter uprated by this series has its last value (or start_instant) in 2020 or later: 2021 for SOI rental, farm, farm-rent and alimony income, 2020 for the SOI AGI bands, and 2027 for gov.hhs.smi.amount. None of them moves.

Household inputs. Core uprates a variable forward from its latest earlier known period, and holds the index at its first value before the index begins. So an AGI-uprated input dated 2018 or earlier changes when it is projected to a later year. Projected to 2019 or later, the old factors were too high by:

Input year	Old factor to 2026	New factor to 2026	Old result too high by
2015 or earlier	1.927	1.705	13.0%
2016	1.944	1.718	13.2%
2017	1.958	1.606	21.9%
2018	1.971	1.529	28.9%

Projected only into 2016-2018, the old factors were too low, by up to 12% for a 2018 target. An input dated 2015 projected to 2018 was ×0.978 and is now ×1.115.

Microsimulation aggregates. These are unchanged. The default dataset (populace_us_2024) has base year 2024, and its extension projects forward from 2024, so it never reads 2015-2018 index values. Microcosm and policyengine-us-data don't read this series for years before 2019.

Tests.

• 838 YAML test cases in 220 files have an input dated 2018 or earlier, by test period or period key. Only 11 of them have anything dated later. That is the only way a forward uprating can fire here, since no variable formula reads a later period; the offset(1, "year") loops in reforms/ check whether parameters are in effect.
  • Seven of the 11 are IRMAA lookback cases in gov/hhs/medicare/eligibility/medicare_irmaa_magi_two_years_prior.yaml. Each takes lag-year income from inputs dated in the lag year, which are read as stored arrays (holder.get_array) for lag years before 2015, so nothing is projected.
  • Four are Social Security earnings histories on employment_income, which has no uprating.
• Local runs of the HHS SMI, IRMAA, uprating and calibration tests, plus every YAML shard, are queued under the heavy-test lock (results to follow). I added instrumentation that logs every uprating on this series starting in 2018 or earlier.

Medicare IRMAA lookback (#10106, open). That PR deflates lookback income by this per-capita series, and its docstring documents the 2016-2018 gap. For a household entered for a single benefit year Y, the deflator is index(Y−2)/index(Y):

Benefit year	Old	New
2017	1.016	0.942
2018	1.014	0.890
2019	0.765	0.932
2020	0.721	0.930
2021 and later	unchanged	unchanged

Once #10106 lands, this PR will update that docstring note, and any of its expectations the fix moves. Not to be merged before that step.

Invariants

• Each per-capita series equals its total divided by population at every instant. The existing test_every_per_capita_series_is_its_total_over_population checks this, and check_per_capita_series_is_current enforces it at runtime.
• The default total has a value for every year from its first to its last. The new guard checks every year, so it covers the whole domain rather than a sample, and a property-based test would add nothing.
• An input projected from year X to year Y grows by index(Y)/index(X). The existing test_simulation_uprating_tracks_national_totals checks this.

Follow-up

Every IRS SOI series that inputs follow (calibration.gov.irs.soi.*, 88 variables) has the same defect. Each has 2015 values and then nothing until 2020-2022. A separate task will fill those years and generalize the guard.

axiom: n/a: calibration uprating series, not a legal provision

🤖 Generated with Claude Code

[MaxGhenis]

The CI run on this pull request was cancelled at 23:27Z on 2026-10-11 to free the organization's shared GitHub runners for an urgent fix in another repository. The US + core hub will re-run it automatically (cancelled jobs only; jobs that had passed are kept). Nothing in this pull request changed. Please don't push or reopen just to restart CI.

[DTrim99]

PR Review: Fill CBO total AGI for 2015-2018 with IRS SOI actuals

PR #10114 (author: MaxGhenis, head aeabda50bc, merge-base ae3782f4d7). 3 files, 3 commits.

Summary

The PR fixes the early years of calibration.gov.cbo.income_by_source.adjusted_gross_income, the default uprating series.

• Old data. The 2015 value was tax-year 2013 AGI excluding dependent filers, and there were no values for 2016-2018.
• New data. 2015 is replaced and 2016-2018 are added, all from IRS SOI Table 1.1 ("All returns", adjusted gross income less deficit).
• Why it matters. The derived adjusted_gross_income_per_capita series is the default uprating index for dollar inputs. It drifted down through 2018 and then jumped 31.7% into 2019.
• New guard. A test now checks that the default series has a value for every year.

The data change itself is correct:

• All four values match the SOI files to the thousand dollars.
• The units match the CBO series.
• The SOI-to-CBO splice is level: SOI tax year 2019 is within 0.04% of CBO's 2019 value.
• Every number in the PR body's per-capita, growth, factor and IRMAA tables reproduces.
• No parameter that is uprated by this series changes, and microsimulation aggregates don't move.

One blocking problem comes from timing:

• The PR body itself says it is "Not to be merged before" it updates Fall back to the earliest input year's MAGI for Medicare IRMAA, and add a same-year income switch #10106's docstring and expectations.
• Fall back to the earliest input year's MAGI for Medicare IRMAA, and add a same-year income switch #10106 merged into main on 2026-10-12. This PR hasn't been updated since.
• CI is green only because its merge ref predates Fall back to the earliest input year's MAGI for Medicare IRMAA, and add a same-year income switch #10106.
• On the merged tree, three of Fall back to the earliest input year's MAGI for Medicare IRMAA, and add a same-year income switch #10106's YAML expectations fail, and the new IRMAA docstring on main describes the old gap as current.

Critical (must fix)

1. Once merged with main, three Medicare IRMAA lookback expectations fail, and a docstring on main becomes false.

• Why the series now matters for IRMAA. Fall back to the earliest input year's MAGI for Medicare IRMAA, and add a same-year income switch #10106 (merged 2026-10-12 01:03 UTC) added LOOKBACK_DEFLATOR = "calibration.gov.cbo.income_by_source.adjusted_gross_income_per_capita" to medicare_irmaa_magi_two_years_prior.py.
  • A household with inputs only for benefit year Y gets that year's MAGI times index(Y−2)/index(Y).
  • So the 2015-2018 values this PR changes now feed household results for benefit years 2016-2020.
• Reproduced. I built the merge with git merge-tree --write-tree PolicyEngine/main aeabda50bc and ran the IRMAA YAML files on it with policyengine-core 3.33.3. Result: 3 failed, 24 passed.
  • policyengine_us/tests/policy/baseline/gov/hhs/medicare/eligibility/medicare_irmaa_magi_two_years_prior.yaml:33 (Case 2): [503654.4] differs from 504425.34375 (margin 0.01).
  • Same file :91 (Case 7): [100730.88] differs from 100885.0703125. The Part D surcharge stays 0, because 100,730.88 is still under the 103,001 threshold.
  • policyengine_us/tests/policy/baseline/gov/hhs/medicare/senior_household_2015_2016_integration.yaml:56 (Case 2): [10073.089] differs from 10088.509765625 (margin 1).
• Hand check. The new index is 10,210,310,102,000 / 326,078,000 = 31,312.48 for 2015 and 10,225,938,479,000 / 328,964,000 = 31,085.28 for 2016, a ratio of 1.0073088.
  • 500,000 × 1.0073088 = 503,654.40
  • 100,000 × 1.0073088 = 100,730.88
  • 10,000 × 1.0073088 = 10,073.09
• Stale text on main once this lands:
  • policyengine_us/variables/gov/hhs/medicare/eligibility/medicare_irmaa_magi_two_years_prior.py:62-67: "CBO AGI has no values for 2016 to 2018 and is held at its 2015 value ... one entered for 2019 or 2020 alone gets that year's income times 0.765 or 0.721." With this PR those factors are 0.932 and 0.929, and 2016-2018 get 1.007, 0.942 and 0.890 rather than "slightly more than that year's income".
  • medicare_irmaa_magi_two_years_prior.yaml:5-12: the header shows 2015: 9,033,840,000,000 / 326,078,000 = 27,704.54 and 2016: ... = 27,461.49.
  • The per-case comments at :22, :32, :90 and senior_household_2015_2016_integration.yaml:35, :55 use the same old index values.
• The Python IRMAA tests pass on the merged tree. They compute the deflator from the parameter. 59 passed: test_irmaa_first_modeled_year.py, the new guard, and the per-capita invariant.
• Fix. Merge main into the branch, then:
  • Set the three expectations to 503,654.40, 100,730.88 and 10,073.09 (or 10,073 within the 1-dollar margin).
  • Update the index comments to 31,312.48 (2015) and 31,085.28 (2016).
  • Rewrite the docstring sentence. For example: "CBO AGI uses IRS SOI actuals for 2015-2018, so a household entered for 2017, 2018, 2019 or 2020 alone gets that year's income times about 0.942, 0.890, 0.932 or 0.929." Or drop the numbers.

Should address

1. The changelog and the PR body's test-impact analysis predate #10106.

• Changelog. changelog.d/fix-cbo-agi-2016-2018-soi.fixed.md describes only uprated dollar inputs. On main the change also moves the imputed Medicare IRMAA lookback MAGI (and Part B/D IRMAA) for households entered for a single benefit year from 2016 to 2020. The 2019 factor goes from 0.765 to 0.932 and the 2020 factor from 0.721 to 0.929.
• PR body. The "Tests" section says the only way the fix can move a YAML case is forward uprating, "since no variable formula reads a later period". It also says the IRMAA lookback cases read stored arrays, "so nothing is projected".
  • On main, medicare_irmaa_magi_two_years_prior reads this index directly for earlier years, and that is how the three cases in Critical 1 move.
  • The "Local runs ... queued under the heavy-test lock (results to follow)" line was never filled in.
• Fix.
  • Add one changelog sentence on the IRMAA lookback effect.
  • Update the PR body's "Tests" and "Medicare IRMAA lookback" sections to say Fall back to the earliest input year's MAGI for Medicare IRMAA, and add a same-year income switch #10106 has merged, and list the expectations changed.
  • Fill in or remove the "results to follow" line.

Suggestions

1. Pin the largest IRMAA move with a test. No case covers a benefit year of 2017-2020 entered alone, where the deflator moves most. An example: a 66-year-old with only 2019 wages of 500,000.
   • Expected IRMAA MAGI two years prior: 500,000 × (11,009,900,155,000 / 331,211,000) / (11,962,000,000,000 / 335,329,000) = 500,000 × 0.931850 ≈ 465,924.93 (hand-computed, not run).
   • It would pin the household-level effect of the splice, which the new guard checks only at the parameter level.
2. Sequence with draft Fill IRS SOI uprating series gaps with Publication 1304 actuals #10128 ("Fill IRS SOI uprating series gaps with Publication 1304 actuals"). It also edits policyengine_us/tests/core/test_uprating_series.py, presumably to generalize the new guard as the PR body's "Follow-up" describes. Land this PR first, or fold the guard into Fill IRS SOI uprating series gaps with Publication 1304 actuals #10128's generalized version, to avoid a conflict.
3. Optional wording in the guard's docstring. test_uprating_series.py:127-130 ("CBO AGI once had nothing for 2016 to 2018, so inputs dated 2018 came out 29% too high by 2019") is accurate: 1.317 / 1.022 = 1.289. Consider also naming the IRMAA lookback as a reader of the series, since Fall back to the earliest input year's MAGI for Medicare IRMAA, and add a same-year income switch #10106 made it one.

Verified correct

• Values against IRS SOI Table 1.1 ("All returns" row, column 3 "Adjusted gross income less deficit, Amount"; the header says "money amounts are in thousands of dollars"):
  • TY2015: 10,210,310,102 thousand, which matches 10_210_310_102_000 (https://www.irs.gov/pub/irs-soi/15in11si.xls)
  • TY2016: 10,225,938,479 thousand, which matches 10_225_938_479_000 (https://www.irs.gov/pub/irs-soi/16in11si.xls)
  • TY2017: 11,009,900,155 thousand, which matches 11_009_900_155_000 (https://www.irs.gov/pub/irs-soi/17in11si.xls)
  • TY2018: 11,643,439,106 thousand, which matches 11_643_439_106_000 (https://www.irs.gov/pub/irs-soi/18in11si.xls)
• Units and concept. Both series are in dollars and cover all returns, dependent filers included. SOI TY2019 is 11,966,873,976 thousand (https://www.irs.gov/pub/irs-soi/19in11si.xls), against CBO's 2019 value of 11,962,000,000,000: a 0.04% gap, so the splice is level.
• The old value was wrong, as the PR says.
  • The cited Tax Foundation page is "Summary of Federal Income Tax Data, 2013": "$9,033,840" million AGI on 138,313,155 returns, and "Does not include dependent filers".
  • SOI's TY2013 all-returns figure is 9,093,628,703 thousand (https://www.irs.gov/pub/irs-soi/13in11si.xls).
• PR-body tables reproduced from the YAML values and the CBO population series:
  • Per capita, old to new:
    • 2015: 27,705 to 31,312
    • 2016: 27,461 to 31,085
    • 2017: 27,275 to 33,241
    • 2018: 27,086 to 34,910
    • 2019: 35,672 (unchanged)
  • Growth into 2019 drops from +31.7% to +2.2%.
  • Factor to 2026:
    • from 2015: 1.927 to 1.705 (+13.0%)
    • from 2016: 1.944 to 1.718 (+13.2%)
    • from 2017: 1.958 to 1.606 (+21.9%)
    • from 2018: 1.971 to 1.529 (+28.9%)
  • 2015 to 2018 goes from ×0.978 to ×1.115.
  • IRMAA deflators:
    • 2017: 1.016 to 0.942
    • 2018: 1.014 to 0.890
    • 2019: 0.765 to 0.932
    • 2020: 0.721 to 0.929 (the PR body rounds 0.92950 up to 0.930)
• Parameters uprated by this series don't move. Core parameter uprating extends only after a parameter's last value:
  • calibration.gov.irs.soi.{alimony,farm,farm_rent,rental}_income: last value 2021
  • calibration.gov.irs.soi.agi.total_agi bands: last value 2020
  • gov.hhs.smi.amount: last value 2026-10-01
• Microsimulation is unaffected.
  • Dataset extension runs forward from the 2024 base year.
  • The IRMAA deflator for 2024 and 2025 benefit years reads 2022-2024 CBO values, which are unchanged.
• Tests that read the series compute expectations from the parameter rather than pinning 2015-2018 values:
  • test_per_capita_uprating.py (the per-capita invariant at 2015 and 2017)
  • test_foreign_earned_income_exclusion_leaves.py (2025-2026)
  • the IRMAA Python tests
• The new guard (test_uprating_series.py:126-139) checks parameter structure, which YAML can't express, so a Python test is justified. It fails on main (years 2015, 2019-2036) and passes at the head.
• References. There is one reference per value, each pointing to its own tax year's SOI workbook. The workbooks are XLS files, so no #page anchor applies, and the titles contain no page text.
• Description and changelog.
  • The node description now names CBO for 2019-2036 and SOI for 2015-2018.
  • The changelog fragment is top-level and typed fixed.

Axiom line assessment

Valid n/a. The line reads axiom: n/a: calibration uprating series, not a legal provision.

• The PR changes a calibration data series and its test. No statute, regulation or legal parameter changes.
• Its new reach into the IRMAA lookback, through Fall back to the earliest input year's MAGI for Medicare IRMAA, and add a same-year income switch #10106, affects a model imputation for missing prior-year income, not an encoded legal rule.

CI status

All 26 checks pass (gh pr checks 10114), but the run doesn't cover current main.

• The run is a pull_request event created 2026-10-11 21:42 UTC. Attempt 2 re-ran the jobs cancelled for the org runner freeze.
• It checked out merge ref 80419bc0 ("Merge aeabda5 into 065d0e9"). Base 065d0e9560 is Drop the fixed Illinois AABD entry from the person-adds ratchet #10092, from 2026-10-11, and doesn't contain Fall back to the earliest input year's MAGI for Medicare IRMAA, and add a same-year income switch #10106 (8df4dffb2f, merged 2026-10-12 01:03 UTC).
• The three failures in Critical 1 therefore never ran in CI.
• The PR touches no partner test files.

Branch status

• Position: 54 commits behind main, 3 ahead.
• Merge: git merge-tree --write-tree PolicyEngine/main aeabda50bc has no textual conflicts. It does have a semantic conflict with Fall back to the earliest input year's MAGI for Medicare IRMAA, and add a same-year income switch #10106: three YAML expectations fail on the merged tree (Critical 1).
• Overlapping open PR: draft Fill IRS SOI uprating series gaps with Publication 1304 actuals #10128 also edits policyengine_us/tests/core/test_uprating_series.py (Suggestion 2).

Verdict: REQUEST_CHANGES

[DTrim99]

All four values match SOI Table 1.1 exactly, and the splice into CBO is smooth. One critical (details in the comment above): #10106, merged 10/12, deflates IRMAA two-year-lookback MAGI by this series, so merged with main the PR breaks three of its expectations (two_years_prior.yaml:33 and :91, senior_household_2015_2016_integration.yaml:56), and the medicare_irmaa_magi_two_years_prior docstring and index comments become false. Please merge main and update them. One should item: the changelog and PR body should state the IRMAA lookback effect (2019 factor 0.765 → 0.932, 2020 0.721 → 0.929).