Uprate dollar inputs by national totals per capita

[MaxGhenis]

Fixes a double count of population growth in uprating. Stacked on #9525, which corrects the docs paragraph this PR then updates; merge that first and this diff shrinks to one commit.

The defect

Household weights grow with total population (calibration.gov.census.populations.total). Dollar inputs that follow a national total, an IRS SOI series or a CBO income projection, grew by the full total as well. The weighted total of each such input therefore grew by the national total times population growth.

Measured on the default dataset (populace_us_2024.h5 at populace-us-2024-spm-20260909), growth from 2024:

Input	Year	National total	Weighted total before	Weighted total after
employment_income_before_lsr	2025	1.04935	1.05909	1.04935
	2030	1.28365	1.32823	1.28365
	2035	1.54243	1.62042	1.54243
social_security_retirement	2025	1.08312	1.09317	1.08312
	2035	1.80954	1.90104	1.80954
pre_subsidy_rent	2025	1.06904	1.07897	1.06904
	2035	1.54729	1.62553	1.54729

The overshoot equalled weighted population growth in every case: 1.00928 in 2025, 1.03473 in 2030, 1.05056 in 2035. It applied to all 33 dataset columns on IRS SOI series and all 44 on the CBO total AGI default.

The fix

policyengine_us/tools/per_capita_uprating.py runs at the end of system init. For each national total that a variable uprates by, it adds a sibling parameter holding the total divided by population (<path>_per_capita) and points the variable at it. A national total is any path under calibration.gov that does not already say per_capita and is not the population series.

• Both uprating paths are covered. Simulation uses policyengine-core's variable-level uprating, which reads variable.uprating. Microsimulation extends the single-year dataset through extend_single_year_dataset, which also has its own override table; those overrides now resolve to the per-capita sibling too.
• Unchanged: CMS per-capita out-of-pocket spending (17 variables), CPI-U (23), and the population series on household_weight and family_weight.
• Runs after reforms, so a reform to a national total or to the population series reaches the per-capita series. System init applies reforms twice; rebuilding is idempotent.
• Variable files are untouched. They still declare the national total, and a variable added later with a national total is converted automatically. A test fails if any variable ends init pointing at a national total.
• The derived parameters carry a label, unit, economy: false and household: false.

Trustees 2025 long-run scenario

apply_trustees_2025_soi_income_projection extends the SOI totals past 2036 by the average wage index so that each record's income follows wages. With per-capita uprating it now multiplies by population growth as well, so per-record growth still equals average-wage growth and the weights carry population. A new test asserts this for 2040 and 2050. The baseline extension past 2036 compounds the last CBO growth rate, which is aggregate growth, so it needs no change.

Legacy data pipeline

policyengine_us_data/utils/uprating.py already divides national totals by population growth and skips any path containing per_capita. It will read the new paths as already per capita, which is correct. That repository is archived.

Tests

• New tests/core/test_per_capita_uprating.py, 24 tests: path classification, no variable left on a national total, each per-capita series equals total over population, overrides resolve, weighted totals track national totals in an extended dataset for 2025, 2030 and 2035, the same through Simulation, a population reform reaches the series, idempotence, and the Trustees scenario.
• Updated three existing tests that pinned series names to expect the per-capita siblings.
• Partner contract tests: 620 passed locally with no expected-output changes.
• Run locally: tests/microsimulation/data, tests/policy/contrib/ssa, tests/policy/baseline/parameters/test_uprating_extensions.py, tests/test_system_import.py. The full suite is left to CI.

Effect on results

Population results for the dataset year do not change. For later years, every input on a national total is lower than before by cumulative population growth: 0.9 percent in 2025, 5.1 percent in 2035. Household calculations that supply inputs for the year they calculate do not change. I have not measured the effect on budgetary or poverty estimates.

Review findings addressed (second commit)

An independent adversarial review confirmed the identity exactly (weighted-total growth equals national-total growth to ten decimal places at person, household and SPM-unit level, including the two negative totals) and requested changes on six points. All are in the second commit.

1. HHS state median income outran incomes. gov.hhs.smi.amount is projected from 2027 by aggregate AGI; with incomes now per capita, the threshold drifted above them by population growth, 2.8 percent by 2035, and moved childcare and energy eligibility in projection years. It now follows AGI per capita, and that series is built before parameter uprating runs. This is the only parameter outside the calibration tree that uprates by a national total.
2. Reforms applied after init left the series stale. Core's Simulation applies a reform once more after the country system is built, and a non-idempotent reform to a total would then not reach uprating. modify_parameters now refreshes the derived series after every call.
3. Silent fallback. A tree holding a national total but not its sibling now raises in dataset extension instead of quietly double counting. The mock fixtures carry the sibling, and a test checks that the sibling's growth, not the total's, is what gets applied. A second guard raises when a derived series no longer equals its total over population, which catches a parameter edited behind the system's back.
4. Classifier by tree location. National totals are now the two families in use, calibration.gov.cbo. and calibration.gov.irs.soi.. Any other calibration path on a variable or override raises at init, since a state total or a head count needs a different denominator.
5. Refresh heuristic could overwrite a real series. Derived series carry derived_from; only those refresh, and a hand-written series that shares the derived name stops the build.
6. At-instant caches. Ancestor caches clear after a derived series is added, so parameters(instant) views resolve it.

Also from the review: the unused import is gone, the docs paragraph now attributes the per-capita default correctly, and tests cover the boundary after the population series ends in 2055.

One test reprojected

tests/policy/baseline/gov/states/wa/dcyf/eceap/wa_eceap_income_eligible.yaml has two 2031 cases one dollar either side of the 50 percent state-median-income floor. That floor is a projection, and with SMI now projected per capita the floor is 69,317.41 rather than 70,405.39, the old value divided by population growth from 2027. The cases now sit one dollar either side of the new floor. Every other projection-year test of an SMI-based program (11 files) passes unchanged.

Not in this PR

• Recipient counts still grow uniformly with total population. Reweighting to projected age structure, with factors solved against the reweighted totals, is the fuller fix and belongs with the dataset build.
• Series mappings are unchanged, including unemployment compensation being extended by CBO's taxable Social Security series while calibration.gov.cbo.unemployment_compensation sits unused, and rent and child care expenses following total AGI.

🤖 Generated with Claude Code