Skip to content

Measure SPM uprating effects and reconcile housing replay - #11

Draft
MaxGhenis wants to merge 1 commit into
masterfrom
max/spm-uprating-validation-20260919
Draft

MaxGhenis wants to merge 1 commit into
masterfrom
max/spm-uprating-validation-20260919

Conversation

@MaxGhenis

Copy link
Copy Markdown
Contributor

The September 18 uprating changes had not been tested against the age-specific SPM forecast misses, and the saved housing-cap validation compared the model against a reference that omitted household subsidy allocation. This PR adds a dated, reproducible diagnostic for both questions.

The controlled comparison holds the population and all 84 non-country dependencies fixed, changing only PolicyEngine-US from 238321d3 to 44001a4c (#9526 and #9527, plus the intervening version bump). Six sequential runs cover 2024, 2025 and a 2025 threshold anchor in each arm.

Group Annual change before Annual change after Census annual change
All people +0.164 pp +0.286 pp +0.07 pp
Children +0.857 pp +1.026 pp −0.06 pp
Age 65+ −0.729 pp −0.678 pp +0.24 pp

All recorded 2024 summaries match exactly. The fixes raise 2025 poverty by 0.121 pp overall, 0.170 pp for children and 0.051 pp for seniors. National and child point-estimate fit worsens; senior fit improves modestly but retains the opposite sign. This compares the controlled before arm with the after arm, not the original forecast runtime. Census's three observed annual changes are not statistically significant at 90% confidence.

The successor housing replay matches all 16 saved cases exactly at both native-unit and person levels. Census documentation supports household subsidy proration. The model's tenant-payment proxy remains an explicit assumption; matching this model contract does not establish Census parity or replace the original NOT_REQUALIFIED record.

The analysis directory includes aggregate results, full-file and omitted-metadata hashes, exact installed-source verification receipts, dependency comparison, scripts, and reproduction instructions. The national-threshold decomposition labels the remainder as the change with national bases held constant in real terms, which includes resources, geography, composition and weights.

Validation:

  • Six population runs completed; raw and compact summaries agree exactly.
  • 57 repository tests and 23 housing fixtures passed locally; paper drift check passed.
  • Installed country files match both exact Git trees; all non-country dependencies match.
  • Registered forecast, grade and population SHA-256 hashes remain unchanged.
  • Added lightweight diagnostic CI coverage without population loads.

These are explicitly uncertified development calculations: the released wrapper's data certificate does not cover the two country revisions. The original forecast and grade remain unchanged. No production model formula or app is changed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant