Rebase congressional-district capital gains onto Table 1.4 and drop mislabeled CD SALT and PTC columns on the full surface

[MaxGhenis]

Fixes #1038. Stacked on #1036. The base is main so that CI runs, since test.yml runs only on PRs into main. Until #1036 merges, the diff also shows #1036's commits 0f343249f and 6ebfcf7f4 and the main merge 2c811ab1c. This PR's own change is commit 011a63b1a: git diff 2c811ab1c..us-cd-capital-gains-concept.

Summary

On the default --target-surface full, the congressional-district file (22incd.csv) had three families summing past the national target of the same model quantity. Two more of its measures read the wrong IRS column:

CD-file family on full (pinned feed)	CD states = CD districts = CD US row	National target	Ratio	Cause
net_capital_gains_returns	29,845,710	12,392,020 (Table 1.4 col 37)	2.41x	concept: N01000 is Form 1040 line 7 (gain, capped loss or distributions-only)
limited_state_local_taxes_amount	$272.3B	$137.4B (HT2 A18460)	1.98x	wrong column: A18425 is state and local income taxes (Schedule A line 5a)
net_capital_gains_amount	$1,522.2B	$1,270.9B (Table 1.4 col 38, aged)	1.198x	TY2022 data stamped ty2023, plus the line-7 concept
premium_tax_credit_returns	6,848,330	7,841,370	0.87x	wrong column: N85530 is additional Medicare tax (Form 8959 line 24)

This PR:

1. Capital gains. Congressional-district capital-gains rows become shares of the file's own US row, scaled to the Table 1.4 control, exactly as Choose the capital-gains rebase control by concept and period, not feed order #1036 does for Historic Table 2 (option (a)). The file's US row retires. CD children keep scaling to their CD state parent.
2. SALT and PTC. The four mislabeled columns go into a reviewed, receipted US_FISCAL_TARGET_SOURCE_COLUMN_EXCLUSIONS register keyed by (measure id, IRS column). It replaces the hard-coded CD PTC-amount check.
3. Invariant. A pinned-feed test holds that no CD-file family sums past its national target on full.

This moves published full-surface target values, so it waits on Max's ruling before merge. national_state (what Build P shipped) keeps every value and membership.

Option (a) or (b)

I quantified both options on the pinned feed and the sha-matched IRS files. Scripts and JSON are in ~/PolicyEngine/_reviews/cd-cg-concept-20260927/lane-b/.

(a) Rebase as shares. This is what the PR does.

• Share error. The assumption is the one Choose the capital-gains rebase control by concept and period, not feed order #1036 already makes for states: a district's share of line-7 returns stands in for its share of Schedule D gain returns. AGI composition is the measurable source of error. Reweighting each of the 428 numbered districts' N01000 by AGI class, using Table 1.4's per-class gain share, moves its share by:
  • counts: median 2.1%, 95th percentile 4.7%, max CA-18 +9.7%;
  • amounts: median 0.5%, 95th percentile 2.4%.
• Agreement with the HT2 state rows already on full:
  • Returns differ by a median 0.6% (95th percentile 2.1%). That is tighter than the existing return_count pair (1.6% / 2.4%).
  • Amounts differ by a median 2.8% (95th percentile 15.4%, worst MT −27.0%), from item suppression in CD A01000; MT is 32.5% below HT2. That is inside the existing qualified-dividends pair (11.6% / 21.7%).

(b) Exclude. This would throw away the strongest geographic signal on the surface.

• District capital-gains incidence runs from 8.3% to 31.5% of returns (5th to 95th percentile), against 13.7% to 23.4% across states.
• The within-state district signal left after AGI has log SD 0.227 for counts and 0.223 for amounts, about 10x the share proxy's AGI error (0.024 and 0.009).
• Without these rows, district capital gains would be unconstrained, apart from the 9 HT2 at-large proxies.

One accepted difference. The CD US row equals the sum of its 51 states, because the file has no other areas. So rebased CD states and CD districts each sum to the control itself. The HT2 US row includes other areas, so HT2 states sum to 99.2% of the control for returns and 98.0% for amounts.

Interplay

The congressional_district_children_scale_to_state_parent profile rule.

• The rebase runs before the profile and applies one factor per family and quantity to states and districts alike.
• The CD district rows already summed exactly to their CD state parents (22incd.csv: state = sum of districts), so all 872 CD capital-gains children still reconcile with factor 1.
• The rule never used the CD US row; parents are state rows.

#1030 (source-vintage correction).

• A share divides CD rows by the CD US row of the same restamped package, so the stamp cancels. The value lands at the control's period (2023) and ages from there.
• A new CI test (test_congressional_district_capital_gains_ignore_the_package_stamp) compiles and ages the same package stamped ty2023 and truthfully at 2022, and asserts identical values. Its facts carry the pinned 22incd.csv digest and raw key.
• Read restamped Chronicle facts at their data year before aging #1030 as written refuses any restamped fact that was rebased (source_vintage.py, "was rebased; review the rebase"). So Read restamped Chronicle facts at their data year before aging #1030 plus this PR refuses every compile.
• The fix belongs in Read restamped Chronicle facts at their data year before aging #1030: admit a rebased restamp only when its soi_share_total_source_record_id (new here) is a restamp of the same package with the same stamp. Move source_period/uprating_from_period to the data year, and never move uprating_to_period.
  • Starting aging at 2022 instead would apply the Table 1.4 2022→2023 fall twice (×0.761).
  • The patch is at ~/PolicyEngine/_reviews/cd-cg-concept-20260927/lane-a/source_vintage_share_narrowing.patch. With it, all 21 of Read restamped Chronicle facts at their data year before aging #1030's test_us_source_vintage.py tests pass.
    • A trial merge of Read restamped Chronicle facts at their data year before aging #1030 onto this branch, with and without the narrowing, is running now. Its result will be added here.
• Read restamped Chronicle facts at their data year before aging #1030 already needs a rebase after Choose the capital-gains rebase control by concept and period, not feed order #1036 (its pinned CD-control test cannot pass once CD rows are never controls). That follow-up is filed separately.

What changes

fiscal_targets.py:

• _rebase_stale_soi_capital_gains_distributions rebases every row of a share family: a registered family whose concept is not the model's, today historic_table_2 and congressional_district (_soi_capital_gains_share_family).

  • Each row is divided by its own family's national all-AGI row at its own period (_soi_capital_gains_share_national_totals, keyed by family) and scaled to the control.
  • Each row declares soi_source_concept, soi_control_concept and the new soi_share_total_source_record_id.
  • Two different national totals for one family, quantity and period raise AmbiguousSoiCapitalGainsShareTotalError.

• US_FISCAL_TARGET_SOURCE_COLUMN_EXCLUSIONS drops SOI facts whose (measure id, layout.source_column_id) is listed:

  • limited_state_local_taxes_{returns,amount} read from N18425/A18425;
  • premium_tax_credit_{returns,amount} read from N85530/A85530.

  The IRS column is the definition, so a corrected package that reads N18460/N85770 passes untouched. These are the only CD measures whose column differs from their HT2 twin's; all 56 were compared.

• us_fiscal_target_exclusion_receipt gains a source_column_concept_exclusion rule.

• The release coverage report lists the register (tools/build_us_fiscal_refresh_release.py).

• _is_soi_cd_premium_tax_credit_amount_conflict is removed. Its comment called A85530 "not the same gross annual PTC control"; A85530 is not PTC at all.

Docs:

• docs/us-soi-capital-gains-concepts.md: the CD section, the (a)/(b) analysis and the effect table.
• docs/us-chronicle-feed-repin.md: a follow-up note with the new pins.
• The changelog fragment.

Effect on the pinned feed

Release compile: target period 2024, aging, packaged CD crosswalk, Medicaid substitution. Before is #1036 merged with main (2c811ab1c).

	Before	After
Compiled targets (= full)	32,842 (e02123644d42)	31,376 (059dc56d78db)
national_state	5,694 (65e4dde11c83)	5,694 (47dce807b412)
CD capital-gains returns, states / districts	29,845,710 / 29,845,710	12,392,020 / 12,392,020
CD capital-gains amount, states / districts	$1,522.19B / $1,522.19B	$1,270.86B / $1,270.86B
CA CD returns / amount	3,808,930 / $214.08B	1,581,478 / $178.73B
CD-file families past their national target on full	3 of 38 compared	0 of 35

• Removed: 1,466 specs, all on full:
  • 976 CD SALT specs;
  • 488 CD PTC-returns specs;
  • the 2 CD capital-gains US rows.
• Values moved: 974 CD capital-gains specs, by 0.415202720927 (returns) and 0.834893818418 (amounts).
• Nothing added.
• national_state: membership and every value are unchanged. The only difference is one metadata key, soi_share_total_source_record_id, on the 102 HT2 capital-gains rows, each naming the HT2 US row. That key alone moves the registry version.

Invariants

• Shares conserve the control. Every CD capital-gains row equals raw × control / CD US. The states, and the districts of each state, sum to control × (Σ states / CD US), never past the control. No CD capital-gains row ships at the country level or without the Schedule D bridge and its denominator. This is a Hypothesis property over random packages, controls and feed orders, so it also checks order-freedom.
• Stamp-invariance. The package stamped ty2023 compiles and ages to exactly the values it gives stamped truthfully at 2022, and amounts age from the control's 2023 period (differential test).
• Denominators are unambiguous. Two national totals for one share family, quantity and period refuse the compile.
• Column exclusion is exact. A fact drops if and only if its (measure id, column) is registered; the corrected columns compile. The receipt lists exactly the dropped facts for any sub-feed (existing Hypothesis receipt-completeness property, extended to the new rule).
• No CD-file family sums past its national target on full (pinned feed, 35 compared families). Before this change, 3 of 38 violated it.
• national_state values and membership are unchanged (pinned diff).

Tests

All runs below were local on the pinned feed, sequential, with peak RSS at most 3.5 GB. CI has no feed and skips the pinned-feed tests.

• engine_free/us/test_us_fiscal_targets.py: all 205 tests passed, before the order-free denominator test was added (that test passes on its own). The 6 pinned-feed tests passed under -rA, not skipped:
  • the exclusion receipt;
  • the fences and version pins;
  • Choose the capital-gains rebase control by concept and period, not feed order #1036's HT2 control test;
  • the HT2 state invariant;
  • the new CD control test;
  • the new CD invariant.
• engine/us/test_us_fiscal_targets.py, engine_free/shared/test_release_target_parity.py, engine_free/us/test_us_chronicle_feed.py, test_us_release_gate_preflight.py and test_us_fiscal_refresh_builder.py, run with MICROCOSM_US_CHRONICLE_FACTS set: 467 passed.
• tools/ci_test_groups.py --verify passes. ruff check and ruff format --check are clean on the touched files.

New tests:

• CD rows are shares, and HT2 rows still divide by the HT2 US row.
• With no control, no CD row compiles.
• Conflicting national totals refuse the compile.
• Equal totals name the smaller denominator id in either order.
• Share-family classification.
• A Hypothesis property: share conservation and order-freedom over random packages.
• The stamp-invariance differential.
• Column exclusion: exact drop, corrected columns compile, and every reason names both columns.
• The receipt rule is added to the synthetic five-rule feed and its Hypothesis completeness property.
• Pinned-feed:
  • the CD control test;
  • the CD-family invariant;
  • the receipt, 4 × 488 ids;
  • the fences, 31,376 / 059dc56d78db / 47dce807b412.

Not changed here (follow-ups filed)

• Chronicle: remap the CD package's four columns (to N18460/A18460 and N85770/A85770). A chip is filed. After a re-pin on the fix, the four register entries stop matching, and test_pinned_feed_exclusion_receipt flags them for deletion.
• Read restamped Chronicle facts at their data year before aging #1030: it needs the share narrowing above, plus its post-Choose the capital-gains rebase control by concept and period, not feed order #1036 rebase and re-pin (a chip will be filed).
• CD taxable interest is left as TY2022 levels on full, at 0.40x the national target. HT2 interest rows are rebased; CD rows are not. Same structure as capital gains, in the other direction; it needs its own analysis.
• The CD vintage guard _is_soi_congressional_district_record_set matches only congressional_district_2022. Another session is making it vintage-free. The new share-family code already uses the vintage-free _soi_record_set_family.

axiom: n/a: microsim target compilation, no policy rule changes.

🤖 Generated with Claude Code