Label IRS SOI N01000/A01000 as Form 1040 line 7 capital gain or loss

docs/architecture.md

@@ -141,8 +141,9 @@ Consumer-facing source vocabulary is a compatibility contract. Keep
 concept has not changed; put publication-specific wording in `source_table` and
 labels instead. Within a `source_name`, `measure_id` must distinguish different
 statistical measures even when their publisher columns share labels such as
-`band_a` or `total`. A deliberate rename requires a migration note and a
-regression test for the affected consumer selector.
+`band_a` or `total`. A deliberate rename requires a migration note in
+[concept migrations](concept-migrations.md) and a regression test for the
+affected consumer selector.
 
 The `policyengine_chronicle.normalization` package owns low-assumption representation helpers:
 

docs/concept-migrations.md

@@ -0,0 +1,76 @@
+# Concept migrations
+
+Consumer-facing source vocabulary is a compatibility contract
+(see [architecture](architecture.md#source-facts-and-microcosm-targets)). A
+deliberate concept rename is recorded here, with the regression test that pins
+the affected consumer selector. Changing a concept changes each affected
+consumer row's `observed_measure.source_concept`, `observed_measure_key`,
+`aggregate_fact_key`, `semantic_fact_key`, `legacy_fact_key` and generated
+`label`; a label change also moves `layout.measure_label`. `measure_id`,
+`source_record_id`, values and lineage do not change.
+
+## 2026-09-27: IRS SOI Form 1040 line 7 capital gain or (loss)
+
+| Packages | Columns | Old concept | New concept |
+|---|---|---|---|
+| `soi-historic-table-2`, `soi-historic-table-2-state-broad-2022` | `N01000` | `irs_soi.returns_with_taxable_net_capital_gains` | `irs_soi.returns_with_form_1040_capital_gain_or_loss` |
+| same | `A01000` | `irs_soi.taxable_net_capital_gains` | `irs_soi.form_1040_capital_gain_or_loss` |
+| `soi-congressional-district-2022` | `N01000` | `irs_soi.returns_with_net_capital_gains` | `irs_soi.returns_with_form_1040_capital_gain_or_loss` |
+| same | `A01000` | `irs_soi.net_capital_gains` | `irs_soi.form_1040_capital_gain_or_loss` |
+
+Labels become the IRS wording, "Returns with net capital gain (less loss)" and
+"Net capital gain (less loss)". `measure_id` stays `net_capital_gains_returns` /
+`net_capital_gains_amount`.
+
+**Why.** The IRS documentation guides for Historic Table 2 (TY2020–TY2023) and
+for the TY2022 congressional-district, ZIP and county files define `N01000` as
+"Number of returns with net capital gain (less loss)" and `A01000` as "Net
+capital gain (less loss) amount", both from Form 1040 line 7, "Capital gain or
+(loss)". That line holds a Schedule D gain, a Schedule D loss limited to $3,000
+($1,500 married filing separately), or capital gain distributions reported
+without a Schedule D.
+
+- The Historic Table 2 packages had borrowed the concept of Table 1.4 columns
+  37/38 (25/26 before TY2022), "Sales of capital assets reported on Form 1040, Schedule D: Taxable
+  net gain", which counts only Schedule D returns with a gain. For TY2022 the
+  two counts are 30,465,850 and 12,915,122 returns. Historic Table 2's US count
+  equals Table 1.4's capital-gain-distribution, taxable-net-gain and
+  taxable-net-loss returns combined to within 0.25% in every year TY2020–TY2023
+  (the TY2022 identity is checked in CI against Chronicle's registered files).
+- The congressional-district ids read as a gain-only amount. IRC section
+  1222(11) and the Publication 1304 Explanation of Terms both use "net capital
+  gain" for a positive amount only.
+- The ids name the Form 1040 line rather than repeat the IRS phrase "net
+  capital gain (less loss)". Publication 1304 Table A uses that phrase for
+  Schedule D gain and loss returns without the distribution-only returns
+  (TY2022: 26,480,998), so the phrase alone does not identify the population.
+
+**Semantic keys.** Historic Table 2 and the congressional-district file now
+share one concept for the same IRS variable. Before this change that held for 26
+of the 38 IRS variables the national package shares with the
+congressional-district file (24 of 36 for the state package); the capital-gains
+pair was 2 of the same 12 exceptions in both. Their TY2022 state and US rows therefore share semantic
+keys, which adds 104 semantic-duplicate keys to the default bundle (51 states
+and the US, returns and amount). The values differ: the IRS guide for the
+congressional-district file says its state totals "may not be comparable to
+State totals published elsewhere by SOI because of disclosure protection
+procedures or the exclusion of returns that did not match based on the ZIP
+code." Historic Table 2 and Table 1.4 capital-gain rows for the same year can
+no longer share a semantic key. In the default bundle they did not collide,
+because Historic Table 2 builds at TY2022 and Table 1.4 at TY2023, but a TY2022
+build of both did.
+
+**Unchanged.** Table 1.4 keeps `irs_soi.returns_with_taxable_net_capital_gains`
+/ `irs_soi.taxable_net_capital_gains`. `measure_id` stays
+`net_capital_gains_returns` / `net_capital_gains_amount` in all four packages
+because Microcosm selects on it, so one `measure_id` still names two different
+measures; chronicle#307 tracks giving them distinct ids together with the
+Microcosm selector change, and the 10 other IRS columns that Historic Table 2
+and the congressional-district file still label with different concepts.
+
+**Pinned by** `tests/test_chronicle_soi_capital_gain_concepts.py`: every
+`irs_soi` measure (any vintage) that names `N01000`/`A01000` in its
+`source_column_id` or `expected_column_header` guard must declare the new
+concept, only Table 1.4 may declare the Schedule D gain concept, the retired
+ids may not reappear, and the consumer selector (`source_measure_id`) and
+published values are unchanged.

packages/irs_soi/congressional_district_2022/source_package.yaml

@@ -14124,24 +14124,24 @@ record_sets:
     value_scale: 1000
     expected_cell_type: number
   - measure_id: net_capital_gains_returns
-    label: Returns with net capital gains
+    label: Returns with net capital gain (less loss)
     ordinal: 13
     column: AL
     source_column_id: N01000
     expected_column_header_row: 1
     expected_column_header: N01000
-    concept: irs_soi.returns_with_net_capital_gains
+    concept: irs_soi.returns_with_form_1040_capital_gain_or_loss
     unit: count
     aggregation: sum
     expected_cell_type: number
   - measure_id: net_capital_gains_amount
-    label: Net capital gains
+    label: Net capital gain (less loss)
     ordinal: 14
     column: AM
     source_column_id: A01000
     expected_column_header_row: 1
     expected_column_header: A01000
-    concept: irs_soi.net_capital_gains
+    concept: irs_soi.form_1040_capital_gain_or_loss
     unit: usd
     aggregation: sum
     value_scale: 1000

packages/irs_soi/historic_table_2/source_package.yaml

@@ -559,23 +559,23 @@ record_sets:
         value_scale: 1000
 
       - measure_id: net_capital_gains_returns
-        label: Returns with taxable net capital gains
+        label: Returns with net capital gain (less loss)
         ordinal: 27
         column: AK
         source_column_id: N01000
-        concept: irs_soi.returns_with_taxable_net_capital_gains
+        concept: irs_soi.returns_with_form_1040_capital_gain_or_loss
         unit: count
         aggregation: sum
         expected_cell_type: number
         expected_column_header_row: 1
         expected_column_header: N01000
 
       - measure_id: net_capital_gains_amount
-        label: Taxable net capital gains
+        label: Net capital gain (less loss)
         ordinal: 28
         column: AL
         source_column_id: A01000
-        concept: irs_soi.taxable_net_capital_gains
+        concept: irs_soi.form_1040_capital_gain_or_loss
         unit: usd
         aggregation: sum
         expected_cell_type: number

packages/irs_soi/historic_table_2_state_broad_2022/source_package.yaml

@@ -368,22 +368,22 @@ record_sets:
     expected_column_header: A00900
     value_scale: 1000
   - measure_id: net_capital_gains_returns
-    label: Returns with taxable net capital gains
+    label: Returns with net capital gain (less loss)
     ordinal: 17
     column: AK
     source_column_id: N01000
-    concept: irs_soi.returns_with_taxable_net_capital_gains
+    concept: irs_soi.returns_with_form_1040_capital_gain_or_loss
     unit: count
     aggregation: sum
     expected_cell_type: number
     expected_column_header_row: 1
     expected_column_header: N01000
   - measure_id: net_capital_gains_amount
-    label: Taxable net capital gains
+    label: Net capital gain (less loss)
     ordinal: 18
     column: AL
     source_column_id: A01000
-    concept: irs_soi.taxable_net_capital_gains
+    concept: irs_soi.form_1040_capital_gain_or_loss
     unit: usd
     aggregation: sum
     expected_cell_type: number

tests/test_chronicle_bundle.py

@@ -144,8 +144,11 @@ def test_build_bundle_writes_merged_consumer_contract(tmp_path):
         # file's state-total and US rows share semantic keys with the Historic
         # Table 2 rows for the same TY2022 cells (two IRS publications of one
         # cell): 1,560 new duplicate keys among the changed packages' own
-        # builds, for a bundle-wide net of +1,555.
-        "semantic_duplicate_key_count": 2022,
+        # builds, for a bundle-wide net of +1,555. The HT2 and CD N01000/A01000
+        # rows then took one Form 1040 line 7 concept
+        # (docs/concept-migrations.md), adding 104 more: 51 states and the US
+        # for returns and amount.
+        "semantic_duplicate_key_count": 2126,
         "skipped_source_count": 10,
         "source_count": 50,
         "source_package_count": 227,
@@ -1231,7 +1234,7 @@ def test_build_bundle_writes_merged_consumer_contract(tmp_path):
         "tax_unit": 41368,
     }
     assert not coverage["duplicates"]["aggregate_fact_keys"]
-    assert len(coverage["duplicates"]["semantic_fact_keys"]) == 2022
+    assert len(coverage["duplicates"]["semantic_fact_keys"]) == 2126
     assert Counter(warning["code"] for warning in summary["warnings"]) == {
         "conflicting_geography_name_across_packages": 50,
         "conflicting_groupby_value_label": 16,