Document every result's baseline and make the projection baseline selectable (TR2008, TR2026, CBO 2026)

.gitattributes

@@ -33,5 +33,10 @@ data/external/psid_modern_job_context_raw_extraction_specs_v1.json text eol=lf
 data/external/snapshots/** binary
 data/external/snapshots/covered_earnings_methodology_capture1/** binary
 
+# Baseline input captures: SHA-256-pinned source bytes, never normalized
+data/external/tr2026/sources/** binary
+data/external/cbo2026/sources/** binary
+data/external/*.source.html.gz binary
+
 # The depletion-cut sources are byte-pinned to their fetch records (SHA-256).
 docs/analysis/pe_us_depletion_cut_20261001/sources/** binary

data/external/cbo2026/provenance.md

@@ -0,0 +1,146 @@
+# CBO 2026 baseline input provenance
+
+These are baseline demographic and economic inputs. No Social Security
+cost, income, balance or policy-option table was parsed. Source captures
+were provided read-only under `followup/inputs/baseline-inputs/`, captured
+2026-10-01. This builder fetched nothing from the web.
+
+## Captures and identities
+
+`sources.json` records original URLs, captured relative paths, locators,
+byte counts, full SHA-256, computed SHA-1/base32, archive attribution and
+retained compressed-source hashes. Every CBO file came from Internet
+Archive raw `id_` captures after direct requests served a DataDome CAPTCHA.
+The captured research lane recorded the CDX digests; this builder computed
+and matched those digests from the provided bytes. It did not re-query
+CDX or assume that the live files still match the captured vintages.
+
+| File | Wayback capture ID | Captured CDX SHA-1/base32 |
+| --- | --- | --- |
+| `57059-2026-01-Demographic-Projections.zip` | `20260211074115` | `AUU73LM3P3IH4ITJSNMHQM7T7UEHJCDD` |
+| `57054-2026-02-LTBO-econ.xlsx` | `20260726122828` | `OWABBCNJ5O2UJGAZDVBAK2SLWSCHOFLK` |
+| `51135-2026-02-Economic-Projections.xlsx` | `20260802034021` | `SLIHKFJTIHK62CO3YN5CYBRU4PIQCB4D` |
+| `62556-2026-Additional-Info.xlsx` | `20260919143154` | `DTGXZD5RVUA4T4SH3HTVHBHFOFS3DR3I` |
+
+The two demographic CSVs and `61879-Demographic-Projections.xlsx` were
+checked byte-for-byte against their members of the verified archive.
+Only these input rows are transcribed:
+
+- `fertilityRates_byYearAgePlace.csv`: single ages 14-49, 2021-2099, all,
+  native-born and foreign-born women; births per 1,000 females. JSON keeps
+  printed units; the accessor divides by 1,000 and returns births per
+  woman. It does not convert rates to Bernoulli probabilities.
+- `mortalityRates_byYearAgeSex.csv`: ages 0-119, both sexes, 2021-2099;
+  deaths per 1,000 people. The accessor divides by 1,000 to obtain q(x),
+  following the empirical life-expectancy check below.
+- `61879-Demographic-Projections.xlsx`, `1. Population and growth`: row 8
+  years, row 39 TFR, row 41 period life expectancy at birth, row 42 at 65;
+  2026-2099. This workbook remains external with a pinned hash.
+- `57054-2026-02-LTBO-econ.xlsx`, `1. Econ Vars_Annual Rates`: row 8
+  years, row 31 growth of real earnings per worker, row 40 CPI-U growth;
+  `3. Econ Vars_Annual Levels`: row 7 years, row 13 CPI-U (1982-84=1).
+  JSON converts the latter to 1982-84=100. Coverage is 1996-2056.
+- `51135-2026-02-Economic-Projections.xlsx`, `2. Calendar Year`: row 7
+  years, row 53 CPI-U levels (1982-84=100); 2023-2036.
+- `62556-2026-Additional-Info.xlsx`, `1. Covered Workers`: numeric year
+  rows, male/female/total workers in thousands; `2. Covered and Taxable
+  Earnings`: numeric year rows, covered earnings column B in trillions
+  of dollars; 2026-2100. Beneficiary, interest-rate and taxable-payroll
+  columns are not transcribed; workbook footnotes are not displayed.
+  The workbook's sheets are `Contents`, `1. Covered Workers`,
+  `2. Covered and Taxable Earnings`, `3. OASDI Beneficiaries` and
+  `4. Average Interest Rates`; only sheets 1 and 2 are read. It holds
+  no outlay, revenue, cost-rate or balance table (CBO's main
+  62556 projections workbook, which does, was not captured).
+- The AWI anchor is CY2024 historical AWI, captured TR2026 single-year
+  VI.G1 (`tr2026/lr6g1.html`), SHA-256
+  `2e0214278e0b2271616093af02280277f6e363d112c47a90b49fe7168e795e75`.
+  Nonempty cell order is year, adjusted CPI, AWI. An AWI header check and
+  a differential test against the separate TR2026 accessor guard the
+  column selection. The SSA capture lane used User-Agent `Wget/1.21.4`
+  after a browser User-Agent returned 403. HTML has per-request Akamai
+  markup; exact captured bytes and parsed values are pinned, without
+  assuming future fetches will hash identically.
+
+Stable raw CSV/XLSX/PDF captures are retained under `sources/` only when
+the payload is below 2 MB and deterministic gzip is below 500 KB, the
+repository's stricter precommit limit. Compression uses
+`GzipFile(filename='', mtime=0)`; hashes of payload and compressed bytes
+are recorded. Larger inputs and HTML carry URL, captured path and hash
+only. Rebuilding external inputs requires the captures via `--inputs`.
+
+## Mortality interpretation
+
+CBO labels mortality "deaths per 1,000 people" without an explicit q/m
+definition. Both interpretations were checked against every published
+2026-2099 period life expectancy at birth and 65:
+
+1. q(x) = printed rate / 1,000.
+2. m(x) = printed rate / 1,000, converted using q = m/(1 + m/2).
+
+For each sex, a period life table uses uniform deaths within each year,
+including infancy (`f0=0.5`). Arithmetic mean male/female life expectancy
+reproduces the combined-sex summary. This averaging convention and q
+interpretation are empirical findings, not quoted CBO definitions. Every
+source row has q(119)=1 in the q interpretation, with terminal
+person-years l(119)/2.
+
+`transcription_check.json` records all published/recomputed values and
+both interpretations' errors. Maximum q error is 0.000703364 year;
+minimum m error is 0.241771105 year. q passes every summary within
+0.0011 year; every m interpretation fails. The tolerance allows rounded
+CSV rates and three-decimal published life expectancy. Tests compare
+forward life-table columns with independent backward recursion as well.
+
+## Fertility check
+
+Summed all-women ASFR over ages 14-49, divided by 1,000, matches each
+published TFR within 0.0023 births per woman. The observed maximum error
+is 0.0005. The conservative rounding bound is 36 age rates at half of
+0.1 birth per 1,000 plus half of 0.001 in the summary TFR. Native-born
+and foreign-born schedules remain published schedules; no unweighted
+averaging creates the all-women schedule. Before 2026, TFR is explicitly
+derived from the all-women ASFR sum.
+
+## Builder defaults awaiting ratification
+
+The module docstring and per-value metadata name these model-builder
+defaults, each awaiting ratification:
+
+- `ten_year_cpiu_then_long_term_growth`: ten-year CPI-U levels through
+  2036, growth from adjacent levels; 2023 uses the 2022 LT denominator.
+  After 2036 compound published LT growth on the final ten-year level.
+- `hold_2056_economic_growth`: final LT CPI-U and real earnings growth
+  held through 2100, with CPI levels compounded continuously. No
+  demographic extension after 2099 is supplied.
+- `cpiw_equals_cpiu_growth` and `cola_equals_annual_cpiu_growth`: annual
+  CPI-U growth proxies for series CBO does not publish. This is not a
+  statutory Q3-to-Q3 COLA computation. Derived COLA covers 2024-2100.
+- `awi_bridge_2025_2026_real_earnings_times_cpiu`: compound the actual
+  2024 AWI with `(1 + real_growth/100) * (1 + cpiu_growth/100)` in 2025
+  and 2026. Both bridge years are needed because covered inputs begin
+  in 2026 and have no 2025 denominator.
+- `awi_growth_of_covered_earnings_per_worker`: from 2027 apply growth in
+  covered earnings divided by covered workers to the 2026 bridge.
+  1e12 dollars/1e3 workers gives nominal dollars per worker; the unit
+  factor cancels in growth ratios. Covered inputs continue to 2100;
+  AWI needs no post-2056 economic extrapolation. Every analog value is
+  tagged derived; only the 2024 anchor is actual.
+- `tfr_sum_asfr_before_2026`: use all-women ASFR sums in 2021-2025,
+  when no summary TFR is published; published TFR thereafter.
+
+## Deterministic reproduction
+
+From the repository root:
+
+```sh
+PYTHONPATH=src /Users/maxghenis/PolicyEngine/microcosm-dynamics/.venv/bin/python scripts/extract_cbo2026_parameters.py --inputs /path/to/baseline-inputs --check
+```
+
+The builder re-parses every pinned source, verifies stated CDX digests
+and demographic archive membership, and recomputes mortality/fertility
+checks. `--check` compares every JSON byte and retained compressed hash,
+and writes nothing. Without `--check`, it rebuilds JSON and eligible raw
+captures. The accessor separately verifies `FILE_SHA256` for every
+consumed JSON on first load. A changed source requires an intentional
+rebuild and repin; stale or outside-coverage data are refused.

data/external/cbo2026/sources.json

@@ -0,0 +1,16 @@
+{
+  "capture_method": "CBO raw Internet Archive id_ captures; original cbo.gov served DataDome CAPTCHA; no live download attempted by this builder. Source identities from captured research summary; computed SHA-1/base32 matches every stated CDX digest.",
+  "compression": "gzip mtime=0; hashes refer to uncompressed bytes",
+  "retention": "Raw sources retained only if payload <2MB and deterministic gzip <500KB (repository precommit limit); otherwise URL/hash only",
+  "schema_version": "cbo2026_sources.v1",
+  "sources": {
+    "actual_awi_anchor": {"archive_source_key": null, "bytes": 226764, "capture_date": "2026-10-01", "captured_relative_path": "tr2026/lr6g1.html", "cdx_digest_matches": null, "cdx_sha1_b32": null, "committed_file": null, "compressed_sha256": null, "computed_sha1_b32": "JOIWG64Q3FURI4SHN2BQB36EPOCMLJSQ", "locator": "TR2026 Table VI.G1, Historical data, CY2024 AWI column", "original_url": "https://www.ssa.gov/oact/TR/2026/lr6g1.html", "sha256": "2e0214278e0b2271616093af02280277f6e363d112c47a90b49fe7168e795e75", "wayback_timestamp": null},
+    "covered_earnings": {"archive_source_key": null, "bytes": 36896, "capture_date": "2026-10-01", "captured_relative_path": "cbo/62556-2026-Additional-Info.xlsx", "cdx_digest_matches": true, "cdx_sha1_b32": "DTGXZD5RVUA4T4SH3HTVHBHFOFS3DR3I", "committed_file": "sources/62556-2026-Additional-Info.xlsx.gz", "compressed_sha256": "27d44ad6297248558c2034754f3b08e1a9752a7cb22e2255f8617fa7454740fc", "computed_sha1_b32": "DTGXZD5RVUA4T4SH3HTVHBHFOFS3DR3I", "locator": "'1. Covered Workers': numeric year rows,male/female/total (thousands); '2. Covered and Taxable Earnings': numeric year rows,covered earnings column B (trillions of dollars)", "original_url": "https://www.cbo.gov/system/files/2026-09/62556-2026-Additional-Info.xlsx", "sha256": "7a7247651f3ae6820c9263a16f11b9cc9cf24cd7dcb9e18bb76e44711924d73b", "wayback_timestamp": "20260919143154"},
+    "demographic_archive": {"archive_source_key": null, "bytes": 4904312, "capture_date": "2026-10-01", "captured_relative_path": "cbo/57059-2026-01-Demographic-Projections.zip", "cdx_digest_matches": true, "cdx_sha1_b32": "AUU73LM3P3IH4ITJSNMHQM7T7UEHJCDD", "committed_file": null, "compressed_sha256": null, "computed_sha1_b32": "AUU73LM3P3IH4ITJSNMHQM7T7UEHJCDD", "locator": "Archive containing the demographic CSV files and workbook", "original_url": "https://www.cbo.gov/system/files/2026-01/57059-2026-01-Demographic-Projections.zip", "sha256": "083ea91701c1af901cd378211a34c8c8a941c8833c2507564af89551d63392bf", "wayback_timestamp": "20260211074115"},
+    "demographic_workbook": {"archive_source_key": "demographic_archive", "bytes": 2237636, "capture_date": "2026-10-01", "captured_relative_path": "cbo/extracted/57059-2026-01-Demographic-Projections/61879-Demographic-Projections.xlsx", "cdx_digest_matches": null, "cdx_sha1_b32": null, "committed_file": null, "compressed_sha256": null, "computed_sha1_b32": "ZNHSZNBNLIBOBZECU27AQRN4P53LOU5O", "locator": "61879-Demographic-Projections.xlsx: '1. Population and growth'; row 8 years; r39 TFR; r41 life expectancy at birth; r42 at 65", "original_url": "https://www.cbo.gov/system/files/2026-01/57059-2026-01-Demographic-Projections.zip", "sha256": "aca3322e0a74fee39e669d953201075887da13e5e3792e08b8b6f7d1f86d1006", "wayback_timestamp": null},
+    "fertility": {"archive_source_key": "demographic_archive", "bytes": 206838, "capture_date": "2026-10-01", "captured_relative_path": "cbo/extracted/57059-2026-01-Demographic-Projections/CSV Files/fertilityRates_byYearAgePlace.csv", "cdx_digest_matches": null, "cdx_sha1_b32": null, "committed_file": "sources/fertilityRates_byYearAgePlace.csv.gz", "compressed_sha256": "903fff07de1aecea2dee059f8246d8230f0a81dbba9f95a05aff473bd08406f7", "computed_sha1_b32": "MGZI7VZYHYFWFEXSXCJESRUFGWKSZDEW", "locator": "CSV Files/fertilityRates_byYearAgePlace.csv: year,age,place_of_birth,births_per_1000_females", "original_url": "https://www.cbo.gov/system/files/2026-01/57059-2026-01-Demographic-Projections.zip", "sha256": "8ed8615ed7561d6c08c3005dc516b8dd78c16072020e8c106a5738a5563b0f1c", "wayback_timestamp": null},
+    "long_term_economics": {"archive_source_key": null, "bytes": 572138, "capture_date": "2026-10-01", "captured_relative_path": "cbo/57054-2026-02-LTBO-econ.xlsx", "cdx_digest_matches": true, "cdx_sha1_b32": "OWABBCNJ5O2UJGAZDVBAK2SLWSCHOFLK", "committed_file": "sources/57054-2026-02-LTBO-econ.xlsx.gz", "compressed_sha256": "13caae990b76e28a36bf5c6fde34ad6e9b014ee8752ff013ae38b283c0391021", "computed_sha1_b32": "OWABBCNJ5O2UJGAZDVBAK2SLWSCHOFLK", "locator": "'1. Econ Vars_Annual Rates': r8 years,r31 real earnings growth,r40 CPI-U growth; '3. Econ Vars_Annual Levels': r7 years,r13 CPI-U", "original_url": "https://www.cbo.gov/system/files/2026-02/57054-2026-02-LTBO-econ.xlsx", "sha256": "6f6e66d7e2aaf187e0d13aa90767a39e682a6a5ab2be87b0cc2221b7370e6929", "wayback_timestamp": "20260726122828"},
+    "mortality": {"archive_source_key": "demographic_archive", "bytes": 410627, "capture_date": "2026-10-01", "captured_relative_path": "cbo/extracted/57059-2026-01-Demographic-Projections/CSV Files/mortalityRates_byYearAgeSex.csv", "cdx_digest_matches": null, "cdx_sha1_b32": null, "committed_file": "sources/mortalityRates_byYearAgeSex.csv.gz", "compressed_sha256": "58d5114d019ff91fa0ecf4938a523af4023f8ed2131feb3e0726d4c9fbeab61f", "computed_sha1_b32": "R3756O6XYAYQGFEUGRPJOWZLIQJ3LBTM", "locator": "CSV Files/mortalityRates_byYearAgeSex.csv: year,age,sex,deaths_per_1000_people", "original_url": "https://www.cbo.gov/system/files/2026-01/57059-2026-01-Demographic-Projections.zip", "sha256": "378defc1d6eee7b6870cc7ddaf534dd782d51bf3f686a39a8fd7cd7cba2beb63", "wayback_timestamp": null},
+    "ten_year_economics": {"archive_source_key": null, "bytes": 1503579, "capture_date": "2026-10-01", "captured_relative_path": "cbo/51135-2026-02-Economic-Projections.xlsx", "cdx_digest_matches": true, "cdx_sha1_b32": "SLIHKFJTIHK62CO3YN5CYBRU4PIQCB4D", "committed_file": null, "compressed_sha256": null, "computed_sha1_b32": "SLIHKFJTIHK62CO3YN5CYBRU4PIQCB4D", "locator": "'2. Calendar Year': r7 years; r53 CPI-U (1982-84=100)", "original_url": "https://www.cbo.gov/system/files/2026-02/51135-2026-02-Economic-Projections.xlsx", "sha256": "ae8f4920702fabf8fb3136bc94a42c53466cff5e890dec22396d8dc49dc2f776", "wayback_timestamp": "20260802034021"}
+  }
+}