Fill CBO total AGI for 2015-2018 with IRS SOI actuals

changelog.d/fix-cbo-agi-2016-2018-soi.fixed.md

@@ -0,0 +1 @@
+Fill total AGI in calibration.gov.cbo.income_by_source.adjusted_gross_income for 2015-2018 with IRS Statistics of Income actuals. The 2015 value was tax-year 2013 AGI excluding dependent filers and was held flat through 2018, so dollar inputs dated 2018 or earlier came out 13% to 29% too high when uprated to 2019 or later, and up to 12% too low when uprated to a year within 2016-2018.

policyengine_us/parameters/calibration/gov/cbo/income_by_source.yaml

@@ -1,4 +1,4 @@
-description: Historical data and projections of detailed adjusted gross income (AGI) by source (2019-2036)
+description: Historical data and projections of detailed adjusted gross income (AGI) by source from CBO (2019-2036), with IRS Statistics of Income actuals for total AGI in 2015-2018
 metadata:
   propagate_metadata_to_children: true
   unit: currency-USD
@@ -19,12 +19,33 @@ adjusted_gross_income:
   metadata:
     label: Total AGI
   values:
+    # IRS Statistics of Income actuals, all returns: adjusted gross income
+    # less deficit.
     2015-01-01:
-      value: 9_033_840_000_000
+      value: 10_210_310_102_000
       metadata:
         reference:
-        - title: Tax Foundation
-          href: https://taxfoundation.org/data/all/federal/summary-latest-federal-income-tax-data-2015-update/
+        - title: IRS SOI Tax Stats - Publication 1304 Table 1.1, tax year 2015, adjusted gross income less deficit
+          href: https://www.irs.gov/pub/irs-soi/15in11si.xls
+    2016-01-01:
+      value: 10_225_938_479_000
+      metadata:
+        reference:
+        - title: IRS SOI Tax Stats - Publication 1304 Table 1.1, tax year 2016, adjusted gross income less deficit
+          href: https://www.irs.gov/pub/irs-soi/16in11si.xls
+    2017-01-01:
+      value: 11_009_900_155_000
+      metadata:
+        reference:
+        - title: IRS SOI Tax Stats - Publication 1304 Table 1.1, tax year 2017, adjusted gross income less deficit
+          href: https://www.irs.gov/pub/irs-soi/17in11si.xls
+    2018-01-01:
+      value: 11_643_439_106_000
+      metadata:
+        reference:
+        - title: IRS SOI Tax Stats - Publication 1304 Table 1.1, tax year 2018, adjusted gross income less deficit
+          href: https://www.irs.gov/pub/irs-soi/18in11si.xls
+    # CBO from 2019.
     2019-01-01: 11_962_000_000_000
     2020-01-01: 12_650_000_000_000
     2021-01-01: 14_791_339_000_000

policyengine_us/tests/core/test_uprating_series.py

@@ -123,6 +123,22 @@ def test_series_is_defined_across_the_projection_window(series):
         assert _value(series, year) not in (None, 0)
 
 
+def test_default_series_has_a_value_for_every_year():
+    """A year with no value holds the previous one, so the per-capita index
+    drifts down with population and then jumps when values resume. CBO AGI
+    once had nothing for 2016 to 2018, so inputs dated 2018 came out 29% too
+    high by 2019."""
+    total = get_parameter(system.parameters, _authored_series(DEFAULT))
+    years = sorted(
+        {
+            int(value.instant_str[:4])
+            for value in total.values_list
+            if value.instant_str.endswith("-01-01")
+        }
+    )
+    assert years == list(range(years[0], years[-1] + 1))
+
+
 @pytest.mark.parametrize("year", (2025, 2030, 2035, 2036))
 def test_unemployment_compensation_grows_with_cbo_unemployment_outlays(year):
     soi = "calibration.gov.irs.soi.unemployment_compensation"