Repository navigation
Backdate parameters before applying reforms - #9628
Merged
Merged
Conversation
CountryTaxBenefitSystem.__init__ backdated every parameter to 2015 (copying its earliest value back) after applying the reform set. For a parameter first dated after 2015, a reform starting on or before that first date was copied back to 2015, and a reform ending before it left the years in between undefined, which dropped brackets from scales and could raise ParameterNotFoundError. Backdate the baseline tree right after the parameter pipeline instead, before the reform is applied. Reforms still apply after uprating and interpolation (#9075), and structural-reform detection still reads reformed values, now on a backdated tree, so a start_instant before 2024 no longer raises ParameterNotFoundError. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Independent review found that a reform adding a parameter (guarded, because __init__ applies a reform twice) was backdated on main but not after moving backdating ahead of the reform, so a formula reading it before its first date raised ParameterNotFoundError. Backdate again after the first reform application: every baseline parameter already has a value dated by 2015 and Parameter.update never removes earlier values, so this pass reaches only parameters the reform added. Tests now check two invariants over every backdated numeric and boolean parameter outside gov.contrib (46,615): a reform changes a parameter only inside its own period, and building the system with a reform matches applying the same reform to the finished system. Both fail on main for 45,937 parameters. The targeted tests compare whole series, cover an app-style reform starting before a breakdown parameter's first date, a formula reading the year after a bounded reform, the Simulation path with start_instant, and parametrized guards on each test parameter's first dated value. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Resolves the conflict in policyengine_us/system.py with main's FIRST_MODELED_YEAR (#9626): the PR's two backdating passes now use f"{FIRST_MODELED_YEAR}-01-01" instead of "2015-01-01", and main's call after structural-reform detection is the one this PR moves before the reform. The second-pass comment now says it reaches parameters the reform added or replaced (review round 2, finding 3). Correction to a7fd4ad's message: on the pre-merge base 9e5852a, both exhaustive tests fail on main's ordering for 45,941 parameters, not 45,937. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review round 2 (findings 2, 4 and 5): - test_reform_at_construction_matches_afterwards_for_every_parameter compares every parameter, not only the ones the reform sets, between building with the reform and applying it afterwards. The derived <national total>_per_capita series are skipped: construction rebuilds them from the reformed totals and Reform.from_dict does not. - The selection-size guard moves into the shared fixture, so neither exhaustive test can pass on an empty selection. - shifted() moves Feb 29 to Feb 28 in a non-leap year instead of raising; test_shifted_moves_february_29_to_february_28 covers it. - Locals no longer shadow the module's reformed fixture. - The first backdated year comes from FIRST_MODELED_YEAR (#9626). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
backdated_first_date picks a parameter whose 2015-01-01 value repeats its next value. That is every backdated parameter, and also any parameter whose YAML dates 2015-01-01 itself with the same value as its next date: 178 of the 42,489 selected at a50e1e3, mostly state sales tax table cells. The invariants hold for those too, so the selection stays; the comments now say so. The module docstring says the second pass reaches parameters a reform added or replaced (review round 2, finding 3). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 6, 2026
The changelog fragment now says parameters a reform adds or replaces are backdated afterwards, as the second-pass comment in system.py does (review round 2, finding 3). Correction to 743b6df's message: 176 of the 42,489 selected parameters only look backdated (139 state sales tax table cells, 36 Arkansas low-income tax table entries and the SSA PIA threshold), not 178. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
US + core hub merge audit, us#9628 at
|
This was referenced Oct 6, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CountryTaxBenefitSystem.__init__backdated every parameter to 2015 after applying the reform.backdate_parameters(policyengine_us/tools/parameters.py) copies each parameter's earliest value back to 2015-01-01, so for a parameter first dated after 2015 it copied whatever the reform had just made earliest. 44,424 parameters are first dated after 2015-01-01 (715 of them later in 2015), counted ata50e1e3a7c. They include most parameters OBBBA added, such as the tip deduction cap. OBBBA switches with a0000-01-01sentinel or an earlier value, such asgov.irs.deductions.itemized.limitation.obbb.applies(first dated 2009), are never backdated. Two things went wrong:2026-01-01.2100-12-31) does this to every one of the 1,863govparameters first dated on or after 2026-01-01.Parameter.updatewritesNoneafter the reform's stop date when no earlier value exists. Scales lost that bracket, and formulas reading the parameter raisedParameterNotFoundError. When the reform started after 2015-01-01, its value was also copied back to 2015.This PR backdates the baseline tree immediately after the parameter pipeline, before the reform is applied. Parameters a reform adds or replaces are still backdated after it is applied, as before. Reforms are still applied after uprating and interpolation, so #9075 stays fixed, and structural-reform detection still reads reformed values.
Side effect:
CountryTaxBenefitSystem(start_instant=...)andSimulation(reform=..., start_instant=...)now build with any start from 2015-01-01 through 2023-12-31. Onmainthey raiseParameterNotFoundErrorfor every such start, even with no reform. The cause is that detection read contrib parameters (a switch, a threshold and a rate) first dated after the start before they were backdated. Starts before 2015 still raise under both orderings.Found while reviewing #9604, which documents the ordering as pre-existing and out of scope.
Repro
The change
The order in
__init__was: pipeline (load, homogenize, interpolate, uprate, default uprating) → apply reform → detect structural reforms → backdate → resetmodifiedflags → apply reform and structural reforms again.It is now: pipeline → backdate → apply reform → backdate → detect structural reforms → reset
modifiedflags → apply reform and structural reforms again.Both passes backdate to
FIRST_MODELED_YEAR(2015, from #9626).[2015-01-01, first dated value)with each baseline parameter's first dated value. Uprating has already run, so a backdated value cannot act as a "next defined value" and freeze uprating the way Reforms starting at a future date freeze uprating of the reformed parameter for intervening years #9075 did. The existing Reforms starting at a future date freeze uprating of the reformed parameter for intervening years #9075 tests pass.values_listas a replacement. A parameter the reform only updated already has a value dated by 2015, andParameter.updatenever removes values dated before the period it sets, so the second pass leaves it alone.a50e1e3a7c, I wrappedbackdate_parametersto record what each pass reaches:Reformclass that adds one parameter, the second pass reached exactly that parameter.main:Reformclass replaces (gov.irs.deductions.tip_income.phase_out.rate);values_listaReformclass reassigns (gov.irs.deductions.auto_loan_interest.phase_out.increment).__init__applies a reform twice) left that parameter undefined before its first date. Onmainit was backdated, so a 2024 formula reading it worked; without the second pass it raisedParameterNotFoundError.Invariants
These hold for every reform that sets values of existing parameters, other than the 19 derived
<national total>_per_capitaseries, and switches on nogov.contribstructural reform. That covers dict, app and API reforms that leave the contrib switches off, whichReform.from_dictapplies throughParameter.update.CountryTaxBenefitSystem.__init__, and again inpolicyengine_us.Simulation.__init__andMicrosimulation.__init__. It is not installed whenReform.from_dictor the YAML runner applies a reform to a finished system. So building with the reform and applying it afterwards can differ in what that contrib reform installs; the tests leavegov.contribout for this reason.Reformclass that adds a parameter or bracket, or replaces a parameter object or itsvalues_list, has its first dated value copied back to 2015 by the second pass, exactly as onmain, so 1 and 3 don't hold for what it adds or replaces.None. This follows from 1.CountryTaxBenefitSystem(reform=r)gives the same values asReform.from_dict(r)applied to the finished system. The YAML runner (throughset_parameter) and household-api likewise apply reforms to a built system throughParameter.update. The exception is the derived<national total>_per_capitaseries: construction rebuilds them from the reformed totals (add_per_capita_uprating), whileReform.from_dictapplied afterwards sets values throughParameter.updatealone. Under the exhaustive tests' reform, 1 of the 19 derived series differs on Jan 1 of 2015–2035:calibration.gov.cbo.income_by_source.employment_income_per_capitain 2018 is 37,207.11 at construction and 24,804.74 afterwards. That gap predates this PR and has nothing to do with backdating; see "Related" below.main's.Three tests check 1 and 3. They share one reform covering the 42,489 numeric and boolean parameters outside
gov.contribthat look backdated, meaning their 2015-01-01 value repeats their next value.The three tests:
test_every_backdated_parameter_changes_only_in_its_reform_periodchecks 1.test_reform_at_construction_matches_reform_applied_afterwardschecks 3 for the reformed parameters.test_reform_at_construction_matches_afterwards_for_every_parameter(new in this round) checks 3 for every parameter in the tree, apart from the 19 derived series: 104,727 of the 104,746 leaves.For dict reforms, 1 also follows from
Parameter.update's code once every parameter has a value from 2015.main(ea79c39fb7): all three fail for 41,635 parameters. An earlier version of this description said 45,937; round 2 measured 45,941 on the older base9e5852a337. The count has fallen because main has since given about 6,000 late-dated parameters values from 2015-01-01. Almost all are the 5,358 cells of the IRS state sales tax tables, which main has extended back to 2015 (Correct the IRS Optional State Sales Tax Table and add 2022-2025 values #9616, Add the IRS Optional State Sales Tax Tables for 2018-2021 #9631, Add the IRS Optional State Sales Tax Tables for 2015-2017 #9651), and 650 entries of Arkansas's low-income tax tables (Give Arkansas 2015-2020 income tax parameters their own values #9615). Main has also added about 1,900 parameters first dated after 2015, over half of them under Kansas LIHEAP (gov.states.ks.dcf). Net, 44,424 parameters are first dated after 2015 now, against 48,730 then; both counts fell by 4,306.For (4), the first round replayed
main's__init__verbatim in a subclass ate8bf23a958. Across all 102,627 parameter leaves at Jan 1 and Jul 1 of 2010–2035 there were 0 differences. There were also 0 differences in any variable's formulas,defined_for,addsorsubtracts. This round compared every parameter's full value list betweenmainand this branch on the default-dataset runs: see "Impact" below.Impact
Four microsimulations on the default dataset (
populace_us_2024.h5@populace-us-2024-spm-20260909), run one at a time:main(ea79c39fb7, this branch's merge base) and this branch, each with no reform and with the Georgia reform from the first row of "Results that change" below, for 2025 and 2026. Each run saved every record's value of eight variables (household_net_income,household_tax,household_benefits,income_tax,state_income_tax,spm_unit_net_income,child_care_subsidies,ga_child_care_subsidies) and every parameter's full value list, and the runs were compared record by record and leaf by leaf. The branch's baseline run was at743b6dfd3dand its reform run at81e427cf85; the only change between those commits is the changelog fragment's wording.Baseline: no change.
Reform: the Georgia CAPS family fee rate for income above 100% FPL, 7% → 3% as
2026-01-01.2100-12-31. The parameter is first dated 2026-03-02.mainand this branch. Its 2015-01-01 entry is 0.03 onmain, the reform value copied back, and 0.07 here, the baseline value.main, and nothing on this branch:mainga_child_care_subsidiesandchild_care_subsidies(SPM unit)household_benefitsincome_tax(tax unit)state_income_tax(tax unit)household_taxhousehold_net_incomespm_unit_net_incomemainand this branch give the same value for every record of every variable. The same SPM unit gets +$1,612.00 ofga_child_care_subsidies, and its household +$1,443.98 of net income, under both.mainadded $3.99 to 2025's weighted Georgia child care subsidies of $72.1 million. The household example under "Results that change" shows the size for one family.Other things that could change, and what I checked
These checks ran in round 1, on
e8bf23a958(thenmain) plus the first version of this change, and were not re-run on later heads. The exception is the "Related" reproductions at743b6dfd3d, which re-confirm the five EITC crashes andut_hb210_s2on this branch. This round re-ran the tests and the counts ata50e1e3a7c, and the impact runs at743b6dfd3dand81e427cf85. The two later commits change only comments and docstrings in the test file and the changelog fragment's wording.create_structural_reforms_from_parameterscalls 140 creators, which read 143 parameters, starting atstart_instant(some also read the next four years). 121 have a0000-01-01sentinel, which backdating never touches.falseor0, andcbo.payroll.secondary_earnings_thresholdstarts at infinity, which its creator treats as off. So backdating them cannot switch a reform on.in_effectinside the formula.gov.contribin_effectflags that build under both orderings, as2026-01-01.2100-12-31and asyear:2026:1. Triggered reforms, variable formulas and parameter values were identical.2019-01-01.2100-12-31,mainraised ondc_kccatc.active. So I also pinned five detection parameters (dc_kccatc.active,cbo.payroll.secondary_earnings_threshold, the twobiden.budget_2025rates,wyden_smith.actc_lookback) to their baseline values from 2019. Then the triggered reforms were identical. The only parameter differences were the 24in_effectflags first dated 2019–2025, whichmainset totruefor 2015–2018 by copying the reform's value back.policyengine_us/reforms/that callmodify_parameters. Passed asreform=, 22 give identical parameter trees and variables under both orderings;ut_hb210_s2crashes under both (see "Related" below).modifiedflags: still reset after backdating and the first reform application, and set again by the second. A test checks that reformed parameters carry the flag and untouched backdated ones do not.e8bf23a958): I perturbed all 48,127 non-list late-dated parameters in five reform shapes, one build per shape, and read each at Jan 1 and Jul 1 of 2015–2035:NonegapNonegap<year before>-01-01.2100-12-31(app format)Who is affected
Every path that builds the system with the reform in it.
Simulation(reform=...)andMicrosimulation(reform=...)go throughspm.pytoCountryTaxBenefitSystem(reform=...). That covers:Simulationsignature check falls back toreform=for the US;These are not affected, because they apply the reform to an already backdated system:
tax_benefit_system=.policyengine-api and the api-v2 executor both pin
policyengine[models]==6.2.1, which pinspolicyengine-us==2.2.1(checked 2026-10-06). That release has the same ordering. policyengine.org gets this fix once they move to a release that contains it.Results that change (household calculations,
mainvs this PR)Measured in round 1 (2026-09-25), on
e8bf23a958(thenmain) and that plus the first version of this change; not re-run onea79c39fb7.in_effectswitched on: +$800 at743b6dfd3d.main2026-01-01in_effect, amount and refundable amount from2026-01-01)in_effecthas a0000-01-01sentinel)2026-01-01.2026-12-31(app yearly mode)in_effectis dated from 2000)wa_income_taxandhousehold_net_incomeraiseParameterNotFoundErrorin 20272025-01-01tip_income_deductionis $45,000 in 2015–2024, but no tax effect, because the deduction only enters the deduction lists from 2025In all four cases, the years each reform value covers match under both orderings. In the Washington case, 2027 is inside the
in_effectperiod but after the one-year rate, andmainraises there.Formulas that read the poverty guidelines at the prior October 1 (SNAP and others) are the one path I found where the leak also changes the reform's own years: the Guam, Puerto Rico and US Virgin Islands amounts are first dated 2024-01-01. As a contrived demonstration, a Guam guideline reform from
2024-01-01pays $667 a month of SNAP in January–September 2024 onmain, against $0 on this branch (round 1; SNAP has changed onmainsince, and this was not re-run).Tests
policyengine_us/tests/policy/baseline/parameters/test_reform_backdating.pyhas 20 tests and does 4 system builds during them:modified;Simulation(reform=...)keeps the pre-start year at baseline, and a formula reading the year after a bounded reform computes instead of raising;Simulation(..., start_instant="2019-01-01")is detected;shiftedmoves a Feb 29 date to Feb 28 in a non-leap year (new in this round).Before and after, at
a50e1e3a7c. These are local runs, one pytest process at a time, on Python 3.13.9 with policyengine-core 3.32.15, on a shared machine running other sessions' work.a50e1e3a7c)main(ea79c39fb7) with this test file743b6dfd3d) and the changelog fragment's wording (81e427cf85).mainare:modifiedtest, which guards the new order and passes on the old too;main;shiftedtest, which builds no system.shiftedtest, before and after: the helper ata7fd4ad171raisesValueError: day 29 must be in range 1..28 for month 2 in year 2025forshifted("2024-02-29", years=1); now it returns2025-02-28.Cost. This round adds no system build:
shiftedtest builds nothing.The file runs in the
policygroup of Full Suite - Rest (Python + variables).CI at
81e427cf85(run 37434830792, which tests the merge of81e427cf85intomainatb812bd46a0): all 35 checks pass. In Full Suite - Rest (Python + variables), all 20 tests in this file passed in about 69 s. The slowest steps were the start-instant structural-reform test (29.0 s), the exhaustive tests' fixture (14.6 s), theSimulationtest (10.5 s) and the first fixture build (9.9 s).Earlier rounds, before the rebase and the second pass (
e8bf23a958plus the first version of this change):main, because they download microdata and the run was offline.The YAML runner applies reforms to the already built baseline system, never through
CountryTaxBenefitSystem(reform=...), which is the only place the second pass runs. CI at81e427cf85also re-ran every Baseline and Contrib YAML shard and the variables YAML tests, all passing.ruff format --check .andruff check .(ruff 0.15.5, theuv.lockpin) are clean at743b6dfd3d.Review round 2
Round 2 reviewed
a7fd4ad171. It found the code and tests ready and asked for corrections to this description only. This round:The
mainfailure count (finding 1). It said 45,937; round 2 measured 45,941 on the older base9e5852a337. Re-measured on today'smain, it is 41,635 (see "Invariants"). The message of merge commita40fa9aa7acarries the correction fora7fd4ad171's commit message.The invariants claimed too much (finding 2). "Invariants" now:
gov.contribstructural reform, and says 1 and 3 don't hold for what aReformclass adds or replaces;test_reform_at_construction_matches_afterwards_for_every_parameter(bb010100ff) checks invariant 3 over the whole tree. It fails onmainfor 41,635 parameters and passes here."Added or replaced" (finding 3). The second-pass comment in
system.py, the test module docstring, the changelog fragment (81e427cf85) and this description now say the second pass reaches parameters a reform added or replaced.Test robustness (finding 4):
shiftedhandles Feb 29, with a test;Shadowed fixture (finding 5). Locals renamed so they no longer shadow the
reformedfixture.Tracking issues (finding 6). The "Related" items now have issues: Utah CTC contrib reform changes years in which its in_effect switch is off #9842, AL, KY, MS, ND and OH dashboard EITC contrib reforms raise 'mixes computation modes' when switched on #9843, ut_hb210_s2 cannot be passed as reform= (VariableNameConflictError) #9844, Derived _per_capita series are not rebuilt by reforms applied to a built system, and hold their value through a None total #9845 and _normalise_period_key drops the stop date of start.stop reform keys policyengine-sim-api#724. I re-ran the first four on this branch before filing; the fifth is from reading the code. The stale
scratchpad/pr_body.mddraft was in the temporary scratchpad of the session that worked in the oldquizzical-bouman-016455worktree. Neither exists any more, and this description was rebuilt from the live one.Merged
maintwice:a40fa9aa7aresolved a conflict insystem.pywith Keep Medicare IRMAA's income lag within the modeled years #9626'sFIRST_MODELED_YEAR. Both passes use it, andmain's single call after structural-reform detection is the one this PR moves before the reform.a50e1e3a7cmerged cleanly. After Judge each SPM branch by the county_fips it reads itself #9751's change tospm.py,test_simulation_reform_leaves_earlier_yearsstill passes here and fails onmain, soSimulation(reform=...)still builds the reform into the system.Related, not fixed here
I re-ran the first four on this branch (
743b6dfd3d) on 2026-10-06, except the YAML-runner case under the per-capita item, which comes from reading policyengine-core'stools/test_runner.py. The fifth is from reading the code. Each has an issue.in_effectswitch is off (Utah CTC contrib reform changes years in which its in_effect switch is off #9842).in_effect.gov.contrib.states.ut.ctc.in_effectfrom 2026 adds $800 to the example family's net income in 2024 and 2025 on this branch.mainadds $2,800 in those years, because the reformed amounts also leak.in_effectcould behave the same way; only Utah has been checked.child_poverty_impact_dashboardstate EITC contrib reforms (AL, KY, MS, ND and OH dashboard EITC contrib reforms raise 'mixes computation modes' when switched on #9843). AL, KY, MS, ND and OH raiseVariable "<st>_refundable_credits" mixes computation modes: formula and upratingwhen switched on.ut_hb210_s2cannot be passed asreform=(ut_hb210_s2 cannot be passed as reform= (VariableNameConflictError) #9844).__init__applies the user reform twice, so itsadd_variableraisesVariableNameConflictError._per_capitaseries (Derived _per_capita series are not rebuilt by reforms applied to a built system, and hold their value through a None total #9845):Reform.from_dict, the YAML runner's inline reforms) does not rebuild them. This is why invariant 3 excludes them.Nonefor a year leaves its derived series at the previous value._normalise_period_keykeeps only the start ofstart.stopkeys.axiom: n/a: infrastructure (order of backdating and reform application in
CountryTaxBenefitSystem), not a policy rule.🤖 Generated with Claude Code