You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
to_input_dataframe() drops dataset inputs for variables with formulas, such as state_pension_reported #2297
Simulation.to_input_dataframe() leaves out values a dataset supplies for a variable that has a formula. A simulation built from the exported frame then recalculates that variable instead of using the data, so it does not give the original's results. state_pension_reported is one such variable: the reloaded simulation pays no State Pension.
Reproduction
Run on #2270's branch with policyengine-core 3.32.27 and with 3.33.2; both print the same. The frame has one person, so it also loads on main:
to_input_dataframe(include_computed_variables=True), taken before the simulation calculates anything, keeps the column, and the reloaded simulation then matches on every variable (tested in #2270).
Cause, read in the code
policyengine-core's to_input_dataframe exports a variable only if Variable.is_input_variable() is true: no formulas, and no adds or subtracts (_is_exportable_input_variable in policyengine_core/simulations/simulation.py). state_pension_reported has formula_2022, which returns the previous year's value, so the value the dataset supplied is not exported. Any dataset column for a variable with a formula or adds is dropped the same way, such as a supplied consumption.
Options
Give the UK Simulation its own to_input_dataframe that exports every value the simulation was given as an input and still stores, whatever the variable's formula. policyengine_uk.utils.supplied_inputs already tracks those (and Give UK simulations a working subsample #2265 adds supplied_input_arrays).
Change the rule in policyengine-core to export supplied inputs of formula variables.
Either needs a test that a dataset-built simulation's default export reloads to the same results when the dataset supplies a formula variable.
Found while fixing #2266 (#2270 documents the limit in its tests and does not change the export).
Activity