Drop sums and twelfths an own-period input makes stale

[MaxGhenis]

Fixes #579. Stacked on #581 (base fix-set-input-helper-order), whose helper it reuses. The diff below is this PR's change only.

What this fixes

set_input for one of a variable's own periods (a month of a monthly variable, a year of a yearly one) replaced the stored value. It left what calculate had derived from the old value at periods of another size: the sum calculate_add stores for a monthly flow over a year, and the twelfth calculate_divide stores for a yearly flow at a month. A later calculate returned the old result, so it depended on whether that period was read before the input was set. The audit's witness: annual flow read as 0, then January set to 1, and the annual read stays 0.

#581 already drops such calculated values when a set_input helper spreads a longer period (_drop_calculated_overlapping). #579 is the remaining path, an input that needs no helper. This PR applies the same rule there:

• Holder.set_input, after storing an own-period input of a non-ETERNITY variable, calls _drop_calculated_over_own_period. That drops what the simulation calculated for the variable at overlapping periods, on the input's branch and the branches it reads. Inputs are kept.
• It also evicts calculate's fast-cache entries for the variable at overlapping periods, the input's own period included. The fast cache never holds the sums and twelfths themselves (I checked; calculate_add and calculate_divide store only in the holder). The eviction matters for inputs set on the holder directly, where Simulation.set_input's exact-key pop never runs.
• It does not touch Simulation.set_input, whose fast-cache block draft Drop every contained period from the fast cache when deleting or setting a period #566 rewrites.

#581's property test_inputs_do_not_depend_on_what_was_calculated_before excluded other-size reads for variables given an own-period input, because of #579. This PR removes that exclusion.

Invariants

• For any sequence of inputs (own-period and helper), any calculate calls in between on the simulation and up to two nested branches, and memory or disk storage: every plain read, at the variable's own periods and at periods of another size, equals the same read on a simulation that calculated nothing first. This is Count only inputs as already set when a set_input helper splits a longer period #581's property with the set_input for one period leaves the variable's cached sum or twelfth over an overlapping period in place #579 exclusion removed.
• An own-period input never drops another input.

Tests

• New test_own_period_input_overlapping_cache.py, 7 examples:
  • issue set_input for one period leaves the variable's cached sum or twelfth over an overlapping period in place #579's table, in memory and on disk;
  • the audit witness, and a sum over two years;
  • an input for another month kept;
  • a branch input updating the branch's sum while the parent keeps its own;
  • an input set through the holder.
• test_set_input_helper_order_property.py: the set_input for one period leaves the variable's cached sum or twelfth over an overlapping period in place #579 exclusion removed.

Before, on #581's head 917f0c23:

test_own_period_input_overlapping_cache.py:  7 failed (e.g. annual [0, 0] after setting January to [1, 1])
test_set_input_helper_order_property.py:     1 failed, 2 passed
  shrunk: requests_by_level=[[], [], [('flow_m', '2013')]], depth=2, one own-period input:
  AssertionError: (('flow_m', '2013'), array([0., 0.]), array([0., 1.]))

After:

test_own_period_input_overlapping_cache.py:  7 passed
test_set_input_helper_order_property.py:     3 passed

Ruff format and ruff check pass. I ran these files one at a time, because the shared host is saturated. This repository's PR workflow runs only for PRs into master, so CI runs once this PR is retargeted, after #581 lands or after this PR is merged into #581's branch.

Impact

Outputs can change wherever an own-period input is set after the simulation has calculated a sum or twelfth of that variable. policyengine-us marginal_tax_rate, for example, sets inputs on branches after calculating. Dataset loading sets inputs before any calculation, so there is nothing to drop there.

PE-US A/B (core 917f0c23 vs this head, same policyengine-us checkout and data, 2026; household_net_income, income_tax, state_income_tax, snap, marginal_tax_rate): pending. It is queued behind the shared microsim lock, and I will add results here when it finishes.

Known limits (from the independent review, which approved at 1fa8b034)

• Inherited, not introduced here: the rule trusts the simulation's record of inputs, _user_input_keys, and Simulation.clone (so get_branch) still shares that set with the parent. An input the parent sets after branching can therefore make the branch keep a stale calculated value. Executed witness:

  • read flow_m for 2013 (0);
  • branch;
  • the parent sets twelve months as month:2013-01:12;
  • the branch sets May to 50.

  The branch's annual then reads [0, 0] instead of [50, 0]. Giving each clone its own copy of the record fixes it. That belongs with the record's own fixes (Keep the record of set_input values in step with storage #561), not this PR.

• Cost: each own-period input scans the variable's stored keys, so N successive inputs to one variable check N(N+1)/2 keys. The reviewer measured 0.012 s for 100 inputs, 0.040 s for 200 and 0.152 s for 400. Dataset loading sets few periods per variable and starts with an empty fast cache, so that is negligible; long panels would merit a cheaper index.

Landing

Either merge this into #581's branch, so #581 (decision d899) lands both, or land it after #581. GitHub retargets it to master when #581's branch is deleted. #581 currently conflicts with master in holders/helpers.py and tools/simulation_dumper.py. This PR adds no conflict of its own: git merge-tree against master reports the same two files.

axiom: n/a: engine cache fix, no policy change

Other audit findings, handled elsewhere

The 2026-10-06 invariants audit of master 757147c7 also confirmed defects that this PR leaves alone, because other work owns them:

• Stock ADD changing later plain reads: draft Cache ADD and DIVIDE results only where a plain read returns them #571.
• Year deletion leaving monthly fast-cache entries: draft Drop every contained period from the fast cache when deleting or setting a period #566.
• Disk deletion of contained periods, and disk branch-name parsing and prefix collisions: draft Delete the periods within a deleted period from disk storage #565 (it inherits Keep branch period reads, dumps, and disk deletion consistent #552).
• Clone aliases bound to the parent: fixed by Bind a clone's calc and df to the clone, and give it its own system's entities #586.
• Serialization and subsample input provenance: Restore inputs as inputs, evict the fast cache on holder writes, reject non-numeric uprating and defined_for #576 and Rebuild a subsampled simulation from its inputs, not its calculated values #573.
• Restored entity counts: Fix restore_simulation entity counts #548.

Branch staleness after set_input and raw-file restore are documented behaviour, not defects.

🤖 Generated with Claude Code