Delete the periods within a deleted period from disk storage

[MaxGhenis]

Fixes #564.

Stacked on #552. This branch carries #552's two commits; review only the last commit, bfa78cb. I'll rebase onto master once #552 merges. It uses #552's _split_key (split on the last _) instead of a second key parser.

What was wrong

Holder.delete_arrays(period) documents that it removes "all values for any period included in period". InMemoryStorage.delete does that. OnDiskStorage.delete(period, branch_name) removed only the file keyed exactly f"{branch_name}_{period}". So a disk-backed simulation (one with a MemoryConfig) kept values the caller had deleted. Here is the issue's reproduction with delete_arrays("rent", "2025"):

	in memory	on disk
before (#552 head)	[]	['2025-01', '2025-02']
this PR	[]	[]

Change

OnDiskStorage.delete(period, branch_name) now deletes each file key whose parsed branch is branch_name and whose period period.contains(...). This is the same rule as InMemoryStorage.delete. Eternal storage still deletes the branch's single ETERNITY key, and delete(None, branch_name) keeps #552's whole-branch-name match. As before, delete only forgets the mapping and leaves the .npy file in place.

Invariants (tested)

1. Memory and disk agree. For any sequence of put, delete(period, branch) and delete(None, branch), eternal or not, both storages hold the same (branch, period) keys with the same values. This is a differential test, checked after every step.
2. Deletion follows the documented rule. After delete(p, b), branch b has no key whose period p contains, and every other key is unchanged. After delete(None, b), branch b has no keys. Both storages are checked against a reference model of this rule after every step.
3. Branches are matched by their whole name. A branch name that contains _, or that starts with another branch's name plus _ (pre / pre_tcja / pre_tcja_ctc, y / y_2019, trailing_), never loses or keeps another branch's keys.

Tests

• tests/core/test_disk_storage_delete_contained_periods.py runs without dev extras. It covers:

  • exact, containing, excluding, other-branch, underscore-branch, eternal-storage, delete-ETERNITY and multi-unit cases on both backends;
  • an exhaustive grid comparing memory and disk for every stored and deleted period over 3 branches;
  • Simulation.delete_arrays and Holder.delete_arrays on a disk-backed holder (the OnDiskStorage.delete(period) deletes only that exact period, not the periods within it #564 reproduction).

  Without the fix, 29 of the 59 fail: every disk case where a period contains another.

• tests/core/test_disk_storage_delete_property.py holds the Hypothesis differential and model test for invariants 1–3. It starts with pytest.importorskip("hypothesis") because the smoke job installs no dev extras. It runs 200 examples in CI; I also ran 5,000 locally with up to 40 steps. Without the fix it shrinks to put(2023, "default") followed by delete(ETERNITY, "default"): disk keeps 2023.

• On Windows, the cases whose period string contains : (multi-unit and rolling-year periods) are skipped. A Windows file name cannot hold that period's key, which predates this PR. Keep branch period reads, dumps, and disk deletion consistent #552's round-trip test now skips them for the same reason (5fe8b61; that was Keep branch period reads, dumps, and disk deletion consistent #552's Windows failure).

• Full suite on master + Keep branch period reads, dumps, and disk deletion consistent #552 + this PR: 1,735 passed, 4 skipped, 1 xfailed. Country-template YAML: 39 passed. ruff format --check and ruff check are clean.

Open PRs that touch the same code

Each was merged with this branch on top of master and the full suite run:

PR	merge	full suite
#561 (579c375)	clean	1,766 passed
#558 (a9a0b40)	clean (on_disk_storage.py composes)	1,767 passed
#562 (912a4ac)	simulation_dumper.py conflicts with #552, not this PR. Resolved as #562's body says (#552's branch passed to is_derived)	1,792 passed; its _derived.intersection_update sits after this PR's comprehension
#557 (97d4582)	clean	1,979 passed

#561: its disk tests compare the record with what storage still holds, so they pass before and after this fix, and its Holder._forget_deleted_inputs drops an entry only when neither storage still holds the value. One sentence in that method's docstring goes stale once both are merged: "disk storage deletes only the period it is given, not the periods within it (policyengine-core#564), so a value deleted from memory can survive on disk, and its entry stays." Whichever of #561 and this PR merges second edits it.

Impact

Holders get disk storage only when simulation.memory_config is set (Holder.__init__). policyengine-us (4e4999a3), policyengine.py (6a9c878), policyengine-canada, -il and -ng never set it, and policyengine-uk (c7e826ea) sets it to None. So this changes no country-package or policyengine.py result.

Not run: make documentation, and Windows locally (CI covers Windows).

axiom: n/a: core storage fix, no policy encoded.

🤖 Generated with Claude Code