Restore inputs as inputs, evict the fast cache on holder writes, reject non-numeric uprating and defined_for

[MaxGhenis]

Restored inputs must survive apply_reform; direct holder writes must invalidate the calculation fast cache; non-numeric uprating and defined_for configurations must receive named validation errors before arithmetic or entity mapping.

Draft: this changes core behaviour, so the merge decision remains Max's. No merge or draft-status change is part of this repair.

Behaviour

• Dumps use master's derived_periods.txt and write __entities__/dump_format.txt on every successful dump. There is no parallel inputs.txt record.
• Restore preserves derived marks and registers every non-derived value through the same input context as set_input, without running input-splitting helpers again. Explicit inputs, including already split months, ETERNITY inputs and Enum inputs, remain replayable after a reform.
• Storage inputs written with put_in_cache(derived=False) were not necessarily in the original simulation's explicit input registry. Restore registers those values too. This intentionally follows storage provenance, so universal reform equivalence is not claimed for simulations whose explicit input registry disagrees with storage marks.
• Ambiguous legacy dumps lacking both the format marker and calculated-period provenance warn that any calculated values restored as inputs will survive a reform. Marker-bearing all-input dumps and older dumps listing calculated periods restore without that ambiguity warning.
• Holder writes and deletes evict affected fast-cache entries in their own simulation, respecting visible branch names, ETERNITY values and contained-period deletion. Master's helper-subperiod eviction, shared-key bookkeeping, disk directories, derived-write protection and clone/baseline fixes are retained.
• Enum, string and date uprating or defined_for conditions remain rejected at registration and, for late changes, before uprating or condition mapping. Numeric conditions remain accepted. Rejected variable replacements preserve the previous variable.

Invariants

1. Dump/restore preserves default-branch stored bytes, Enum metadata and derived marks; restored replay keys identify exactly the non-derived stored values.
2. Explicit set_input values survive dump/restore and reform; calculated values remain derived and are dropped on reform. Reform round-trip equality is conditional on the original replay registry agreeing with storage provenance.
3. Cached and uncached simulation trees return the same results for holder writes/deletes and input operations, in memory and on disk. A holder never evicts another simulation's fast cache or entries for unrelated variables.
4. Master input-only uprating and derived-overwrite protection remain intact. Calculated helper subperiods do not count as explicit inputs.
5. Non-numeric condition validation is independent of registration order and runs before entity mapping.

This repair

Merged canonical master 5d68130be487f0b87d47fc75ef300f36ad46f7ca with a history-preserving merge. Resolved holder, simulation and dumper conflicts without dropping master changes. Consolidated this PR's four fragments into one changed fragment.

Folded #566's unique coverage into this PR: its seven examples, two Hypothesis properties and fixture are copied byte-for-byte from c825cab722bf202f1815e63e63e245fe7b1b3ff2. No #566 production change or changelog was imported. #566 remains open and its branch is unchanged.

Added legacy-format, all-input-format, unrecorded storage-input, stale input-record, Enum replay, disk clone/branch and shared-key regressions. Expanded restore and disk-cache differential coverage, removing the obsolete calculated-month exception now addressed by master.

Remaining stack work

• Branch-visible dumps remain Keep branch period reads, dumps, and disk deletion consistent #552/Make set_input on a branch drop values calculated from the input it replaces #560 work; this repair preserves the default-branch dump contract.
• Branch input invalidation remains Make set_input on a branch drop values calculated from the input it replaces #560 work, and input-registry drift after deletions remains Keep the record of set_input values in step with storage #561 work.
• Branch spiral purging remains Evaluate recursions over periods in full when they reach an input or the start of a formula #572 work. If Evaluate recursions over periods in full when they reach an input or the start of a formula #572 lands after this PR, its new exact-period Holder.delete_array must call _evict_fast_cache.
• The old disk-clone overwrite and shared clone-registry findings are addressed by master's isolated disk writes and copied input set. Current-head execution of their focused regressions remains pending.
• Count only inputs as already set when a set_input helper splits a longer period #581 must adopt this derived-period dumper rather than restore its old inputs.txt hunk; retain its Keep the record of set_input values in step with storage #561/Cache ADD and DIVIDE results only where a plain read returns them #571 ordering and existing decision hold.

Verification

Changed Python files pass Ruff formatting and lint checks; git diff --check passes. The three #566 file blobs exactly match their originals.

The required foreground wrapper was attempted for the focused restore, holder-cache and non-numeric validation tests. It exited 75 before pytest started because the sandbox denied ps used by its priority check. No tests or country loads were run outside the wrapper; no full suite or new microsimulation was run. Current-head focused tests, country-load checks and CI are pending. Prior-head test counts are historical and do not validate this resolution. A fresh independent source review is recorded in the build report; execution remains a separate gate.

Downstream impact

Re-run pending at b8d98660a03d3fc3aa45d90b477737c759ec4938; the following measurements are historical, from 83597c22 against core master b78b0ba9, with identical country code/data on both arms:

• policyengine-uk (main c7e826ea, Enhanced FRS 2024-25, year 2026): 36 of 36 arrays identical, including marginal_tax_rate; income tax £313.139bn on both.
• policyengine-us (main fbe24ad1, Enhanced CPS, 3,000-household subsample, 2026, including marginal_tax_rate): 25 of 25 arrays identical; income tax $1,962.456bn on both.
• Historical harness: ~/reviews/core-restore-cache-enum-2026-10-02/ab/.

Partner-impact gate: this core PR's workflow runs core and country smoke jobs; it does not run US partner baseline snapshots. The hub's downstream country-load/A/B checks and any partner-impact assessment remain for Max before merge. No partner baseline file was changed.

axiom: n/a: core engine, no policy change

🤖 Generated with Claude Code

[MaxGhenis]

US + core hub merge audit, core#576 at b8d98660a03d3fc3aa45d90b477737c759ec4938 (squash)

• Max's ruling d899 (2026-10-08): merge the core order Restore inputs as inputs, evict the fast cache on holder writes, reject non-numeric uprating and defined_for #576 → Keep the record of set_input values in step with storage #561 → Cache ADD and DIVIDE results only where a plain read returns them #571 → Count only inputs as already set when a set_input helper splits a longer period #581, with Give each disk-backed holder storage its own directory #558 before or alongside, each on the hub's gates. Restore inputs as inputs, evict the fast cache on holder writes, reject non-numeric uprating and defined_for #576 is first.
• Change: restore-registry holder cache and enums.
  • bdba182 merges current master and resolves the holder, simulation and dumper conflicts.
  • 5d868a4 restores input replay using derived-period marks, and adds a format marker and a legacy warning.
  • 69bc771 folds Drop every contained period from the fast cache when deleting or setting a period #566's seven examples, two properties and fixture in unchanged.
• Review: independent review (Subfleet) APPROVE at this head, with no P1 and no P2 introduced. The P2s disclosed in the body belong to other PRs in the stack.
• Gates checked live at merge:
  • gh pr checks exits 0;
  • MERGEABLE;
  • not a draft;
  • no CHANGES_REQUESTED review;
  • head pinned.
• Impact: storage and restore semantics only; no formula or parameter changes.