Skip to content

Update the subsample policy-binding test for policyengine-core 3.33.1 - #10069

Merged
MaxGhenis merged 1 commit into
mainfrom
hub-fix-spm-subsample-test-core-3331
Oct 10, 2026
Merged

MaxGhenis merged 1 commit into
mainfrom
hub-fix-spm-subsample-test-core-3331

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 10, 2026 •

Copy link
Copy Markdown
Contributor

The SPM policy-binding property test assumed calculated reform values became inputs during every subsample rebuild. Core #573, released in policyengine-core 3.33.1, now rebuilds from surviving recorded inputs instead, so indexing an absent calculated reform_only_income holder raised KeyError and failed the Microsimulation job's Run SPM Python tests step.

Failing CI evidence: run 38074706313, job 114279224752 and run 38074181952, job 114277668349. Upstream contract: core#573.

After each subsample, the test recalculates both the reform-only and redefined variables, then verifies their holders have known periods and non-eternal storage. It retains every existing ownership assertion and the explicit repeated-subsample example, rechecks the newly populated reform holders, and explicitly excludes the reform-only holder from the baseline. Populations, holders, and recorded input names must remain bound to each arm's own policy under either core behavior.

Read-only audit of system.py and spm.py found no production dependency on calculated caches surviving as inputs. Holder rebinding and baseline cleanup operate on existing holders, and input transformations use recorded set_input calls. The obsolete export comments were confined to this test and are corrected here. No production code or partner tests changed.

Validation:

  • CI at this head, policyengine-core 3.33.2: the Microsimulation job passed. Run SPM Python tests reported 115 passed, including 76 passed in test_spm_system.py; the subsequent microsimulation tests reported 130 passed, 4 skipped. The merge gate still requires the latest Pull request run at exact head 9b06402dd1291b7dd199a979a5adc1b83c0fd965 to conclude success before the hub merges.
  • Independent review (GPT-6.1 Sol) approves 9b06402dd1. It traced the test against the core 3.33.1 source and checked both holder expectations by hand; it ran no tests.
  • Exact-version local targeted checks, Python 3.14.4: core 3.33.1 completed with 1 passed, 75 deselected, 8 warnings in 1900.63s (31:40); core 3.32.29 completed with 1 passed, 75 deselected, 8 warnings in 2098.81s (34:58). Both exited 0. The full test_spm_system.py file on core 3.33.1 remains pending; its counts will be added to this body after completion.
  • Formatting and whitespace: make format (Ruff formatting) and git diff --check pass.

axiom: n/a: test-only change, no policy change

Hub microsimulation impact: none (test-only). No local microsimulation was run. The diff at 9b06402dd1 changes two files, policyengine_us/tests/core/test_spm_system.py and changelog.d/hub-fix-spm-subsample-test-core-3331.fixed.md. No variable, parameter, reform or data file changes.

Opened by the US + core hub: this failure blocks CI on every open PE-US pull request.

🤖 Generated with Claude Code

@MaxGhenis
MaxGhenis merged commit 761aa83 into main Oct 10, 2026
26 checks passed
@MaxGhenis
MaxGhenis deleted the hub-fix-spm-subsample-test-core-3331 branch October 10, 2026 20:42
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

US + core hub merge audit, us#10069 at 9b06402dd1291b7dd199a979a5adc1b83c0fd965 (squash): update the subsample policy-binding test for policyengine-core 3.33.1.

  • Why: policyengine-core 3.33.1 (core#573, Max's d1026 and d1118) rebuilds a subsample from recorded inputs and no longer exports calculated values as inputs. One PE-US test asserted the old export, so every PE-US pull request has failed Python tests / Full Suite - Microsimulation since 2026-10-10T17:41Z.
  • Change: test-only. The test recalculates the reform's variables after each subsample and asserts the binding invariant on both core behaviours. No production code changes; a read-only audit of system.py and spm.py found no dependence on calculated values surviving a subsample.
  • Review: independent GPT-6.1 Sol review approves this exact head (no P1 or P2; its two body notes, the pending validation lines and the impact receipt, were fixed in the PR body before merge). The review ran no tests; CI at this head is the test.
  • Gates checked live at merge:
    • gh pr checks exits 0;
    • the latest Pull request run at the head concluded success, on core 3.33.1;
    • MERGEABLE, not a draft, no CHANGES_REQUESTED;
    • the semantic-overlap guard;
    • head pinned.
  • Impact: none (test-only).
  • Axiom: n/a: test-only change, no policy change.
  • --admin: used only because a required approving review is the sole block.

@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Local two-core results for this change, as promised in the description (run by the build job at 9b06402dd1, each in its own Python environment with an editable PolicyEngine-US install):

  • policyengine-core 3.33.1: test_subsampling_never_changes_which_policy_a_holder_belongs_to passed (1 passed, 75 deselected).
  • policyengine-core 3.32.29: the same test passed (1 passed, 75 deselected).

So the updated test holds under both the recorded-input rebuild (3.33.1) and the earlier behaviour that exported calculated values. The full test_spm_system.py file was not finished locally on either core before the job ended; CI ran the whole file on 3.33.1 at this head and passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant