Repository navigation
Conversation
A chained vector parameter lookup such as node[keys][keys][keys] with empty key arrays raised IndexError once it reached numeric leaves: after the empty selection each leaf held zero values, and the scalar-per-child path read the first one. Leaves with one value still take that path; leaves with zero values now take the N-element path, which yields an array of length zero. Adds regressions at every level of the fixture tree and on the audit's single-path witness, plus a Hypothesis property: for trees of depth 1 to 3 and 0 to 20 paths, the vector lookup returns one value per path, each equal to the scalar lookup. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Addresses the independent review of f842617 (one should-fix finding), and a second defect the extended property found. - After a selection of no rows, a one-element unknown key broadcast over those rows and returned an empty array, so the NaN check that raises ParameterNotFoundError never fired. The check now also fires when any key index is the missing-child sentinel, whatever the result holds. - At a node (structured level), zero rows took the one-row path: a valid one-element key returned one row of NaN, and node[none][["b"]][["c"]] read NaN where it should give no rows (on master too). Zero rows now give zero rows at nodes as at leaves, broadcasting the key the same way (a key of two or more over zero rows raises a shape mismatch). Adds a regression for both, and a Hypothesis property: a key naming no child raises ParameterNotFoundError at any level, among valid keys for the selected rows or alone after selecting none. Both fail on f842617. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this fixes
A chained vector parameter lookup that selects no rows raises once it reaches numeric leaves. The audit's witness is a tree with one leaf
a.b.c = 0and three empty key arrays:After the empty selection every numeric child holds zero values. The "scalar per child" path, meant for 0-d and length-1 children, read the first value of each. That path now takes only 0-d and length-1 children. Length-0 children go to the existing N-element path, which stacks the children into a
(K+1, N)matrix and returns a length-0 float array.The independent review found two more cases in the same code, now fixed:
node[none][["missing"]]returned an empty array. The unknown-key check only fired when the result held NaN, so nothing raised. It now also fires when any key index is the missing-child sentinel, and the lookup raisesParameterNotFoundErroras it does for rows that exist.node[none][["b"]][["c"]]read[nan](master too). Zero rows now give zero rows at nodes as at leaves, broadcasting the key the same way: no rows for a one-element key, and a shape-mismatch error for a key of two or more.Invariants
For any parameter tree of depth 1 to 3 (1 to 3 children per node, numeric leaves) and any 0 to 20 paths through it:
(number of paths,). Each element equals the scalar lookup of its path. Zero paths give an empty array.ParameterNotFoundErrorat any level, among valid keys for the selected rows, or alone when no row is selected.Tests
test_fancy_indexing.py::test_empty_fancy_indexing_at_every_level(leaf, node and triple lookups on the existing fixture tree) and::test_empty_fancy_indexing_on_a_single_path(the audit witness).test_fancy_indexing.py::test_one_key_after_an_empty_selection: one-element keys after an empty selection, at a node and at a leaf.test_fancy_indexing_property.py: the two properties above, 300 examples each.Before the fix, on master's
policyengine_core/parameters:Round 2 (after the independent review), the new tests on the first head
f8426179:After:
Ruff format and ruff check pass. I ran these files one at a time and left the full suite to CI, because the shared host is saturated.
Impact
No country output can change. The only inputs that take a different path are selections of zero rows, which used to raise or return NaN rows, and unknown keys, which now raise in one more case. Children with one value, which includes every first-level lookup, still take the scalar path. Children with two or more values still take the N-element path. A microsimulation never looks up parameters for zero rows, so no A/B run is needed.
axiom: n/a: engine fix in vector parameter lookup, no policy change
Other audit findings, handled elsewhere
The 2026-10-06 invariants audit of master
757147c7also confirmed defects that this PR leaves alone, because other work owns them:Branch staleness after
set_inputand raw-file restore are documented behaviour, not defects.🤖 Generated with Claude Code