Unsorted benefit unit or household tables give each group another group's inputs #2278

What happens

When a dataset's benefit unit or household table is not sorted by identifier, each group's inputs go to a different group, without an error.

Reproduction

On main (a375230, 2.126.1):

import pandas as pd
from policyengine_uk import Microsimulation
from policyengine_uk.data import UKMultiYearDataset, UKSingleYearDataset

person = pd.DataFrame(
    {"person_id": [1, 2], "person_benunit_id": [1, 2], "person_household_id": [1, 2], "age": [40, 40]}
)
benunit = pd.DataFrame({"benunit_id": [1, 2]})
household = pd.DataFrame(
    {
        "household_id": [2, 1],
        "household_weight": [20.0, 10.0],
        "region": ["LONDON", "WALES"],
        "rent": 0.0,
        "tenure_type": "OWNED_OUTRIGHT",
        "council_tax": 0.0,
    }
)
sim = Microsimulation(
    dataset=UKMultiYearDataset(
        datasets=[UKSingleYearDataset(person=person, benunit=benunit, household=household, fiscal_year=2025)]
    )
)
print(sim.populations["household"].ids.tolist())                          # [1, 2]
print(sim.calculate("household_id", 2025).values.tolist())                  # [2, 1]
print(sim.calculate("region", 2025, map_to="person").values.tolist())      # ['LONDON', 'WALES']

Person 1 lives in household 1, which the table puts in Wales with weight 10. The simulation gives person 1 London and weight 20, household 2's values.

Cause, read in the code

Simulation.build_from_ids (policyengine_uk/simulation.py) declares the groups as np.unique(benunit_id) and np.unique(household_id), which sorts them, and core's join_with_persons assigns people to groups in that sorted order. build_from_multi_year_dataset (and the other loaders) then call set_input with each group table's columns in the table's own row order. The two orders agree only when the table is sorted by identifier.

policyengine-uk-data sorts its tables for this reason (see the comments in policyengine_uk_data/datasets/frs.py above its sort_values("benunit_id") and sort_index() calls). Other callers, and the docs example in docs/book/usage/simulations.md, are not told to.

Suggested fix

Either reorder each group table by identifier before setting its inputs, or raise when it is not sorted. Reordering keeps the stored benunit_id and household_id equal to populations[...].ids. A loader that takes one row per person (build_from_dataframe) needs the np.unique as it is.

Found during review of #2273 (issue #2267), which does not change this.

🤖 Generated with Claude Code