Repository navigation
Wealth and consumption shares divide by a weighted total that can be zero #2296
Description
What happens
shareholding, consumption_shareholding and consumer_incident_tax_revenue_change return inf or NaN for a simulation built from data when some household holds corporate wealth (or consumption) but the weighted total is zero. corporate_tax_incidence and household_tax inherit it.
Reproduction
Run on #2270's branch, policyengine-core 3.32.27; #2270 does not touch the dataset-built path this uses, so main gives the same:
import pandas as pd
from policyengine_uk import Simulation
from policyengine_uk.data import UKSingleYearDataset
# The only household with corporate wealth and consumption has zero weight.
data = UKSingleYearDataset(
person=pd.DataFrame(
{"person_id": [1, 2], "person_benunit_id": [1, 2], "person_household_id": [1, 2], "age": [40, 50]}
),
benunit=pd.DataFrame({"benunit_id": [1, 2]}),
household=pd.DataFrame(
{
"household_id": [1, 2],
"household_weight": [0.0, 1.0],
"region": ["LONDON", "WALES"],
"council_tax": [0.0, 0.0],
"rent": [0.0, 0.0],
"tenure_type": ["OWNED_OUTRIGHT"] * 2,
"corporate_wealth": [10_000.0, 0.0],
"food_and_non_alcoholic_beverages_consumption": [3_000.0, 0.0],
}
),
fiscal_year=2025,
)
sim = Simulation(dataset=data)
print(sim.calculate("shareholding", 2025)) # [inf nan]
print(sim.calculate("consumption_shareholding", 2025)) # [inf nan]
print(sim.calculate("corporate_tax_incidence", 2025)) # [inf nan]
print(sim.calculate("household_tax", 2025)) # [nan nan]Cause, read in the code
Each formula tests the unweighted total and divides by the weighted one. In policyengine_uk/variables/household/wealth/shareholding.py:
if (
household.simulation.dataset is not None
and household("corporate_sector_wealth", period).sum() != 0
):
weight = household("household_weight", period)
return wealth / (wealth * weight).sum()consumption_share.py and contrib/policyengine/consumer_incident_tax_revenue_change.py do the same with consumption. When every household that holds the value has zero weight, the first total is not zero and the second is.
Suggested fix
Test the weighted total ((wealth * weight).sum() != 0), and take the existing no-data branch otherwise. Add a test with a zero-weight holder.
Scope
A full dataset has positive weighted totals, so its results are not affected. Subsamples and small frames can hit it: #2265 gives households that are not drawn a weight of zero, and from #2270 a simulation built from a DataFrame takes this branch too. Found in review of #2270, which leaves these formulas alone.
Activity