Skip to content

Update PiWindPostcode test_4/test_5 for per-building sampling - #60

Open
sstruzik wants to merge 2 commits into
mainfrom
update/piwindpostcode-building-hash-samples
Open

sstruzik wants to merge 2 commits into
mainfrom
update/piwindpostcode-building-hash-samples

Conversation

@sstruzik

@sstruzik sstruzik commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

Expected-results update for OasisLMF#2145, which makes samples the default disaggregation.

Why these two tests move

Under samples a location's buildings are multiplexed into the sample dimension and each draws its own random numbers. Under items with the default hash (PortNumber, AccNumber, LocNumber) every building of a location shared one group_id — so they were perfectly correlated.

test_4 and test_5 are the only PiWindPostcode cases with NumberOfBuildings > 1.

Evidence this is the more correct simulation, not a different one

test_4 run three ways on the same exposure:

run selt rows EPT median |diff| vs samples p95
main, default hash 3135 4.12% 36.9%
main + building_id in the damage hash 3814 0.26% 3.0%
branch, samples 3801 — —

Adding building_id to the hash on main reproduces the branch's result to Monte Carlo noise. That identifies the correlation assumption as the cause, rather than any change to the losses.

Total loss is conserved — test_4 +0.012%, test_5 −0.092%. It is the spread that changes.

test_5 shows the effect more weakly: only 2 of its 6 locations have more than one building, so most of it is unchanged by construction.

Merge ordering

This PR and OasisLMF#2145 must land together. Each one makes run-model-tests fail against the
other repo's current main, so merging either alone turns that job red on every other open PR
there until the second follows. Verified green as a pair on all five Python versions via
workflow_dispatch (run): 22 passed, 5 skipped on 3.10 through 3.14.

Second commit: summary-info TIV

OasisLMF#2145 also fixes summary-info reporting a packed location's TIV as one building's share
rather than the location total — number_of_buildings was dropped when the summary map is read
back and narrowed to MAP_SUMMARY_DTYPES, leaving the multiplication that restores the location
total with nothing to work from.

Only the reported exposure moves, which is why that commit touches just the four summary-info
files and no loss output:

expected before now
test_4 82,896.27 250,000.00
test_5 28,429,687.5 32,781,250.0

Both land at 92.59% of their source exposure total — the same keys-coverage ratio — against the
old test_4 figure being 31% of its exposure.

Not included

test_1/2/3 were regenerated and reverted — they produced only last-bit float32 differences (2868654.000000 → 2868654.250000), below the check tolerance and unrelated to this change.

🤖 Generated with Claude Code

OasisLMF#2145 makes 'samples' the default disaggregation. A location's buildings are then
multiplexed into the sample dimension and each draws its own random numbers, where
'items' with the default hash gave every building of a location the SAME group_id -- so
they were perfectly correlated. These two tests are the only PiWindPostcode cases with
NumberOfBuildings > 1, which is why only they move.

The new results are the more correct simulation, not a different one. Checked by running
test_4 three ways on the same exposure:

  main, default hash          selt 3135 rows   EPT median |diff| vs samples  4.12%, p95 36.9%
  main + building_id in hash  selt 3814 rows   EPT median |diff| vs samples  0.26%, p95  3.0%
  branch, samples             selt 3801 rows

Adding building_id to the damage hash on main reproduces the branch's result to Monte
Carlo noise, which is what identifies the correlation assumption as the cause rather than
a change in the losses themselves. Total loss is conserved: test_4 +0.012%, test_5
-0.092%; it is the spread that changes.

test_5 shows the same effect more weakly because only 2 of its 6 locations have more than
one building, so most of it is unchanged by construction.

test_1/2/3 were left alone: regenerating them produced only last-bit float32 differences
(2868654.000000 -> 2868654.250000), which are below the check tolerance and unrelated.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
OasisLMF#2145 fixes summary-info reporting a packed location's TIV as one building's
share rather than the location total: number_of_buildings was dropped when the summary
map is read back and narrowed to MAP_SUMMARY_DTYPES, so the multiplication that restores
the location total had nothing to work from.

Only the reported exposure moves -- no loss file changes, which is why just the four
summary-info files are touched.

  test_4   82,896.27    -> 250,000.00
  test_5   28,429,687.5 -> 32,781,250.0

Both land at 92.59% of their source exposure total, the same keys-coverage ratio; the
old test_4 figure was 31% of its exposure.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Waiting for Review

Development

Successfully merging this pull request may close these issues.

2 participants