Per-location IL depends on accounts.csv row order under ktools_alloc_rule_il=2 (layer_id follows row order; rule 2 back-allocates all layers by layer 1) #413

Summary

With the default ktools_alloc_rule_il=2, per-location IL results depend on the row order of accounts.csv whenever policy layers carry layer-specific financial terms at coverage granularity. Totals and per-policy ILs are stable and correct — only the back-allocation of each policy's loss across locations changes with row order.

This is distinct from OasisLMF/OasisLMF#2040 (and unaffected by the fix in OasisLMF/OasisLMF#2044, which stabilises the FM structure, totals and per-policy losses). It is the combination of two things that are each individually "by design":

1. fmcalc -a 2 back-allocates every layer's losses to items using layer 1's prior-level loss proportions.
2. layer_id is assigned in accounts.csv row order, so reordering the file changes which policy is layer 1 — and therefore which policy's loss distribution is used to allocate everyone's losses.

ktools_alloc_rule_il=3 (per-layer back-allocation) is fully order-independent and matches the analytically correct per-layer allocation.

Reproduction

One account, four single-layer policies. The non-binding "big" coverage limit (1e9) rotates across policies (B→Pol1, O→Pol2, C→Pol3, BI→Pol4) with a binding 100k limit elsewhere, so each policy concentrates its loss in a different coverage.

location.csv:

PortNumber,AccNumber,LocNumber,IsTenant,BuildingID,CountryCode,Latitude,Longitude,StreetAddress,PostalCode,OccupancyCode,ConstructionCode,LocPerilsCovered,BuildingTIV,OtherTIV,ContentsTIV,BITIV,LocCurrency,OEDVersion
1,ACC1,L1,1,1,US,34.0,-118.0,addr1,90001,1050,5000,AA1,1000000,500000,300000,200000,USD,4.0.0
1,ACC1,L2,1,1,US,34.1,-118.1,addr2,90001,1050,5000,AA1,2000000,1000000,700000,800000,USD,4.0.0

account.csv:

PortNumber,AccNumber,AccCurrency,PolNumber,PolPerilsCovered,PolLimit1Building,PolLimit2Other,PolLimit3Contents,PolLimit4BI,PolDed5PD,PolDedType5PD,PolDed6All,PolDedType6All,LayerParticipation,LayerLimit,LayerAttachment,OEDVersion
1,ACC1,USD,Pol1,AA1,1e9,1e5,1e5,1e5,25000,0,0,0,0.02,1e12,0,4.0.0
1,ACC1,USD,Pol2,AA1,1e5,1e9,1e5,1e5,0,0,50000,0,0.0925,1e12,0,4.0.0
1,ACC1,USD,Pol3,AA1,1e5,1e5,1e9,1e5,0,0,50000,0,0.1675,1e12,0,4.0.0
1,ACC1,USD,Pol4,AA1,1e5,1e5,1e5,1e9,0,0,50000,0,0.044,1e12,0,4.0.0

Run with a deterministic loss factor, once with the file as-is and once with the four policy rows reversed:

from oasislmf.computation.run.exposure import RunExposure
RunExposure(src_dir=src, run_dir=run, output_file=out, output_level="loc",
            loss_factor=[1.0], fmpy=False, check_oed=False,
            ktools_alloc_rule_il=2).run()

Observed (release/2.4.13 + PR OasisLMF/OasisLMF#2044)

Per-location IL under alloc_rule_il=2:

Under alloc_rule_il=3 every ordering gives L1 = 148,786.82, L2 = 342,963.40, which matches the hand-calculated per-layer allocation. Rule 1 is also order-independent (GUL-proportional: L1 = 151,307.26). Totals match the hand calculation (491,750) under every rule and every ordering.

Analysis

Within a single rule-2 run, all four policies show the identical L1:L2 split, and that ratio exactly equals the prior-level (post-terms) loss proportions of whichever policy is layer 1:

• original order → layer 1 = Pol1 (big Building limit): L1:L2 = 1,075,026 : 2,199,974 ≈ 0.4886 → L1 = 161,418
• reversed order → layer 1 = Pol4 (big BI limit): L1:L2 = 296,667 : 1,003,333 ≈ 0.2957 → L1 = 112,220

Both observed results are reproduced analytically from layer 1's distribution alone, confirming the mechanism.

The FM input files across orderings are semantically identical up to a layer permutation (verified by resolving fm_policytc profile IDs to terms: in the reversed run, layer 1 carries exactly Pol4's terms at every level). So the preparation output is consistent — the row-order sensitivity enters only through which policy receives layer_id 1, interacting with rule 2's layer-1-proportional back-allocation.

The effect needs layers whose loss distributions across items differ at the level below (e.g. layer-specific coverage limits with multiple coverage items per location). With a single item per location and blanket (*6All) terms, all layers distribute proportionally to GUL within an aggregation cell and the quirk is invisible — which is why the OasisLMF/OasisLMF#2040 regression fixtures don't trip it.

Possible resolutions

1. Canonical layer_id assignment in preparation (e.g. numbering layers by sorted (PolNumber, LayerNumber) rather than accounts.csv row order). This makes the generated FM files — and therefore rule-2 output — deterministic regardless of input row order, and seems desirable independently of this issue. It would change which policy is layer 1 for existing books whose row order isn't already sorted, so per-location rule-2 results would shift once at upgrade.
2. Document that per-location allocation under rule 2 is layer-1-proportional and recommend ktools_alloc_rule_il=3 where per-location/per-layer allocation matters.

These aren't mutually exclusive; (1) fixes the non-determinism, (2) addresses the underlying allocation approximation.

Context

Found while adding the sub-level regression fixtures requested in the PR OasisLMF/OasisLMF#2044 review (tests/preparation/test_il_ordering.py). Those end-to-end tests pin ktools_alloc_rule_il=3 specifically to avoid this effect; see the comment in test_il_losses_are_order_independent_sub_levels.

• oasislmf: branch fix/issue-2040-polnumber-2.4.13 (based on release/2.4.13)
• Related: Ordering changes values in oasislmf exposure run OasisLMF#2040, fix: IL outputs are non-deterministic when accounts CSV rows are reordered OasisLMF#2044