Repository navigation
Calibrate the UK national build from Ledger-backed targets #623
Description
Depends on: #622 (compiled registry + parity gate) and a UK national calibration seam, which does not exist yet — the national staging path is gate-only by design. This issue is the convergence point, filed now so the seam work and the target migration land against a shared contract.
Context
Today's UK national build (tools/build_uk_national_dataset.py → build_uk_national_dataset) runs two stages (frs_hmrc_retained_leaves, hmrc_spi_income), enforces coverage/terminal gates, and stages an H5 — explicitly without calibration: the terminal report omits the target-surface/target-fit/parity trio "instead of inventing passes". The June production release's 149-target national calibration ran from code that never reached a pushed branch; its surface is being ported to Ledger (PolicyEngine/chronicle#132, PolicyEngine/chronicle#133) and its compilation to populace (#622).
The local lane already shows the target shape end to end (ladder targets → local_doctrine solve → diagnostics). This issue gives the national build the same property, with Ledger as the only source of target values.
Scope
- National calibration seam — a stage (or post-stage step) in the UK national build that accepts a compiled
TargetRegistryand produces calibrated household weights, advancinghousehold_weight_kindand appending the canonical mass-change record, mirroring the rowwise candidate path's conventions (build_uk_rowwise_candidate.py). - Ledger-backed surface only — the seam consumes the registry compiled in Consume Ledger UK facts: uk_target_references, mapping module, and a compile-parity gate #622 (hash-pinned consumer artifact +
uk_target_references.jsonactive subset). No target value may enter from any other path; the uk-data package is not a build dependency. - Solver doctrine — reuse
populace.calibrate(the shared L0/solve stack) under a declared national doctrine analogous tolocal_doctrine.py: uniform operator, declared caps, no per-target knobs. The doctrine constants land in a reviewed module, not as CLI flags. - Evidence becomes real — with a calibration present, the terminal-gate battery gains the previously-omitted trio: target-surface, target-fit (existing
UK_MAX_TARGET_ABS_RELATIVE_ERROR = 0.25fence), and parity evidence;uk_calibration_diagnostics_payloadgets wired with the Ledger pin (facts sha256, profile ids) in its provenance block. - Acceptance against the incumbent — the US base v2: one CPS+ACS+PUF-detail pool; datasets labeled by exact record count (dense = full pool; exact-k L0 selection) #578 acceptance rule stands: the first Ledger-calibrated candidate must not regress the certified June artifact on battery observables (ESS fraction ≥ 0.01 fence, weight-ratio fence, target-fit). Score against the frozen reference exactly as the terminal gates already encode it.
Explicitly out of scope
- Base-table construction, imputations, or any change to what the input H5 is (separate workstream).
- Ledger authoring or profile changes.
- Retiring uk-data's own calibration pipeline — uk-data remains the production builder until a Ledger-calibrated populace candidate passes the acceptance battery; this issue produces that candidate, not the switchover decision.
Acceptance
- A UK national staging build with the seam armed produces a calibrated H5 whose terminal report carries the full gate battery (no omitted trio), with every target value traceable through
ledger_facts_sha256→ consumer artifact → source package → publisher bytes. - The build fails hard if the compiled registry and the release's pinned Ledger artifact disagree (no mixed-provenance surfaces).
- Candidate-vs-incumbent scoring reproduces the production diagnostics schema (
score_vs_enhanced_frs-style block) so the release decision is comparable to June's.
Activity
juaristi22 commentedon Aug 14, 2026
Adjudication request from WS-E planning: the incumbent's post-calibration aggregate scalers (2026-08-14)
The incumbent runs four aggregate adjustments after calibration, on weighted sums: rail-subsidy scaling to the ORR total, bus-subsidy scaling to the DfT total, fuel-litre-proxy scaling to HMRC road-fuel clearances, and the bus-fare rescale that also reshapes the income gradient to NTS trips-per-quintile. Because they read calibrated weights, they cannot live in the WS-E source stages — the ports (#682 consumption/services, #685 bus fares) deliberately exclude them and impute incidence + amounts only.
They need a home on this seam. Two candidate dispositions, per family:
Related: the WS-E/WS-D grain adjudication is #688; the WS-E child map is on #145. No action needed until #685 opens, but flagging now so the target-registry design (#622) can reserve the families if disposition 1 wins.
MaxGhenis commentedon Aug 20, 2026
Picking up the core of this per #701: the national calibrate step consuming activated ledger-backed references, with loud resolution failures (no silent drops — the policyengine-uk-data#456 defect class) and the existing gate discipline. The 2026-08-14 post-calibration aggregate-scaler adjudication stays explicitly OUT of scope — that ruling remains open and yours; the calibrate step lands without porting any incumbent scalers.
Add the UK national calibration step over ledger-backed target refere…
juaristi22 commentedon Aug 22, 2026
Status: narrowed to run-readiness — delivered by #743 (draft); the licensed armed run is held
Per #736 item 10 and the adjudicated plan (2026-08-21), mapping this issue's scope to delivery:
household_weight_kind_chain+calibration_mass_changemanifest block.frs_release.calibration_year(2025) through the shared adapter, fail-closed.uk_runtime/national_doctrine.py(v1 = Add the UK national calibration step over ledger-backed target references #729's values, declared + tamper-tested; revision requires a receipt).uk_export_surface/uk_target_surface/uk_target_fitverdicts from independently sourced sides (frozen eFRS instrument + declared registry at name@period grain); unarmed builds keepevidence_absent. Diagnostics are US-format-identical (shared producer, format-parity pin) with the chronicle pin (ledger_facts_sha256, manifest sha, profile ids) in the build block.tools/score_uk_national_candidate.pyrescores both artifacts on one frozen register (US base v2: one CPS+ACS+PUF-detail pool; datasets labeled by exact record count (dense = full pool; exact-k L0 selection) #578 rule 1), June-schemascore_vs_enhanced_frsblock; the real score is produced by the held run.The run itself is held until the WS-E spine completes E8 (#684) + E10 (#686), or an explicitly named different base — runbook in
docs/uk-national-calibration-runbook-623.md. The 2026-08-14 post-calibration aggregate-scaler adjudication remains open on this issue (per Max's 2026-08-20 comment); first-run evidence on those families becomes input to that ruling. This issue stays open until the held run executes and its acceptance evidence lands.Add the spine-posture national calibration runner
Re-derive the spec and manifest digests on the rebased base
Re-derive the spec and manifest digests on the rebased base
Merge pull request #743 from PolicyEngine/uk-national-first-calibrate…
juaristi22 commentedon Aug 26, 2026
Status against this issue's own acceptance criteria (2026-08-26)
Measured after #747/#743 merged and the #757 swap branch (PR #787) retired the June path.
Delivered in full — scope items 1–3:
household_weight_kind_chain). Production path:tools/calibrate_uk_national_dataset.py→calibration_run.py.national_doctrine.py— declared constants, per-run overrides refused for release candidates.Delivered in substance — scope item 4: the parity trio evaluates for real on armed builds from independently sourced sides; diagnostics are US-format-identical with chronicle provenance (
ledger_facts_sha256→ artifact → source, acceptance a's traceability half). Live proofs on the migrated seam: 403 targets bound at 2025 (+25 over the assessment campaign, none dropped), correctly blocking atuk_target_fiton the known catalogue; four binding defects found and fixed from those runs (obr.ni−76.7% → −2.9% among them).Open — the three pieces keeping this issue alive:
tools/score_uk_national_candidate.pyexists with the June-schema block, but no production seam run has been scored against the incumbent —score_vs_enhanced_frsis still reservednullin every build record. Produced by the first certified cut (Swap the microcosm-built UK spine into the national build and retire the incumbent feed #757 follow-up, Phase 3).shippable: falseuntil the certification exists. Same Swap the microcosm-built UK spine into the national build and retire the incumbent feed #757 follow-up.obr.fuel_dutiesdisposition (frame carries ~37% of implied litres) are the same question from two sides.Closure path: items 1–2 land with the #757 follow-up's certified cut; this issue can close on that PR, with #790 carrying the scaler ruling and #736 the remaining deferred-decision registry.
🤖 Posted with Claude Code
Round-1 docs: deviations, runbook, changelog, receipts R6, the R5 com…