Repository navigation
UK source stages on the shared manifest: source_stages.json, operation handlers, and the StagePlan migration #677
Description
Workstream E1 of the UK-migration master epic #665; child of #145. Lands alone, before every other WS-E child.
Why
The UK country package declares hmrc_income_source_stages.json — a bespoke manifest whose 10 operation kinds are absent from ALLOWED_SOURCE_OPERATION_KINDS and which only uk_runtime/hmrc_source_contract.py validates — so load_country_spec("uk").sources is None and country_stage_plan refuses UK. UK stages run as UKNationalStage (no produces/consumes/donor declarations) instead of the shared plan.Stage/StagePlan. Every WS-E stage needs the shared machinery: the closed operation vocabulary, produces/consumes enforcement, StageRecord.nonzero_share (the per-stage parity evidence carrier), and the donor graph.
Scope
- New
build/uk/source_stages.json— a realSourceManifest, initially carrying the two existing stages, declarations copied fromhmrc_income_source_stages.json. Absorb by copy, not move: the old file stays byte-identical (the coverage manifest and the frozen June evidence sidecars pin it); a lockstep test fails if the copies diverge; the original retires at the FRS 2024-25 refresh. - Add the 10 existing HMRC operation kinds to
ALLOWED_SOURCE_OPERATION_KINDSinbuild/source_manifest.py. - New
uk_runtime/source_runtime.py::uk_source_operation_handlers()mirroringus_runtime/source_runtime.py— handlers delegate to the existing executable modules, so dispatch moves but behavior does not. - Migrate
build_uk_national_datasetfromSequence[UKNationalStage]toplan.Stage/StagePlanviacountry_stage_plan(load_country_spec("uk"), implementations). Stage names unchanged so checkpoint identities remain valid;UKNationalStagekept as a deprecated shim for one release. - Thin, country-parameterized rules-engine adapter:
frame/adapters/policyengine_uk.pyimplementing the sharedframe/rules.pyprotocol (variable metadata + materialization only for now), plus a sharedmaterialize_rules_engine_predictorsoperation kind. The engine-country-must-match-dataset-country check lives in the shared layer; each stage declares its materialized predictors in the manifest so the dependency stays attested. - Known landmine:
FORBIDDEN_SOURCE_DEPENDENCIESsubstring-matchespolicyengine-uk-datainside the HMRC manifest'sreviewed_sourcecitation strings. Rephrase the citations in the copy (keep the DOI and UKDS study number); do not weaken the rejection.
Acceptance
load_country_spec("uk").sourcesis non-None;country_stage_planassembles the two-stage plan.- A staging rebuild from the same inputs is payload-identical to a pre-migration build (
tools/compare_uk_h5_payload.pyexit 0); all 13 gate verdicts unchanged. - The lockstep test pins the copied declarations to the frozen original.
UK-specificity note (required in the PR)
Expected UK-specific surface: the adapter module (country parameterization is shared), the UK handler map, and the copied stage declarations. Everything else (op-kind allowlist entries, the materialize_rules_engine_predictors op, StagePlan use) is country-agnostic. The PR lists anything beyond this and why.
Coordination
#666 (Logbook adoption) touches the same driver — coordinate merge order.
Activity