Scope diagnoses, notes and construction premises to their engine

[MaxGhenis]

Stacked on #160. WIP: W6 of the Autumn Budget 2026 CI plan.

Why

compare_uk_event.py attached ten review diagnoses established on policyengine-uk 2.89.2 to every comparison row, whatever engine produced the row, with "engine_version": "2.89.2" hardcoded. Its construction notes also stated 2.89.2 facts as facts about the run: the £3.15 Class 2 rate, "the pin still has the 3% surcharge scale". diagnose_uk_event_models.py refused any other engine. A rerun on a new certified variant would therefore present stale findings as current.

What changes

• Version-scoped diagnoses. On 2.89.2, output is byte-identical. On another engine, each review diagnosis moves to prior_diagnoses with established_on: ["2.89.2"] and status: "not_reverified", and is rendered under its own heading. Authored notes are prefixed "Authored against policyengine-uk 2.89.2", and the 2.89.2-only corrections are dropped.
• Variant-aware reports. Event reports and the summary state the run's execution (engine, wrapper, data build, data years) from its RUN_MANIFEST.json, bound in COMPARISON_PROVENANCE.json. They show unsupported years ("complete on supported years; unsupported: 2023").
• Re-verifiable measured candidates. diagnose_uk_event_models.py --reverify <prior> --expect-engine <v> reruns all ten small synthetic cases on the installed engine. Each case's own predicate records persists, resolved or inconclusive. A moved file or missing variable is inconclusive, never resolved.
  • On 2.125.1, three are resolved: CGT commencement (now day-weighted), the Class 4 freeze, and employer NICs above State Pension age.
  • Seven persist: Carer's Allowance earnings test, SDLT surcharge rise, private-school VAT lever, dividend lag, UC LCWRA cutoff, pension taper, and the AA charge rate.
• Construction premises. pipeline/audit_uk_event_premises.py records, per engine and parameter tree only, what each executed world holds at every annual lookup. It checks four premises: forward deltas change current law; reversals differ from it; authored constructions re-derive identically; and annualisation is a 30 April sample.
  • Between 2.89.2 and 2.125.1, 15 of 16 constructions keep every premise, including the derived UC counterfactuals.
  • The AB2024 CGT reversal does not. 2.125.1 day-weights gov.hmrc.cgt.*_rate (fiscal_year_blend, utils/parameters.convert_to_fiscal_year_parameters), so CY2024 current law is 0.1346 in both worlds.
  • docs/uk_replay/CONSTRUCTION_REVISIONS.md proposes a revision. Running it changes the construction and waits on cos d1344. Nothing here changes a registered construction.

Committed outputs:

• results/uk/events/CONSTRUCTION_PREMISES.json (2.89.2)
• results/uk/replays/pe6.2.6_uk2.125.1_policyengine-uk-data-1.58.0/{CONSTRUCTION_PREMISES,MODEL_DIAGNOSTICS}.json (2.125.1)

Invariants and tests

tests/test_uk_event_version_scope.py and tests/test_uk_event_premises.py check:

• Legacy review diagnoses and notes are unchanged. On another engine, no diagnosis is current and no note asserts a 2.89.2-only fact (marker list).
• Every predicate returns persists on its founding 2.89.2 observation and resolved on the corresponding fixed observation. A missing file or variable gives inconclusive. Predicates are total (Hypothesis over arbitrary observations).
• Premise checks hold or fail correctly on stub engines: inert forward deltas and reversals, in-force boundaries, October dates moving to the next January, blend detection only where the engine reads the metadata, and authored-construction drift.
• Output is canonical and independent of dictionary order, with no NaN or infinity.
• The committed premise files regenerate byte-for-byte on their engines.

Legacy byte preservation: all five events re-staged and re-compared, plus the summary, give a clean git diff on results/uk/events. The 2.89.2 diagnostics run is identical to origin/main's script output.

Status

• Implementation and tests (250 replay tests passing)
• Independent review

axiom: n/a: replay reporting and audit tooling in scorecard, no policy encoding.

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
policyengine-scorecard	Ready	Preview	Oct 11, 2026 9:01pm UTC