Skip to content

Qualify each new policyengine-uk or uk-data release automatically before promotion #573

Description

@MaxGhenis

UK certification is started by hand today (scripts/bundle.py certify-data, provenance/certification.py). Its CLI checks manifests, compatibility claims and whether the default artifact exists. It runs no simulation and does not check core compatibility.

The last three certifications took 10h37m (#555), about 20h43m (#536) and about 8d6h (#491) from model release to merge. Manual review caught real defects:

With no model freeze through the UK Autumn Budget (28 October 2026), every new pair needs to be qualified automatically and promoted only if it passes.

Scope

  1. Trigger. .github/workflows/certify-uk-release.yaml runs on:

    • a policyengine-uk wheel publish, by repository_dispatch;
    • a uk-data release, by repository_dispatch;
    • a scheduled catch-up.

    Candidates are deduplicated by pair identity.

  2. Candidate commit. A candidate commit regenerates the manifest, pins, uv.lock, TRO and test constants together. That commit is what qualification runs, what the replay corpus replays, and what promotion merges.

  3. Mandatory gates (scripts/check_uk_candidate.py), none of which can be skipped:

    • installed wrapper, model and core versions;
    • wheel hashes, by RECORD verification;
    • model/core compatibility;
    • authenticated data access;
    • dataset bytes;
    • supported years;
    • required Budget inputs;
    • W1's reform contract;
    • W3's metrics under review bounds against the previous certificate;
    • W6's replay corpus, via an immutable pre-promotion request and an artifact-bound receipt.
  4. Ledger. Qualification receipts are recorded in a ledger with a verdict that is a pure function of the gates, and failure evidence is kept.

  5. Promotion. Promotion is serialized and ordered: a stale completion never displaces a newer accepted candidate. Qualification, consumer adoption and deployment are tracked separately.

  6. Compatibility claims. Exact tested compatibility claims (==version) are backed by the qualification receipt and never broadened to get a green check.

  7. Consumer dispatch after release goes to sim-api, the dashboard and the scorecard, with a richer payload.

Automatic promotion and the review bounds are methodology calls under cos d1344. Until that is ruled, promotion is a reviewed PR merge.

Refs #462, #570.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions