Repository navigation
Conversation
A receipt names exactly what produced a published result. It records the installed wrapper, model and core versions; the data release and the SHA-256 of the simulated dataset bytes; the certificate digest, attached only when it covers that model and those bytes; reform and baseline digests bound to their representation; and the fiscal-year convention. Digests use RFC 8785 canonical JSON, so the dashboard and scorecard can recompute them. The JSON Schema ships in the wheel. Fixes #570 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ificate Independent review of #571 found ways a receipt could claim more than what ran. Reform digests no longer collide for Scenario.from_reform closures, lambdas, partials or bound methods: those are refused, and a module-level modifier is bound by source, bytecode and revision. Digests are over strict JSON, so numpy scalars, sets, Decimals and colliding keys are converted exactly or refused instead of stringified. canonical_json writes numpy floats and int subclasses as their values. Certification now requires the installed core version, records whether the certificate was packaged or caller-supplied, and verify_certificate checks it against a published manifest. Wheel hashes are recorded only when the builder verified the installed files against the wheel RECORD. The execution identity no longer depends on how the builder was called, reform and baseline are required, models parse strictly, overlay datasets get uncertified receipts, and v1 covers UK results. Refs #570 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- verify_certificate rebuilds the certificate from the bundle record with the builder's own derivation and compares every identity member plus the whole data identity; a forged receipt that keeps the genuine record digest is now reported. The "or require certificate_source packaged" rule is gone. - Modifier identity no longer digests marshalled bytecode or the Python version, so the digest is the same in any checkout. The source is recompiled and compared with the loaded code (ignoring file and line numbers), defaults and the module source are bound, and code_revision needs a full commit sha. - Wheel verification hashes every archive member against the wheel's own RECORD (one to one) and exempts only the launchers an installer generates for the wheel's declared console/GUI scripts; stray .pyc and INSTALLER files outside their installer locations fail. - The public metadata_module hook is removed; versions always come from importlib.metadata. - scenario_reform_identity refuses anything that is not a Scenario. - strict_json_value refuses numpy longdouble and deep nesting with ReceiptError; content_sha256 raises ReceiptError for unsafe integers. - The schema fixes package names and the UK fiscal-year convention as consts, and drops the never-populated release_manifest_sha256. Refs #570 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Before binding a modifier, the module's source as read now must match what was imported: every module-scope function in that file compiles to its loaded code, and every module-scope name bound once to a literal still holds that value. Edited helpers, edited constants and constants reassigned at run time are refused instead of named by the digest. - Functions defined in module-level blocks bind (source is dedented before compiling); methods are refused with a clear reason. - Unreadable wheels (not a zip, duplicate RECORD entries) raise ReceiptError from installed_package_identity. - Tests now kill the round-3 surviving mutants: each forgery alone, the record-version and basis checks, operator-only and nested-code edits, forged function globals and duplicated wheel members. Refs #570 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Wheel verification now checks bytecode: every __pycache__ file this interpreter would load for a verified source (any optimisation level, listed in RECORD or not, under sys.pycache_prefix) must compile from that source. Caches the interpreter would ignore are reported as inert. Launchers are exempt only when they exist in bin/Scripts outside the site directory. - The microsimulation adapter requires the file to still hash to the SHA-256 verified at load time, and the receipt to name the dataset, URI, revision and package recorded then; unmanaged loads cannot be receipted. - Data identity is resolved with the materializer's own resolver (overlays, per-dataset package and revision, release manifest revision), and a receipt never gives an artifact's name to other bytes. - Modifier binding: helper defaults, block-scoped literals and comprehension scope are checked; every module function's defaults are bound with their types; foreign helper globals and conditionally defined functions with different code are refused. - Only .whl direct-URL archive hashes are recorded as wheel_sha256. - Masked, 0-d, complex, datetime and structured NumPy values are refused. - Years must be integers and schema_version may not be a boolean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Receipt v1 gains a required supported_years: the years the installed policyengine-uk projects the dataset to, from its own year, computed as the qualification supported_years gate does (extend_single_year_dataset, no formula run), cached per file bytes and model version and never taken from the caller. A reported year outside it is unsupported (the unsupported_years property), and the published rules say its results must be shown as provisional or not at all. The certified 1.58.0 dataset projects to 2024-2030, so a 2031-32 figure on it is flagged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Bytecode: a cache with flags 2 is timestamp-based (importlib reads the
check-source bit only for hash-based caches), unknown flag bits and
unreadable caches are inert, and a cache reached through a link is
checked once for each source it stands for.
- Code constants compare by exact type and float bits, item by item, so
(1,) and (1.0,), True and 1, or 0.0 and -0.0 are different code.
- Reform and scenario digests keep nulls: Scenario.from_reform reads
{"2027": None} unlike {}.
- Module binding: the module is compiled whole (a function compiled alone
can compile differently, as math.copysign did on 3.13); a name the
source binds only with def must still be that function or a wrapper of
it (through __wrapped__); functions count as module-level by their code
object's own qualified name, which functools.wraps cannot copy.
- Binding analysis follows lexical scopes: a lambda body is its own scope;
a def's defaults, decorators and annotations bind in the module.
- Float defaults are encoded by their bits; NumPy booleans are refused as
years and as schema_version.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
added a commit
that referenced
this pull request
Oct 12, 2026
- Compositions use the null-preserving payload digest of #571's round 5, as every reform digest now does (composition payloads hold no nulls, so existing composition digests are unchanged). - Tests: a tuple and a list of members give the same composition, and an invalid phase is refused even when no member changes anything. - Docs: a baseline shared by several members (parameter changes only) appears once, at the first sharing member's position, which cannot affect the run (from the dashboard's composer, workstream W5). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A module-level modifier that called a module-level functools.partial got one digest whatever the partial's captured values were: W5's review of uk-autumn-budget-dashboard-2026#23 edited PAYMENT.keywords["amount"] from 120 to 240 and paid twice as much under the same reform_sha256. Every module global the modifier reads, directly or through functions defined in its file (from the bytecode, nested code and decorator wrappers included), must now be a literal constant (checked by value), a function defined in the file (checked against the source), an imported module or callable, a builtin or a name the import system sets. A partial or other object built at import, a value computed from the environment, a file or the clock, a class defined in the file, an imported value, a name set on the module after import, and code that assigns module globals are refused, naming the global. State in other modules stays bound by the code revision or installed versions, as documented. Invariant (property-tested): an identity is issued if and only if every global the modifier reads binds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Float and complex code constants compare by their IEEE 754 bytes: float.hex() wrote both signs of NaN as "nan", so a cache with a NaN of the other sign passed wheel verification (H1). - A def name must still be bound to the function of that name, not to another function of the same file that matches its own source (M2). - A constant bound by unpacking is no longer a checked literal and is now refused when read, through the computed-globals rule (M3). - Two defaults that are one mutable object (in any function of the module, nested, or a container holding itself) are refused: the value encoding cannot tell them from two equal objects (M4). - Module literals compare exactly by type, float bits and key order, for every literal kind, so unedited bytes, set and complex constants bind (L5); a := in an evaluated module annotation unbinds a constant (L6). - The reform digest's description says nulls are kept (L7). - Tests kill the three round-6 mutation survivors: the exception table, unknown pyc header flags and the projection's first-year filter. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Round 7 reproduced state a digest missed through channels the bytecode scan could not see: globals()/vars()/getattr on the module, decorator closures and defaults, conditional literals and imports, rebound imports, function attributes and aliased constants. Rather than chase each channel, the module holding a bound modifier now needs a shape whose state its source fixes: - At module level: only docstrings, pass, imports, one-name literal assignments (annotated or not), undecorated defs and a main guard with no else; each name bound once; no := or lambda in defaults or annotations (H/M3/M4). - Every namespace name is one the source binds: literals (exact value), functions (code, no attributes), imports (still what the import binds, holding a module or a non-partial callable; star-imported names judged the same way) (M2/M4/M5). - No two literal constants or defaults are one mutable object, and no code tells equal values apart by identity (is other than against None/True/False/.../type(...), or id()) (M6). - globals(), vars(), locals(), exec(), eval(), compile() and __import__() are refused in what the modifier runs (M2). - Bytecode constants compare with their sharing across the code tree, so a cache splitting ((1,), (1,))'s shared tuple fails verification (H1); 1,689 caches Python wrote for six installed packages still match. - Tests for class-body LOAD_NAME, class type-alias LOAD_FROM_DICT_OR_GLOBALS and DELETE_GLOBAL (L8); a class inside the modifier reads its own names (L7). Tests that exercised the superseded per-binding analyses now assert the shape refusals. The unbindable test modifiers moved to their own module. Mutation screening: 18 of 18 mutants of the new checks are killed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
added a commit
that referenced
this pull request
Oct 12, 2026
- Compositions use the null-preserving payload digest of #571's round 5, as every reform digest now does (composition payloads hold no nulls, so existing composition digests are unchanged). - Tests: a tuple and a list of members give the same composition, and an invalid phase is refused even when no member changes anything. - Docs: a baseline shared by several members (parameter changes only) appears once, at the first sharing member's position, which cannot affect the run (from the dashboard's composer, workstream W5). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Python 3.14 compiles annotations to module-level __annotate__ functions; they are the compiler's, not defs, so an annotated constant and an annotated modifier no longer fail as "two definitions" (M5). - A main guard never runs on import, so its body binds nothing (L6). - Module-wide AST rules replace the bytecode read scan, which every name in the namespace being checked had made redundant: id, operator.is_ and is_not, globals, vars, locals, exec, eval, compile, __import__ and breakpoint are refused by name however imported; importing builtins is refused; type may not be rebound, so `is type(...)` stays safe; global statements are refused; dunder attributes other than __init__/__name__ and dunder names other than __name__/__file__ are introspection (M3, M4, M1/M2 reads). - Each function's name, qualified name, docstring (as the compiler stores it), annotated names and builtins must be what its def made; the module's docstring and builtins too; only the import system's and the annotation machinery's dunders may sit in the namespace unbound (M1, M2). Annotations may only name types (no calls); dunder defs are refused. - Regressions for the twelve round-8 survivors, including sharing into nested code and frozensets (skipped on Python 3.14, which merges equal constants in every code object, so a cache cannot split them there). Mutation screening: 30 of 30 mutants of the round-8 checks are killed. Tests: 412 pass on 3.13; on 3.14 all pass except the two that need policyengine-uk's dependencies, absent from that local environment. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
added a commit
that referenced
this pull request
Oct 12, 2026
- Compositions use the null-preserving payload digest of #571's round 5, as every reform digest now does (composition payloads hold no nulls, so existing composition digests are unchanged). - Tests: a tuple and a list of members give the same composition, and an invalid phase is refused even when no member changes anything. - Docs: a baseline shared by several members (parameter changes only) appears once, at the first sharing member's position, which cannot affect the run (from the dashboard's composer, workstream W5). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Receipts refuse a run whose installed versions changed after loading (H1). policyengine records its version when imported, and each country model records its model, core and wrapper versions when it loads the country package (policyengine.provenance.loaded, first seen kept); both builders compare them with what is installed when the receipt is built, so a notebook that upgrades a package mid-session is told to restart. - typing.get_type_hints, inspect.signature, getfullargspec and get_annotations are refused: annotation values are not bound, only which names carry one (M2). - A generic function (def helper[T], Python 3.12+) is found inside the wrapper the compiler makes for its type parameters, so it binds and is still checked against its source (L3). - Regressions for the nine round-9 survivors: every code_shape field, main-guard naming, identity-call operands and the installed RECORD comparison (L4). Mutation screening: 19 of 19 mutants of the round-9 changes are killed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis
added a commit
that referenced
this pull request
Oct 13, 2026
- Compositions use the null-preserving payload digest of #571's round 5, as every reform digest now does (composition payloads hold no nulls, so existing composition digests are unchanged). - Tests: a tuple and a list of members give the same composition, and an invalid phase is refused even when no member changes anything. - Docs: a baseline shared by several members (parameter changes only) appears once, at the first sharing member's position, which cannot affect the run (from the dashboard's composer, workstream W5). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Round-10 review of #571: - M1: version drift no longer depends on import order. Besides the recorded-version comparison, a receipt is refused when the wrapper's, model's or core's install metadata was written after this process started (psutil process start time against the .dist-info files). A missing version record is allowed: an install that predates the process has one version for the whole process. Known false refusal: installing and importing a package in the same process. - M2: a scenario identity keeps a process-local re-check handle, outside pydantic fields, equality, dumps and digests. Receipt builders recompute reform and baseline identities and refuse if the modifier's module changed since the identity was computed. Loaded identities have no handle and are not re-checked. - M3: PackageIdentity gains optional install_origin (archive, directory, editable, vcs; None for an index install) from PEP 610 direct_url.json. A receipt is certified only if no package is a directory, editable or VCS install without installed_record_verified evidence; the model validator, CROSS_FIELD_RULES and verify_certificate enforce it. Malformed direct_url.json is refused. - L1: from_callable is refused; getattr, hasattr, setattr and delattr need a literal, non-dunder name (except __init__ and __name__). - L2: tests kill review mutants A, B, C, G, H, N, O and V. - L3: literal arithmetic, TYPE_CHECKING blocks, type aliases and *args: *Ts are documented as refused, each with a pinned test. Invariants: identity equality, dumps, schema and digests do not depend on the re-check handle; certified implies no directory, editable or VCS package without verified wheel evidence; a receipt is refused if a named package's install metadata postdates the process. Co-Authored-By: GPT-6.1 Sol via Subfleet <noreply@openai.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tion Round-11 review of #571: - M1: every annotation a module's functions hold must be what its source annotation names (identity for named objects; equality for subscripted generics and unions; under future annotations, the compiler's own string). An edited annotation that changed singledispatch dispatch is refused, and both receipt builders surface the refusal through the identity re-check. - M2: the loaded policyengine, policyengine_uk and policyengine_core packages must come from the distribution importlib.metadata selected: for index, archive and VCS installs the loaded __init__.py is the file RECORD lists; for directory and editable installs it lies under the PEP 610 directory. A checkout earlier on sys.path than the installed release is refused. - L1: a wheel's RECORD must list itself with empty hash and size; RECORD.jws and RECORD.p7s are the spec's only exceptions. Co-Authored-By: GPT-6.1 Sol via Subfleet <noreply@openai.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
CI installs this checkout editable and then installs a released policyengine over it, so the tests run this checkout's code under policyengine 6.3.4's metadata. The loaded-package check added in round 11 rightly refuses that, which failed every receipt-building test in CI. An autouse fixture now takes the real policyengine, policyengine_uk and policyengine_core modules as loaded from the distributions they name; synthetic distributions and modules that tests place in sys.modules are still checked by the real function. The test of the real editable install is marked real_install, runs the check unchanged, and is skipped where the installed policyengine is not an editable install of this checkout. Reproduced locally with a venv that installs the checkout editable and then policyengine 6.3.4: before, 86 failed and 66 errors; after, 670 passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Round-12 review of #571: - M1: annotations are compared structurally and in order (origin, then each argument by the same rule), so Union[Decimal, float] and Union[float, Decimal] differ even when typing's cache hands back an earlier, differently ordered object. - M2: a PEP 610 dir_info install without editable=true copied its files into site-packages, so the loaded __init__.py must equal its RECORD entry, as for index installs; directory containment applies only to editable installs. A plain "pip install ." is accepted and an edited checkout earlier on sys.path is refused. - L1: unary minus or plus on a numeric constant is allowed inside annotations (Literal[-1, 0, 1]), nothing broader. - L2: RECORD.jws and RECORD.p7s are accepted only as members of the verified wheel itself, and uv_build.json only inside the distribution's .dist-info directory. Both install layouts pass: editable 786 passed, 1 skipped; this checkout's src over a released policyengine 6.3.4, 785 passed, 2 skipped. Co-Authored-By: GPT-6.1 Sol via Subfleet <noreply@openai.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #570
What this adds
policyengine.provenance.receipt.ExecutionReceipt(schema v1) is a strict, immutable, JSON round-trippable record of exactly what produced a UK result. It is the contract the UK Budget dashboard (W5) uses for per-measure receipts, and the scorecard (W6) binds to it. Both need it by 14 October.wrapper,model,corepolicyengine,policyengine-ukandpolicyengine-core, read from the installed distributions. The schema fixes the names as consts. A wheel SHA-256 appears only with evidence: eitherwheel_paths=(the builder checks the wheel's members against its RECORD one to one, then the installed files against the wheel), or the installer's PEP 610 archive hash.datacertification_status,certificatecertifiedonly when the bundle certificate covers the installed model and core, those dataset bytes and name, and the running wrapper. Otherwise the receipt isuncertified. The certificate digest is RFC 8785 overdata_releases.uk.reform,baselinecurrent_lawor a digest of{representation, payload}, never normalized across representations. AScenariowith a module-levelsimulation_modifieris bound bycode_revision(see below).fiscal_yearuk_fiscal_year: year Y is the fiscal year starting 6 April Y (2027-28). It is a const in v1.outputs_sha256,runtimeAlso added:
provenance.canonical(RFC 8785);provenance.wheels;build_execution_receiptandbuild_execution_receipt_for_microsimulation;verify_certificate;x-policyengine-rulesandx-policyengine-digests;docs/receipts.md.Invariants (each tested)
verify_certificate(receipt, bundle)rebuilds the certificate and the data identity from the bundle record, using the builder's own derivation. Any consistent forgery of a certified member is reported, even one that keeps the genuine record digest. Tests cover each of 16 single-field forgeries, Hypothesis-generated combinations, and a builder/verifier differential over bundle edits.certificate_sourceis the producer's statement, not evidence.Decimal, arrays, NaN, datetimes, longdouble and deep nesting are refused withReceiptError.rfc8785package (differential) and the W6 golden fixtures byte for byte.code_revisionwith a full commit sha.entry_points.txtare exempt, and stray.pycorINSTALLERfiles fail. The real policyengine-core 3.33.0 install verifies: 199 members and 199 files.Review history
Independent adversarial review by an Opus 5.5 agent; the Codex lanes were unavailable. Each round executed reproductions and mutation runs.
Breaking against earlier heads (v1 is unreleased):
metadata_module=is removed;release_manifest_sha256is removed fromdata;ModelIdentityandCoreIdentity.Re-vendor from this head.
Checks run locally
generate_receipt_schemas.py --checkpasses, and the schema output is identical on pydantic 2.10.6 and 2.14.0.axiom: n/a: provenance/infrastructure contract, no policy change.
🤖 Generated with Claude Code