Skip to content

Resolve catalog supersede links against every row's effective assertion id - #306

Open
MaxGhenis wants to merge 3 commits into
codex/thesis-ledger-factsfrom
fix/catalog-effective-assertion-ids
Open

MaxGhenis wants to merge 3 commits into
codex/thesis-ledger-factsfrom
fix/catalog-effective-assertion-ids

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Defect

scripts/build_series_catalog.py build_catalog built its assertion-version id set only from rows that carry assertionVersion.id. It then refused any correction whose supersedes names a pre-versioning row, which covers every row of the 128-line immutable prefix, with assertionVersion X supersedes unknown version Y.

receipt 0.6.2's append gate works differently. check_rows gives every row one effective id and reserves all of them. For a versioned row that is its explicit id, which must equal the recomputed av2 content address. For any other row it is that address. A correction must name the active version's effective id, so the gate accepts a correction that names a prefix row's address. effective_current_rows applies the same id rule through _effective_assertion_id.

The preconditions added in 3df42b9 were meant to mirror the gate ("standalone --observations runs are as safe as gate-validated ones"). The Thesis resolver runs this generator on every append: at ThesisInstitute/thesis@95ca41c8, ledger_catalog_refusals (scripts/resolve_pending.py:13815) and _regenerate_series_catalog (:13025, called from _prepare_release_files at :13178) run the branch-head build_series_catalog.py in write mode on the ledger plus the run's rows. If the base alone fails, the first screens nothing, and a non-zero exit in the second aborts the proposal. So one gate-accepted correction merged to the ledger would have made every later append fail.

Repro (2026-09-29). I took six synthetic corrections of the June 2026 Table A-19 prefix rows (ledger lines 119-124), each with supersedes set to that row's av2 address. receipt check_rows accepted them and the generator exited 1. Evidence: ~/chief-of-staff/state/thesis-resolution-backlog-2026-09-19/scratch/exp-0929/. With this change, both correction cases exit 0. The August-only case still refuses on its millions/thousands unit conflict; that is decision d397 and this PR does not touch it.

Fix

check_assertion_versions(rows), extracted from build_catalog, now does four things:

  • It computes every row's effective id the way receipt does, via effective_assertion_version_id: the explicit assertionVersion.id, else check_thesis_facts_append.expected_assertion_version_id(row).
  • It reserves all of those ids for the duplicate check. Two identical unversioned rows now collide here as they already do at the gate, and so does a versioned row that reissues an unversioned row's address.
  • It resolves supersedes links against those ids.
  • It keeps the refusals for malformed ids and links, self-links and cycles unchanged.

A row whose content address cannot be computed (for example a non-object measure) is refused with a message instead of a traceback.

No ledger rows are added or changed. Which six-row remedy to use for A-19 is still Max's open decision d397. The June 2026 A-19 targets are registered in thousands, so restating June in millions conflicts with June's contract.

Invariants (tested)

  1. Gate soundness. If receipt 0.6.2 check_rows accepts a journal, the generator's assertion-version preconditions accept it too. Property-tested over gate-valid journals built to include corrections of unversioned (prefix) rows. This is the direction the defect broke.

  2. Effective-id agreement. For every row, check_assertion_versions returns receipt's own _effective_assertion_id(row, APPEND_GATE_SPEC). This is a differential test against receipt's function, run on the generated journals and on the live ledger.

  3. Reason agreement ("refuses a link iff the gate does for that reason"). This holds on journals with at most one defect, whose explicit ids are their content addresses (the gate's first row rule). There the generator refuses for reason R exactly when the gate's refusal classifies as R. The reasons are duplicate id, self-link, unknown target and cycle. The classifier reads the gate's first refusal and places its target among the gate's own effective ids: the row itself, no row, or a row whose chain leads back to it.

    Some refusals belong to the gate alone, and there the generator must accept:

    • a link to a non-active, other-record or later version outside a cycle
    • a correction with no link

    The defects are unknown, self-link, cycle of two or three rows, arbitrary relink, unlink, duplicate row and A→B→A restore. At the committed 400 derandomized examples, the single-defect property reaches every class: cycle 12, duplicate 46, self 60, unknown 38, gate-only 13, accept 231. Unversioned prefix-style rows appear in 74% of the journals.

  4. Refusal containment. With up to three defects, every generator refusal is also a gate refusal. When the gate refuses for a generator reason, the generator refuses too. This includes rows whose explicit id is rewritten away from their content address (the "reid" defect). The gate refuses those outright. The generator trusts explicit ids, so it can still refuse such a journal for a duplicate or orphaned link, which is why invariant 3 is not claimed for them.

  5. No-op on the live ledger. The rendered catalog is byte-identical before and after this change (sha256 226174876fce03074161dad38d1d0c04dcad3843008a53911e7b9a05d275425a, equal to the committed ledger/series_catalog.json). test_committed_catalog_is_current_and_valid pins it.

The generator's precondition set is deliberately a subset of the gate's. The gate's other rules stay the gate's alone: an explicit id must equal the content address, and a link must name the active version of the same record on an earlier line, so no version is superseded twice. A fork, where two corrections supersede one version, passes the generator and is refused at the gate.

Tests

tests/test_build_series_catalog.py, 162 passed locally (Python 3.14, receipt 0.6.2, ruff 0.14.10):

  • test_live_ledger_reserves_the_gates_ids: the gate accepts the live ledger, the 128 prefix rows carry no assertionVersion, and the generator's ids equal the gate's.
  • test_correction_of_a_prefix_row_is_accepted_like_the_gate: the A-19 case on the live ledger. The gate accepts, the preconditions accept, and a full build_catalog gives no mints, supersedes or drops, the six occupation series in millions, and the current row count from effective_current_rows. The corrections are synthetic and each supersedes its record's active version. Their archive digests are synthetic too, so no real correction can collide with them.
  • test_every_rows_effective_id_is_reserved: identical unversioned rows, a versioned reissue of an unversioned row's address, and A→B→A restores (from an unversioned and from a versioned A) are all refused, matching the gate's "restates".
  • test_link_refusals_fire_where_the_gate_refuses: unknown target, a near-miss address, a self-link and cycles of two and three rows.
  • Three Hypothesis properties for invariants 1-4 (400 examples each, derandomized, no example database). The adversarial review also re-ran them randomized at 5,000 examples each, and all passed.

Mutation check. I ran seven mutants of check_assertion_versions against the suite and every one fails it:

  • the original skip-unversioned-rows logic (6 tests fail)
  • unversioned ids not reserved
  • no duplicate check
  • no self check
  • no unknown check
  • no cycle check
  • a wrong content address

Thesis coupling (follow-up, not a blocker)

At ThesisInstitute/thesis@95ca41c8, bls_ledger_unit_conflict (scripts/resolve_pending.py:9058) builds its current view by dropping only rows whose explicit assertionVersion.id is superseded. So after a correction of a prefix row it still counts the June rows. Its docstring (:9073-9078) says that is deliberate and fails closed until "a reviewed change follows". docs/anchor-verifications.md (:1531-1541) already says the change that lifts the A-19 hold must update that guard.

After this merges, three things in Thesis describe the old generator: that docstring, the doc paragraph ("learns version ids only from rows that carry one ... exits 1"), and the comment in tests/test_bls_api_registrable.py::test_the_guard_follows_chronicles_supersede_links (:1100-1101). I'll open a Thesis follow-up for them. Nothing in Thesis parses the generator's messages.

Adversarial verification

A four-lens workflow ran against ab986a4. The integration lens replayed the resolver's two generator calls with the stage file set and refusal logic taken from Thesis:

  • With this PR, the corrected base passes, the registry is unchanged and _validate_catalog_binding passes.
  • With 598f394, the base fails ("supersedes unknown version") and every append would abort.
  • August A-19 rows in millions are still refused individually.

The review lens found no code bug. Its wording findings are fixed in ee04eac.

Also in this PR

  • 7a409f8: pin the three September claims-week spellings. CI "Arch checks" fails on the branch head 598f394. That resolver append first spelled week_2026-08-29, week_2026-09-05 and week_2026-09-12, and the stripped-segment pin didn't list them. Each is its row's own week_ending period (us.dol.initial_claims.sa, dol.eta.continued_claims.sa), so the strips stand.
  • Dependencies. hypothesis>=6.100,<7 is added to the dev extra and the dev group. uv.lock gains only the hypothesis 6.168.3 entry. A full uv lock (uv 0.11.7 and 0.12.20) would also have moved pandas 2.3.3 to 3.0.6 and policyengine-us 1.670.2 to 2.17.1, so I didn't take it. uv lock --check passes on both uv versions, and a locked all-extras sync plan installs hypothesis with pandas and policyengine-us unchanged.

Neither ledger/** nor the gate surface is touched, so the append gate sees an unclassified-only change.

axiom: n/a: catalog-generator infrastructure, no policy change

🤖 Generated with Claude Code

MaxGhenis and others added 3 commits September 29, 2026 16:28
The 598f394 first-print append spelled week_2026-08-29, week_2026-09-05
and week_2026-09-12 for the first time, and the stripped-segment pin in
test_committed_catalog_is_current_and_valid did not list them, so Arch
checks failed on the branch head. Each is the row's own week_ending
period (us.dol.initial_claims.sa and dol.eta.continued_claims.sa), so
the strips stand and the pin is extended.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
build_catalog collected assertion version ids only from rows that carry
assertionVersion.id, so a correction superseding a pre-versioning row
(every row of the 128-line immutable prefix) was refused as superseding
an "unknown version". receipt 0.6.2's append gate addresses such a row by
its recomputed av2 content address and accepts the correction, and the
Thesis resolver runs this generator on every append, so one
gate-accepted correction would have failed every later append.
Reproduced 2026-09-29 with six synthetic corrections of the June 2026
Table A-19 prefix rows.

check_assertion_versions now gives every row the gate's effective id
(the explicit id, else expected_assertion_version_id(row)), reserves
all of them for the duplicate check, and resolves links against them.
The self-link and cycle refusals are unchanged. Two identical
unversioned rows, and a versioned row reissuing an unversioned row's
address, are now duplicates here as they already are at the gate.

Tests: the live ledger gets the gate's ids (catalog bytes unchanged:
sha256 226174876fce...425a before and after); the A-19 correction case
passes the gate, the preconditions and a full build; duplicate, unknown,
self and two- and three-row cycle refusals match the gate's; Hypothesis
properties check gate-accepted journals pass, and on single-defect
journals the generator refuses for a reason iff the gate's refusal
classifies as that reason. hypothesis is added to the dev extras and
group; uv.lock gains only its entry (a full relock would have moved
pandas and policyengine-us a major version).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of #306 found wording that overstated the change: the gate-only
list in _gate_reason included an explicit id that is not the content
address, but the generator trusts explicit ids and can refuse such a
journal for a duplicate or orphaned link. The single-defect "iff" is now
stated for journals whose explicit ids are their content addresses, and
the containment property also rewrites ids ("reid") to hold the rest to
"every generator refusal is a gate refusal". check_assertion_versions
now says it enforces the part of the gate's rules the current view
relies on, names the gate-only rules (content address, active version,
earlier line, no version superseded twice), and says a duplicate needs
an exact restatement of addressed content, not just a restored value.
The receipt import comment says check_rows reserves the same id inline,
and the import-failure message names the content address too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant