Repository navigation
ci: fail a PR that deletes an e2e attestation receipt - #1052
Merged
Merged
Conversation
Adds the failing unit, integration and e2e tests for a new `dirsql-checks attestation-guard` check, against inert stubs, plus the workflow that will invoke it. The `e2e-verify` gate is diff-based and asks only whether a branch adds or updates ITS OWN receipt, so a diff that deletes other branches' receipts passes it green. Receipts are append-only and nothing can regenerate a merged branch's, so the rule asserted here is total: any deletion under an `e2e-attestations/` directory fails, with an `allow-receipt-deletion: <reason>` commit-body bypass mirroring changelog-gate's `skip-changelog:`. 30 tests fail against the stubs; the implementation lands next.
Fills in the stubs the previous commit left inert: receipt-path detection, the `allow-receipt-deletion: <reason>` bypass and its near-miss diagnostic, and the failure report naming every deleted receipt plus the exact `git checkout <base> -- <paths>` that restores them. `--no-renames` on the diff so a receipt moved to another path counts as a deletion of the original — a receipt's path is its branch slug, so renaming one loses the record as surely as removing it.
Kills a surviving `*` -> `/` mutant on the signature: the mutant turns the positional-only marker on, which every keyword-passing test still tolerates.
8 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1043
The hole
e2e-verifyis diff-based and asks only whether a branch adds or updates its own receipt. A diff that deletes other branches' receipts is invisible to it — #1036 merged exactly that, 28 receipts, fully green with 65 passing checks. Anything that deletes a receipt still slips through: a stale globaltesting-conventionsinvoked by hand, a badgit add -A, a botched rebase.This adds a
dirsql-checks attestation-guardcheck and its own workflow lane that fails a PR whosebase...HEADdiff deletes any e2e attestation receipt.This is a repo-side guard, not a tool-version workaround. It holds at any
testing-conventionsversion: nothing should ever delete a receipt it does not own, and nothing can regenerate a merged branch's.Design decisions
The rule is total, not ownership-scoped. The issue title says "a receipt it does not own"; the check fails on any receipt deletion. Reasons:
agents/reference/e2e-attestation.md) — a merged branch's receipt is the standing record that its suite ran.Renames count as deletions. The diff runs with
--no-renames, so a receipt moved to another path registers as a deletion of the original. A receipt's path is its branch slug, so renaming one loses the record as surely as removing it.Escape hatch follows
changelog-gate. Anallow-receipt-deletion: <reason>line anywhere in any commit body bypasses the gate, scanned over raw%Bbodies rather than git's trailer parser (same reason asskip-changelog:— a blank line must not split it off). The reason is mandatory, so a bare word cannot disarm the gate by accident.Near-miss diagnostics. A bypass that was attempted but misspelled (wrong verb, wrong noun, pluralised, reason omitted) is quoted back with the exact accepted form rather than falling through to a generic failure.
Actionable output. Every deleted receipt gets its own
::error file=annotation, followed by the literalgit checkout <base-sha> -- <paths…>that restores them all. Verified below: the printed command was pasted verbatim and turned the check green.Wiring
Its own workflow,
.github/workflows/attestation-guard.yml, filtered on'**/e2e-attestations/**'(plus the check's source and the workflow file).packages/python/,packages/ts/,plugins/dirsql-plugin-embeddings/andinternals/checks/, so no single package lane sees them all.changelog-check.ymlexplicitly filterse2e-attestationsout of its paths, so it was not a candidate.**filter is the point: a deletion can ride in any diff, and the receipt path itself is the only reliable trigger.CI Gatewaiting for a run that never appears.The
run:step is one invocation; all logic (receipt-path detection, bypass scanning, the restore instructions) lives ininternals/checks/src/checks/attestation_guard/, split into a puredecide.py/report.pyand a thingate.pycomposition root, per AGENTS.md "CI Workflows".Red/green
RED commit
c620e58pushed on its own against inert stubs. CI went red for the right reason:Red run: https://github.com/thekevinscott/dirsql/actions/runs/33935055548 — 25 unit failures + 1 integration failure, every one of them a new
attestation_guardtest asserting the guarded behavior:Unit-test coverage (python)—25 failed, 433 passeddecide_test.py::describe_deleted_receipts::{keeps_only_paths_under_an_e2e_attestations_directory, spans_every_package_root_that_holds_receipts, sorts_the_result}—assert [] == ['packages/py...laude-a.json']decide_test.py::describe_has_allow_line::{accepts_the_bypass_with_a_reason, accepts_it_regardless_of_case, accepts_it_from_any_line_not_only_a_formal_trailer}—assert False is Truedecide_test.py::describe_near_miss_lines::{names_a_bypass_spelled_with_the_wrong_verb, …_wrong_noun, …_missing_its_reason, …_missing_its_colon, strips_surrounding_whitespace}—assert [] == ['skip-receip...ion: cleanup']gate_test.py::describe_run::{fails_when_the_diff_deletes_a_receipt, names_the_restore_command_with_the_base_sha, fails_and_names_a_near_miss_bypass, reads_commit_messages_over_the_same_range}anddescribe_run::passes_when_a_bypass_line_is_present—assert 0 == 1gate_test.py::describe_verdict::reports_every_deleted_receipt—assert 0 == 1report_test.py::describe_restore_command::names_the_base_sha_and_every_pathand 6describe_reportcases —assert '' == 'git checkout...a.json b.json'internals/checks integration tests—1 failed, 7 passed:tests/integration/attestation_guard_test.py::describe_run_against_a_real_repo::it_fails_when_the_branch_deletes_another_branchs_receipt—AssertionError: assert 0 == 1GREEN commit
4e22dbefills the stubs in; CI flips red → green.f1179a1adds one further assertion to kill a surviving mutant thatunit mutationfound:ReplaceBinaryOperator_Mul_Divongate.py:30turns the keyword-only*marker into the positional-only/, which every keyword-passing test tolerated. Killed with a new assertion (run("base", "head", <positional>)must raiseTypeError), not by weakening anything.Manual exercise
Against a real clone of
mainwith real receipts, using the packageddirsql-checksconsole script.1. A branch that adds its own receipt and prunes three siblings — fails, names each file:
2. The printed restore command, pasted verbatim — legitimate diff, passes:
3. A misspelled bypass — named as a near miss, not honoured:
4. The correct bypass — honoured:
5. Dogfooded on this branch (
base = origin/main,head = f1179a1):No e2e attestation receipts deleted.EXIT=0. Everye2e-attestationsentry in this PR's file list isadded.Gates
just preflight --conventions .github/workflows/internals-checks-ci.yml— 0 failing pairs, 0 skipped:uv-sync,declared-deps,colocated-test,one-function-per-file,unit-lint,integration-lint,unit-coverage,mutation(34 mutants, no survivors),e2e-verify. No exemptions added.E2E Verification
packages/pythonchangepackages/tschangepackages/rustchangepackages/python/e2e-attestations/<branch>.jsonreceipt written ifpackages/pythonchanged — N/Apackages/ts/e2e-attestations/<branch>.jsonreceipt written ifpackages/tschanged — N/Ajust e2e-attest-internals-checks(cd internals/checks && uvx testing-conventions e2e attest 'uv run python -m pytest tests/e2e -x -q')internals/checks/e2e-attestations/claude-1043-receipt-deletion-guard.jsoninb4f20fb, whichgit show --statconfirms is a single added file.Changelog / Migrations
changelog_gate/decide.pyscopes to_ROOTS = ("packages", "plugins");internals/is outside it, andchangelog-check.yml's path filter does not match this diff. Verified, not assumed.PARITY.md: unchanged — no SDK public API touched.