diff --git a/assets/quickstart/wizard/release-management.svg b/assets/quickstart/wizard/release-management.svg index bc94356a8..b5cb54174 100644 --- a/assets/quickstart/wizard/release-management.svg +++ b/assets/quickstart/wizard/release-management.svg @@ -42,7 +42,7 @@ pmc-roster.md Who is binding release-build.md - How this project builds and signs artefacts: build… + How this project builds and signs artefacts:… release-management-config.md Vote window and pass rule, distribution backend and… release-trains.md diff --git a/docs/designs/2026-09-20-reproducible-releases.md b/docs/designs/2026-09-20-reproducible-releases.md new file mode 100644 index 000000000..b238ab5dc --- /dev/null +++ b/docs/designs/2026-09-20-reproducible-releases.md @@ -0,0 +1,240 @@ + + + + +**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)* + +- [Reproducible releases](#reproducible-releases) + - [What was wrong](#what-was-wrong) + - [Decisions](#decisions) + - [The source artefact is a function of the tag](#the-source-artefact-is-a-function-of-the-tag) + - [What ships is decided once, in the open, and committed before the tag](#what-ships-is-decided-once-in-the-open-and-committed-before-the-tag) + - [The record: commit, SWHID, origin, epoch, digest](#the-record-commit-swhid-origin-epoch-digest) + - [Convenience artefacts are the project's, and reproducibility is their acceptance test](#convenience-artefacts-are-the-projects-and-reproducibility-is-their-acceptance-test) + - [The vote text carries the verification path](#the-vote-text-carries-the-verification-path) + - [Automated signing is an ASF option, gated on all of the above](#automated-signing-is-an-asf-option-gated-on-all-of-the-above) + - [What was built](#what-was-built) + - [Alternatives considered](#alternatives-considered) + - [Known limits](#known-limits) + - [Related work outside this repository](#related-work-outside-this-repository) + + + + + +# Reproducible releases + +| | | +|---|---| +| **Status** | Built, in [apache/magpie#1296](https://github.com/apache/magpie/pull/1296). The ASF automated-signing option is designed and wired but no project has exercised it end to end; the ATR side of the SWHID comparison depends on ATR exposing the value it computes. | +| **Created** | 2026-09-20 | +| **Origin** | Magpie's own `0.1.0-rc1` got a `-1` for a source artefact nobody could regenerate; the follow-up asked for the full reproducible-builds.org treatment, an education step for what a source release should contain, and a path to CI-signed releases. | + +A signature proves who packed an archive. It says nothing about whether the +archive is the tree that was voted on. Reproducibility is what closes that +gap: if two people can regenerate the same bytes from the same tag, the +release manager's machine stops being a trust boundary. This design makes +that property the default for the source artefact, the acceptance test for +anything shipped besides it, and the precondition for letting CI sign. + +## What was wrong + +- The release skills said *"build the artefact"* and left the how to each + project. Magpie's own first RC was a `zip -r` of a working tree: it carried + `__pycache__`, dangling agent-view symlinks, and a shape no voter could + reproduce. `git archive` fixed the contents, but its bytes still depend on + the `git` version that runs it, so two voters could not compare digests. +- Nothing in the process asked a project, once, what belongs in its source + release. `.gitattributes` `export-ignore` was known folklore, not a step. +- "Binary" meant one global rebuild command in the config and a hard-coded + list of the framework's own validators in `release-verify-rc`. Both were + Magpie-specific; neither fitted a project shipping wheels, jars and a + container image. +- The `[VOTE]` mail told voters where the artefacts were and nothing about + how to check them. ATR's default vote text links only the candidate page. + +## Decisions + +### The source artefact is a function of the tag + +The source archive is exported from the tag, never packed from a working +tree, and the export is normalised by a tool the framework ships +([`tools/reproducible-archive`](../../tools/reproducible-archive/README.md)) +rather than by whatever `tar`, `zip`, `gzip` or `git` the release manager +has. The tool applies every rule on +[reproducible-builds.org § Archive metadata](https://reproducible-builds.org/docs/archives/) +— one `SOURCE_DATE_EPOCH` (the tag's committer time), locale-independent +ordering, uid/gid 0, `a=rX,u+w` modes, no PAX `atime`/`ctime`, `gzip -n`, +`zip -X` — and pins the two inputs the page does not mention that still vary +between machines: the builder's `core.autocrlf` / `core.eol` (which `git +archive` would apply to `text` files) and the archive writer itself. It is +stdlib-only and single-file so a CI workflow or a project that does not adopt +Magpie can embed it verbatim. + +The reproducibility of the *source* is the property that matters most, not +the least. The `xz` compromise was a source tarball that did not match the +repository at its stated commit; a distribution that could rebuild the +tarball from the tag and compare would have caught it. Convenience binaries +are the place where "build it yourself" is the traditional advice; the source +archive is where nobody used to check, because it looked like it could not be +tampered with. + +### What ships is decided once, in the open, and committed before the tag + +`git archive` reads `export-ignore` from the tree it archives, so the +decision about what a source release contains has to be in the tree before +the RC tag exists. That put the review into `release-prepare prep`, as an +education step on the first release: list what would ship, classify every +top-level path (source, legal files, inputs to the checks voters run, +foundation metadata, VCS / CI / editor / lint / agent-view metadata, scratch), +check references before proposing an exclusion, confirm each entry with the +release manager, land `.gitattributes` in the prep PR, and record that the +review happened. `release-rc-cut` refuses to cut while it is outstanding. +Later releases get a drift check on new top-level paths. + +### The record: commit, SWHID, origin, epoch, digest + +Every RC records, on the planning issue and in the `[VOTE]`: + +- the commit the tag points to, and the URL of the repository it lives in; +- the [Software Heritage identifier](https://swhid.org/) of the archive's + expanded content, `swh:1:dir:`, with `origin=` and + `anchor=swh:1:rev:` qualifiers; +- `SOURCE_DATE_EPOCH` and the sha512 of the archive. + +The SWHID is the load-bearing one. A directory SWHID is computed exactly as +git computes a tree id — from names, modes and contents alone — so it is +intrinsic to the files: a voter recomputes it from the staged bytes without +git, ATR computes the same value at compose time, and it equals `git +rev-parse ^{tree}` unless `.gitattributes` altered the export (in which +case the difference is itself the record of what was left out). Unlike a +digest of the archive it does not depend on the container, so a `.tar.gz` +and a `.zip` of the same tree carry the same SWHID, and a convenience +artefact can name the source SWHID it was built from. Recording the SWHID +next to the commit and the origin, rather than the commit alone, gives a +reference that outlives the repository's hosting and that ATR can be checked +against directly. + +### Convenience artefacts are the project's, and reproducibility is their acceptance test + +What a project ships besides the source — a binary tarball, wheels, jars, a +container image, a chart — is project-specific by nature, so the framework +assumes none. `release-build.md § Convenience artefacts` declares each one +with its own build command, staging target, reproducibility mode, vote scope +and publish channel, and every lifecycle step reads the list: `rc-cut` builds +and stages, `verify-rc` rebuilds and compares, `vote-draft` lists, +`promote` publishes, `announce-draft` names channels. + +A convenience artefact is *good* only if it is demonstrably built from the +voted source. A binary cannot be reviewed, only rebuilt; rebuilding from the +tag under the same `SOURCE_DATE_EPOCH` and comparing — bit for bit, or with +every difference written down — is the one check that establishes what it +contains. `release-promote` withholds the publish command for an artefact +whose `verify-rc` rebuild did not reproduce, and never holds the source +promotion back for it: the source is the release, the artefact is a courtesy, +and a courtesy that cannot be verified is withheld, not shipped. + +### The vote text carries the verification path + +Every `[VOTE]` body carries a *How to verify this candidate* section: the +record above, the agentic one-liner (`verify-rc`), the project's +human-readable verification page at the RC tag, the convenience artefacts and +their vote scope, the ATR candidate page, and the voter-obligation sentence +from the release policy. Under ATR the drafted body is what the release +manager supplies to the platform; the default text is not allowed to stand. + +### Automated signing is an ASF option, gated on all of the above + +[Infra § Automated release signing](https://infra.apache.org/release-signing.html#automated-release-signing) +lets an ASF project have CI sign what it builds, provided every signed +artefact is reproducible, CI stages only, and a committer re-validates every +artefact bit-for-bit on trusted hardware before publication. The framework +offers it only under `organization: ASF` (resolved from the organization +manifest, `null` elsewhere), as a one-time drafting sub-command that produces +the Infra ticket, the Security Team notification and a workflow PR with no +key material, and it changes the lifecycle only once enabled: the RM pushes a +signed tag, CI builds and stages, `verify-rc` becomes mandatory at a +byte-identical bar with the committer's own trusted-hardware assertion, and +`promote` blocks without that attestation. The agent still holds no key. + +## What was built + +| Piece | Where | +|---|---| +| `repro-archive build` / `check` / `compare` / `swhid` / `recipe` / `epoch` | [`tools/reproducible-archive`](../../tools/reproducible-archive/README.md) | +| `§ Source archive`, `§ Convenience artefacts`, `§ Source-tree validators`, `§ Reproducibility checks` | [`projects/_template/release-build.md`](../../projects/_template/release-build.md) | +| `vote_verification_doc_url`, `reproducibility_doc_url`, `vote_verification_skill`, `automated_release_signing` | [`projects/_template/release-management-config.md`](../../projects/_template/release-management-config.md) | +| First-release `.gitattributes` review (prep Step 2e); `automated-signing` sub-command | [`release-prepare`](../../skills/release-prepare/SKILL.md) | +| Reproducible source build, per-artefact builds, self-check (Step 2b), CI-signed flow (Step 2c), the record | [`release-rc-cut`](../../skills/release-rc-cut/SKILL.md) | +| Config-driven Step 7 validators; Step 9 rebuild-and-compare with SWHID check | [`release-verify-rc`](../../skills/release-verify-rc/SKILL.md) | +| *How to verify* section with the record and the artefact list | [`release-vote-draft`](../../skills/release-vote-draft/SKILL.md) | +| Per-artefact publish, reproducibility gate, trusted-hardware gate | [`release-promote`](../../skills/release-promote/SKILL.md) | +| Rationale for adopters and voters | [`docs/release-management/reproducibility.md`](../release-management/reproducibility.md) | +| CI workflow template (ASF automated signing) | [`projects/_template/workflows/release-candidate.yml`](../../projects/_template/workflows/release-candidate.yml) | + +## Alternatives considered + +- **Plain `git archive` as the build command.** Correct contents, but the + bytes depend on the `git` version: the zip writer's compression and the + tar's PAX handling have changed between releases. A voter on another + version gets `content-identical`, never `identical`, and cannot compare + digests with the RM or with ATR. Kept as the documented fallback recipe + (`repro-archive recipe` prints the GNU tar / Info-ZIP equivalent). +- **Shell recipe instead of a Python tool.** The reproducible-builds.org + recipe needs GNU tar ≥ 1.28 and Info-ZIP flags that macOS's stock tools + lack, and a shell script cannot be unit-tested against git's own tree + hashing. The tool is stdlib-only and single-file so it costs nothing to + embed; the shell recipe stays available. +- **One global `binary_rebuild_command`.** Fitted a project with one binary + and none with several; said nothing about where each goes or whether it is + in the vote. Replaced by the per-artefact list. +- **Holding the source promotion until every convenience artefact + reproduces.** Rejected: the source is the release and its vote is what + passed; a courtesy artefact that cannot be verified is withheld on its own. +- **Extracting the archive and running `swh identify` / `asfswhid` for the + SWHID.** Same value, but it adds a dependency and an extraction step to a + tool that already has every member in memory. The in-memory tree hashing is + tested against `git write-tree` over the extracted tree, so the two agree + by construction; a voter who prefers `asfswhid` gets the same identifier. +- **Recording only the commit and repository URL.** A commit id identifies a + repository object; the SWHID of the expanded content identifies what + shipped, survives a repository move, is the same across archive formats, + and is what ATR computes. Both are recorded; the SWHID is the one to compare. +- **Offering automated signing to every organization.** The policy, the key + provisioning and the approval body are ASF Infra's. Other organizations can + add an equivalent block to their manifest; without one the option does not + exist in the skills. + +## Known limits + +- **Compression bytes.** `identical` requires the same deflate output. zlib's + output at a given level has been stable for years, but zlib-ng or a + different Python build can differ. The SWHID and `compare`'s + `content-identical` verdict are container-independent; the record carries + both so a voter can tell "different bytes, same tree" from "different tree". +- **`export-subst`.** Deterministic for a given commit but it rewrites + content, so the archive's SWHID differs from the repository tree's; the + note `repro-archive build` prints says which case applies. +- **Registry-staged artefacts.** Container images and packages staged in a + registry are compared by digest against a local rebuild; the mechanics of + pulling by digest are the project's, not the framework's. +- **The ATR comparison** depends on ATR showing its SWHID for the candidate; + the value is computed today but not yet exposed in the interface. + +## Related work outside this repository + +- [apache/tooling-actions#37](https://github.com/apache/tooling-actions/pull/37), + an `upload-source-to-atr` action that builds a `git archive | gzip -n` + source archive on CI, computes its SWHID with + [`asfswhid`](https://github.com/apache/tooling-asfswhid), signs it and + uploads to ATR. Same rules, same identifier; the discussion there is where + the "SWHID next to the commit" decision comes from, and where binaries were + agreed to be each project's concern — which is what `§ Convenience + artefacts` encodes. +- [reproducible-builds.org § Archive metadata](https://reproducible-builds.org/docs/archives/) + and [§ `SOURCE_DATE_EPOCH`](https://reproducible-builds.org/docs/source-date-epoch/). +- [SWHID specification](https://swhid.org/) (ISO/IEC 18670:2025) and the + [Software Heritage persistent identifiers](https://docs.softwareheritage.org/devel/swh-model/persistent-identifiers.html) + documentation. diff --git a/docs/designs/README.md b/docs/designs/README.md index 95d0b6811..ba831158b 100644 --- a/docs/designs/README.md +++ b/docs/designs/README.md @@ -23,6 +23,7 @@ what was designed and deliberately not built. |---|---| | [Install, adopt, upgrade](2026-09-13-install-adopt-upgrade.md) | Built, bar two items it names | | [Body-owned configuration layers](2026-09-17-body-owned-config-layers.md) | Proposed — depends on the Incubator PMC and ComDev | +| [Reproducible releases](2026-09-20-reproducible-releases.md) | Built (apache/magpie#1296); the ASF automated-signing path and the ATR SWHID comparison await first use | One document per subject, describing the result rather than the phases it was built in. While a design is being implemented it may be split into plans; when diff --git a/docs/labels-and-capabilities.md b/docs/labels-and-capabilities.md index 92dbbbf4d..61d099eac 100644 --- a/docs/labels-and-capabilities.md +++ b/docs/labels-and-capabilities.md @@ -144,6 +144,7 @@ framework substrate: | `substrate:action-guard` | substrate | Deterministic pre-tool-use command guards. | | `substrate:privacy` | substrate | PII redaction / approved-LLM gating. | | `substrate:framework-dev` | substrate | Build / validate / eval the framework itself. | +| `substrate:release` | substrate | Release-artefact helpers an adopter's release process runs: reproducible-archive build, lint and comparison. | ### Coverage qualifiers @@ -337,6 +338,7 @@ or a contract-free mix of substrates (e.g. `tools/spec-inventory` is | [`tools/spec-status-index`](../tools/spec-status-index/) | `substrate:framework-dev` + `substrate:analytics` | Index of spec / RFC implementation status — framework-dev substrate that also doubles as a governance/stats view (`analytics`) | | [`tools/vendor-neutrality-score`](../tools/vendor-neutrality-score/) | `substrate:framework-dev` + `substrate:analytics` | Deterministic vendor-neutrality score — reads each contract tool's `**Kind:**` / `**Vendor:**` metadata and scores per-contract + per-skill neutrality (`analytics`); backs the score block in [`docs/vendor-neutrality.md`](vendor-neutrality.md) | | [`tools/spec-validator`](../tools/spec-validator/) | `substrate:framework-dev` | Spec-frontmatter and body-section validator — counterpart to `skill-and-tool-validator` for `tools/spec-loop/specs/` | +| [`tools/reproducible-archive`](../tools/reproducible-archive/) | `substrate:release` | `repro-archive` — build, lint and compare reproducible source archives from a git ref (`git archive` + `.gitattributes` `export-ignore`, every reproducible-builds.org archive rule applied); used by `release-rc-cut` Step 2/2b and `release-verify-rc` Step 9 | | [`tools/symlink-lint`](../tools/symlink-lint/) | `substrate:framework-dev` | Self-adoption symlink hygiene — rejects cyclic symlinks, misdirected skill relays (canonical/relay target-correctness), and incomplete self-adoption symlink sets | | [`tools/pilot-report-validator`](../tools/pilot-report-validator/) | `substrate:framework-dev` | Adopter pilot-report validator — required frontmatter keys, no unfilled placeholders, valid profile, and required body sections; counterpart to `spec-validator` for `docs/pilot-report-template.md` | | [`tools/skill-reconciler-diff`](../tools/skill-reconciler-diff/) | `substrate:framework-dev` | Deterministic structural diff between two skill trees — parses frontmatter, section headings, step inventory, placeholders, support files, and safety-baseline clauses into a JSON diff object for the `skill-reconciler` skill | diff --git a/docs/mode-economics.md b/docs/mode-economics.md index 4cf9745bf..db2049709 100644 --- a/docs/mode-economics.md +++ b/docs/mode-economics.md @@ -84,7 +84,7 @@ history separately provides its publication revision and date. -Measured on (UTC): 2026-09-19. +Measured on (UTC): 2026-09-20. Tokenizer: **tiktoken 0.14.0, `cl100k_base`**. Method: full UTF-8 file, including frontmatter and comments; line endings normalized to LF; @@ -92,7 +92,7 @@ special-token spellings counted as ordinary text. Coverage: **75 of 75 local `skills/*/SKILL.md` files**. External `source.md` redirects and harness symlinks are excluded. -Measurement manifest SHA-256: `316bec2ddc04557527ec9ab4010987220709ccced9df8acc2f3e7a27dcaf439d`. +Measurement manifest SHA-256: `0ef06f4874382739a4738d35ec92e689f0483db6e99292c4accc72836d0619ad`. | Skill file | Measured tokens | Source SHA-256 (first 16 characters) | |---|---:|---| @@ -131,15 +131,15 @@ Measurement manifest SHA-256: `316bec2ddc04557527ec9ab4010987220709ccced9df8acc2 | [pr-management-triage](../skills/pr-management-triage/SKILL.md) | 12,685 | `bdd0cae06e589165` | | [pr-stale-sweep](../skills/pr-stale-sweep/SKILL.md) | 7,804 | `c63539a5c0662d52` | | [pre-first-pr-check](../skills/pre-first-pr-check/SKILL.md) | 4,525 | `1a901a80838e83b0` | -| [release-announce-draft](../skills/release-announce-draft/SKILL.md) | 6,991 | `7fd2b0720eaae5ca` | +| [release-announce-draft](../skills/release-announce-draft/SKILL.md) | 7,053 | `dfd3058bc1b8034f` | | [release-archive-sweep](../skills/release-archive-sweep/SKILL.md) | 5,604 | `6e30100ea5a633dd` | | [release-audit-report](../skills/release-audit-report/SKILL.md) | 6,774 | `ce849ac8e2a217d1` | | [release-keys-sync](../skills/release-keys-sync/SKILL.md) | 5,945 | `3c11551de0e1e5f9` | -| [release-prepare](../skills/release-prepare/SKILL.md) | 7,960 | `5b76de4222c13166` | -| [release-promote](../skills/release-promote/SKILL.md) | 7,174 | `6f6bff9d291fab91` | -| [release-rc-cut](../skills/release-rc-cut/SKILL.md) | 7,614 | `e557be43124d9c5b` | -| [release-verify-rc](../skills/release-verify-rc/SKILL.md) | 8,777 | `1561f8f82f15a5eb` | -| [release-vote-draft](../skills/release-vote-draft/SKILL.md) | 6,327 | `73c6490d4351da9e` | +| [release-prepare](../skills/release-prepare/SKILL.md) | 11,986 | `78f302ca28e1aa40` | +| [release-promote](../skills/release-promote/SKILL.md) | 8,044 | `c17053e63005b55b` | +| [release-rc-cut](../skills/release-rc-cut/SKILL.md) | 12,942 | `25d4f5d7b6d5a1ab` | +| [release-verify-rc](../skills/release-verify-rc/SKILL.md) | 11,881 | `07eb370e2462f1b5` | +| [release-vote-draft](../skills/release-vote-draft/SKILL.md) | 7,821 | `d0bc0fd3c11914a9` | | [release-vote-tally](../skills/release-vote-tally/SKILL.md) | 6,696 | `c848e809e2d877cd` | | [report-framework-issue](../skills/report-framework-issue/SKILL.md) | 5,703 | `30b2ce8b774ea68c` | | [reviewer-routing](../skills/reviewer-routing/SKILL.md) | 6,272 | `dcd75b720d42af34` | diff --git a/docs/release-management/README.md b/docs/release-management/README.md index 9cf12562d..f7c5a64e2 100644 --- a/docs/release-management/README.md +++ b/docs/release-management/README.md @@ -123,7 +123,7 @@ says which file is missing. | File | What it carries | Read by | |---|---|---| | [`pmc-roster.md`](../../projects/_template/pmc-roster.md) | Who is binding. Read wherever a vote is counted or a PMC-only action is gated. | `promote`, `vote-tally` | -| [`release-build.md`](../../projects/_template/release-build.md) | How this project builds and signs artefacts: build command, artefact names, checksum algorithm, signing-key expectations. | `rc-cut`, `verify-rc` | +| [`release-build.md`](../../projects/_template/release-build.md) | How this project builds and signs artefacts: reproducible source-archive recipe (`git archive` + `.gitattributes`), build command, the project's optional convenience artefacts (each with its own build, staging, reproducibility mode, vote scope and publish channel), artefact names, checksum algorithm, optional reproducibility checks. | `rc-cut`, `verify-rc` | | [`release-management-config.md`](../../projects/_template/release-management-config.md) | Vote window and pass rule, distribution backend and paths, announce/vote list addresses, retention rule. | `announce-draft`, `archive-sweep`, `audit-report`, `keys-sync`, `prepare`, `promote`, `rc-cut`, `verify-rc`, `vote-draft`, `vote-tally` | | [`release-trains.md`](../../projects/_template/release-trains.md) | Active release branches, release-manager attribution per cut, rotation rosters, security-team roster. | `archive-sweep`, `prepare` | diff --git a/docs/release-management/atr-release-runbook.md b/docs/release-management/atr-release-runbook.md index d84b89eef..fd61168ea 100644 --- a/docs/release-management/atr-release-runbook.md +++ b/docs/release-management/atr-release-runbook.md @@ -374,7 +374,19 @@ the binding votes. It produces the subject (`[VOTE] Release Apache Magpie from -rcN`) and body — pointing voters at the ATR candidate page and its check - results. + results, and carrying the **How to verify this candidate** section + every Magpie `[VOTE]` has: the reproducibility record (source + commit, `SOURCE_DATE_EPOCH`, sha512 from the planning issue), the + agentic one-liner + (`/magpie-release-management:verify-rc -rcN`), the + human-readable page + ([`manual-release-process.md` § Manual verification](manual-release-process.md#manual-verification--what-a-voter-runs-before-1) + at the RC tag), the [reproducibility background](reproducibility.md), + and the voter-obligation sentence. ATR's default vote text links + only the candidate page, so **the drafted body is what the RM + supplies to ATR** (the client's body option or the vote form on the + candidate page; confirm with `atr vote start --help`) — do not let + the default stand. 2. **Start the vote in ATR.** The RM triggers the vote for the composed candidate; ATR sends the `[VOTE]` email to `dev@magpie.apache.org` and opens the tabulation. Starting the @@ -393,6 +405,22 @@ the binding votes. [release-policy § release approval](https://www.apache.org/legal/release-policy.html#release-approval); the Magpie config may lengthen but not shorten it ([`release-management-config.md` § Vote](../../projects/_template/release-management-config.md#vote)). + What a PMC member does during the window, in either order: + - **Agentic:** `/magpie-release-management:verify-rc -rcN` + from any Magpie-enabled agent — read-only; it fetches the staged + artefacts, checks signature / checksum / RAT / LICENSE-NOTICE / + binaries / links / version strings, rebuilds the source archive + from the tag and reports `identical`, `content-identical` or + `differs`. + - **Manual:** the linked + [verification page](manual-release-process.md#manual-verification--what-a-voter-runs-before-1) + at the RC tag — the same checks longhand, ending with a build and + test run from the unpacked source. + - **Reply** on the thread using the + [reply template](manual-release-process.md#reply-template--what-to-put-in-your-vote): + the commit you verified, the `compare` verdict, and what you built + and tested on. A binding `+1` is the voter's own statement, not the + tool's. 4. **Tally** (Step 9). ATR tabulates the replies; cross-check with [`release-vote-tally`](../../skills/release-vote-tally/SKILL.md), which classifies each reply binding-vs-non-binding against the @@ -473,7 +501,13 @@ This does **not** move the signing key into CI unless the project has adopted a reproducible-build + trusted-publishing model the PMC has explicitly signed off on. For Magpie's first releases, prefer the local `atr` client path above (Step C); revisit CI-driven compose once -the build is demonstrably reproducible. See the +the build is demonstrably reproducible. When it is, `release-prepare +automated-signing` drafts the Infra key request, the Security Team +notification and the workflow PR (template: +[`projects/_template/workflows/release-candidate.yml`](../../projects/_template/workflows/release-candidate.yml)) +under the conditions in +[Infra § Automated release signing](https://infra.apache.org/release-signing.html#automated-release-signing); +see [`reproducibility.md`](reproducibility.md#automated-release-signing--asf-specific-optional). See the [`tooling-asf-example`](https://github.com/apache/tooling-asf-example) repository for a worked GitHub Actions example. diff --git a/docs/release-management/manual-release-process.md b/docs/release-management/manual-release-process.md index 211a61844..a28bb1b77 100644 --- a/docs/release-management/manual-release-process.md +++ b/docs/release-management/manual-release-process.md @@ -24,6 +24,7 @@ - [5. Confirm the source matches the tagged commit](#5-confirm-the-source-matches-the-tagged-commit) - [6. License headers (Apache RAT)](#6-license-headers-apache-rat) - [7. Optional — reproduce the project's own checks from pristine source](#7-optional--reproduce-the-projects-own-checks-from-pristine-source) + - [Reply template — what to put in your vote](#reply-template--what-to-put-in-your-vote) - [Caveats hit during rc1 / rc2](#caveats-hit-during-rc1--rc2) - [Release Manager checklist](#release-manager-checklist) - [Cross-references](#cross-references) @@ -339,6 +340,19 @@ the `diff` output. (If your `git` version happens to produce byte-identical but the tree `diff` is the version-independent check.) If the tag is signed, `git tag -v "${VERSION}-${RC}"` also confirms it has not moved. +The framework's `repro-archive` tool does the same check in one step and +tells the two cases apart — `identical` (same bytes), `content-identical` +(same tree, archive metadata differs) or `differs` — using the +`SOURCE_DATE_EPOCH` the RM recorded on the planning issue +([reproducibility.md](reproducibility.md)): + +```bash +uv run --project tools/reproducible-archive repro-archive build \ + --ref "${VERSION}-${RC}" --format zip --prefix "apache-magpie-${VERSION}" \ + --epoch "" -o /tmp/rebuilt.zip +uv run --project tools/reproducible-archive repro-archive compare "../${ARTIFACT}" /tmp/rebuilt.zip +``` + ### 6. License headers (Apache RAT) Run [Apache RAT](https://creadur.apache.org/rat/) over the unpacked tree @@ -370,6 +384,34 @@ Only after these pass should a voter post `+1` (binding voters: your `+1` carries the release). Report any failure on the `[VOTE]` thread with the exact command and output. +The whole sequence above is what +[`release-verify-rc`](../../skills/release-verify-rc/SKILL.md) runs for +you (`/magpie-release-management:verify-rc -rcN` from any +Magpie-enabled agent): it emits each command, records the results, and +adds the source rebuild-and-compare from +[`reproducibility.md`](reproducibility.md). Use whichever path you +prefer; the `[VOTE]` email links both. + +### Reply template — what to put in your vote + +State what you verified, so the tally reads as evidence rather than a +count: + +```text ++1 (binding) + +Verified apache-magpie--source.zip from dist/dev at r: +- signature OK against KEYS (), sha512 matches +- tag -rcN = commit ; rebuilt with repro-archive at + SOURCE_DATE_EPOCH : identical (or: content-identical / differs — say which) +- RAT clean; LICENSE + NOTICE present; no binaries, no dangling links +- built and ran the test suite from the unpacked source on +``` + +Under automated release signing add *"rebuilt on my own hardware"* — +that line is the trusted-hardware validation the policy requires +([`reproducibility.md` § Automated release signing](reproducibility.md#automated-release-signing--asf-specific-optional)). + ## Caveats hit during rc1 / rc2 Real friction from the `0.1.0` iterations, recorded so the next RM expects diff --git a/docs/release-management/process.md b/docs/release-management/process.md index d1b041f1b..4d5bd79a4 100644 --- a/docs/release-management/process.md +++ b/docs/release-management/process.md @@ -189,6 +189,19 @@ clears. Output: a single PR proposed against the release branch, RM merges after their own review. +On a project's **first release** (or whenever +`release-build.md § Source archive` has no `export_ignore_reviewed` +marker) the same prep PR carries the **source-archive contents +review**: a guided walk through every top-level path of the +repository that classifies what the `git archive` source artefact +ships, proposes the `.gitattributes` `export-ignore` entries that keep +VCS / CI / editor metadata out (never `LICENSE`, `NOTICE`, build +inputs, or a path a shipped file references), and records the +decision. The attributes must be committed before the RC tag exists, +which is why the review lives here and why Step 4 blocks while it is +outstanding. See +[`reproducibility.md` § The first-release `.gitattributes` review](reproducibility.md#the-first-release-gitattributes-review). + > [!NOTE] > The Step 2 `LICENSE` / `NOTICE` draft is *provisional*. It is > drafted before the build, so it covers only content the skill can @@ -226,18 +239,49 @@ Agentic Drafting. The skill emits a paste-ready command sequence: 1. `git tag -s -rcN -m "..."` (signed tag, RM's key). -2. Build invocation, project-specific - (`/release-build.md`). -3. `gpg --detach-sign --armor ` for each artefact. -4. `sha512sum > .sha512` for each artefact. +2. Build invocation. The **source artefact** is, by default, a + reproducible export of the tag — `repro-archive build`, which is + `git archive` (tracked files only, `.gitattributes` `export-ignore` + honoured) with every + [reproducible-builds.org archive rule](https://reproducible-builds.org/docs/archives/) + applied, so a voter rebuilding from the tag gets byte-identical + bytes. **Convenience artefacts** — whatever the project ships + besides the source (binary tarball, wheels, jars, a container + image, a chart) — are project-specific by nature and are declared + one by one in + [`/release-build.md` § Convenience artefacts](../../projects/_template/release-build.md); + each entry's own `build_command` follows, under the same + `SOURCE_DATE_EPOCH`. A source-only project declares none. +3. *Optional* reproducibility self-check (`release-build.md + § Reproducibility checks`): lint the archive, rebuild it, compare; + rebuild every convenience artefact and compare. A `differs` stops + the cut before anything is signed — a convenience artefact that + cannot be rebuilt from the tag is not known to be what the source + produces. +4. `gpg --detach-sign --armor ` for each artefact. +5. `sha512sum > .sha512` for each artefact. The skill writes nothing to disk and runs nothing locally. The RM runs every command on their own machine, with their own key, in their own checkout. The skill's output is the *recipe*; correctness of the recipe is reviewable independently from execution. After the -RM reports back the artefact list + checksums + sig filenames, the -skill records them in the planning issue's audit-trail comment for -Step 13. +RM reports back the artefact list + checksums + sig filenames — plus +the source commit, `SOURCE_DATE_EPOCH` and sha512 that make the +artefact reproducible — the skill records them in the planning +issue's audit-trail comment for Step 13. + +> [!NOTE] +> **🪶 ASF-specific option — automated release signing.** An ASF +> project may, after the one-time setup in `release-prepare +> automated-signing` (Infra-provisioned key, Security Team approval, +> reproducible-build workflow), let CI sign and stage the artefacts +> ([Infra § Automated release signing](https://infra.apache.org/release-signing.html#automated-release-signing)). +> Step 4 then reduces to pushing the RM-signed tag; steps 3 and 5 run +> in CI; Step 6's reproducibility check becomes mandatory as the +> policy's validation on trusted hardware; and Step 10 refuses to +> promote without it. See +> [`reproducibility.md` § Automated release signing](reproducibility.md#automated-release-signing--asf-specific-optional). +> The option is offered only under `organization: ASF`. > [!NOTE] > Detached `.asc` signatures and `.sha512` checksums are the ASF @@ -292,6 +336,19 @@ Read-only. The skill fetches the staged artefacts from binary-exclusion list). - **Version string consistency** between artefact filename, embedded manifests, and tag. +- **Reproducibility** (optional, per + [`/release-build.md` § Reproducibility checks](../../projects/_template/release-build.md); + mandatory under automated release signing): the source artefact is + rebuilt from the tag with `repro-archive build` at the recorded + `SOURCE_DATE_EPOCH` and compared with the staged one — + `identical`, `content-identical` (only archive metadata differs) or + `differs` (not the tagged tree, a `-1`); every convenience artefact + is rebuilt and compared byte-for-byte, or against the project's + documented divergences. For a convenience artefact this is the + check that decides whether it is *good*: a binary cannot be + reviewed, only rebuilt, and one that does not reproduce from the + voted source is withheld from publication in Step 10. See + [`reproducibility.md`](reproducibility.md). The skill emits a pass/fail report to the planning issue. A failure does not auto-flip any label; the RM decides whether to roll a new @@ -380,6 +437,24 @@ commit` under their own ASF credentials. This is **the moment of release**. The skill writes nothing and runs nothing; the human commit is the act. +When the project declares **convenience artefacts** +([`release-build.md` § Convenience artefacts](../../projects/_template/release-build.md)), +the skill follows the source promotion with each artefact's own +`publish_command` to its declared channel (PyPI, Maven Central, a +container registry, a chart repository, …) — project-specific +commands the config supplies, never invented. It emits a publish +command only for an artefact that `release-verify-rc` reproduced from +the voted tag (`identical`, or documented divergence only); an +artefact that did not reproduce gets a HOLD note instead. The source +promotion is never held back by a convenience artefact. + +🪶 ASF-specific: under `automated_release_signing: enabled` the skill +additionally requires the planning issue to carry a `release-verify-rc` +attestation that every artefact was rebuilt bit-by-bit identical on a +committer's own hardware — the validation step +[Infra § Automated release signing](https://infra.apache.org/release-signing.html#automated-release-signing) +mandates before publication — and blocks without it. + The `dist/release/` tree is PMC-write-only by default ([release-policy.html](https://www.apache.org/legal/release-policy.html)). If the RM is a committer but not a PMC member, the `svn mv` will @@ -531,6 +606,12 @@ state machine participant. - [`spec.md`](spec.md), per-skill scope, state-change boundary, hand-off protocol, adopter knobs. - [`projects/_template/release-management-config.md`](../../projects/_template/release-management-config.md), adopter contract scaffold. +- [`reproducibility.md`](reproducibility.md), the reproducible source + archive (`git archive` + `.gitattributes` + the reproducible-builds.org + rules), the optional reproducibility checks for source and binaries, + and the 🪶 ASF-specific automated-release-signing option. +- [`tools/reproducible-archive`](../../tools/reproducible-archive/README.md), + the `repro-archive` tool Steps 4 and 6 use. - [`docs/modes.md` § Drafting](../modes.md#drafting), [`§ Triage`](../modes.md#triage), the modes the skills inhabit. - [`MISSION.md` § Initial Goals](../../MISSION.md#initial-goals), diff --git a/docs/release-management/reproducibility.md b/docs/release-management/reproducibility.md new file mode 100644 index 000000000..f7369e70e --- /dev/null +++ b/docs/release-management/reproducibility.md @@ -0,0 +1,270 @@ + + + + +**Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)* + +- [Reproducible source archives, reproducibility checks, and automated signing](#reproducible-source-archives-reproducibility-checks-and-automated-signing) + - [Why](#why) + - [The source archive is `git archive` plus `.gitattributes`](#the-source-archive-is-git-archive-plus-gitattributes) + - [The first-release `.gitattributes` review](#the-first-release-gitattributes-review) + - [Later releases: drift](#later-releases-drift) + - [The archive rules from reproducible-builds.org](#the-archive-rules-from-reproducible-buildsorg) + - [The record: commit, SWHID, origin](#the-record-commit-swhid-origin) + - [Reproducibility checks](#reproducibility-checks) + - [Source (`reproducibility_source`)](#source-reproducibility_source) + - [Convenience artefacts (`reproducibility_binaries`, per-artefact `reproducibility`)](#convenience-artefacts-reproducibility_binaries-per-artefact-reproducibility) + - [Where the checks run](#where-the-checks-run) + - [Automated release signing (🪶 ASF-specific, optional)](#automated-release-signing--asf-specific-optional) + - [What the policy requires](#what-the-policy-requires) + - [One-time setup: `release-prepare automated-signing`](#one-time-setup-release-prepare-automated-signing) + - [What changes in the lifecycle once it is enabled](#what-changes-in-the-lifecycle-once-it-is-enabled) + - [What does not change](#what-does-not-change) + - [Cross-references](#cross-references) + + + + + +# Reproducible source archives, reproducibility checks, and automated signing + +How the release-management family produces a source artefact that is a function of the tag alone, +how a Release Manager (RM) and every voter confirm that the staged artefacts really are that function, +and, for ASF projects, how that opens the door to CI-signed releases. + +The configuration keys this page refers to live in +[`/release-build.md`](../../projects/_template/release-build.md) +(`§ Source archive`, `§ Reproducibility checks`) and +[`/release-management-config.md`](../../projects/_template/release-management-config.md) +(`§ Signing › Automated release signing`). +The tool that does the mechanical work is +[`tools/reproducible-archive`](../../tools/reproducible-archive/README.md) (`repro-archive`). + +## Why + +[`PRINCIPLES.md` § 11](../../PRINCIPLES.md#11-releases-are-reproducible-from-signed-source): +releases are reproducible from signed source to the extent the toolchain permits; +where byte-identical output is achievable it is required, +and where it is not, the process documents the divergence and provides a verification path a contributor can run locally. +Reproducibility is what makes a signature worth verifying: +a `+1` on a source artefact means "I confirmed these bytes are the tagged tree", and that is only checkable if the tagged tree yields those bytes again. + +The framework learnt this the hard way. +Magpie's own `0.1.0-rc1` got a `-1` because the artefact was a `zip -r` of a working tree: +it carried `__pycache__/*.pyc`, its dangling agent-view symlinks pointed at directories the archive had stripped, +and no voter could regenerate it to compare. +`0.1.0-rc2` switched to `git archive` with `.gitattributes` `export-ignore`, and +[`docs/source-release-contents.md`](../source-release-contents.md) records what ships and why. +This page generalises that fix for every adopter. + +## The source archive is `git archive` plus `.gitattributes` + +With `source_archive_method: git-archive` (the default in `release-build.md`) the source artefact is not a build output. +It is an export of the tagged tree: + +- **only tracked files at the tag** — an untracked `.pyc`, a stray editor backup, a local `.env`, cannot ship; +- **minus the paths marked `export-ignore`** in the repository's root + [`.gitattributes`](https://git-scm.com/docs/gitattributes#_creating_an_archive) — VCS, CI and editor metadata a source consumer never needs; +- **wrapped by `repro-archive build`**, which applies the reproducible-builds.org archive rules below so that the output is byte-identical on every machine. + +`.gitattributes` is therefore part of the release definition and must be **committed before the RC tag is cut**; +`git archive` reads the attributes from the tree it archives, not from the working copy. +That is why the review lands in the prep PR (`release-prepare prep`) and why `release-rc-cut` refuses to cut an RC while the review is outstanding. + +### The first-release `.gitattributes` review + +`release-prepare prep` Step 2f runs a guided review on the first release +(or whenever `export_ignore_reviewed` is unset in `release-build.md`, or on `--review-archive`). +It is an education step: the goal is that the operator understands *why* each path ships or does not, not that the agent guesses. +The skill: + +1. **Lists what would ship today** — `git archive --format=tar HEAD | tar -tf -` — and every top-level entry of `git ls-files`. +2. **Classifies each top-level path** into one of these buckets and says which: + + | Bucket | Typical paths | Default | + |---|---|---| + | Source, docs, build descriptors, packaging metadata | `src/`, `docs/`, `pom.xml`, `pyproject.toml`, lock files, `README*` | **ship** | + | Legal files | `LICENSE`, `NOTICE`, `DISCLAIMER` (incubating), `licenses/` | **ship, never excludable** | + | Inputs to the checks voters run | RAT excludes (`.rat-excludes`), in-tree validators | **ship** | + | Foundation / project metadata | `.asf.yaml`, `doap_*.rdf` | ship (project's call; ASF projects conventionally ship them) | + | VCS metadata | `.gitattributes`, `.gitmodules`, `.mailmap` | exclude (`.gitignore` is the project's call) | + | CI and bot configuration | `.github/workflows/`, `.github/dependabot.yml`, `.gitlab-ci.yml`, `.travis.yml`, `.circleci/`, `.pre-commit-config.yaml` | exclude | + | Editor and IDE state | `.idea/`, `.vscode/`, `.devcontainer/` | exclude | + | Linter and formatter configuration not needed to build | `.lychee.toml`, `.markdownlint.json`, `.typos.toml`, `.zizmor.yml`, `.yamllint`, `.codespellrc` | exclude | + | Agent-view directories | `.claude/`, `.agents/`, `.kiro/`, `.cursor/` | exclude the relay symlink dirs; keep a single-hop canonical view if shipped files link into it | + | Large assets not needed to build or verify | screenshots, recordings, demo data | project's call; say what they are for | + | Release-tooling scratch | `.apache-magpie.session-state.json`, `.apache-magpie.local.lock` | exclude | + +3. **Checks references before proposing an exclusion**: `git grep -l ''` over tracked files. + A path that a shipped file links to (a doc, a validator, a symlink target) must not be excluded, + or `release-verify-rc` Step 7 will fail the RC with a dangling reference. + The skill names the referrers and offers the alternative (keep the path, or repoint the reference). +4. **Checks symlinks**: every committed symlink whose target would be stripped is flagged; chained symlinks (link to a link) are flagged because safe extractors reject them. +5. **Proposes the entries**, root-anchored with a leading `/` for root-only files, one rationale comment per entry, + and shows the before/after archive listing diff (`repro-archive build` twice, `repro-archive compare`). +6. **Records the decision**: `.gitattributes` joins the prep PR's file set, and the prep PR also sets + `export_ignore_reviewed: ` in `release-build.md` so the review does not repeat. + +Everything is a proposal; the RM confirms each entry. +The agent never edits `.gitattributes` without that confirmation and never marks the review done on its own. + +### Later releases: drift + +On every subsequent `release-prepare prep` the skill runs a cheap drift check: +top-level entries added since the last reviewed tag (`git diff --name-only HEAD`, top level only) +that fall in an *exclude* bucket are surfaced as candidates. +`--review-archive` forces the full review again. + +## The archive rules from reproducible-builds.org + +`git archive` on its own is only deterministic for one `git` version: +the zip writer's compression, the tar's PAX handling and the mtime source have changed between releases, +so two voters with different `git` versions get different bytes from the same tag. +[reproducible-builds.org § Archive metadata](https://reproducible-builds.org/docs/archives/) +lists what has to be pinned for an archive to be reproducible. +`repro-archive build` applies every rule; `repro-archive check` verifies each one on any archive; `repro-archive recipe` prints the equivalent GNU tar / Info-ZIP shell commands. + +| Rule | Standard-tool form | `repro-archive` | +|---|---|---| +| One modification time for every member | `tar --mtime="@${SOURCE_DATE_EPOCH}"` / `touch --date="@${SOURCE_DATE_EPOCH}"` | mtime = `SOURCE_DATE_EPOCH`, default the committer timestamp of the ref | +| Locale-independent file ordering | `tar --sort=name`, `find … \| LC_ALL=C sort -z` | per-directory byte order | +| No ownership leakage | `--owner=0 --group=0 --numeric-owner` | uid/gid 0, empty names | +| No umask leakage | `--mode=a=rX,u+w` | `0644` / `0755` | +| No PAX `atime`/`ctime`/PID headers | `--pax-option=exthdr.name=%d/PaxHeaders/%f,delete=atime,delete=ctime` | none written; `check` catches `PaxHeaders.` | +| gzip carries no timestamp or filename | `gzip -n` | header mtime 0, no name | +| zip carries no extra attributes | `zip -X`, unzip with `TZ=UTC` | no extra fields, UTC DOS time | +| Static libraries deterministic | `ARFLAGS=Dcvr`, `ranlib -D` | belongs in the *binary* build command (`release-build.md § Build invocation`) | + +`SOURCE_DATE_EPOCH` is the one input two builders must agree on. +Deriving it from the ref's committer timestamp makes it a property of the tag, which is why `release-rc-cut` records it on the planning issue alongside the commit hash and the sha512. +Two inputs the page does not list are pinned as well: the builder's `core.autocrlf` / `core.eol` (which `git archive` would apply to `text` files, so a Windows-configured builder exports different bytes) and the archive writer itself (the tool, not the local `tar` / `zip` / `git` version). + +### The record: commit, SWHID, origin + +Every RC carries a record on the planning issue and in the `[VOTE]`, printed by `repro-archive build` and pasted back by the RM: + +| Line | What it is for | +|---|---| +| `commit ` and the repository URL | Where the tree comes from. A voter rebuilds from this. | +| `swhid_dir swh:1:dir:;origin=;anchor=swh:1:rev:` | The [Software Heritage identifier](https://swhid.org/) (ISO/IEC 18670:2025) of the archive's **expanded content**. Computed from names, modes and contents alone, exactly as git computes a tree id, so it does not depend on the archive format, the compression or who packed it: a voter recomputes it from the staged bytes with `repro-archive swhid ` (or `asfswhid` / `swh identify` after extracting), ATR computes the same value for the candidate at compose time, and a `.tar.gz` and a `.zip` of the same tree carry the same value. | +| `swhid_rev swh:1:rev:;origin=` | The commit as a SWHID, the `anchor` of the content identifier. | +| `swhid_dir_note` | Whether the content SWHID equals `git rev-parse ^{tree}`. It does unless `.gitattributes` altered the export (`export-ignore`, `export-subst`, `text` / `eol`); when it differs, the difference is itself the record that something was left out, and `release-prepare`'s review is where that was decided. | +| `SOURCE_DATE_EPOCH ` | The one input to feed back into a rebuild. | +| `sha512 ` | Byte-level comparison; `identical` in `compare` terms. | + +Why the SWHID and not just the commit: a commit id names a repository object; the content SWHID names what shipped, survives a repository move, is the same across archive formats, and is what ATR computes — so it is the value to compare, with a voter's own recomputation and with the platform. +It also gives a convenience artefact something precise to point at: each one records the source `swh:1:dir:` it was built from. +`release-verify-rc` checks the staged archive against the recorded SWHID (`repro-archive check --swhid …`) and reports `swhid_matches`. + +## Reproducibility checks + +Both checks are **optional** and configured in `release-build.md § Reproducibility checks`. +They are `on` for the source archive by default (it costs one rebuild), `off` for binaries by default (most adopters ship none). +They become **mandatory** under automated release signing (below). + +### Source (`reproducibility_source`) + +`on`: rebuild the source artefact from the tag with `repro-archive build` (same prefix, format and `SOURCE_DATE_EPOCH` as recorded on the planning issue) and `repro-archive compare` it against the staged artefact. + +| `compare` verdict | Meaning | Outcome | +|---|---|---| +| `identical` | Byte-for-byte the same file | `PASS` | +| `content-identical` | Same members, same bytes, same modes; only archive metadata differs | `WARN` (RM-key mode: the RM built with a plain `git archive` or a different tool version; switch to `repro-archive build` for the next RC) / `FAIL` (automated signing) | +| `differs` | Members added, removed or changed | `FAIL` — the artefact is not the tagged tree; a `-1` | + +`repro-archive check --epoch ` on the staged artefact runs alongside and reports any rule the staged archive violates. + +### Convenience artefacts (`reproducibility_binaries`, per-artefact `reproducibility`) + +What a project ships besides the source — a binary tarball, wheels, jars, a container image, a Helm chart — is **project-specific by nature**, so the framework does not assume any. +Each one is declared in `release-build.md § Convenience artefacts` with its own build command, staging target, reproducibility mode, vote scope and publish channel, and every `release-*` skill reads that list: `release-rc-cut` builds and stages them, `release-verify-rc` rebuilds and compares them, `release-vote-draft` lists them, `release-promote` publishes them. +A source-only project leaves the list empty and the skills say so. + +**Reproducibility is what makes a convenience artefact "good".** +The source is the release, and a voter can read it; a binary cannot be read, only rebuilt. +The single check that establishes that a convenience artefact is what the voted source produces is to rebuild it from the tag, under the same `SOURCE_DATE_EPOCH`, and compare. +An artefact that reproduces bit-for-bit is known-good; one whose every difference is written down and explained is acceptably good; one that cannot be reproduced either way is of unknown provenance, whatever the vote said about the source, and `release-promote` withholds its publish command until a `release-verify-rc` run reproduces it. + +| Mode (per artefact; default `reproducibility_binaries`) | What runs | Outcome | +|---|---|---| +| `off` | nothing | `SKIP`, stated explicitly in the report, with the note that the artefact is published on trust | +| `byte-identical` | `export SOURCE_DATE_EPOCH=…` then the entry's `build_command` at the tag; the rebuilt artefact compared byte-for-byte with the staged one | any mismatch is `FAIL` and the artefact is held back from publication | +| `documented-divergence` | the rebuild, then the entry's `verification_command` (for example [`diffoscope`](https://diffoscope.org/)) against the staged artefact | differences that match the entry's `known_divergences` are `WARN` and listed; anything else is `FAIL` | + +`documented-divergence` is the honest mode for toolchains that cannot yet produce identical bytes (a JIT-compiled bundle, a signed installer, a platform that embeds the build host). +The known divergences are part of the release documentation, which is exactly what `PRINCIPLES.md § 11` asks for. +Typical levers for getting to `byte-identical`: honour `SOURCE_DATE_EPOCH` (most build tools do), pin the toolchain, `ARFLAGS=Dcvr` / `ranlib -D` for static libraries, `gzip -n`, sorted inputs, and no absolute build paths. + +### Where the checks run + +- **`release-rc-cut` Step 2b** — the RM's self-check right after building, before signing: `check` the artefact, rebuild into a scratch directory, `compare`. Catching a non-reproducible build here saves a whole RC round-trip. +- **`release-verify-rc` Step 9** — every voter's check against the *staged* artefact, in the same read-only pairing loop as signatures and checksums. The report's step summary carries the verdict and the recorded `SOURCE_DATE_EPOCH`. +- **`release-promote` Step 0** — under automated signing only: refuses to promote unless the planning issue carries a `release-verify-rc` reproducibility attestation from a run on trusted hardware. + +## Automated release signing (🪶 ASF-specific, optional) + +> **Scope.** This section applies to projects whose `project.md` declares `organization: ASF`. +> The skills offer the option only to those projects +> (`organizations/ASF/organization.md` → `release_process.automated_signing`; +> the `independent` organization sets it to `null` and the sub-command is not shown). +> Non-ASF adopters keep the RM-key flow; a foundation with its own CI-signing policy can add an equivalent block to its organization manifest. + +### What the policy requires + +[Infra § Automated release signing](https://infra.apache.org/release-signing.html#automated-release-signing) +lets an ASF project have CI (for example GitHub Actions) sign the artefacts it builds, **provided that**: + +- *all* artefacts being signed can be built reproducibly; +- CI deploys the artefacts to a **staging** environment only; +- the release process contains a **validation step on trusted hardware** (explicitly *not* GitHub Actions) that rebuilds every artefact from source and confirms it is **bit-by-bit identical** to what was staged, before anything is published to end users. + +The Apache Security Team must be notified of the request and approve the workflow before it is used +(the request should spell out the trusted-hardware validation; `INFRA-23996` is the background ticket). +The key is requested through an Infra Jira ticket and is provisioned by Infra: +4096-bit RSA, signing-only, private half held by infra-root and made available to the chosen CI system only, +a PGP-encrypted revocation certificate placed in the project's private repository, +and the public key sent to the project or added to its `KEYS`. +[release-policy § release signing](https://www.apache.org/legal/release-policy.html#release-signing) +allows signatures by "the automated release infrastructure, where the underlying implementation MUST follow the principles outlined by the Apache Security Team". + +### One-time setup: `release-prepare automated-signing` + +The sub-command is a **drafting** step: it produces the artefacts the RM files, and files none of them. + +1. **Eligibility gate.** `organization: ASF`; `reproducibility_source: on`; `reproducibility_binaries: byte-identical` for every convenience binary in `expected_artefacts` (or none); the most recent RC's `release-verify-rc` report shows `identical` for every artefact. If the build is not yet demonstrably reproducible the skill stops here and says what to fix first. +2. **Infra Jira ticket draft** requesting the CI signing key, naming the workflow, the staging target (ATR trusted publishing via [`apache/tooling-actions/upload-to-atr`](https://github.com/apache/tooling-actions), pinned by commit SHA), and the trusted-hardware validation step. +3. **Security Team notification draft** for `security@apache.org` (the mail is drafted, never sent — [spec § Boundary 3](spec.md#boundary-3-agent-never-sends-mail-to-dev-users-announce)). +4. **Workflow PR proposal** from the template + [`projects/_template/workflows/release-candidate.yml`](../../projects/_template/workflows/release-candidate.yml): + build the source archive with the embedded `repro-archive` script, build binaries under `SOURCE_DATE_EPOCH`, self-check reproducibility in CI, upload to ATR with OIDC. The workflow contains no key material; signing is performed by the infra-managed mechanism agreed on the ticket. +5. **Config diff proposal**: `automated_release_signing: requested` now, `enabled` plus `ci_signing_key_fingerprint` and `ci_signing_infra_ticket` once Infra has provisioned the key and the public block is in `KEYS` (`release-keys-sync` handles the `KEYS` diff). + +### What changes in the lifecycle once it is enabled + +| Step | RM-key flow | `automated_release_signing: enabled` | +|---|---|---| +| 4 (`release-rc-cut`) | tag, build, `gpg --detach-sign`, `sha512sum` | tag (still signed by the RM) and push; the push triggers the workflow which builds, uploads to ATR, and stages. The skill emits the tag push plus the commands to watch the run and fetch the staged artefact list. | +| 5 (`release-rc-cut`) | `svn import` to `dist/dev/` | performed by CI; the skill records the run URL and staging URL on the planning issue | +| 6 (`release-verify-rc`) | reproducibility step optional | **mandatory**, `compare --require-identical` for every artefact, run on a committer's own hardware; the `--post-to` comment carries a trusted-hardware attestation | +| 10 (`release-promote`) | promote after `vote-passed` | additionally requires the attestation on the planning issue; blocks without it | + +### What does not change + +- The agent still holds no key of any kind ([spec § Boundary 1](spec.md#boundary-1-agent-never-holds-the-rms-signing-key)) — not the RM's, not the CI key. +- The RM still signs the **tag** with their own key; the RC is still voted on `dev@` by people who downloaded, rebuilt and tested it. +- Nothing is published by the agent or by CI; promotion stays a PMC member's `svn mv` (or ATR finish) after the vote ([spec § Boundary 2](spec.md#boundary-2-agent-never-publishes-the-release)). + +## Cross-references + +- [`tools/reproducible-archive/README.md`](../../tools/reproducible-archive/README.md) — the tool, rule by rule. +- [`docs/source-release-contents.md`](../source-release-contents.md) — Magpie's own `.gitattributes` decisions, the worked example of the review. +- [`process.md` § Step 2, 4, 5, 6, 10](process.md) — where each piece sits in the 14-step lifecycle. +- [`spec.md`](spec.md) — per-skill contract changes (`release-prepare`, `release-rc-cut`, `release-verify-rc`, `release-promote`). +- [`manual-release-process.md` § 5](manual-release-process.md#5-confirm-the-source-matches-the-tagged-commit) — the same source check written out longhand for a voter without the skill. +- [`atr-release-runbook.md` § GitHub Actions path](atr-release-runbook.md#github-actions-path-reproducible-builds) — the ATR trusted-publishing path the workflow template uses. +- [reproducible-builds.org § Archive metadata](https://reproducible-builds.org/docs/archives/) — the rule set. +- [reproducible-builds.org § `SOURCE_DATE_EPOCH`](https://reproducible-builds.org/docs/source-date-epoch/) — the timestamp convention. +- [Infra § Automated release signing](https://infra.apache.org/release-signing.html#automated-release-signing) — the ASF policy. +- [`apache/tooling-actions`](https://github.com/apache/tooling-actions) — ATR trusted-publishing actions (pin by commit SHA). diff --git a/docs/release-management/spec.md b/docs/release-management/spec.md index e0bc32428..66e22ac17 100644 --- a/docs/release-management/spec.md +++ b/docs/release-management/spec.md @@ -89,6 +89,15 @@ Practical consequences: from, ASF-owned infrastructure ([release-policy.html](https://www.apache.org/legal/release-policy.html)). The skill states this in its hand-off text; it cannot enforce it. +- 🪶 ASF-specific automated release signing does not move the + boundary. When an ASF project enables it + ([Infra § Automated release signing](https://infra.apache.org/release-signing.html#automated-release-signing)), + the CI key is provisioned and held by Infra, never by the agent or + the project; the workflow template the skills propose contains no + key material and no signing step; the RM still signs the tag; and + the policy's bit-by-bit validation on trusted hardware is a + committer's `release-verify-rc` run, not the agent's. See + [`reproducibility.md`](reproducibility.md#automated-release-signing--asf-specific-optional). ### Boundary 2: Agent never publishes the release @@ -180,12 +189,29 @@ PR. **Outputs.** - A planning-issue body (markdown), labelled `release-planning`. -- A prep PR (separate invocation), labelled `prep-pr-open`. +- A prep PR (separate invocation), labelled `prep-pr-open`. On the + first release (no `export_ignore_reviewed` in `release-build.md`) + the prep PR also carries the **source-archive contents review**: + `.gitattributes` `export-ignore` entries proposed per top-level + path with a rationale each, after a reference check so nothing a + shipped file links to is stripped, plus the + `export_ignore_reviewed: ` marker. Later releases get a + drift check only. See + [`reproducibility.md`](reproducibility.md#the-first-release-gitattributes-review). - A post-release bump PR (third invocation), unlabelled. +- 🪶 ASF-specific, `automated-signing` sub-command (version-less, + offered only under `organization: ASF`): an Infra Jira ticket + draft requesting the CI signing key, a Security Team notification + draft, a reproducible-build workflow PR rendered from + [`projects/_template/workflows/release-candidate.yml`](../../projects/_template/workflows/release-candidate.yml), + and the `release-management-config.md § Signing` diff — gated on the + build being demonstrably reproducible. Nothing is filed or sent. **State-change boundary.** The skill opens the planning issue and opens the PRs *as drafts*. The RM marks them ready and merges. The -skill never marks ready, never merges, never closes. +skill never marks ready, never merges, never closes, never edits +`.gitattributes` without per-entry confirmation, never files a +ticket and never sends mail. **Hand-off conditions.** @@ -281,15 +307,39 @@ to the adopter's distribution backend (default `svn import` to **Inputs.** - `/release-build.md`, build invocation, digest - set (`sha512`, optionally `sha256`), binary-exclude list. + set (`sha512`, optionally `sha256`), binary-exclude list; `§ Source + archive` (`source_archive_method`, format, prefix, + `export_ignore_reviewed`) and `§ Reproducibility checks`. - Current HEAD of the configured release branch. - The RC number (from the trigger). **Outputs.** - A four-section markdown block: (1) `git tag -s` command, - (2) build command, (3) `gpg --detach-sign` for each expected + (2) build command — for the source artefact, by default, + `repro-archive build --ref -rcN` (a `git archive` export + honouring `.gitattributes` `export-ignore`, with every + [reproducible-builds.org archive rule](https://reproducible-builds.org/docs/archives/) + applied; never an archive of a working tree), then the adopter's + `build_command` for convenience binaries under the tag's + `SOURCE_DATE_EPOCH`; (3) `gpg --detach-sign` for each expected artefact, (4) `sha512sum > artefact.sha512` for each artefact. +- An optional reproducibility self-check block (Step 2b): lint, + rebuild, `repro-archive compare`; binaries rebuilt and compared + per `reproducibility_binaries`. A `differs` stops the cut. +- The planning-issue record of source commit, `SOURCE_DATE_EPOCH` and + sha512, so voters can rebuild. +- 🪶 ASF-specific, under `automated_release_signing: enabled` only + (`organization: ASF`): sections (3) and (4) and the staging block + are replaced by the RM-signed tag push that triggers the CI + workflow, plus the commands to watch the run; the hand-off states + that a committer must validate on trusted hardware before + promotion. + +The skill blocks while the first-release `.gitattributes` review is +outstanding (`export_ignore_reviewed` unset with +`source_archive_method: git-archive`); `--allow-unreviewed-archive` +is the logged override. - A second markdown block with the backend-shaped staging command sequence. For `svnpubsub` (ASF default): `svn import` into `dist/dev//-rcN/`. For `github-releases`: @@ -324,7 +374,13 @@ own ASF credentials. **Adopter knobs.** Inherits `/release-build.md` verbatim, see the [`projects/_template/release-build.md`](../../projects/_template/release-build.md) -scaffold. +scaffold. Convenience artefacts — anything the project ships besides +the source — are project-specific by nature and are declared one by +one under `§ Convenience artefacts` (`build_command`, `staging` / +`stage_command`, `reproducibility`, `vote_included`, +`publish_channel` / `publish_command`); the skill emits each entry's +own build and staging commands under the tag's `SOURCE_DATE_EPOCH` +and never assumes an artefact the config does not declare. ### `release-verify-rc` @@ -352,9 +408,19 @@ loop before posting `+1`. - A pass/fail report per check (signatures, checksums, license headers via Apache RAT, NOTICE / LICENSE presence + diff vs - previous release, no prohibited binaries, version-string - consistency). + previous release, no prohibited binaries, source-tree integrity, + version-string consistency, and — optional per + `release-build.md § Reproducibility checks` — reproducibility: the + source artefact rebuilt from the tag with `repro-archive build` at + the recorded `SOURCE_DATE_EPOCH` and compared (`identical` / + `content-identical` / `differs`), binaries rebuilt and compared + byte-for-byte or against documented divergences). - A summary classification: `PASS`, `PASS-WITH-WARNINGS`, `FAIL`. +- 🪶 ASF-specific, under `automated_release_signing: enabled`: the + reproducibility check is mandatory with a byte-identical bar, and + the `--post-to` comment carries the *validated on trusted hardware* + attestation (only with the committer's `--trusted-hardware` + assertion) that `release-promote` requires. The report is a mechanical aid, not a vote. A `PASS` does not discharge a voter's own ASF obligation to download, build, and @@ -374,8 +440,17 @@ posting. - A binary appears that the binary-exclude list neither permits nor names, the skill reports `FAIL` and points at the file; the RM decides whether to exclude or pull the binary. - -**Adopter knobs.** Inherits `/release-build.md`. +- A convenience artefact does not reproduce from the voted tag + (`DIFFERS`, or differences outside its documented set), the skill + reports `FAIL` for that artefact and names it in + `binaries.differs`; `release-promote` withholds its publication. + Reproducibility is the check that decides whether a convenience + artefact is good, since a binary cannot be reviewed, only rebuilt. + +**Adopter knobs.** Inherits `/release-build.md`, +including `§ Convenience artefacts` (per-artefact `build_command` +and `reproducibility` mode) and `§ Source-tree validators` (the +project's own integrity checks Step 7 runs; none are assumed). ### `release-vote-draft` @@ -542,6 +617,13 @@ For `self-hosted`: the promote half of `release_publish_command_template`. - A markdown block with the `svn` command sequence (`svn mv`, `svn commit -m`, expected mirror-propagation note). +- When `/release-build.md § Convenience artefacts` + declares any: a second block with each artefact's own + `publish_command` to its declared channel (project-specific; PyPI, + Maven Central, a container registry, …), emitted only for artefacts + the recorded `release-verify-rc` run reproduced from the voted tag; + a `HOLD` note for any that did not. The source promotion is never + held back by a convenience artefact. - A proposed next label: `promoted`. The mirror-propagation note also records the earliest time the @@ -568,6 +650,11 @@ hard skill-side denylist; removing it requires a skill PR. ([release-policy.html](https://www.apache.org/legal/release-policy.html)); the skill emits an "ask a PMC member to publish" hand-off instead of the `svn mv` command set. +- 🪶 ASF-specific: `automated_release_signing: enabled` and the + planning issue carries no `release-verify-rc` trusted-hardware + attestation for this RC → hard blocker; the policy's validation + step has not happened + ([Infra § Automated release signing](https://infra.apache.org/release-signing.html#automated-release-signing)). ### `release-announce-draft` diff --git a/docs/release-management/svn-release-runbook.md b/docs/release-management/svn-release-runbook.md index 99418fb3b..2c1439d29 100644 --- a/docs/release-management/svn-release-runbook.md +++ b/docs/release-management/svn-release-runbook.md @@ -154,10 +154,31 @@ build output. The `--prefix` puts everything under a versioned top folder so the unpacked tree is `apache-magpie-/`. ```bash -git archive --format=zip \ - --prefix="apache-magpie-${VERSION}/" \ - -o "${ARTIFACT}" \ - "${RC_TAG}" +# Preferred: git archive wrapped by the framework's reproducible-archive +# tool, which applies every https://reproducible-builds.org/docs/archives/ +# rule so the bytes do not depend on your git version. Prints the commit, +# SOURCE_DATE_EPOCH and sha512 to record on the planning issue. +uv run --project tools/reproducible-archive repro-archive build \ + --ref "${RC_TAG}" --format zip \ + --prefix "apache-magpie-${VERSION}" \ + -o "${ARTIFACT}" + +# Plain git archive gives the same *contents* (a voter's `repro-archive +# compare` reports `content-identical`), but not the same bytes across +# git versions: +# git archive --format=zip --prefix="apache-magpie-${VERSION}/" -o "${ARTIFACT}" "${RC_TAG}" +``` + +Self-check that the archive is reproducible before signing it +([reproducibility.md](reproducibility.md)): + +```bash +uv run --project tools/reproducible-archive repro-archive check "${ARTIFACT}" +mkdir -p rebuild +uv run --project tools/reproducible-archive repro-archive build \ + --ref "${RC_TAG}" --format zip --prefix "apache-magpie-${VERSION}" -o "rebuild/${ARTIFACT}" +uv run --project tools/reproducible-archive repro-archive compare --require-identical \ + "${ARTIFACT}" "rebuild/${ARTIFACT}" ``` Quick sanity check that `LICENSE` and `NOTICE` are present at the diff --git a/docs/setup/marketplace.md b/docs/setup/marketplace.md index 3b58f0969..18224ce4b 100644 --- a/docs/setup/marketplace.md +++ b/docs/setup/marketplace.md @@ -156,7 +156,7 @@ can say so, because it is the floor everything else is managed from. |---|---|---| | `magpie-security` | 15 | ~2.0k | | `magpie-setup` | 10 | ~1.3k | -| `magpie-release-management` | 10 | ~1.0k | +| `magpie-release-management` | 10 | ~1.3k | | `magpie-pr-management` | 8 | ~1.0k | | `magpie-issue` | 8 | ~0.8k | | `magpie-repo-health` | 7 | ~0.7k | diff --git a/docs/source-release-contents.md b/docs/source-release-contents.md index 08ef631ba..1dac87d8e 100644 --- a/docs/source-release-contents.md +++ b/docs/source-release-contents.md @@ -31,7 +31,11 @@ review discussion on the `0.1.0-rc2` `[VOTE]` thread on `release-verify-rc` re-checks the unpacked archive (symlink-lint, validators, no `.pyc`), so a regression in what ships fails the RC before -the vote. +the vote. This page is the worked example of the first-release +source-archive review every adopter goes through in `release-prepare +prep`; the generic rules, the classification buckets, and how the +archive is made byte-reproducible are in +[`release-management/reproducibility.md`](release-management/reproducibility.md). ## Files kept in the source archive diff --git a/docs/vendor-neutrality.md b/docs/vendor-neutrality.md index 84f773a05..cef10247d 100644 --- a/docs/vendor-neutrality.md +++ b/docs/vendor-neutrality.md @@ -591,7 +591,7 @@ Organization scope (declared, orthogonal to vendor): ASF = 14, agnostic = 61. **LLM / agent-integration neutrality** -**Agent harness: 24/24 substrate tools run under any harness unchanged (100%).** Substrate tools are Magpie's own machinery; each declares the agent harness it integrates with (`**Harness:**`), or `agnostic`. A tool is neutral when it is harness-agnostic or supports two or more harnesses; *coupled* when it targets a single harness. +**Agent harness: 25/25 substrate tools run under any harness unchanged (100%).** Substrate tools are Magpie's own machinery; each declares the agent harness it integrates with (`**Harness:**`), or `agnostic`. A tool is neutral when it is harness-agnostic or supports two or more harnesses; *coupled* when it targets a single harness. | Substrate tool | Substrate | Harness support | Verdict | |---|---|---|---| @@ -606,6 +606,7 @@ Organization scope (declared, orthogonal to vendor): ASF = 14, agnostic = 61. | `preflight-audit` | analytics | any | ✅ agnostic | | `privacy-llm` | privacy | any | ✅ agnostic | | `probe-templates` | sandbox | any | ✅ agnostic | +| `reproducible-archive` | release | any | ✅ agnostic | | `sandbox-lint` | sandbox | Claude Code, Codex, Cursor, Gemini CLI, Kiro, OpenCode | ✅ portable | | `security-tracker-stats-dashboard` | analytics | any | ✅ agnostic | | `skill-and-tool-validator` | framework-dev | any | ✅ agnostic | @@ -628,7 +629,7 @@ Harness → substrate tools it supports: - **Gemini CLI** (3): `agent-guard`, `sandbox-lint`, `spec-loop` - **Kiro** (3): `agent-guard`, `sandbox-lint`, `spec-loop` - **OpenCode** (3): `agent-guard`, `sandbox-lint`, `spec-loop` -- **any harness** (21): `agent-isolation`, `dashboard-generator`, `dev`, `egress-gateway`, `permission-audit`, `pilot-report-validator`, `pr-management-stats`, `preflight-audit`, `privacy-llm`, `probe-templates`, `security-tracker-stats-dashboard`, `skill-and-tool-validator`, `skill-evals`, `skill-reconciler-diff`, `skill-token-count`, `spec-inventory`, `spec-status-index`, `spec-validator`, `symlink-lint`, `vendor-neutrality-score`, `vetted-ops` +- **any harness** (22): `agent-isolation`, `dashboard-generator`, `dev`, `egress-gateway`, `permission-audit`, `pilot-report-validator`, `pr-management-stats`, `preflight-audit`, `privacy-llm`, `probe-templates`, `reproducible-archive`, `security-tracker-stats-dashboard`, `skill-and-tool-validator`, `skill-evals`, `skill-reconciler-diff`, `skill-token-count`, `spec-inventory`, `spec-status-index`, `spec-validator`, `symlink-lint`, `vendor-neutrality-score`, `vetted-ops` **Model endpoint: neutral by construction — 4 default-approved endpoint classes across independent trust domains, plus adopter opt-in.** From the [`privacy-llm` registry](../tools/privacy-llm/models.md): the framework keys approval on *endpoint identity*, not on who hosts the model, so no single LLM vendor is privileged. diff --git a/organizations/ASF/organization.md b/organizations/ASF/organization.md index 9c5040f44..fb73a4c35 100644 --- a/organizations/ASF/organization.md +++ b/organizations/ASF/organization.md @@ -252,6 +252,17 @@ release_process: release_dist: https://dist.apache.org/repos/dist # project_wiki: https://cwiki.apache.org/confluence/display/ # announce_list: announce@apache.org # + # Automated (CI) release signing — ASF-specific option, offered by + # `release-prepare automated-signing` only under this organization. + # Policy: https://infra.apache.org/release-signing.html#automated-release-signing + automated_signing: + policy_url: https://infra.apache.org/release-signing.html#automated-release-signing + key_request_channel: infra-jira # https://issues.apache.org/jira/projects/INFRA + key_request_background: INFRA-23996 + approval_body: security@apache.org # Security Team approves the workflow before use + key_spec: "4096-bit RSA, signing-only, private half held by infra-root only" + trusted_publishing_action: apache/tooling-actions/upload-to-atr # pin by commit SHA + validation: "every signed artefact rebuilt bit-by-bit identical on trusted hardware before publication" ``` ## Roster diff --git a/organizations/independent/organization.md b/organizations/independent/organization.md index ce9a2e849..8733bbdd8 100644 --- a/organizations/independent/organization.md +++ b/organizations/independent/organization.md @@ -121,6 +121,7 @@ release_process: release_dist: null # GitHub Releases (no svn dist area) project_wiki: null announce_list: null # announcements via GitHub Releases / Discussions + automated_signing: null # ASF-specific option; not offered here ``` ## Roster / tracker diff --git a/plugins/magpie-release-management/skills/announce-draft/SKILL.md b/plugins/magpie-release-management/skills/announce-draft/SKILL.md index 81298e3c3..4b079eba0 100644 --- a/plugins/magpie-release-management/skills/announce-draft/SKILL.md +++ b/plugins/magpie-release-management/skills/announce-draft/SKILL.md @@ -434,6 +434,9 @@ Release notes / changelog for : Keys used to sign the release artifacts: +Convenience artefacts, built from the released source, are also available: ← include only when release-build.md § Convenience artefacts declares any that release-promote published + : //> + Questions, feedback, and contributions are welcome on the . General user support is available on . diff --git a/plugins/magpie-release-management/skills/prepare/SKILL.md b/plugins/magpie-release-management/skills/prepare/SKILL.md index d25788f09..d4172296f 100644 --- a/plugins/magpie-release-management/skills/prepare/SKILL.md +++ b/plugins/magpie-release-management/skills/prepare/SKILL.md @@ -10,23 +10,33 @@ requires_config: - release-trains.md description: | Draft release preparation artefacts for ``: the planning - issue, the version-bump and changelog prep PR, or the post-release - development-version bump PR. Reads release metadata from - `/release-trains.md` and + issue, the version-bump and changelog prep PR (which, on a project's + first release, includes a guided review of what the `git archive` + source artefact ships and the `.gitattributes` `export-ignore` + entries that keep VCS/CI/editor metadata out), or the post-release + development-version bump PR. For ASF projects, the one-time + `automated-signing` setup drafts the Infra key request, the Security + Team notification and the reproducible-build workflow PR. Reads + release metadata from `/release-trains.md` and `/release-management-config.md`. Every output is a draft confirmed by the Release Manager before filing; the agent never - marks a PR ready, never merges, and never closes any artefact. + marks a PR ready, never merges, never closes any artefact, never files + a ticket and never sends mail. when_to_use: | Invoke when a Release Manager says "prepare the release", "draft the planning issue for ", "open the prep PR for ", "write the version bump for ", "draft the - post-release bump for ", or similar. Covers three lifecycle - moments: planning-issue creation (`/release-prepare `), - version-bump prep PR (`/release-prepare prep `), and - post-release dev-version bump (`/release-prepare post `). - Requires `/release-management-config.md` and + post-release bump for ", "review what goes into the source + release", "set up CI release signing", or similar. Covers three + lifecycle moments: planning-issue creation (`/release-prepare + `), version-bump prep PR (`/release-prepare prep `, + which also runs the first-release source-archive review), and + post-release dev-version bump (`/release-prepare post `); + plus the version-less, 🪶 ASF-only `/release-prepare + automated-signing` setup. Requires + `/release-management-config.md` and `/release-trains.md` to exist. -argument-hint: "[prep | post] " +argument-hint: "[prep | post] [--review-archive] | automated-signing" capability: capability:resolve license: Apache-2.0 --- @@ -311,25 +321,46 @@ For Step 14 (`post`): - **Planning issue labelled `announced`** — confirms Steps 10–11 completed. Accepted via `--planning-issue `. +For Step 2's source-archive review (`prep`, Step 2f) — optional: +- **`/release-build.md § Source archive`** — + `source_archive_method` (default `git-archive`) and + `export_ignore_reviewed`. Absent file or key = the review has not + happened yet, which is exactly when the sub-step runs. +- **A local clone of ``** at the release branch tip (the + resolved `user.md` clone path) — the review lists what `git archive` + would ship from *that* tree. + +For Step A (`automated-signing`, 🪶 ASF-specific): +- **`project.md` declares `organization: ASF`.** The sub-command is + not offered otherwise; see [`organizations/ASF/organization.md`](../../../../organizations/ASF/organization.md) + → `release_process.automated_signing`. +- **`release-build.md § Reproducibility checks`** — `reproducibility_source: on` + and `reproducibility_binaries: byte-identical` (or no binaries). + --- ## Inputs | Selector | Resolves to | |---|---| -| `[prep \| post]` (optional first argument) | Sub-command: `prep` = Step 2, `post` = Step 14, omit = Step 1 | +| `[prep \| post \| automated-signing]` (optional first argument) | Sub-command: `prep` = Step 2, `post` = Step 14, `automated-signing` = Step A (🪶 ASF-only, no ``), omit = Step 1 | | `` (positional) | Target release version string | | `--planning-issue ` | Explicit planning issue URL (auto-detected if omitted) | | `--release-branch ` | Override the base branch for the prep or post PR | | `--previous-tag ` | Override the previous release tag for the merged-PR query | | `--skip-empty-check` | Allow Step 1 with an empty merged-PR set; reason logged on planning issue | +| `--review-archive` | Force the full Step 2f source-archive review even when `export_ignore_reviewed` is already set | --- ## Step 0 — Pre-flight check 1. **Sub-command parsed.** Argument is one of: `` (Step 1), - `prep ` (Step 2), `post ` (Step 14). + `prep ` (Step 2), `post ` (Step 14), or + `automated-signing` (Step A; 🪶 ASF-specific — if `project.md` does + not declare `organization: ASF`, block with *"automated release + signing is an ASF Infra offering; this project's organization does + not provide one"* and do not describe the flow further). 2. **Version argument parseable.** `` matches a semver-ish pattern (`X.Y.Z`, `X.Y.Z.post0`, or similar). 3. **`release-management-config.md` readable.** Required keys present: @@ -354,8 +385,8 @@ Return ONLY valid JSON with this structure: ```json { "verdict": "proceed" | "blocked", - "sub_command": "plan" | "prep" | "post", - "version": "", + "sub_command": "plan" | "prep" | "post" | "automated-signing", + "version": "", "blockers": [""], "release_branch_base": "", "previous_tag": "" @@ -577,7 +608,100 @@ Changelog coverage must be ≥ 90% of the merged-PR set. If fewer than 90% of PRs can be categorised, surface the uncategorised set and ask the RM to classify before the PR is opened. -### 2e — Compose the prep PR +### 2e — Source-archive contents review (first release, or on drift) + +With `source_archive_method: git-archive` (the default in +`release-build.md § Source archive`) the source artefact is an export +of the tagged tree that honours `.gitattributes` `export-ignore`. The +attributes are read from the tree being archived, so they have to be +committed **before** the RC tag — which is why this review lands in +the prep PR and why `release-rc-cut` blocks while it is outstanding. +Full rationale and the classification buckets: +[`docs/release-management/reproducibility.md` § The first-release `.gitattributes` review](../../../../docs/release-management/reproducibility.md#the-first-release-gitattributes-review). + +**When the full review runs:** `export_ignore_reviewed` is unset in +`release-build.md`, or `--review-archive` was passed, or +`source_archive_method` is `git-archive` and the file has no +`§ Source archive` at all. **Otherwise** run only the drift check +(below). With `source_archive_method: custom` skip the sub-step and +say so (`archive_review: "skipped"`). + +This is an **education step**: the operator ends up knowing why every +top-level path ships or does not. Do not guess; show, classify, +explain, and ask. + +1. **List what would ship today** from the local clone at the release + branch tip, and what is tracked: + + ```bash + git archive --format=tar HEAD | tar -tf - | sort > /tmp/would-ship.txt + git ls-files | cut -d/ -f1 | sort -u # top-level tracked entries + cat .gitattributes 2>/dev/null | grep export-ignore # what is already excluded + ``` + +2. **Classify every top-level entry** into one bucket and say which — + *ship* (source, docs, build descriptors, lock files, `README*`), + *ship, never excludable* (`LICENSE`, `NOTICE`, `DISCLAIMER`, + `licenses/`), *ship, input to voter checks* (RAT excludes, in-tree + validators), *project's call* (`.asf.yaml`, `doap_*.rdf`, + `.gitignore`, large assets), *exclude: VCS metadata* + (`.gitattributes`, `.gitmodules`, `.mailmap`), *exclude: CI / bot + config* (`.github/workflows/`, `.github/dependabot.yml`, + `.gitlab-ci.yml`, `.travis.yml`, `.circleci/`, + `.pre-commit-config.yaml`), *exclude: editor / IDE* (`.idea/`, + `.vscode/`, `.devcontainer/`), *exclude: lint config not needed to + build* (`.lychee.toml`, `.markdownlint.json`, `.typos.toml`, + `.zizmor.yml`, `.yamllint`, `.codespellrc`), *agent-view dirs* + (`.claude/`, `.agents/`, `.kiro/`, `.cursor/` — exclude relay + symlink dirs, keep a single-hop canonical view if shipped files link + into it), *exclude: release-tooling scratch* + (`.apache-magpie.session-state.json`, `.apache-magpie.local.lock`). + Look inside `.github/` and the agent-view dirs; part of a directory + may ship (issue templates a shipped skill links to) while the rest + is excluded. + +3. **Check references before proposing any exclusion**: + `git grep -l -- ''` over tracked files. A path that a shipped + file links to must not be excluded or `release-verify-rc` Step 7 + fails the RC on a dangling reference; name the referrers and offer + the alternative (keep it, or repoint the reference). Flag every + committed symlink whose target would be stripped, and every symlink + that points at another symlink (safe extractors reject chains). + +4. **Propose the entries** — root-anchored (`/.pre-commit-config.yaml`) + for root-only files, directory form (`.idea/`) for directories, one + rationale comment per entry — and show the before/after listing + diff: + + ```bash + # "before": the attributes committed at HEAD + uv run --project /tools/reproducible-archive repro-archive build \ + --ref HEAD --prefix p --format tar.gz -o "$TMPDIR/before.tar.gz" + # "after": the proposed .gitattributes as edited in the working tree + uv run --project /tools/reproducible-archive repro-archive build \ + --ref HEAD --prefix p --format tar.gz --worktree-attributes -o "$TMPDIR/after.tar.gz" + uv run --project /tools/reproducible-archive repro-archive compare \ + "$TMPDIR/before.tar.gz" "$TMPDIR/after.tar.gz" # 'removed' = exactly what the review strips + ``` + + Walk the RM through each proposed entry; each one is confirmed or + dropped individually. Never exclude `LICENSE`, `NOTICE`, + `DISCLAIMER`, a build descriptor, the RAT excludes, or a referenced + path, even if asked — say why and keep it. + +5. **Record the decision.** `.gitattributes` joins the prep PR file set + (2f), and the prep PR sets `export_ignore_reviewed: ` in + `release-build.md § Source archive` so the full review does not + repeat. If the RM confirms an existing `.gitattributes` unchanged, + still set the marker (`archive_review: "confirmed-existing"`). + +**Drift check (later releases).** Top-level entries added since the +last reviewed tag — `git diff --name-only HEAD | cut -d/ +-f1 | sort -u` — that fall in an *exclude* bucket are surfaced as +candidates with the same confirm-each flow; nothing new → `archive_review: +"skipped"` with the note *"no new top-level paths since ``"*. + +### 2f — Compose the prep PR The prep PR touches: 1. Each file in `version_manifest_files` — replace current dev @@ -587,6 +711,9 @@ The prep PR touches: 3. `NOTICE` — apply the justified attribution changes (if any). 4. `LICENSE` — apply any required Category-B attribution additions (if any). +5. `.gitattributes` and `/release-build.md` + (`export_ignore_reviewed`) — only when 2e proposed or confirmed the + review. Present the full set of file diffs to the RM for confirmation before opening the PR. @@ -614,11 +741,15 @@ Entry added for covering merged PRs since . ### NOTICE/LICENSE +### Source archive contents +". Otherwise omit this section.> + ## Checklist (RM) - [ ] Version bump is correct in all manifest files - [ ] Changelog entry covers the intended scope - [ ] NOTICE attribution changes are justified - [ ] No Category-X dependency appears in the diff +- [ ] (first release) every `export-ignore` entry was reviewed; LICENSE / NOTICE / build inputs still ship Generated by `release-prepare` (magpie-release-prepare). ``` @@ -637,6 +768,7 @@ Return ONLY valid JSON with this structure: "category_x_hit": false, "notice_removal_unjustified": false, "changelog_coverage_pct": , + "archive_review": "proposed" | "confirmed-existing" | "skipped", "proposed": true } ``` @@ -644,6 +776,11 @@ Return ONLY valid JSON with this structure: `proposed` is always `true` at the point this JSON is returned. `category_x_hit` and `notice_removal_unjustified` are `false` because the skill would have stopped in 2b or 2c if they were `true`. +`archive_review` is `"proposed"` when 2e proposed `.gitattributes` +entries (and `.gitattributes` appears in `files_in_scope`), +`"confirmed-existing"` when the RM confirmed the existing entries +unchanged (only `release-build.md` joins the file set), `"skipped"` +when the review was not due or `source_archive_method` is `custom`. --- @@ -712,6 +849,113 @@ Return ONLY valid JSON with this structure: --- +## Step A — Automated release signing setup (sub-command: `automated-signing`, 🪶 ASF-specific) + +> **Scope.** Only for a project whose `project.md` declares +> `organization: ASF`. The option is an ASF Infra offering +> ([Infra § Automated release signing](https://infra.apache.org/release-signing.html#automated-release-signing)) +> and is resolved from +> [`organizations/ASF/organization.md`](../../../../organizations/ASF/organization.md) +> → `release_process.automated_signing`; for any other organization +> the value is `null`, Step 0 blocks, and the flow is not described. +> Non-ASF adopters keep the RM-key flow. + +A one-time, version-less **drafting** step. Under the policy an ASF +project may let CI sign the artefacts it builds with an +Infra-provisioned key **provided that** every signed artefact is built +reproducibly, CI deploys to staging only, and a committer re-validates +every artefact **bit-by-bit identical on trusted hardware** before +publication; the Apache Security Team approves the workflow before use. +Background: +[`docs/release-management/reproducibility.md` § Automated release signing](../../../../docs/release-management/reproducibility.md#automated-release-signing--asf-specific-optional). + +### A1 — Eligibility gate + +All of the following, else stop and list what is missing: + +- `project.md` → `organization: ASF`. +- `release-build.md` → `source_archive_method: git-archive`, + `reproducibility_source: on`, and `reproducibility_binaries: + byte-identical` for every convenience binary in `expected_artefacts` + (or none). +- The most recent RC's `release-verify-rc` report on its planning issue + shows Step 9 `PASS` with every artefact `identical`. If no such report + exists the build is not *demonstrably* reproducible yet: tell the RM + to cut and verify one RC with the checks on first. +- `release_vote_backend: atr` or `release_dist_backend: atr` — ATR + trusted publishing is the staging target the workflow template uses. + +### A2 — Draft the Infra Jira ticket + +Draft (never file) an `INFRA` ticket titled *"CI release signing key +for Apache "* that: requests the key per the policy (4096-bit +RSA, signing-only, private half held by infra-root, public block to +`KEYS`, encrypted revocation certificate to the project's private +repo); names the workflow (`ci_release_workflow`) and the staging +target (ATR via `apache/tooling-actions/upload-to-atr`, pinned by +commit SHA); **highlights the trusted-hardware validation step** — +`release-verify-rc` Step 9 with `--trusted-hardware`, `repro-archive +compare --require-identical` for every artefact, recorded on the +planning issue, gating `release-promote`; and references the +background ticket in +`release_process.automated_signing.key_request_background`. + +### A3 — Draft the Security Team notification + +Draft (never send — [spec § Boundary 3](../../../../docs/release-management/spec.md#boundary-3-agent-never-sends-mail-to-dev-users-announce)) +a mail to `release_process.automated_signing.approval_body` +(`security@apache.org`) from the RM, pointing at the ticket, the +workflow PR and the validation step, asking for the approval the +policy requires before the workflow is used. Plain text, real links, +per the repository's email rules. + +### A4 — Propose the workflow PR + +From +[`projects/_template/workflows/release-candidate.yml`](../../../../projects/_template/workflows/release-candidate.yml), +rendered with the project's slug, artefact prefix, source format and +`build_command`, placed at `ci_release_workflow`. The template builds +the source archive with the embedded `repro-archive` script (copy +`tools/reproducible-archive/src/reproducible_archive/__init__.py` to +`release/reproducible_archive.py` in the upstream repo), builds +binaries under `SOURCE_DATE_EPOCH`, builds twice and compares, +checksums with sha512 only, and uploads to ATR with OIDC. It contains +**no key material and no signing step**; the signing mechanism is what +Infra agrees on the ticket. Pin every action to a commit SHA. Open as a +draft PR via `gh pr create --web` after RM confirmation. + +### A5 — Propose the config diff + +`release-management-config.md § Signing`: `automated_release_signing: +requested`, `ci_release_workflow`, `ci_signing_infra_ticket` (once the +ticket exists). Tell the RM that `enabled` is set only after Infra has +provisioned the key, its public block is in `KEYS` +(`release-keys-sync`), the Security Team has approved, and the workflow +PR is merged — and that from then on `release-rc-cut` emits the tag +push instead of local signing, `release-verify-rc` Step 9 is mandatory, +and `release-promote` blocks without the attestation. + +Return ONLY valid JSON with this structure: + +```json +{ + "organization": "ASF", + "eligible": true | false, + "missing_conditions": [""], + "infra_ticket_draft": "", + "security_notification_draft": "", + "workflow_pr": {"path": "", "title": "", "proposed": true} | null, + "config_diff": [""], + "filed_or_sent": false, + "proposed": true +} +``` + +`filed_or_sent` is always `false`: the skill drafts the ticket and the +mail and proposes the PR; the RM files, sends and marks ready. + +--- + ## Step N+1 — Hand-back artefact The AI-driven part ends with a hand-back artefact containing: @@ -733,6 +977,10 @@ The AI-driven part ends with a hand-back artefact containing: - **NOTICE/LICENSE summary** — confirmed clean (or the removals that required justification). - **Changelog coverage** — percentage and any uncategorised PRs. +- **Source-archive review** — the `export-ignore` entries proposed or + confirmed with their reasons, the paths kept because shipped files + reference them, and the `export_ignore_reviewed` marker; or the + one-line reason the review was skipped. - **Label to apply** — `prep-pr-open` on the planning issue after the RM merges the prep PR. - **Next steps** — `release-keys-sync` (Step 3), then `release-rc-cut @@ -744,6 +992,15 @@ The AI-driven part ends with a hand-back artefact containing: - **Next development version** — restated for clarity. - **Scope** — confirmed only `version_manifest_files` were modified. +**For Step A (`automated-signing`, 🪶 ASF-specific):** + +- **Eligibility** — met, or the conditions still missing. +- **Infra ticket draft** and **Security Team notification draft** — + for the RM to file and send. +- **Workflow PR** — URL if opened as a draft, or the rendered file. +- **Config diff** — the `release-management-config.md § Signing` + changes, and what has to happen before `enabled`. + --- ## Hard rules @@ -763,6 +1020,17 @@ The AI-driven part ends with a hand-back artefact containing: - **Never emit signing commands.** `gpg`, `git tag -s`, and `svn` commands belong to other skills (`release-keys-sync`, `release-rc-cut`). +- **Never edit `.gitattributes` without per-entry confirmation**, and + never propose excluding `LICENSE`, `NOTICE`, `DISCLAIMER`, a build + descriptor, the RAT excludes, or a path a shipped file references. +- **Never mark the archive review done on the skill's own authority.** + `export_ignore_reviewed` is set only in a prep PR the RM confirmed. +- **Never file the Infra ticket, never send the Security Team mail, + never add key material to the workflow.** Step A drafts; the RM + files and sends. +- **Never offer automated release signing outside `organization: + ASF`.** The option is an ASF Infra offering; for other organizations + it does not exist in this skill. --- @@ -779,6 +1047,11 @@ The AI-driven part ends with a hand-back artefact containing: | Changelog coverage low | Many PRs lack standard labels | RM classifies uncategorised PRs before the prep PR opens | | Scope violation (prep) | A proposed file is outside the expected set | Confirm the extra file explicitly or remove it from the diff | | Scope violation (post) | A proposed file is outside `version_manifest_files` | Confirm the extra file explicitly or remove it | +| 2e: proposed exclusion is referenced | A shipped file links to the path (`git grep` hit) | Keep the path, or repoint the reference; never exclude it as-is | +| 2e: symlink chain | A committed symlink points at another symlink | Exclude the relay dir, keep the single-hop canonical link, or replace the relay with a real link | +| 2e: no local clone | `user.md` names no `` clone | Set the clone path in `user.md`, or clone and rerun `prep` | +| Step A blocked — not ASF | `project.md` organization is not `ASF` | No action; automated signing is an ASF Infra offering | +| Step A blocked — not demonstrably reproducible | No `release-verify-rc` report with every artefact `identical`, or `reproducibility_*` not set as required | Enable the checks in `release-build.md`, cut and verify an RC, then rerun | --- @@ -788,6 +1061,16 @@ The AI-driven part ends with a hand-back artefact containing: Steps 1, 2, and 14 context. - [`docs/release-management/spec.md`](../../../../docs/release-management/spec.md) — `release-prepare` per-skill specification. +- [`docs/release-management/reproducibility.md`](../../../../docs/release-management/reproducibility.md) — + the source-archive review (2e) buckets and rationale, and the 🪶 + ASF-specific automated-signing setup (Step A). +- [`/release-build.md`](../../../../projects/_template/release-build.md) — + `§ Source archive` (`source_archive_method`, `export_ignore_reviewed`) + and `§ Reproducibility checks`. +- [`projects/_template/workflows/release-candidate.yml`](../../../../projects/_template/workflows/release-candidate.yml) — + the workflow template Step A renders. +- [`tools/reproducible-archive`](../../../../tools/reproducible-archive/README.md) — + `repro-archive build` / `compare` used for the before/after listing. - [`/release-management-config.md`](../../../../projects/_template/release-management-config.md) — adopter keys this skill reads (`release_branch_base`, `version_manifest_files`, `category_x_dependencies`, diff --git a/plugins/magpie-release-management/skills/promote/SKILL.md b/plugins/magpie-release-management/skills/promote/SKILL.md index 7ed7cc989..7b9ae319c 100644 --- a/plugins/magpie-release-management/skills/promote/SKILL.md +++ b/plugins/magpie-release-management/skills/promote/SKILL.md @@ -342,8 +342,21 @@ non-blocking. the roster. If the RM is a committer but not a PMC member, set `rm_is_pmc = false`; the skill continues to emit non-command outputs but replaces the svn command set with a hand-off note. -6. **Drift check** — see *Snapshot drift* above. -7. **Override consultation** — see *Adopter overrides* above. +6. **Trusted-hardware validation recorded** (🪶 ASF-specific; only when + `release-management-config.md` sets `automated_release_signing: + enabled` under `organization: ASF`). The planning issue must carry a + `release-verify-rc` comment with the **Reproducibility validated on + trusted hardware** attestation for *this* `-rc` (every + artefact `identical`, `--trusted-hardware` asserted by the committer). + Absent → hard blocker: *"automated release signing requires every + artefact to be rebuilt bit-by-bit identical on trusted hardware before + publication ([Infra § Automated release signing](https://infra.apache.org/release-signing.html#automated-release-signing)); + run `release-verify-rc -rc --trusted-hardware --post-to + ` on your own machine first"*. Not applicable (and + never mentioned) when the key is `off`, `requested`, or the project + is not ASF. +7. **Drift check** — see *Snapshot drift* above. +8. **Override consultation** — see *Adopter overrides* above. If any check fails (except the PMC gate, which downgrades to hand-off), stop and surface what is missing. @@ -388,6 +401,8 @@ Read the following from the planning issue and | `rc_commit_sha` | git / planning issue body | commit the `-rc` tag points to; the final `` tag is cut on this SAME commit (no rebuild). `git rev-list -n1 -rc` | | `rm_gpg_fingerprint` | RM `user.md` | `release_manager.gpg_fingerprint`; the release key the final `` tag is signed with | | `git_upstream_remote` | `release-management-config.md` | `git_upstream_remote`; the remote the final `` tag is pushed to | +| `convenience_artefacts` | `release-build.md § Convenience artefacts` | the project's optional, project-specific artefacts with their `publish_channel` / `publish_command`; empty for a source-only project | +| `verify_rc_binaries` | planning issue body | the `release-verify-rc` Step 9 result for this RC: which convenience artefacts reproduced (`identical` / documented `WARN`) and which `differs` | Surface the loaded metadata to the RM for a brief sanity check before proceeding to Step 2. @@ -484,6 +499,44 @@ stop. --- +### Convenience artefacts (optional, project-specific) + +Only when `convenience_artefacts` is non-empty. The source promotion +above is the release; this block publishes what the project ships +*besides* the source, to wherever the project declared. Emit it +**after** the dist promotion and the final tag, as its own section, +one entry per artefact: + +- `publish_channel: dist-release` — nothing to emit: the artefact + moved with the source in the promotion above; say so. +- any other channel — render the entry's `publish_command` verbatim, + with `` substituted (for example `twine upload + dist/apache_-*`, `mvn nexus-staging:release + -DstagingRepositoryId=`, `docker push + /:`, `helm push …`). These are the + project's own commands; the skill never invents a channel or a + command the config does not declare. + +**Reproducibility gate — the artefact must be good before it is +published.** A convenience artefact is publishable only if the +`release-verify-rc` run recorded on the planning issue rebuilt it from +the voted tag and it reproduced: `identical`, or `WARN` with every +difference matched by its `known_divergences`. For an artefact that +reported `DIFFERS`, or that no verify-rc run covered, emit a **HOLD** +note in place of its publish command: + +```text +HOLD: — not published. release-verify-rc Step 9 did not +reproduce it from -rc (). A binary that +cannot be rebuilt from the voted source is not known to be what the vote +approved. Fix the build or document the divergence in release-build.md, +re-run `release-verify-rc -rc`, then re-run this skill. +``` + +The source promotion is not held back by a convenience artefact; the +source is the release, the artefact is a courtesy, and a courtesy that +cannot be verified is withheld, not shipped. + ### Mirror note (required for all backends) After the backend command block, always include: @@ -514,13 +567,17 @@ Return ONLY valid JSON with this structure: "rm_is_pmc": true | false, "handoff_note": "", "proposed_label": "promoted", - "mirror_note_present": true + "mirror_note_present": true, + "convenience_publish_commands": [": "], + "convenience_held": [": "] } ``` `handoff_note` is non-null only when `rm_is_pmc = false`; the command block is still populated (a PMC member can copy and run it). `mirror_note_present` is always `true` — the mirror and timing note is never omitted. +`convenience_publish_commands` and `convenience_held` are both empty for a +source-only project; every declared artefact appears in exactly one of them. --- diff --git a/plugins/magpie-release-management/skills/rc-cut/SKILL.md b/plugins/magpie-release-management/skills/rc-cut/SKILL.md index cae780c6c..9e3abfc44 100644 --- a/plugins/magpie-release-management/skills/rc-cut/SKILL.md +++ b/plugins/magpie-release-management/skills/rc-cut/SKILL.md @@ -9,12 +9,18 @@ requires_config: - release-build.md - release-management-config.md description: | - Emit the paste-ready command sequence to tag an RC, build artefacts, - sign each artefact, generate checksums, and stage them to the adopter's - distribution backend. Covers Steps 4–5 of the release-management - lifecycle. Never runs any command locally — all sequences are emitted - for the Release Manager to execute on their own machine with their own - key and ASF credentials. + Emit the paste-ready command sequence to tag an RC, build artefacts + (the source archive reproducibly, via `git archive` + `.gitattributes` + `export-ignore` + the framework's `repro-archive` tool), optionally + self-check reproducibility, sign each artefact, generate checksums, and + stage them to the adopter's distribution backend. Covers Steps 4–5 of + the release-management lifecycle. Never runs any command locally — all + sequences are emitted for the Release Manager to execute on their own + machine with their own key and ASF credentials. Blocks while the + first-release `.gitattributes` review (`release-prepare prep`) is + outstanding. For ASF projects with `automated_release_signing: enabled`, + emits the tag push that triggers the CI build instead of local + sign/stage commands. when_to_use: | Invoke when a Release Manager says "cut rc1 for ", "prepare rc for ", "tag the release candidate", "stage the RC to @@ -257,6 +263,30 @@ Any path that includes `dist/release/` is on a hard denylist (when `release_dist skill refuses to emit a command that stages to `dist/release/` (`release_dist_backend = svnpubsub`) regardless of input. Promotion is `release-promote`'s responsibility. +**Golden rule 6 — the source artefact is an export of the tag, never +an archive of a working tree.** +With `source_archive_method: git-archive` (the default) the source +artefact is `repro-archive build --ref -` — `git archive` +(tracked files at the tag only, `.gitattributes` `export-ignore` +honoured) with every +[reproducible-builds.org archive rule](https://reproducible-builds.org/docs/archives/) +applied (one `SOURCE_DATE_EPOCH` mtime, sorted members, uid/gid 0, +`a=rX,u+w`, no PAX `atime`/`ctime`, `gzip -n`, `zip -X`). The skill +never emits `zip -r`, `tar czf `, or any command that packs a +working directory; a working tree carries `__pycache__`, editor state +and untracked files, and no voter can regenerate it. Rationale and the +rule-by-rule mapping: +[`docs/release-management/reproducibility.md`](../../../../docs/release-management/reproducibility.md). + +**Golden rule 7 — an unreviewed `.gitattributes` blocks the cut.** +`git archive` reads `export-ignore` from the tree it archives, so the +first-release review of what ships (`release-prepare prep` Step 2f) +must have landed *before* the RC tag exists. While +`export_ignore_reviewed` is unset in `release-build.md` and +`source_archive_method` is `git-archive`, Step 0 blocks and points at +`release-prepare prep `; `--allow-unreviewed-archive` is the +explicit, logged override. + --- ## Adopter overrides @@ -295,10 +325,20 @@ non-blocking. already exist on the remote; if it does, the skill blocks and the RM decides whether to bump RC or delete the existing tag. - **`/release-build.md` readable** — `build_command`, - `expected_artefacts`, `digest_set`, optional `binary_exclude_list`. + `expected_artefacts`, `digest_set`, optional `binary_exclude_list`; + `§ Source archive` (`source_archive_method`, `source_archive_format`, + `source_archive_prefix`, `export_ignore_reviewed`) and + `§ Reproducibility checks` (`reproducibility_source`, + `reproducibility_binaries`, `binary_rebuild_command`). - **`/release-management-config.md` readable** — `release_dist_backend`, `release_dist_url_template`, - optional `release_publish_command_template`. + optional `release_publish_command_template`; `§ Signing › + automated_release_signing` (🪶 ASF-specific; read only when + `project.md` declares `organization: ASF`). +- **`.gitattributes` reviewed** — when `source_archive_method` is + `git-archive`, `export_ignore_reviewed` is set (the first-release + review in `release-prepare prep` Step 2f has landed and is in the + tree the tag will point at). --- @@ -311,6 +351,8 @@ non-blocking. | `--planning-issue ` | Explicit planning issue URL (auto-detected if omitted) | | `--release-branch ` | Override release branch (default from `release_branch_base` in config) | | `--remote ` | Override the git remote name pointing at the upstream repo (default from `git_upstream_remote` in config, else `origin`) | +| `--allow-unreviewed-archive` | Cut the RC although `export_ignore_reviewed` is unset; the override is recorded in the Step 4 planning-issue comment | +| `--skip-repro-check` | Do not emit Step 2b's optional reproducibility self-check (has no effect when `automated_release_signing: enabled`, where the check is mandatory) | --- @@ -333,8 +375,26 @@ non-blocking. (`release_dist_backend`, `release_dist_url_template`) are present. 7. **Digest set valid.** `digest_set` in `release-build.md` contains at least `sha512` and does not contain `md5` or `sha1`. -8. **Drift check** — see *Snapshot drift* above. -9. **Override consultation** — see *Adopter overrides* above. +8. **Source-archive contents reviewed.** When `source_archive_method` + is `git-archive` (or unset — that is the default), `release-build.md + § Source archive` must set `export_ignore_reviewed`. If it is unset + and `--allow-unreviewed-archive` was not passed, block with + `archive_reviewed: false` and the remediation *"run + `release-prepare prep ` — its Step 2f walks you through + what ships in the source archive and lands `.gitattributes` in the + prep PR"*. With the override, proceed with `archive_reviewed: false` + and carry the override into Step 4. With `source_archive_method: + custom` the check does not apply (`archive_reviewed: true`). +9. **Signing mode consistent** (🪶 ASF-specific). When + `automated_release_signing` is `enabled`, `project.md` must declare + `organization: ASF`, `reproducibility_source` must be `on`, and + `reproducibility_binaries` must be `byte-identical` for every + convenience binary in `expected_artefacts` — the policy conditions in + [Infra § Automated release signing](https://infra.apache.org/release-signing.html#automated-release-signing). + Any other combination blocks. For a non-ASF project the key is + ignored and never mentioned. +10. **Drift check** — see *Snapshot drift* above. +11. **Override consultation** — see *Adopter overrides* above. If any check fails (and is not overridable), stop and surface what is missing with the exact key name or API path that failed. @@ -346,11 +406,15 @@ Return ONLY valid JSON with this structure: "verdict": "proceed" | "blocked", "blockers": [""], "rc_tag_exists": true | false, - "prep_pr_merged": true | false + "prep_pr_merged": true | false, + "archive_reviewed": true | false } ``` `verdict` is `"proceed"` only when all hard blockers resolve. +`archive_reviewed` is `true` when `export_ignore_reviewed` is set or the +check does not apply; `false` when the review is outstanding (blocked, +or overridden with `--allow-unreviewed-archive`). --- @@ -370,6 +434,13 @@ Read the following from `/release-build.md` and | `signing_key_fingerprint` | user.md or `release-management-config.md` | `rm_key_fingerprint` | | `release_branch` | `release-management-config.md` | `release_branch_base` (or `--release-branch` override) | | `git_upstream_remote` | `release-management-config.md` | `git_upstream_remote` — git remote name pointing at the upstream repo (default `origin`, or `--remote` override) | +| `source_archive_method` | `release-build.md § Source archive` | `git-archive` (default) or `custom` | +| `source_archive_format` | `release-build.md § Source archive` | `tar.gz` or `zip` | +| `source_archive_prefix` | `release-build.md § Source archive` | top-level directory inside the archive, rendered with `` | +| `reproducibility_source` | `release-build.md § Reproducibility checks` | `on` (default with `git-archive`) or `off` | +| `reproducibility_binaries` | `release-build.md § Reproducibility checks` | `off` (default), `byte-identical`, `documented-divergence`; with `binary_rebuild_command` | +| `signing_mode` | `release-management-config.md § Signing` | `rm-key` (default) or `ci-automated` when `automated_release_signing: enabled` **and** `project.md` → `organization: ASF`; non-ASF projects always resolve to `rm-key` | +| `convenience_artefacts` | `release-build.md § Convenience artefacts` | the project's optional, project-specific artefacts besides the source — each with `build_command`, `staging` / `stage_command`, `reproducibility`, `vote_included`; empty for a source-only project | Surface the loaded configuration to the RM for confirmation before proceeding to Step 2. @@ -388,7 +459,13 @@ Return ONLY valid JSON with this structure: "staging_url": "", "signing_key_fingerprint": "", "release_branch": "", - "git_upstream_remote": "" + "git_upstream_remote": "", + "source_archive_method": "git-archive" | "custom", + "source_archive_format": "tar.gz" | "zip", + "source_archive_prefix": "", + "reproducibility_source": "on" | "off", + "reproducibility_binaries": "off" | "byte-identical" | "documented-divergence", + "signing_mode": "rm-key" | "ci-automated" } ``` @@ -414,16 +491,77 @@ git push - **Section 2 — Build command.** -The exact `build_command` from `release-build.md`, emitted verbatim (run at -the tag). First **gitignore the RC artefacts** (`` + `.asc`/`.sha512`, +First **gitignore the RC artefacts** (`` + `.asc`/`.sha512`, e.g. a committed glob like `*-source.zip*`) so a stray `git add` never commits -an RC build: +an RC build. Then, depending on `source_archive_method`: + +*`git-archive` (default).* The source artefact is exported from the tag +with the framework's +[`reproducible-archive`](../../../../tools/reproducible-archive/README.md) +tool (`` is `.apache-magpie` in an adopting project, `.` in +the framework checkout; `python3 /tools/reproducible-archive/src/reproducible_archive/__init__.py` +is the no-`uv` equivalent). It packs only tracked files at the tag, +honours `.gitattributes` `export-ignore`, and applies every +reproducible-builds.org archive rule, so the bytes are a function of +the tag alone. It prints the **record** the RM pastes back for the +Step 4 comment: the commit, the `SOURCE_DATE_EPOCH` it used (the tag's +committer timestamp), the sha512, and the +[Software Heritage identifiers](https://swhid.org/) — `swh:1:rev:` of +the commit and `swh:1:dir:` of the archive's expanded content, both +qualified with the repository URL (`--origin`, rendered from +``) — plus a note saying whether the content SWHID equals +the repository tree at the commit (nothing `export-ignore`d) or not. +`build_command` (if any) follows, for convenience binaries only, with +the same `SOURCE_DATE_EPOCH` exported so embedded timestamps are fixed: + +```text +# Run at the release tag - +uv run --project /tools/reproducible-archive repro-archive build \ + --ref "-" --format \ + --prefix "" \ + --origin "https://github.com/" \ + -o "" +# → prints: commit , SOURCE_DATE_EPOCH , sha512 , +# swhid_rev swh:1:rev:;origin=…, swhid_dir swh:1:dir:;origin=…;anchor=…, +# swhid_dir_note the repository tree + +# Convenience binaries (only when build_command is set): +export SOURCE_DATE_EPOCH="$(uv run --project /tools/reproducible-archive repro-archive epoch --ref "-")" + +``` + +`` is the canonical source artefact from +`expected_artefacts`; its extension must match `source_archive_format`. + +*`custom`.* The exact `build_command` from `release-build.md`, emitted +verbatim (run at the tag), with `SOURCE_DATE_EPOCH` exported first: ```text # Run at the release tag - +export SOURCE_DATE_EPOCH="$(git log -1 --format=%ct "-")" ``` +Under either method, never emit `zip -r`, `tar czf ` or any +other command that packs a working directory (Golden rule 6). + +*Convenience artefacts (optional, project-specific).* When +`convenience_artefacts` is non-empty, follow the source archive with +one block per entry — the entry's own `build_command` verbatim, under +the same `SOURCE_DATE_EPOCH`, so the artefact is a function of the tag +and a voter can rebuild it in `release-verify-rc` Step 9 (the check +that decides whether a binary is good). The framework does not know +how a project builds its wheels, jars or images; the config does: + +```text +# Convenience artefact: () — built from the tagged source +export SOURCE_DATE_EPOCH="$(uv run --project /tools/reproducible-archive repro-archive epoch --ref "-")" + +``` + +For a source-only project say *"no convenience artefacts declared"* +rather than emitting a build block. + **Section 3 — Sign commands.** For each artefact in `expected_artefacts`: @@ -466,10 +604,149 @@ Return ONLY valid JSON with this structure: or `sha1` digest command was emitted. `proposed` is always `true` at the point this JSON is returned — the RM has not yet confirmed execution. +When `signing_mode` is `ci-automated`, Sections 3 and 4 are **not** +emitted (CI signs and checksums); return them as empty lists and +continue with Step 2c instead of Step 3. + +--- + +## Step 2b — Emit reproducibility self-check commands (optional) + +Skipped when `reproducibility_source` is `off` **and** +`reproducibility_binaries` is `off`, or when `--skip-repro-check` was +passed and `signing_mode` is `rm-key`. Mandatory (the flag is ignored) +when `signing_mode` is `ci-automated`. Run **after** the build and +**before** signing: a non-reproducible build found here costs a rebuild, +found by a voter it costs an RC. + +**Source (`reproducibility_source: on`).** Lint the artefact against +the reproducible-builds.org checklist, rebuild it into a scratch +directory from the same tag, and compare: + +```text +# 1. every archive rule holds (single SOURCE_DATE_EPOCH mtime, sorted, uid/gid 0, a=rX,u+w, no PAX atime/ctime, gzip -n / zip -X) +uv run --project /tools/reproducible-archive repro-archive check \ + "" --epoch "" +# 2. rebuild from the tag and require byte-identical output +mkdir -p rebuild +uv run --project /tools/reproducible-archive repro-archive build \ + --ref "-" --format \ + --prefix "" -o "rebuild/" +uv run --project /tools/reproducible-archive repro-archive compare --require-identical \ + "" "rebuild/" +``` + +With `source_archive_method: custom` the `check` still runs (it lints +any `.tar`, `.tar.gz` or `.zip`); the rebuild step re-runs +`build_command` into `rebuild/` and compares with +`repro-archive compare`. `content-identical` is then a warning to +switch the build to `repro-archive build` or `repro-archive recipe`; +`differs` is a stop. + +**Convenience artefacts.** One block per entry in +`convenience_artefacts`, using the entry's `reproducibility` mode +(default `reproducibility_binaries`). `byte-identical` — re-run the +entry's `build_command` into `rebuild/` under the same +`SOURCE_DATE_EPOCH` and compare bytes: + +```text +export SOURCE_DATE_EPOCH="" +( cd rebuild && ) +cmp "" "rebuild/" \ + && echo "identical: " || echo "DIFFERS: " +``` + +`documented-divergence` — the same rebuild, then the entry's +`verification_command` (for example `diffoscope +rebuild/`); any difference not listed under the +entry's `known_divergences` is a stop, listed ones are reported. +`off` — state `SKIP` explicitly for that artefact. An artefact that +does not reproduce here is not good to sign: it is not known to be +what the tagged source produces, and `release-promote` will withhold +its publication until a verify-rc run reproduces it. + +The RM runs the block and reports the outcome. Any `differs` / +`DIFFERS` stops the cut: the RM fixes the build (or documents the +divergence) and rebuilds before signing anything. + +Return ONLY valid JSON with this structure: + +```json +{ + "source_check_enabled": true | false, + "binary_check_mode": "off" | "byte-identical" | "documented-divergence", + "mandatory": true | false, + "source_check_commands": ["", "", ""], + "binary_check_commands": [""], + "stop_on": ["differs", "DIFFERS"], + "proposed": true +} +``` + +`mandatory` is `true` only when `signing_mode` is `ci-automated`. +`source_check_commands` is empty when `source_check_enabled` is +`false`; `binary_check_commands` is empty when `binary_check_mode` is +`off`. `stop_on` always lists the verdicts that halt the cut. +`proposed` is always `true`. + +--- + +## Step 2c — CI-signed flow (🪶 ASF-specific, `signing_mode: ci-automated`) + +Only for a project whose `project.md` declares `organization: ASF` and +whose `release-management-config.md` sets `automated_release_signing: +enabled` after the one-time setup in `release-prepare automated-signing` +(Infra-provisioned key, Security Team approval, workflow merged). For +every other project this step does not exist and is never mentioned. + +Under +[Infra § Automated release signing](https://infra.apache.org/release-signing.html#automated-release-signing) +CI builds, signs and **stages** the artefacts; a committer re-validates +them bit-by-bit on trusted hardware before anything is published. The +RM still signs the **tag** with their own key (Section 1). Instead of +Sections 3–4 and Step 3, emit: + +```text +# 1. Push the signed tag — this triggers +git push - +# 2. Watch the run; it builds reproducibly (repro-archive), self-compares, +# checksums, and uploads to ATR (OIDC trusted publishing). It publishes nothing. +gh run list --repo --workflow --branch - +gh run watch --repo +# 3. Confirm the staged candidate and its checks in ATR +atr check status --verbose +# 4. Record the run URL and the SOURCE_DATE_EPOCH from the run log for Step 4 +``` + +Then hand off: *"Before this RC can be promoted, a committer must run +`release-verify-rc -` on their own hardware; its Step 9 +rebuilds every artefact and requires `identical`. `release-promote` +refuses to promote without that attestation on the planning issue."* + +Return ONLY valid JSON with this structure: + +```json +{ + "signing_mode": "ci-automated", + "organization": "ASF", + "ci_release_workflow": "", + "trigger_commands": ["git push -", "gh run list …", "gh run watch …"], + "local_sign_commands_omitted": true, + "trusted_hardware_validation_required": true, + "proposed": true +} +``` + +`local_sign_commands_omitted` and `trusted_hardware_validation_required` +are always `true` in this mode. + --- ## Step 3 — Emit staging commands +Skipped when `signing_mode` is `ci-automated` (CI stages; Step 2c +recorded the run). Otherwise: + Compose the backend-shaped staging command sequence based on `release_dist_backend`. @@ -541,6 +818,16 @@ This block is a proposal like the rest; the RM runs it on their machine under their own ATR credentials. `atr_platform_url` from `release-management-config.md` is the target platform. +**Convenience artefacts with `staging: registry-staging`** (optional, +project-specific) get one more block after the dist-backend staging: +the entry's `stage_command` verbatim (for example `twine upload -r +testpypi …`, `mvn nexus-staging:deploy`, `docker push +/:-`). Entries staged as `dist-dev` or +`atr` travel with the source in the blocks above and need nothing +extra; say so. Never emit a command that publishes to the artefact's +final `publish_channel` here — publication is `release-promote`'s +step, after the vote. + Present the staging command block to the RM and ask for confirmation before proceeding to Step 4. @@ -582,6 +869,24 @@ The comment must include: - The staging URL (where verifiers can download artefacts). - The expected artefact list with filenames (not yet public checksums — those are confirmed once the RM has run the commands). +- **Reproducibility record** — everything `repro-archive build` + printed: the source commit hash, the repository URL + (`https://github.com/`), the SWHIDs (`swh:1:rev:` of the + commit and `swh:1:dir:` of the archive content, with their `origin` + and `anchor` qualifiers) and the note on whether the content SWHID + equals the repository tree, the `SOURCE_DATE_EPOCH`, the sha512, the + `source_archive_format` and `source_archive_prefix`, and the outcome + of Step 2b (or `skipped`). A voter needs the commit and epoch to + rebuild in `release-verify-rc` Step 9, the sha512 to compare bytes, + and the `swh:1:dir:` to compare trees — with their own recomputation + and with the value ATR computes for the candidate. Under + `ci-automated` also the workflow run URL. +- **Convenience artefacts** (when declared) — one line each: name, + kind, where it is staged, its `reproducibility` mode and Step 2b + outcome, whether it is `vote_included`, and the source `swh:1:dir:` + it was built from. +- If `--allow-unreviewed-archive` was used: a line saying the source + archive contents were **not** reviewed and why. - The proposed next label: `rc-staging`. Present the proposed comment to the RM. Ask for confirmation before @@ -612,14 +917,22 @@ The AI-driven part ends with a hand-back artefact containing: - **RC identifier** — `-`. - **Tag command** — the `git tag -s` + `git push` sequence to copy and run. -- **Build command** — the `build_command` to run after the tag. -- **Sign commands** — one `gpg --detach-sign --armor` per expected artefact. +- **Build command** — `repro-archive build` for the source artefact + (or `build_command` under `custom`), plus `build_command` for any + convenience binaries under `SOURCE_DATE_EPOCH`. +- **Reproducibility self-check** — Step 2b's `check` / rebuild / + `compare` block, or the explicit reason it was skipped. +- **Sign commands** — one `gpg --detach-sign --armor` per expected artefact + (omitted under `ci-automated`, where Step 2c's tag push replaces them). - **Checksum commands** — sha512 (and sha256 where configured) per artefact. - **Staging commands** — backend-shaped `svn import` / `gh release` / `aws s3 cp`. -- **Planning-issue comment** — the proposed comment body (pending RM confirmation). +- **Planning-issue comment** — the proposed comment body (pending RM + confirmation), including the reproducibility record. - **Next step** — run `release-verify-rc -` against the - staging URL to verify signatures, checksums, license headers, and - artefact completeness before opening the `[VOTE]` thread. + staging URL to verify signatures, checksums, license headers, + artefact completeness and reproducibility before opening the `[VOTE]` + thread. Under `ci-automated` that run is the policy-required + validation on trusted hardware and must report `identical`. --- @@ -637,6 +950,22 @@ The AI-driven part ends with a hand-back artefact containing: RC tag already exists. - **Never invent artefact names.** All artefact filenames must come from `/release-build.md`; do not derive or guess. +- **Never pack a working tree.** No `zip -r`, no `tar czf `; the + source artefact is `repro-archive build` at the tag (or the adopter's + `custom` build command), never an archive of the checkout. +- **Never cut past an unreviewed `.gitattributes` silently.** Block, or + proceed only on `--allow-unreviewed-archive` and say so in the Step 4 + comment. +- **Never offer automated release signing to a non-ASF project**, and + never emit the CI-signed flow unless `automated_release_signing` is + `enabled` *and* the reproducibility conditions in Step 0 check 9 hold. +- **Never add key material or a signing step to the CI workflow.** The + agent holds neither the RM's key nor the CI key; the workflow template + contains no `gpg` invocation. +- **Never invent a convenience artefact, its build, or its channel.** + Everything about an artefact besides the source comes from + `release-build.md § Convenience artefacts`; a project that declares + none gets none, and no build command runs outside `SOURCE_DATE_EPOCH`. --- @@ -650,6 +979,11 @@ The AI-driven part ends with a hand-back artefact containing: | Staging command uses `dist/release/` (`release_dist_backend = svnpubsub`) | Config error in `release_dist_url_template` | Correct the template; staging target must be `dist/dev/` | | `release-build.md` missing or incomplete | Adopter has not filled out the template | Complete `/release-build.md` before running this skill | | `signing_key_fingerprint` empty | `rm_key_fingerprint` not set in user.md or config | Add `rm_key_fingerprint` to user.md (preferred) or `release-management-config.md` | +| Pre-flight blocked — `archive_reviewed: false` | `export_ignore_reviewed` unset in `release-build.md § Source archive` | Run `release-prepare prep `; its Step 2f reviews what ships and lands `.gitattributes` in the prep PR. `--allow-unreviewed-archive` is the logged override | +| Pre-flight blocked — signing mode inconsistent | `automated_release_signing: enabled` without `organization: ASF`, or without `reproducibility_source: on` / `reproducibility_binaries: byte-identical` | Set `automated_release_signing: off` (or `requested`) until the conditions hold; see `release-prepare automated-signing` | +| Step 2b `compare` → `content-identical` | Source artefact built with a plain `git archive` or another tool version, not `repro-archive build` | Rebuild with `repro-archive build` (or `repro-archive recipe`); under `ci-automated` this is a stop | +| Step 2b `compare` → `differs` | The artefact is not the tagged tree (built from a dirty checkout, wrong ref, or a non-deterministic `custom` build) | Rebuild at the tag from a clean checkout; fix the build; do not sign | +| Step 2b binary `DIFFERS` | Build embeds timestamps, paths or a non-pinned toolchain | Honour `SOURCE_DATE_EPOCH`, pin the toolchain, `ARFLAGS=Dcvr`; or switch to `documented-divergence` and list the divergence in `release-build.md` | --- @@ -661,7 +995,17 @@ The AI-driven part ends with a hand-back artefact containing: `release-rc-cut` per-skill specification. - [`/release-build.md`](../../../../projects/_template/release-build.md) — adopter keys this skill reads (`build_command`, `expected_artefacts`, - `digest_set`, `binary_exclude_list`). + `digest_set`, `binary_exclude_list`, `source_archive_*`, + `export_ignore_reviewed`, `reproducibility_*`). +- [`docs/release-management/reproducibility.md`](../../../../docs/release-management/reproducibility.md) — + the source-archive contract, the reproducible-builds.org rule mapping, + the reproducibility checks, and the ASF automated-signing option. +- [`tools/reproducible-archive`](../../../../tools/reproducible-archive/README.md) — + `repro-archive build` / `check` / `compare` / `recipe` / `epoch`. +- [reproducible-builds.org § Archive metadata](https://reproducible-builds.org/docs/archives/) — + the archive rules the source artefact satisfies. +- [Infra § Automated release signing](https://infra.apache.org/release-signing.html#automated-release-signing) — + 🪶 ASF-specific policy behind Step 2c. - [`/release-management-config.md`](../../../../projects/_template/release-management-config.md) — adopter keys this skill reads (`release_dist_backend`, `release_dist_url_template`, `release_publish_command_template`, diff --git a/plugins/magpie-release-management/skills/verify-rc/SKILL.md b/plugins/magpie-release-management/skills/verify-rc/SKILL.md index 31fbb46c6..50d165be1 100644 --- a/plugins/magpie-release-management/skills/verify-rc/SKILL.md +++ b/plugins/magpie-release-management/skills/verify-rc/SKILL.md @@ -14,10 +14,15 @@ description: | checksums), Apache RAT licence headers, NOTICE/LICENSE completeness, prohibited-binary absence (including `.pyc` / `__pycache__`), source-tree integrity (no dangling symlinks or broken internal - references), and version-string consistency. Emits a - structured PASS / PASS-WITH-WARNINGS / FAIL report. Makes no state - change; a `--post-to ` flag proposes a comment for - explicit RM confirmation before any posting. + references), version-string consistency, and — optionally, per + `release-build.md § Reproducibility checks` — reproducibility: the + source archive is rebuilt from the tag with `repro-archive` and + compared byte-for-byte with the staged artefact, and convenience + binaries are rebuilt and compared (mandatory for ASF projects with + automated release signing, as the policy's validation on trusted + hardware). Emits a structured PASS / PASS-WITH-WARNINGS / FAIL report. + Makes no state change; a `--post-to ` flag proposes a + comment for explicit RM confirmation before any posting. when_to_use: | Invoke when a Release Manager or voter says "verify rc N for ", "run pre-flight on -rcN", "check the RC @@ -25,7 +30,7 @@ when_to_use: | pre-flight phase — before the `[VOTE]` thread is opened (RM's self-check) or during the vote window (any voter's dev loop). Can be run standalone with no other release-* skill in the session. -argument-hint: "-rcN [--post-to ]" +argument-hint: "-rcN [--post-to ] [--skip-repro] [--trusted-hardware]" capability: capability:triage license: Apache-2.0 --- @@ -287,10 +292,15 @@ non-blocking. `version_manifest_files`. - **`/release-build.md` readable** — expected artefact list, digest set, binary-exclude list, RAT configuration - path. + path; `§ Source archive` and `§ Reproducibility checks` for Step 9 + (both optional — absent keys mean the defaults `git-archive` / + `reproducibility_source: on` / `reproducibility_binaries: off`). - **Network reachable** — the staging URL and the `KEYS` file URL must be fetchable. If either is unreachable, the skill stops at the inventory step and reports `FAIL` with the URL that failed. +- **A clone of `` reachable** for Step 9 — the resolved + `user.md` local clone path, or a fresh `git clone` the recipe emits. + The rebuild happens in the voter's checkout, on the voter's machine. --- @@ -300,6 +310,8 @@ non-blocking. |---|---| | `-rcN` (positional) | RC identifier to verify (e.g. `2.11.0-rc1`) | | `--post-to ` | Planning issue URL; if present, draft a comment for RM confirmation (never auto-posts) | +| `--skip-repro` | Skip Step 9 even when `reproducibility_source` is `on`; ignored (Step 9 stays mandatory) when the project has `automated_release_signing: enabled` | +| `--trusted-hardware` | The committer asserts this run executes on hardware they control, not on CI. 🪶 ASF-specific: required for the trusted-hardware attestation the `--post-to` comment carries under `automated_release_signing: enabled`; the skill can state the assertion, never make it | --- @@ -644,32 +656,30 @@ tarball was not exported clean from the tag. ## Step 7 — Source-tree integrity (dangling symlinks + broken references) -The rc1 `-1` came from this class of defect, not from signatures or -RAT: committed agent-view symlinks (`.claude/skills/*`, `.kiro/skills/*`, -`.github/skills/*`) relayed into a directory (`.agents/`) that the -release stripped, so **every relay dangled**; and shipped files linked -to other stripped paths (`.github/` templates, `projects/_template/.gitignore`, -`.claude/skills/magpie-*/SKILL.md`), so the framework's own validators -failed. This step runs the same checks the framework applies to its own -tree, but **against the unpacked tarball**, so a packaging regression -(an `export-ignore` that strips a still-referenced path) fails the RC -before the `[VOTE]` rather than during it. - -Emit the paste-ready recipe (run from the unpacked dir; adapt tool -paths to the project — for Magpie the tools ship in-tree under -`tools/`): +A source archive can be signed, checksummed and licence-clean and +still be broken: a committed symlink whose target was stripped by +`export-ignore`, or a shipped file that links to a path the release +no longer contains. The framework's own first RC failed on exactly +this (relay symlinks into a stripped directory, docs linking stripped +templates), so this step runs the project's own integrity checks +**against the unpacked archive**, where a packaging regression fails +the RC before the `[VOTE]` rather than during it. + +Read `source_tree_validators` from +`/release-build.md § Source-tree validators` — the +project's own commands, run from the unpacked directory (the adopter +chooses them; the framework does not assume any). Emit: ```bash cd -# 1. Dangling symlinks — every symlink must resolve inside the tarball. +# 1. Dangling symlinks — every symlink must resolve inside the archive. find . -type l ! -exec test -e {} \; -print # any output = FAIL -# 2. Internal reference / link integrity — run the project's own -# validators against the unpacked source (Magpie ships these): -uv run --project tools/symlink-lint symlink-lint . -uv run --project tools/skill-and-tool-validator skill-and-tool-validate -uv run --project tools/spec-validator spec-validate # if the project ships specs +# 2. Internal reference / link integrity — the project's own validators +# from release-build.md § Source-tree validators, one per line: + + ``` Classify: @@ -679,8 +689,9 @@ Classify: internal link / missing referenced file. This is a hard `FAIL`: a release whose own files reference content that was stripped from the artefact is incomplete. -- `SKIP` — the project ships no symlinks and no in-tree validators - (state this explicitly; do not silently pass). +- `SKIP` — the project ships no symlinks and declares no validators + (state this explicitly; do not silently pass — the dangling-symlink + scan still runs whenever the archive contains a symlink). Do **not** post-filter the `find`; surface every dangling link so the voter sees the full set. When a validator is not shippable in the @@ -740,7 +751,151 @@ Return ONLY valid JSON with this structure: --- -## Step 9 — Hand-back verification report +## Step 9 — Reproducibility checks (optional) + +Confirm the staged artefacts are a function of the tag alone. Read +`release-build.md § Source archive` and `§ Reproducibility checks`, +and the reproducibility record `release-rc-cut` left on the planning +issue (source commit, `SOURCE_DATE_EPOCH`, sha512, format, prefix). +Background and the rule-by-rule mapping: +[`docs/release-management/reproducibility.md`](../../../../docs/release-management/reproducibility.md). + +**When it runs.** `reproducibility_source: on` (the default with +`source_archive_method: git-archive`) or `reproducibility_binaries` +not `off`. `--skip-repro` skips it and the report says so. 🪶 +ASF-specific: when `release-management-config.md` sets +`automated_release_signing: enabled` (only meaningful under +`organization: ASF`) the step is **mandatory** and `--skip-repro` is +ignored — this run *is* the validation on trusted hardware that +[Infra § Automated release signing](https://infra.apache.org/release-signing.html#automated-release-signing) +requires before publication, and the bar is byte-identical. + +**Source.** Emit the paste-ready recipe (`` is +`.apache-magpie` in an adopting project, `.` in the framework checkout; +`python3 /tools/reproducible-archive/src/reproducible_archive/__init__.py` +works without `uv`): + +```bash +# 1. The tag resolves to the commit recorded on the planning issue +git -C fetch --tags +git -C rev-parse "^{commit}" # expect: +git -C tag -v "" # signed tag verifies against KEYS + +# 2. The staged archive satisfies every reproducible-builds.org rule, and its +# content is the recorded tree (the swh:1:dir: on the planning issue — and, +# under ATR, the SWHID the candidate page shows) +uv run --project /tools/reproducible-archive repro-archive check \ + "" --epoch "" \ + --swhid "" + +# 3. Rebuild from the tag with the recorded epoch, prefix and format, then compare +uv run --project /tools/reproducible-archive repro-archive build \ + --repo --ref "" --format \ + --prefix "" --epoch "" \ + -o rebuilt/ +uv run --project /tools/reproducible-archive repro-archive compare \ + "" rebuilt/ # add --require-identical under automated signing +``` + +With `source_archive_method: custom`, step 3 re-runs the adopter's +`build_command` at the tag under the recorded `SOURCE_DATE_EPOCH` and +compares its output the same way. + +Classify the source result: + +| `compare` verdict | RM-key mode | `automated_release_signing: enabled` | +|---|---|---| +| `identical` | `PASS` | `PASS` | +| `content-identical` (same members and bytes, archive metadata differs) | `WARN` — the RM did not build with `repro-archive build`; note the metadata differences | `FAIL` — the policy requires bit-by-bit identity | +| `differs` (members added / removed / changed) | `FAIL` — the artefact is not the tagged tree | `FAIL` | +| tag commit ≠ recorded commit | `FAIL` — the tag moved | `FAIL` | +| content `swh:1:dir:` ≠ recorded (or ≠ ATR's) | `FAIL` — the staged tree is not the recorded one, whatever the bytes | `FAIL` | +| `check` reports another rule `FAIL` | `WARN`, listed | `FAIL` | + +**Convenience artefacts.** Read `convenience_artefacts` from +`release-build.md § Convenience artefacts` (project-specific; an empty +list means `SKIP`, stated explicitly). For a voter this is the check +that decides whether a convenience artefact is *good*: a binary cannot +be reviewed, so the only way to establish that it is what the voted +source produces is to rebuild it from the tag and compare. Per +artefact, using its own `reproducibility` mode (default +`reproducibility_binaries`): + +- `byte-identical` — rebuild with the entry's `build_command` under + the recorded `SOURCE_DATE_EPOCH`, compare with `cmp`; any difference + is `FAIL`. +- `documented-divergence` — rebuild, run the entry's + `verification_command` (for example `diffoscope`); differences that + match its `known_divergences` are `WARN` and listed, any other + difference is `FAIL`. +- `off` — `SKIP` for that artefact, stated explicitly with the note + that it is being published on trust. + +```bash +export SOURCE_DATE_EPOCH="" +git -C checkout "" +# one block per convenience artefact, its build_command verbatim: +( cd && ) +cmp "/" "//" \ + && echo "identical: " || echo "DIFFERS: " +# documented-divergence entries instead: + "/" "//" +``` + +Container images and other registry-staged kinds (`staging: +registry-staging`) are pulled by digest from the staging registry and +compared the same way against the local rebuild; say which digest was +pulled. + +Do not post-filter any output; the voter sees every difference. Never +report a verdict the commands did not produce. + +Return ONLY valid JSON with this structure: + +```json +{ + "step": "reproducibility", + "status": "PASS" | "WARN" | "FAIL" | "SKIP", + "mandatory": true | false, + "source": { + "enabled": true | false, + "verdict": "identical" | "content-identical" | "differs" | "tag-moved" | null, + "recorded_commit": "", + "source_date_epoch": , + "swhid_dir": "", + "swhid_matches": true | false | null, + "rule_failures": [""], + "metadata_differences": [""], + "content_differences": [""] + }, + "binaries": { + "mode": "off" | "byte-identical" | "documented-divergence", + "identical": [""], + "differs": [""], + "known_divergences_hit": [": "] + }, + "trusted_hardware_asserted": true | false, + "paste_recipe": "" +} +``` + +`status` is `"FAIL"` per the table above, `"WARN"` when only warnings +occurred, `"SKIP"` when nothing was enabled or `--skip-repro` applied, +else `"PASS"`. `mandatory` is `true` only under +`automated_release_signing: enabled`. `trusted_hardware_asserted` +mirrors `--trusted-hardware`; the skill never sets it on its own. +`binaries.mode` is the mode applied (when entries differ, the +strictest one in use); `binaries.differs` names every convenience +artefact that did not reproduce — `release-promote` reads this list +and withholds the publish command for each of them. `swhid_matches` +is `true` when the staged archive's `swh:1:dir:` equals the recorded +one (qualifiers ignored), `false` when it does not (a `FAIL`), `null` +when the planning issue recorded no SWHID — then the report states +the computed value so the RM can add it. + +--- + +## Step 10 — Hand-back verification report Aggregate the per-step results into a final report. @@ -768,9 +923,25 @@ Aggregate the per-step results into a final report. 5. **WARN detail** — for each warning step, the observation and the RM review requirement. 6. **Overall verdict** — `PASS`, `PASS-WITH-WARNINGS`, or `FAIL`. -7. **`--post-to` proposal** (only when `--post-to` was supplied) — +7. **Reproducibility record** — Step 9's verdict, the commit and + `SOURCE_DATE_EPOCH` it rebuilt with, and the sha512 of the rebuilt + source artefact, so another voter can cross-check without rerunning. +8. **`--post-to` proposal** (only when `--post-to` was supplied) — a formatted comment suitable for posting to the planning issue, - pending RM confirmation. + pending RM confirmation. 🪶 ASF-specific: under + `automated_release_signing: enabled`, when Step 9 is `PASS` with + every artefact `identical` **and** `--trusted-hardware` was passed, + the comment carries the attestation block `release-promote` Step 0 + looks for: + + > **Reproducibility validated on trusted hardware** — `` at + > commit ``, `SOURCE_DATE_EPOCH `; every staged artefact + > rebuilt on `@`'s own hardware and confirmed bit-by-bit + > identical (`repro-archive compare --require-identical`). Per + > [Infra § Automated release signing](https://infra.apache.org/release-signing.html#automated-release-signing). + + Without `--trusted-hardware`, or with any non-`identical` result, the + comment carries no attestation and says why. Return ONLY valid JSON with this structure: @@ -790,13 +961,16 @@ Return ONLY valid JSON with this structure: ], "fail_details": [""], "warn_details": [""], + "reproducibility_attestation": true | false, "post_to_comment": "" } ``` `voter_obligation_reminder` is always `true`; it confirms the reminder -was included. `post_to_comment` is non-null only when `--post-to` was -supplied and the RM has not yet confirmed posting. +was included. `reproducibility_attestation` is `true` only when the +attestation block above is included in `post_to_comment`. +`post_to_comment` is non-null only when `--post-to` was supplied and +the RM has not yet confirmed posting. --- @@ -817,6 +991,15 @@ supplied and the RM has not yet confirmed posting. - **Never invent check results.** All step outputs must reflect what is actually returned by the commands shown in the paste recipes, not assumed or predicted outcomes. +- **Never treat a `differs` rebuild as a warning.** A source artefact + whose members differ from the tagged tree is always `FAIL`; so is a + tag that no longer resolves to the recorded commit. +- **Never assert trusted hardware on the committer's behalf.** The + attestation block appears only with `--trusted-hardware`, passed by + the person running the skill; the skill cannot know where it runs. +- **Never downgrade a mandatory reproducibility check.** Under + `automated_release_signing: enabled` `--skip-repro` is ignored and + `content-identical` is `FAIL`. --- @@ -838,6 +1021,12 @@ supplied and the RM has not yet confirmed posting. | Step 7 FAIL — dangling symlink | A committed symlink's target was stripped by `export-ignore` (or is otherwise absent) | RM fixes `.gitattributes` to ship the target (or drops the symlink), cuts new RC | | Step 7 FAIL — broken internal reference | A shipped file links to a path stripped from the artefact | RM stops stripping the referenced path, or repoints the reference at shipped content, cuts new RC | | Step 8 FAIL — version mismatch | Version bump missed one manifest file | RM fixes the manifest and cuts a new RC | +| Step 9 WARN — `content-identical` | RM built with a plain `git archive` or a different tool version instead of `repro-archive build` | Accept for this RC in RM-key mode; RM switches to `repro-archive build` for the next one. Under automated signing this is `FAIL` | +| Step 9 FAIL — `differs` | Artefact built from a dirty checkout, a different ref, or a non-deterministic `custom` build | `-1`; RM rebuilds at the tag from a clean checkout (`release-rc-cut` Step 2b catches this before signing) | +| Step 9 FAIL — tag moved | `` no longer points at the commit recorded on the planning issue | `-1`; the RM explains and cuts a new RC number — never re-point an RC tag | +| Step 9 FAIL — convenience artefact `DIFFERS` | The artefact is not what the voted source produces: the build embeds timestamps, host paths or an unpinned toolchain, or was built from a different tree | The artefact is not good to publish. RM honours `SOURCE_DATE_EPOCH`, pins the toolchain (`ARFLAGS=Dcvr`, `ranlib -D`), or documents the divergence under the entry's `known_divergences`; `release-promote` withholds its publish command until a verify-rc run reproduces it | +| Step 7 SKIP — no validators declared | `release-build.md § Source-tree validators` is empty | Fine for a project with no in-tree link or symlink checks; declare the project's own validators if it has them | +| Step 9 SKIP but `automated_release_signing: enabled` | Misconfiguration — the check cannot be skipped in that mode | Re-run without `--skip-repro`; the report refuses to carry an attestation | --- @@ -851,7 +1040,15 @@ supplied and the RM has not yet confirmed posting. adopter keys this skill reads (`keys_file_url`, `keyserver`, `release_dist_url_template`, `version_manifest_files`). - [`/release-build.md`](../../../../projects/_template/release-build.md) — - expected artefact list, digest set, binary-exclude list, RAT config. + expected artefact list, digest set, binary-exclude list, RAT config, + `§ Source archive`, `§ Reproducibility checks`. +- [`docs/release-management/reproducibility.md`](../../../../docs/release-management/reproducibility.md) — + Step 9 background: the source-archive contract, the reproducibility + checks, and the 🪶 ASF-specific automated-signing validation. +- [`tools/reproducible-archive`](../../../../tools/reproducible-archive/README.md) — + `repro-archive check` / `build` / `compare`. +- [reproducible-builds.org § Archive metadata](https://reproducible-builds.org/docs/archives/) — + the rules `repro-archive check` verifies. - `release-keys-sync` (proposed) — remediation path when a signing key is not yet in the project `KEYS` file. - `release-vote-draft` (proposed) — downstream step; opens the diff --git a/plugins/magpie-release-management/skills/vote-draft/SKILL.md b/plugins/magpie-release-management/skills/vote-draft/SKILL.md index 84135d62d..00fbc01f0 100644 --- a/plugins/magpie-release-management/skills/vote-draft/SKILL.md +++ b/plugins/magpie-release-management/skills/vote-draft/SKILL.md @@ -359,6 +359,13 @@ Read the following from the planning issue body and | `atr_platform_url` | `release-management-config.md` | `atr_platform_url` (only when `vote_backend = atr`) | | `atr_revision` | *(optional)* | Specific ATR revision to vote on; omit to use the latest uploaded revision (`atr vote start --revision` defaults to latest — do not hard-depend on a `revisions` lookup) | | `canned_body` | `/canned-responses.md` | `[VOTE]` template block, if present | +| `repro_record` | planning issue body | the reproducibility record `release-rc-cut` posted: source commit, repository URL, the `swh:1:dir:` SWHID of the archive content (with its `origin` / `anchor` qualifiers), `SOURCE_DATE_EPOCH`, sha512 of the source artefact (see [`reproducibility.md`](../../../../docs/release-management/reproducibility.md)); if absent, say so and leave the lines out — never invent them; if only some fields are present, include those | +| `atr_candidate_url` | planning issue body | URL of the candidate's ATR page with its check results (only when `vote_backend = atr`) | +| `verification_doc_url` | `release-management-config.md` | `vote_verification_doc_url` — the human-readable "how to verify this RC" page, rendered with `-` so voters read the page at the tree under vote | +| `reproducibility_doc_url` | `release-management-config.md` | `reproducibility_doc_url` — background on the reproducible source archive; rendered the same way | +| `verification_skill` | `release-management-config.md` | `vote_verification_skill` (default `magpie-release-management:verify-rc`) — the agentic one-liner a voter can run | +| `signing_mode` | `release-management-config.md` | `ci-automated` when `automated_release_signing: enabled` under `organization: ASF`, else `rm-key` | +| `convenience_artefacts` | `release-build.md § Convenience artefacts` | the project's optional artefacts besides the source: name, `staging`, `vote_included`, `reproducibility`; empty for a source-only project | Surface the loaded metadata to the RM for confirmation before proceeding to Step 2. @@ -401,6 +408,46 @@ The changelog for this release: Keys to verify artifact signatures: +Convenience artefacts (built from the source above; not the release itself): ← include only when convenience_artefacts is non-empty + — staged at <"— included in this vote" when vote_included> + Each one is verified by rebuilding it from the tag and comparing + (); a convenience artefact that does not + reproduce from the voted source will be withheld from publication. + +How to verify this candidate before voting +------------------------------------------ +Reproducibility record (from the planning issue): ← include only the lines repro_record provides; omit the block when it has none + repository: + source commit: + SWHID (content): + SOURCE_DATE_EPOCH: + sha512: + +Agentic path (any agent with the Magpie release skills, read-only): + / -rcN + It checks the signature against KEYS, the checksum, licence headers + (RAT), LICENSE/NOTICE, prohibited binaries, dangling links, version + strings, rebuilds the source artefact from the tag to confirm it is + byte-identical to what is staged and that its SWHID is the recorded + one, and rebuilds and compares every convenience artefact. + +Manual path (the same checks, longhand): + + Reproducibility background: + +ATR check results for this candidate: ← include only when vote_backend = atr + + +[This candidate was signed by CI under the project's automated +release signing. Policy requires a committer's byte-identical rebuild +on their own hardware before promotion: run the verification with +--trusted-hardware --post-to and say so in your +vote.] ← include only when signing_mode = ci-automated + +A binding +1 means you downloaded the artefact, verified it, and built +and tested it on your own hardware; the tools above are an aid, not a +substitute (https://www.apache.org/legal/release-policy.html#release-approval). + Please vote to release: [ ] +1 Release [ ] +0 @@ -418,6 +465,19 @@ Thanks, ``` +The *How to verify* section is part of every `[VOTE]`, whichever +backend sends it and whether the body came from the default above or +from `canned_body`: a PMC member reading the thread on their phone +must find the agentic one-liner, the human-readable page, and the +reproducibility record without opening the tracker. When +`canned_body` lacks the section, append it and tell the RM the +project's canned block should gain it. The *Agentic path* paragraph +is fixed text describing what `verify-rc` does — keep it verbatim, +including the SWHID and convenience-artefact clauses, even when the +planning issue recorded no SWHID or the project declares no +convenience artefacts; only the *Reproducibility record* lines and the +*Convenience artefacts* block vary with what the report provides. + Present the draft subject + body to the RM. Ask for confirmation before proceeding to Step 3. Allow the RM to edit the body before confirming. @@ -434,7 +494,11 @@ confirming. credentials. The `` in the body must still point at the dist backend's download location (e.g. `dist/dev//…` under the hybrid) so voters fetch the canonical artefacts, even though ATR drives - the thread. Emit: + the thread. ATR's own default vote text links only the candidate + page, so the drafted body — with its *How to verify* section — is + what the RM supplies to ATR (the client's body option, or the vote + form on the candidate page; confirm with `atr vote start --help`). + Emit: ```text # ATR sends the [VOTE] to and tabulates replies. @@ -555,8 +619,14 @@ The AI-driven part ends with a hand-back artefact containing: - **Never draft a `[VOTE]` when verify-rc FAIL** without an explicit `--skip-verify-check ` override. - **Never invent metadata.** All staging URLs, tag URLs, keys URLs, - and changelog URLs must come from the planning issue body or the - project config. Do not derive or guess paths. + changelog URLs, and the reproducibility record (commit, + `SOURCE_DATE_EPOCH`, sha512) must come from the planning issue body + or the project config. Do not derive or guess paths or digests; omit + the record lines when the planning issue has none. +- **Never omit the *How to verify* section.** Every `[VOTE]` carries + the agentic one-liner, the human-readable verification page, and + the voter-obligation sentence, under either backend and with or + without a canned body. --- @@ -568,6 +638,8 @@ The AI-driven part ends with a hand-back artefact containing: | Pre-flight blocked — expedited window | `vote_window_hours` < 72 and no `--expedited` | Pass `--expedited ` or raise `vote_window_hours` | | Metadata field missing | Planning issue lacks staging URL, tag URL, etc. | Provide the missing URL in the planning issue body | | Subject template renders incorrectly | `vote_subject_template` has unsubstituted placeholders | Check `/release-management-config.md` | +| `vote_verification_doc_url` unset | Config predates the *How to verify* section | Add the key (`release-management-config.md § Vote`); until then the body links the framework's `docs/release-management/reproducibility.md` and says the project page is missing | +| Reproducibility record missing from the planning issue | `release-rc-cut` ran before the record was added, or the RM did not paste `repro-archive build`'s output back | Add the commit / `SOURCE_DATE_EPOCH` / sha512 to the planning issue; the body omits the three lines rather than guessing | --- @@ -578,9 +650,16 @@ The AI-driven part ends with a hand-back artefact containing: - [`docs/release-management/spec.md`](../../../../docs/release-management/spec.md) — `release-vote-draft` per-skill specification. - [`/release-management-config.md`](../../../../projects/_template/release-management-config.md) — - adopter keys this skill reads. + adopter keys this skill reads (`vote_*`, `vote_verification_doc_url`, + `reproducibility_doc_url`, `vote_verification_skill`). +- [`docs/release-management/reproducibility.md`](../../../../docs/release-management/reproducibility.md) — + what the reproducibility record in the body means and how a voter + uses it. +- [`docs/release-management/manual-release-process.md` § Manual verification](../../../../docs/release-management/manual-release-process.md#manual-verification--what-a-voter-runs-before-1) — + the longhand voter path the framework's own `[VOTE]` links to. - `release-verify-rc` (proposed) — - upstream step; PASS is a prerequisite. + upstream step; PASS is a prerequisite, and the agentic path the body + offers voters. - `release-vote-tally` (proposed) — downstream step; runs after the vote window closes. - [ASF release policy § release approval](https://www.apache.org/legal/release-policy.html#release-approval) — diff --git a/projects/_template/README.md b/projects/_template/README.md index 99ea21cf7..208636dc8 100644 --- a/projects/_template/README.md +++ b/projects/_template/README.md @@ -68,7 +68,7 @@ the rest. | [`release-trains.md`](release-trains.md) | Active release branches, release-manager attribution per cut, rotation rosters, security-team roster. | | [`milestones.md`](milestones.md) | Milestone naming conventions + create-and-assign recipe. | | [`release-management-config.md`](release-management-config.md) | Vote window and pass rule, distribution backend and paths, announce/vote list addresses, retention rule. Read by every `release-*` skill. | -| [`release-build.md`](release-build.md) | How this project builds and signs artefacts: build command, artefact names, checksum algorithm, signing-key expectations. | +| [`release-build.md`](release-build.md) | How this project builds and signs artefacts: reproducible source-archive recipe (`git archive` + `.gitattributes`), build command, the project's optional convenience artefacts (each with its own build, staging, reproducibility mode, vote scope and publish channel), artefact names, checksum algorithm, optional reproducibility checks. | | [`pmc-roster.md`](pmc-roster.md) | Who is binding. Read wherever a vote is counted or a PMC-only action is gated. | ### Scope + product mapping diff --git a/projects/_template/release-build.md b/projects/_template/release-build.md index 0e529b2b8..e86e75d4a 100644 --- a/projects/_template/release-build.md +++ b/projects/_template/release-build.md @@ -6,9 +6,13 @@ **Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)* - [TODO: ``: release-build configuration](#todo-project-name-release-build-configuration) + - [Source archive](#source-archive) - [Build invocation](#build-invocation) + - [Convenience artefacts](#convenience-artefacts) + - [Source-tree validators](#source-tree-validators) - [Expected artefact list](#expected-artefact-list) - [Digest set](#digest-set) + - [Reproducibility checks](#reproducibility-checks) - [Binary-exclude list](#binary-exclude-list) - [Apache RAT configuration](#apache-rat-configuration) @@ -19,32 +23,141 @@ # TODO: ``: release-build configuration -**This file is a placeholder ahead of the release-management -skill family landing.** None of the `release-*` skills exist -yet, see -[`docs/release-management/README.md`](../../docs/release-management/README.md). -The values below are what `release-rc-cut` and `release-verify-rc` -will read. +Per-project source-archive recipe, build invocation, expected +artefact set, digest selection, reproducibility checks, and +license-verification configuration. Read by `release-rc-cut` and +`release-verify-rc` (see +[`docs/release-management/README.md`](../../docs/release-management/README.md)). +Adopters copy this file into their own +`/release-build.md` and fill every TODO with their +project's equivalents. -Per-project build invocation, expected artefact set, digest -selection, and license-verification configuration. Adopters copy -this file into their own `/release-build.md` and -fill every TODO with their project's equivalents. +## Source archive + +The canonical source artefact is the one the `[VOTE]` votes on. +By default it is **not** produced by the project's build tool but +exported straight from the tagged git tree with `git archive`, +which packs only tracked files and honours the `export-ignore` +attributes in the repository's root `.gitattributes`. The +framework's [`reproducible-archive`](../../tools/reproducible-archive/README.md) +tool wraps that export and applies every rule from +[reproducible-builds.org § Archive metadata](https://reproducible-builds.org/docs/archives/) +(one `SOURCE_DATE_EPOCH` mtime, sorted members, uid/gid 0, +`a=rX,u+w` modes, no PAX `atime`/`ctime`, `gzip -n`, `zip -X`), so a +voter rebuilding from the tag gets byte-identical bytes. See +[`docs/release-management/reproducibility.md`](../../docs/release-management/reproducibility.md). + +| Key | Value | Notes | +|---|---|---| +| `source_archive_method` | `git-archive` | `git-archive` (default; `release-rc-cut` emits `repro-archive build`) or `custom` (the source artefact comes out of `build_command` below and `.gitattributes` is not consulted) | +| `source_archive_format` | `tar.gz` | `tar.gz` or `zip` | +| `source_archive_prefix` | `apache--` | top-level directory inside the archive | +| `export_ignore_reviewed` | *(unset)* | set to the `` at which the first-release `.gitattributes` review (`release-prepare prep` Step 2f) was completed; while unset, `release-rc-cut` blocks on an unreviewed archive | + +**What to `export-ignore`.** VCS, CI and editor metadata that a +source consumer never needs (`.github/workflows/`, +`.pre-commit-config.yaml`, linter configs, `.idea/`, agent-view relay +symlinks). **Never** exclude `LICENSE`, `NOTICE`, `DISCLAIMER` +(incubating), the build descriptors, the RAT excludes file, or any +path a shipped file links to. `release-prepare prep` runs a guided +review of every top-level path on the first release and proposes the +entries with a rationale comment each; `release-verify-rc` Step 7 +fails the RC if an exclusion strips a still-referenced path. ## Build invocation -TODO: name the canonical build command that produces the source -artefact (and any convenience binary artefacts the project -publishes). For Maven projects this is typically -`mvn -Papache-release clean install`; for Python projects a -combination of `python -m build` and `twine`; for Cargo projects -`cargo package --list`; etc. +TODO: name the build command that produces the **source artefact** +when `source_archive_method: custom` (a project whose source release is +assembled by its build tool rather than exported from the tag, e.g. +`mvn -Papache-release clean install` with the assembly plugin). Leave +it empty when the source archive comes from `git-archive` (the +default) — the convenience artefacts, if any, are declared per +artefact under § Convenience artefacts, not here. + +Whatever runs here runs at the release tag with the tag's +`SOURCE_DATE_EPOCH` exported (`repro-archive epoch --ref `), so +embedded timestamps are fixed. Example shape: > ```bash > # From the release branch tip, at the release tag: -> mvn -Papache-release clean install +> export SOURCE_DATE_EPOCH="$(git log -1 --format=%ct -)" +> mvn -Papache-release -Dproject.build.outputTimestamp="${SOURCE_DATE_EPOCH}" clean install +> ``` + +## Convenience artefacts + +Optional, and **project-specific by nature**: the framework knows how +to export, sign, verify and promote a source archive, but what a +project ships *besides* the source — a binary tarball, wheels, jars, +a container image, a Helm chart, an npm package — and where that goes +is the project's own decision. Declare each one here; every +`release-*` skill reads the list and emits the project's own +commands at the right lifecycle step. Leave the list empty for a +source-only project and the skills say so instead of guessing. + +Per [release-policy § what must every ASF release contain](https://www.apache.org/legal/release-policy.html#what-must-every-release-contain) +the **source package is the release**; convenience artefacts are +compiled from it for users who will not build from source, are +signed and checksummed under the same regime, and are published only +after the source vote passes. **A convenience artefact is "good" only +if it is demonstrably built from the voted source**: a voter cannot +review a binary, so the one check that establishes what it contains +is rebuilding it from the tag and comparing — bit-for-bit +(`byte-identical`) or with every difference explained +(`documented-divergence`). An artefact that cannot be reproduced +either way is not ready to publish, whatever the vote said about the +source. That is why each entry carries its own reproducibility mode +and why `release-promote` refuses to emit a publish command for an +artefact whose `release-verify-rc` rebuild did not reproduce. + +One entry per artefact: + +```yaml +convenience_artefacts: + - name: apache---bin.tar.gz # filename as staged; is rendered + kind: binary-tarball # binary-tarball | wheel | sdist | jar | container-image | helm-chart | npm-package | other + build_command: | # run at the release tag, SOURCE_DATE_EPOCH exported; must be deterministic + mvn -Papache-release -Dproject.build.outputTimestamp="${SOURCE_DATE_EPOCH}" -DskipTests clean package + staging: dist-dev # dist-dev (alongside the source, default) | atr | registry-staging + stage_command: null # registry-staging only, e.g. `twine upload -r testpypi …`, `mvn nexus-staging:deploy`, `docker push /:-rcN` + reproducibility: byte-identical # byte-identical | documented-divergence (default: reproducibility_binaries) + verification_command: null # documented-divergence only, e.g. `diffoscope ` + known_divergences: [] # documented-divergence only: path/pattern + reason, one per line + vote_included: false # true = the [VOTE] explicitly covers this artefact too (the source is always voted on) + publish_channel: dist-release # dist-release | pypi | maven-central | container-registry | helm-repo | npm | github-release | other + publish_command: null # run by the RM after the vote, e.g. `twine upload dist/*`, `mvn nexus-staging:release`, `docker push …`; dist-release needs none (promoted with the source) +``` + +How each skill uses the list: + +| Skill | Uses | +|---|---| +| `release-rc-cut` | Step 2 emits each `build_command` after the source archive, under the same `SOURCE_DATE_EPOCH`; Step 2b emits the per-artefact rebuild-and-compare self-check; Sections 3–4 sign and checksum every artefact; Step 3 emits `stage_command` for `registry-staging` entries | +| `release-verify-rc` | Step 9 rebuilds every artefact and compares per its `reproducibility` mode — the check that decides whether the artefact is good | +| `release-vote-draft` | Lists the artefacts, where each is staged, which are `vote_included`, and how to rebuild-and-compare them | +| `release-promote` | After the source promotion and the final tag, emits each `publish_command` — only for artefacts whose verify-rc rebuild reproduced | +| `release-announce-draft` | Names the channels the artefacts were published to | + +Keep `expected_artefacts` (below) in sync: it is the flat list of +filenames the RC stages (source + every convenience artefact whose +`staging` is `dist-dev` or `atr`). + +## Source-tree validators + +Optional. Commands `release-verify-rc` Step 7 runs from the unpacked +source archive to confirm that every internal reference resolves +(links, symlink targets, generated indexes) after `export-ignore` +stripped what it strips. Leave empty when the project ships no such +checks; Step 7 then runs only the dangling-symlink scan. + +Example shape: + +> ```yaml +> source_tree_validators: +> - "uv run --project tools/symlink-lint symlink-lint ." +> - "uv run --project tools/skill-and-tool-validator skill-and-tool-validate" > ``` ## Expected artefact list @@ -76,6 +189,36 @@ Example shape: > - `sha512`, required. > - `sha256`, published for downstream-tool compatibility. +## Reproducibility checks + +Optional checks that `release-rc-cut` (Step 2b, RM self-check) and +`release-verify-rc` (Step 9, any voter) run to confirm the staged +artefacts are a function of the tag alone. Per +[`PRINCIPLES.md` § 11](../../PRINCIPLES.md#11-releases-are-reproducible-from-signed-source), +byte-identical output is required where the toolchain permits it and +a documented verification path is required where it does not. + +| Key | Value | Allowed values | +|---|---|---| +| `reproducibility_source` | `on` | `on` (default when `source_archive_method: git-archive`) — rebuild from the tag with `repro-archive build`, `repro-archive compare` against the staged artefact; `off` | +| `reproducibility_binaries` | `off` | the default `reproducibility` mode for entries under § Convenience artefacts that do not set their own: `off` (no convenience artefacts, or checks disabled), `byte-identical` (rebuild with the entry's `build_command`, bytes must match), `documented-divergence` (run the entry's `verification_command`; differences outside its `known_divergences` fail) | + +A project with convenience artefacts should run them at +`byte-identical` wherever the toolchain allows, and at +`documented-divergence` — with the divergences written down — where +it does not; `off` means the artefacts are published on trust, which +[`PRINCIPLES.md` § 11](../../PRINCIPLES.md#11-releases-are-reproducible-from-signed-source) +does not accept as a steady state. + +An ASF adopter that wants +[automated release signing](https://infra.apache.org/release-signing.html#automated-release-signing) +must run with `reproducibility_source: on` and, for every binary it +signs, `reproducibility_binaries: byte-identical` — the policy requires +artefacts that "can be built reproducibly" and a validation step on +trusted hardware that confirms them "bit-by-bit identical" before +publication. In that mode the checks are mandatory and +`release-promote` refuses to promote without the recorded validation. + ## Binary-exclude list TODO: configure `release-verify-rc` Step 6's prohibited-binary check. diff --git a/projects/_template/release-management-config.md b/projects/_template/release-management-config.md index 93b1913df..4c4e4f55e 100644 --- a/projects/_template/release-management-config.md +++ b/projects/_template/release-management-config.md @@ -10,6 +10,7 @@ - [Backends](#backends) - [Distribution URLs](#distribution-urls) - [Signing](#signing) + - [Automated release signing (🪶 ASF-specific, optional)](#automated-release-signing--asf-specific-optional) - [Vote](#vote) - [Approval, non-list variants](#approval-non-list-variants) - [Announce](#announce) @@ -171,6 +172,34 @@ appears in `KEYS` (or draft a `KEYS` diff to add it via See [spec § Boundary 1](../../docs/release-management/spec.md#boundary-1-agent-never-holds-the-rms-signing-key). +### Automated release signing (🪶 ASF-specific, optional) + +Offered only when `project.md` declares `organization: ASF`; the +skills do not mention it to other adopters. Under +[release-signing § Automated release signing](https://infra.apache.org/release-signing.html#automated-release-signing) +an ASF project may let CI (GitHub Actions) sign the artefacts it +builds with an **Infra-provisioned key** (4096-bit RSA, signing-only, +private half never leaves infra-root) *provided that* every signed +artefact is built reproducibly, CI deploys only to a staging area, and +the release process re-validates every artefact **bit-by-bit +identical on trusted hardware** before publication. The Apache +Security Team must approve the workflow before use, and the key is +requested through an Infra Jira ticket. `release-prepare +automated-signing` walks the RM through that one-time setup (drafts +only; nothing is filed or sent by the agent). + +| Key | Value | Allowed values | +|---|---|---| +| `automated_release_signing` | `off` | `off` (default; the RM signs locally), `requested` (Infra ticket open, workflow not yet approved — skills keep the local-signing flow), `enabled` (CI signs; `release-rc-cut` emits the tag push that triggers the workflow instead of local sign/stage commands, `release-verify-rc` Step 9 becomes mandatory with `--require-identical`, `release-promote` blocks without the recorded trusted-hardware validation) | +| `ci_signing_key_fingerprint` | *(unset)* | fingerprint of the Infra-provisioned public key, once it is in `KEYS` | +| `ci_release_workflow` | `.github/workflows/release-candidate.yml` | the workflow that builds, uploads to ATR (OIDC trusted publishing) and stages; template at [`projects/_template/workflows/release-candidate.yml`](workflows/release-candidate.yml) | +| `ci_signing_infra_ticket` | *(unset)* | the `INFRA-` Jira ticket that provisioned the key, for the audit log | + +The agent boundary is unchanged: it holds neither the RM's key nor +the CI key, and the RM still signs the **tag** with their own key. The +`[VOTE]` is still cast on the source artefact rebuilt and compared on +a committer's own hardware. + ## Vote Applies when `release_approval_mechanism = dev-list-vote`. Other @@ -187,6 +216,21 @@ variants* below). | `vote_subject_template` | `[VOTE] Release from -rcN` | | `result_subject_template` | `[RESULT] [VOTE] Release from -rcN` | | `release_approver_roster_path` | `/pmc-roster.md` *(ASF default); non-ASF: e.g. `/release-approvers.md`)* | +| `vote_verification_doc_url` | `https://github.com//blob/-rcN/docs/verifying-a-release-candidate.md` — the human-readable "how to verify this RC" page; `-rcN` is rendered so voters read the page at the tree under vote | +| `reproducibility_doc_url` | `https://github.com//blob/-rcN/.apache-magpie/docs/release-management/reproducibility.md` — or the framework copy on `apache/magpie` | +| `vote_verification_skill` | `magpie-release-management:verify-rc` — the agentic one-liner the `[VOTE]` body offers voters | + +Every `[VOTE]` body `release-vote-draft` produces carries a *How to +verify this candidate* section: the reproducibility record +(commit, `SOURCE_DATE_EPOCH`, sha512), the agentic one-liner +(`/ -rcN`), the two pages above, +the ATR candidate page when ATR runs the vote, and the voter-obligation +sentence from +[`release-policy.html § release approval`](https://www.apache.org/legal/release-policy.html#release-approval). +Point `vote_verification_doc_url` at a page that walks a PMC member +through fetch, signature, checksum, unpack, rebuild-and-compare, RAT and +build-and-test — the framework's own is +[`docs/release-management/manual-release-process.md` § Manual verification](../../docs/release-management/manual-release-process.md#manual-verification--what-a-voter-runs-before-1). The configured `vote_window_hours` is a floor per [`release-policy.html § release approval`](https://www.apache.org/legal/release-policy.html#release-approval). diff --git a/projects/_template/workflows/release-candidate.yml b/projects/_template/workflows/release-candidate.yml new file mode 100644 index 000000000..24ba7c968 --- /dev/null +++ b/projects/_template/workflows/release-candidate.yml @@ -0,0 +1,184 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# +# TEMPLATE — ASF-specific (organization: ASF) automated release signing. +# +# Copy to `.github/workflows/release-candidate.yml` in the upstream repo +# ONLY after `release-prepare automated-signing` has walked you through +# the policy at +# https://infra.apache.org/release-signing.html#automated-release-signing +# and the Apache Security Team has approved the workflow. Until then keep +# `automated_release_signing: off` in release-management-config.md. +# +# What this workflow does: on an RC tag push it builds the source archive +# reproducibly (every rule from https://reproducible-builds.org/docs/archives/, +# via the embedded `repro-archive` script), builds any convenience binaries +# under the tag's SOURCE_DATE_EPOCH, proves the build is reproducible by +# building twice and comparing, generates sha512 checksums, and uploads +# everything to the Apache Trusted Release platform (ATR) with OIDC — +# a STAGING target. It publishes nothing. +# +# What it does NOT do: hold or use any signing key. The RM signs the tag +# with their own key before pushing it. Artefact signing with the +# infra-provisioned CI key happens through the mechanism Infra agreed on +# your INFRA ticket (for ATR trusted publishing, ATR applies the project's +# CI key on upload). Policy still requires a committer to rebuild every +# artefact on trusted hardware and confirm it bit-by-bit identical +# (`release-verify-rc` Step 9, `repro-archive compare --require-identical`) +# before `release-promote` will promote. +# +# Replace every . Pin every action to a commit SHA +# (apache/tooling-actions requires it; so does zizmor). + +name: Release candidate (reproducible build + ATR staging) + +on: + push: + tags: + - "*-rc[0-9]*" # -rcN, the tag release-rc-cut has the RM push + +permissions: {} + +env: + PROJECT: # ASF project slug, e.g. foo + ARTEFACT_PREFIX: apache- # apache---source. + SOURCE_FORMAT: tar.gz # release-build.md → source_archive_format + +jobs: + build: + name: Reproducible build + runs-on: ubuntu-latest + permissions: + contents: read + outputs: + version: ${{ steps.meta.outputs.version }} + source_date_epoch: ${{ steps.meta.outputs.source_date_epoch }} + steps: + - name: Checkout at the RC tag (full history so the tag's commit time resolves) + uses: actions/checkout@ # vX.Y.Z + with: + fetch-depth: 0 + persist-credentials: false + + - name: Set up Python (repro-archive is stdlib-only) + uses: actions/setup-python@ # vX.Y.Z + with: + python-version: "3.12" + + - name: Resolve version and SOURCE_DATE_EPOCH from the tag + id: meta + env: + TAG: ${{ github.ref_name }} + run: | + set -euo pipefail + version="${TAG%-rc*}" + # Embedded copy of tools/reproducible-archive/src/reproducible_archive/__init__.py + epoch="$(python3 release/reproducible_archive.py epoch --ref "$TAG")" + echo "version=$version" >> "$GITHUB_OUTPUT" + echo "source_date_epoch=$epoch" >> "$GITHUB_OUTPUT" + + - name: Build the source archive (git archive + .gitattributes export-ignore, reproducible) + env: + TAG: ${{ github.ref_name }} + VERSION: ${{ steps.meta.outputs.version }} + run: | + set -euo pipefail + mkdir -p dist + # Prints the record: commit, SOURCE_DATE_EPOCH, sha512, swhid_rev, + # swhid_dir (the SWHID of the expanded content, the value ATR + # computes at compose time) and the origin. Keep it in the job + # summary so a voter can compare without the planning issue. + python3 release/reproducible_archive.py build \ + --ref "$TAG" --format "$SOURCE_FORMAT" \ + --prefix "${ARTEFACT_PREFIX}-${VERSION}" \ + --origin "https://github.com/${GITHUB_REPOSITORY}" \ + -o "dist/${ARTEFACT_PREFIX}-${VERSION}-source.${SOURCE_FORMAT}" \ + | tee -a "$GITHUB_STEP_SUMMARY" + python3 release/reproducible_archive.py check \ + "dist/${ARTEFACT_PREFIX}-${VERSION}-source.${SOURCE_FORMAT}" \ + --epoch "${{ steps.meta.outputs.source_date_epoch }}" + + - name: Build convenience binaries under SOURCE_DATE_EPOCH (delete if source-only) + env: + SOURCE_DATE_EPOCH: ${{ steps.meta.outputs.source_date_epoch }} + run: | + set -euo pipefail + # release-build.md → build_command, verbatim. Must be reproducible: + # pinned toolchain, SOURCE_DATE_EPOCH honoured, ARFLAGS=Dcvr for .a. + + # copy the binaries listed in expected_artefacts into dist/ + + - name: Prove reproducibility — build again into a scratch dir and compare + env: + TAG: ${{ github.ref_name }} + VERSION: ${{ steps.meta.outputs.version }} + SOURCE_DATE_EPOCH: ${{ steps.meta.outputs.source_date_epoch }} + run: | + set -euo pipefail + mkdir -p rebuild + python3 release/reproducible_archive.py build \ + --ref "$TAG" --format "$SOURCE_FORMAT" \ + --prefix "${ARTEFACT_PREFIX}-${VERSION}" \ + -o "rebuild/${ARTEFACT_PREFIX}-${VERSION}-source.${SOURCE_FORMAT}" + python3 release/reproducible_archive.py compare --require-identical \ + "dist/${ARTEFACT_PREFIX}-${VERSION}-source.${SOURCE_FORMAT}" \ + "rebuild/${ARTEFACT_PREFIX}-${VERSION}-source.${SOURCE_FORMAT}" + # For binaries: re-run into rebuild/ and compare sha512. + # A CI self-comparison is a smoke test only; the policy's validation + # happens on a committer's trusted hardware in release-verify-rc. + + - name: Checksums (sha512 only; md5 and sha1 are prohibited) + working-directory: dist + run: | + set -euo pipefail + for f in *; do + case "$f" in *.sha512) continue ;; esac + sha512sum "$f" > "$f.sha512" + done + + - name: Keep the artefacts for the upload job + uses: actions/upload-artifact@ # vX.Y.Z + with: + name: rc-artefacts + path: dist/ + if-no-files-found: error + + stage: + name: Upload to ATR (staging only) + needs: build + runs-on: ubuntu-latest + permissions: + id-token: write # OIDC → ATR trusted publishing; no long-lived token + contents: read + steps: + - name: Fetch the artefacts + uses: actions/download-artifact@ # vX.Y.Z + with: + name: rc-artefacts + path: dist/ + + - name: Upload to the Apache Trusted Release platform + # https://github.com/apache/tooling-actions — pin by commit SHA. + uses: apache/tooling-actions/upload-to-atr@ + with: + project: ${{ env.PROJECT }} + version: ${{ needs.build.outputs.version }} + # files: dist/* # confirm the current input names with the action's README + + # No `atr release finish`, no `svn` to dist/release, no GitHub Release: + # publication is release-promote's job, after the [VOTE] passes and the + # trusted-hardware validation is recorded on the planning issue. diff --git a/projects/magpie/release-build.md b/projects/magpie/release-build.md index c15e4c4ea..11e41a67d 100644 --- a/projects/magpie/release-build.md +++ b/projects/magpie/release-build.md @@ -6,9 +6,13 @@ **Table of Contents** *generated with [DocToc](https://github.com/thlorenz/doctoc)* - [Apache Magpie: release build configuration](#apache-magpie-release-build-configuration) + - [Source archive](#source-archive) - [Build invocation](#build-invocation) + - [Convenience artefacts](#convenience-artefacts) + - [Source-tree validators](#source-tree-validators) - [Expected artefact list](#expected-artefact-list) - [Digest set](#digest-set) + - [Reproducibility checks](#reproducibility-checks) - [Binary-exclude list](#binary-exclude-list) @@ -28,30 +32,68 @@ Magpie is a source-first project (skills, docs, and Python tooling). Magpie ships **no convenience binaries** — the signed source artefact is the only release artefact. -## Build invocation - -The canonical source artefact is a deterministic `git archive` of the -tagged tree — no VCS metadata, no build output. **Never `zip -r` a -working directory**: that captures `__pycache__/*.pyc` and other test -cruft and produces a non-reproducible tarball (this was the rc1 `-1`). -`git archive` only ever includes tracked files from the tag, and honours -the `export-ignore` rules in [`.gitattributes`](../../.gitattributes): +## Source archive + +The canonical source artefact is a reproducible export of the tagged +tree — no VCS metadata, no build output. **Never `zip -r` a working +directory**: that captures `__pycache__/*.pyc` and other test cruft and +produces a non-reproducible artefact (this was the rc1 `-1`). The +export is `git archive` (tracked files at the tag only, honouring the +`export-ignore` rules in [`.gitattributes`](../../.gitattributes)) +wrapped by the framework's own +[`reproducible-archive`](../../tools/reproducible-archive/README.md) +tool, which applies every +[reproducible-builds.org archive rule](https://reproducible-builds.org/docs/archives/) +so a voter rebuilding from the tag gets byte-identical bytes regardless +of their `git` version. + +| Key | Value | +|---|---| +| `source_archive_method` | `git-archive` | +| `source_archive_format` | `zip` | +| `source_archive_prefix` | `apache-magpie-` | +| `export_ignore_reviewed` | `0.1.0` — reviewed on the `0.1.0-rc2` `[VOTE]` thread; the per-entry rationale lives in [`.gitattributes`](../../.gitattributes) and [`docs/source-release-contents.md`](../../docs/source-release-contents.md) | ```bash # From the release tag -rcN: -git archive --format=zip \ - --prefix="apache-magpie-/" \ - -o "apache-magpie--source.zip" \ - "-rcN" +uv run --project tools/reproducible-archive repro-archive build \ + --ref "-rcN" --format zip \ + --prefix "apache-magpie-" \ + -o "apache-magpie--source.zip" +# prints the commit, SOURCE_DATE_EPOCH and sha512 to record on the planning issue ``` -Files that must not ship in the source release (CI config, editor -metadata) are marked `export-ignore` in the root -[`.gitattributes`](../../.gitattributes), so `git archive` drops them. [Apache RAT](https://creadur.apache.org/rat/) (run by `release-verify-rc`) is the authoritative check on artefact contents; extend the `export-ignore` set if RAT flags anything on the first RC. +## Build invocation + +None. The source archive above is the whole build. + +## Convenience artefacts + +```yaml +convenience_artefacts: [] +``` + +Magpie ships no convenience artefacts; the skills state that instead of +emitting build, stage or publish commands for any. + +## Source-tree validators + +The checks the framework applies to its own tree, run by +`release-verify-rc` Step 7 against the unpacked archive so an +`export-ignore` that strips a still-referenced path fails the RC before +the vote (the `0.1.0-rc1` `-1` was exactly that): + +```yaml +source_tree_validators: + - "uv run --project tools/symlink-lint symlink-lint ." + - "uv run --project tools/skill-and-tool-validator skill-and-tool-validate" + - "uv run --project tools/spec-validator spec-validate" +``` + ## Expected artefact list - `apache-magpie--source.zip` — canonical source artefact @@ -66,6 +108,18 @@ extend the `export-ignore` set if RAT flags anything on the first RC. [release-distribution § sigs-and-sums](https://infra.apache.org/release-distribution.html#sigs-and-sums) and are never emitted. +## Reproducibility checks + +| Key | Value | +|---|---| +| `reproducibility_source` | `on` — `release-verify-rc` Step 9 rebuilds with `repro-archive build` at the tag and `repro-archive compare`s against the staged `.zip`; anything but `identical` is a `-1` | +| `reproducibility_binaries` | `off` — no convenience artefacts | + +Automated (CI) release signing is not enabled for Magpie +(`automated_release_signing: off` in +[`release-management-config.md`](release-management-config.md)); +the RM signs locally. + ## Binary-exclude list The source artefact must contain no compiled or opaque binary content. diff --git a/projects/magpie/release-management-config.md b/projects/magpie/release-management-config.md index 48c65fc1f..07fdfaa12 100644 --- a/projects/magpie/release-management-config.md +++ b/projects/magpie/release-management-config.md @@ -218,6 +218,9 @@ the last hand-run `svn commit` from the release flow. | `vote_subject_template` | `[VOTE] Release Apache Magpie from -rcN` | | `result_subject_template` | `[RESULT] [VOTE] Release Apache Magpie from -rcN` | | `release_approver_roster_path` | `projects/magpie/pmc-roster.md` | +| `vote_verification_doc_url` | `https://github.com/apache/magpie/blob/-rcN/docs/release-management/manual-release-process.md#manual-verification--what-a-voter-runs-before-1` | +| `reproducibility_doc_url` | `https://github.com/apache/magpie/blob/-rcN/docs/release-management/reproducibility.md` | +| `vote_verification_skill` | `magpie-release-management:verify-rc` | `vote_window_hours` is a floor per [release-policy § release approval](https://www.apache.org/legal/release-policy.html#release-approval). diff --git a/pyproject.toml b/pyproject.toml index 643143de2..8ec2c7e61 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -131,6 +131,7 @@ members = [ "tools/preflight-audit", "tools/privacy-llm/checker", "tools/privacy-llm/redactor", + "tools/reproducible-archive", "tools/sandbox-lint", "tools/security-tracker-stats-dashboard", "tools/skill-and-tool-validator", diff --git a/tools/reproducible-archive/README.md b/tools/reproducible-archive/README.md new file mode 100644 index 000000000..8c018364a --- /dev/null +++ b/tools/reproducible-archive/README.md @@ -0,0 +1,187 @@ + + + + + +- [`reproducible-archive`](#reproducible-archive) + - [The rules it implements](#the-rules-it-implements) + - [Prerequisites](#prerequisites) + - [How to use](#how-to-use) + - [`build` — a source archive that is a function of the tag alone](#build--a-source-archive-that-is-a-function-of-the-tag-alone) + - [`check` — lint an archive against the checklist](#check--lint-an-archive-against-the-checklist) + - [`compare` — did the voter get the same bytes?](#compare--did-the-voter-get-the-same-bytes) + - [`recipe` — the same steps with GNU tar / Info-ZIP](#recipe--the-same-steps-with-gnu-tar--info-zip) + - [`swhid` — the Software Heritage identifier of what shipped](#swhid--the-software-heritage-identifier-of-what-shipped) + - [`epoch` — the `SOURCE_DATE_EPOCH` of a ref](#epoch--the-source_date_epoch-of-a-ref) + - [Embedding the script](#embedding-the-script) + - [Wiring](#wiring) + - [Tests](#tests) + + + + + +# `reproducible-archive` + +**Capability:** substrate:release + +**Harness:** agnostic + +Builds, lints and compares **reproducible source archives**. +The archive is always derived from `git archive `, so only tracked files at the tag are packed and the repository's `.gitattributes` `export-ignore` rules decide what stays out. +On top of that stream the tool applies every rule from [reproducible-builds.org § Archive metadata](https://reproducible-builds.org/docs/archives/), so a Release Manager and a voter on different machines, with different `git`, `tar`, `zip` and `gzip` versions, get **byte-identical** output from the same tag. + +This is what `release-rc-cut` emits for the source artefact, what `release-verify-rc`'s reproducibility step rebuilds and compares against the staged RC, and what an ASF adopter needs before it can request [automated release signing](https://infra.apache.org/release-signing.html#automated-release-signing) (which requires artefacts that "can be built reproducibly" and are re-validated "bit-by-bit identical" on trusted hardware). +See [`docs/release-management/reproducibility.md`](../../docs/release-management/reproducibility.md) for how the pieces fit. + +## The rules it implements + +| # | reproducible-builds.org rule | Standard-tool equivalent | What the module does | +|---|---|---|---| +| 1 | File modification times | `tar --mtime="@${SOURCE_DATE_EPOCH}"`, `touch --date="@${SOURCE_DATE_EPOCH}"` | Every member gets one mtime: `SOURCE_DATE_EPOCH`, defaulting to the committer timestamp of the ref (`git log -1 --format=%ct`), the one value every builder of the same tag agrees on. | +| 2 | File ordering | `tar --sort=name`, or `find … \| LC_ALL=C sort -z` | Members are emitted in per-directory byte order, independent of the packer's locale and filesystem. | +| 3 | Ownership | `--owner=0 --group=0 --numeric-owner` | uid/gid `0`, empty user and group names. | +| 4 | Permissions / umask | `--mode=a=rX,u+w` | Files `0644`, executables and directories `0755`, setuid/setgid/sticky dropped. Symlinks are always packed as `0777`: `lstat` reports them as `0755` on macOS and `0777` on Linux, and the SWHID uses git's fixed `120000` for them, so the platform that packed the archive leaves no trace. | +| 5 | PAX headers | `--pax-option=exthdr.name=%d/PaxHeaders/%f,delete=atime,delete=ctime` | No `atime` / `ctime` headers; `check` also catches the PID-bearing `PaxHeaders.` names GNU tar emits under `POSIXLY_CORRECT`. | +| 6 | gzip | `gzip -n` | gzip header mtime `0`, no embedded filename, no extra field, no comment. | +| 7 | zip extra attributes | `zip -X`, unzip with `TZ=UTC` | No "extra field" per member, no comments, DOS timestamps computed in UTC from `SOURCE_DATE_EPOCH`, Unix create-system so the normalised modes round-trip. | + +`ar` deterministic mode (`ARFLAGS=Dcvr`, `ranlib -D`) and `cpio` are the same page's rules for *binary* artefacts; they belong in the adopter's build command, not here — `release-build.md § Convenience artefacts` records how the project applies them. + +Three more fixes the page does not list but that vary between machines and would otherwise leak into the bytes: + +| Fix | Why | +|---|---| +| `git -c core.autocrlf=false -c core.eol=lf archive` | `git archive` applies the same conversions as a checkout, so a builder with `core.autocrlf=true` exports different bytes for `text` files. The repository's committed `.gitattributes` (`export-ignore`, `export-subst`, `text`, `eol`) stay in force; only the builder's personal config is neutralised. | +| The archive writer is this module, not the local `tar` / `zip` / `git` | `git archive`'s zip and tar output changed between git versions (compression, PAX handling); two voters on different versions get different bytes from the same tag. Python's `tarfile` / `zipfile` output is a function of the inputs above. | +| The commit id travels with the archive | As `git archive` does it: a global PAX header `comment=` in tar, the archive comment in zip. Provenance a reader can extract without the planning issue; `check` accepts exactly that comment and flags any other. | + +And one identifier the page does not cover, which turns "same bytes" into "same tree": the **SWHID** (below). + +## Prerequisites + +- **Runtime:** Python 3.11+ (stdlib only, no third-party dependencies); run via `uv run --project tools/reproducible-archive` or as a plain `python3 `. +- **CLIs:** `git` on `PATH` for `build` and `epoch`; `check`, `compare` and `recipe` work on archive files alone. +- **Credentials / auth:** None. +- **Network:** None — runs fully offline on the local clone and local archive files. + +## How to use + +From the framework root (`` is `.apache-magpie/` in an adopting project, `.` in the framework checkout): + +```bash +uv run --project /tools/reproducible-archive repro-archive --help +``` + +or, with no `uv` at all: + +```bash +python3 /tools/reproducible-archive/src/reproducible_archive/__init__.py --help +``` + +### `build` — a source archive that is a function of the tag alone + +```bash +repro-archive build --ref 2.11.0-rc1 --format tar.gz \ + --prefix apache-foo-2.11.0 -o apache-foo-2.11.0-source.tar.gz \ + --origin https://github.com/apache/foo +# wrote apache-foo-2.11.0-source.tar.gz +# commit 1890a13d… +# SOURCE_DATE_EPOCH 1758326400 +# sha512 … +# swhid_rev swh:1:rev:1890a13d…;origin=https://github.com/apache/foo +# swhid_dir swh:1:dir:3b9f…;origin=https://github.com/apache/foo;anchor=swh:1:rev:1890a13d… +# swhid_dir_note differs from the repository tree swh:1:dir:7c1e… (export-ignore / export-subst / eol attributes applied) +# origin https://github.com/apache/foo +``` + +`--format zip` produces the ZIP equivalent. `--epoch N` overrides `SOURCE_DATE_EPOCH` (the environment variable is honoured too); the default is the committer timestamp of `--ref`. `--origin` is the repository URL recorded as the SWHID origin qualifier. The command refuses to run outside a git repository, refuses a ref it cannot resolve, and refuses a ZIP for an epoch before 1980 (the DOS timestamp cannot encode it). + +Record every printed line on the planning issue: a voter needs the commit and epoch to rebuild, the sha512 to compare bytes, and the SWHID to compare trees — with each other and with what ATR computed. + +### `check` — lint an archive against the checklist + +```bash +repro-archive check apache-foo-2.11.0-source.tar.gz --epoch 1758326400 +# PASS gzip-header mtime 0, no filename (gzip -n) +# PASS file-ordering members are in per-directory byte order +# PASS modification-times single mtime 1758326400 +# PASS ownership uid/gid 0, no user/group names +# PASS permissions every mode is a=rX,u+w +# PASS pax-headers no atime/ctime/PID headers +# SKIP zip-extra-fields not a zip +``` + +Exit code `1` on any `FAIL`. `--json` prints the same table as a list of `{name, status, detail}`. `--epoch` additionally asserts the single mtime *is* the expected `SOURCE_DATE_EPOCH`; without it the check only requires that every member share one mtime. It works on any `.tar`, `.tar.gz` or `.zip`, not only ones this tool built, so it doubles as a lint for an archive produced by a project's own build. + +### `compare` — did the voter get the same bytes? + +```bash +repro-archive compare staged/apache-foo-2.11.0-source.tar.gz rebuilt.tar.gz --require-identical +``` + +Three verdicts, in decreasing strength: + +| Verdict | Meaning | Exit code | +|---|---|---| +| `identical` | Byte-for-byte the same file. The bar ASF automated release signing sets for validation on trusted hardware. | `0` | +| `content-identical` | Every member's bytes, type, normalised mode and link target match; only archive metadata differs (timestamps, owners, ordering, extra fields, compression). What a voter gets from a plain `git archive` with a different `git` version. The metadata differences are listed. | `0`, or `1` with `--require-identical` | +| `differs` | Members were added, removed or changed; each is listed. The artefact does not match the tag. | `2` | + +`--json` prints the `Comparison` record. + +### `recipe` — the same steps with GNU tar / Info-ZIP + +```bash +repro-archive recipe --ref 2.11.0-rc1 --format tar.gz --prefix apache-foo-2.11.0 -o apache-foo-2.11.0-source.tar.gz +``` + +Prints the shell recipe from reproducible-builds.org (GNU tar ≥ 1.28 and `gzip -n`, or `LC_ALL=C sort` + `zip -X` under `TZ=UTC`) adapted to a `git archive` input, for a Release Manager who prefers to run the standard tools. It ends with the `check` invocation that verifies the result. The Python path and the shell path apply the same rules; only the Python path is guaranteed byte-identical across tool versions. + +### `swhid` — the Software Heritage identifier of what shipped + +```bash +repro-archive swhid apache-foo-2.11.0-source.tar.gz --ref 2.11.0-rc1 --origin https://github.com/apache/foo +# swhid_rev swh:1:rev:1890a13d…;origin=https://github.com/apache/foo +# swhid_repo_dir swh:1:dir:7c1e…;origin=https://github.com/apache/foo;anchor=swh:1:rev:1890a13d… +# swhid_dir swh:1:dir:3b9f…;origin=https://github.com/apache/foo;anchor=swh:1:rev:1890a13d… +``` + +A [SWHID](https://swhid.org/) (ISO/IEC 18670:2025) directory identifier is computed from names, modes and contents alone, exactly as git computes a tree id, so it is intrinsic to the files: timestamps, ownership, compression and the archive format play no part, a `.tar.gz` and a `.zip` of the same tree carry the same `swh:1:dir:`, and a voter recomputes it from the staged bytes without git. ATR computes the same value for a candidate at compose time, and it equals `git rev-parse ^{tree}` (`swhid_repo_dir`) unless `.gitattributes` altered the export (`export-ignore`, `export-subst`, `text` / `eol`) — in which case the difference is itself the record of what was left out. `build` prints all three plus a one-line note saying which case applies; the `origin` and `anchor` qualifiers follow the SWH specification. The identifier is computed in memory from the archive members and is tested against `git write-tree` over the extracted tree, so it agrees with `asfswhid` / `swh identify` by construction. + +`check --swhid swh:1:dir:…` asserts the archive content against a recorded value (qualifiers are ignored), and `compare` reports both archives' SWHIDs so "different bytes, same tree" is visible at a glance. Record `swhid_dir`, `swhid_rev` and the origin next to the commit on the planning issue: `release-vote-draft` puts them in the `[VOTE]`, `release-verify-rc` checks them, and a convenience artefact can name the source SWHID it was built from. + +### `epoch` — the `SOURCE_DATE_EPOCH` of a ref + +```bash +SOURCE_DATE_EPOCH="$(repro-archive epoch --ref 2.11.0-rc1)" +``` + +Use it to feed the same timestamp into a binary build (`SOURCE_DATE_EPOCH` is honoured by most build tools that embed dates) so the binary reproducibility check in `release-verify-rc` has a fixed reference. + +## Embedding the script + +`src/reproducible_archive/__init__.py` is a single stdlib-only file with a `__main__` guard. Copy it verbatim into a CI workflow, a release script, or a project that does not adopt the rest of Magpie: + +```bash +python3 reproducible_archive.py build --ref "$GITHUB_REF_NAME" --prefix "apache-foo-${VERSION}" \ + --format tar.gz -o "apache-foo-${VERSION}-source.tar.gz" +``` + +The ASF automated-release-signing workflow template in [`projects/_template/workflows/release-candidate.yml`](../../projects/_template/workflows/release-candidate.yml) does exactly that. + +## Wiring + +- `release-rc-cut` Step 2 emits `repro-archive build` as the source-artefact build command whenever `release-build.md § Source archive` sets `source_archive_method: git-archive` (the default), and Step 2b emits the optional `check` + rebuild-and-`compare` self-check. +- `release-verify-rc` Step 9 rebuilds from the tag with `build` and runs `compare` against the staged artefact; `--require-identical` when the adopter has `automated_release_signing: enabled`. +- The workspace pre-commit hooks (`ruff`, `mypy`, `pytest`) pick the project up from the root `pyproject.toml` `[tool.uv.workspace] members` list. + +## Tests + +```bash +uv run --project tools/reproducible-archive python -m pytest +``` + +Every rule has a positive case (the builder applies it) and a negative case (`check` catches an archive that violates it); `compare` is exercised on all three verdicts; and `build` is shown to be a function of the tag alone: two builds at different wall-clock times under a different umask are byte-identical. diff --git a/tools/reproducible-archive/pyproject.toml b/tools/reproducible-archive/pyproject.toml new file mode 100644 index 000000000..ff61b852c --- /dev/null +++ b/tools/reproducible-archive/pyproject.toml @@ -0,0 +1,89 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +[build-system] +requires = ["hatchling"] +build-backend = "hatchling.build" + +[project] +name = "reproducible-archive" +version = "0.1.0" +description = "Build, lint and compare reproducible source archives from a git ref, applying every rule from reproducible-builds.org/docs/archives/." +readme = "README.md" +requires-python = ">=3.11" +license = { text = "Apache-2.0" } +# stdlib-only on purpose — tarfile, zipfile, gzip and subprocess (git) are +# all it needs, so the module can be copied into a CI workflow verbatim. +dependencies = [] + +[project.scripts] +repro-archive = "reproducible_archive:main" + +[tool.hatch.build.targets.wheel] +packages = ["src/reproducible_archive"] + +[tool.ruff] +line-length = 160 +target-version = "py311" +src = ["src", "tests"] + +[tool.ruff.lint] +select = [ + "E", # pycodestyle errors + "W", # pycodestyle warnings + "F", # pyflakes + "I", # isort + "B", # flake8-bugbear + "UP", # pyupgrade + "SIM", # flake8-simplify + "C4", # flake8-comprehensions + "RUF", # ruff-specific +] +ignore = [ + "E501", # line-too-long — the limit above is already generous +] + +[tool.ruff.lint.per-file-ignores] +"tests/**" = ["B", "SIM"] + +[tool.mypy] +python_version = "3.11" +files = ["src", "tests"] +warn_unused_ignores = true +warn_redundant_casts = true +warn_unreachable = true +check_untyped_defs = true +no_implicit_optional = true +disallow_untyped_defs = true +disallow_incomplete_defs = true + +[[tool.mypy.overrides]] +module = "tests.*" +disallow_untyped_defs = false +disallow_incomplete_defs = false + +[tool.pytest.ini_options] +minversion = "8.0" +addopts = "-ra -q" +testpaths = ["tests"] + +[dependency-groups] +# The shared toolchain (mypy, pytest, ruff) comes from `magpie-dev` +# (tools/dev), declared once for the whole workspace. The checks run each tool +# via `uv run --directory --project . python -m ` — see +# tools/dev/run-workspace-check.sh. +dev = ["magpie-dev"] diff --git a/tools/reproducible-archive/src/reproducible_archive/__init__.py b/tools/reproducible-archive/src/reproducible_archive/__init__.py new file mode 100644 index 000000000..c194ba786 --- /dev/null +++ b/tools/reproducible-archive/src/reproducible_archive/__init__.py @@ -0,0 +1,841 @@ +#!/usr/bin/env python3 +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""Build, lint and compare **reproducible source archives**. + +The archive-level rules come from +. Every rule that page +lists is implemented here, once, so a Release Manager and a voter get +byte-identical output from the same tag regardless of their `git`, `tar`, +`zip` or `gzip` version: + +1. **File modification times** — every member carries the same mtime, + `SOURCE_DATE_EPOCH` (default: the committer timestamp of the ref). +2. **File ordering** — members are emitted in a locale-independent, + per-directory sorted order (what GNU tar's `--sort=name` produces). +3. **Ownership** — uid/gid `0`, empty user/group names (`--owner=0 + --group=0 --numeric-owner`). +4. **Permissions** — modes normalised to `a=rX,u+w` (`0644` files, `0755` + executables and directories), so the packer's umask never leaks in. +5. **PAX headers** — no `atime` / `ctime` / PID-bearing headers + (`--pax-option=...,delete=atime,delete=ctime`). +6. **gzip** — header mtime `0` and no embedded filename (`gzip -n`). +7. **zip** — no "extra" field attributes (`zip -X`), UTC DOS timestamps + from `SOURCE_DATE_EPOCH`, Unix create-system so modes round-trip. + +The input is always `git archive --format=tar `, so only tracked +files at the ref are ever packed and the repository's `.gitattributes` +`export-ignore` rules decide what is left out. Two more inputs that +vary between machines are pinned as well: the builder's +`core.autocrlf` / `core.eol` (which `git archive` would otherwise apply +to `text` files) and the archive tool itself (this module, not the +local `tar` / `zip` / `git` version). The commit id is carried as +provenance the way `git archive` carries it — a global PAX header +`comment` in tar, the archive comment in zip. + +Every archive also gets a **Software Heritage identifier** (SWHID, +ISO/IEC 18670): `swh:1:dir:` of the expanded content, computed +as git computes a tree id, so it is intrinsic to the files — a voter +recomputes it from the staged bytes, ATR computes the same value at +compose time, and it equals `git rev-parse ^{tree}` unless +`.gitattributes` altered the export — plus `swh:1:rev:` and +the origin URL as qualifiers. + +The module is stdlib-only on purpose: it can be run as `python3 ` from any checkout, embedded in a CI workflow, or invoked via +`uv run`. Sub-commands: `build`, `check`, `compare`, `swhid`, `recipe`, +`epoch`. See `tools/reproducible-archive/README.md` for the contract +each one keeps. +""" + +from __future__ import annotations + +import argparse +import gzip +import hashlib +import io +import json +import os +import re +import stat +import subprocess +import sys +import tarfile +import zipfile +from collections.abc import Iterable, Sequence +from dataclasses import asdict, dataclass, field +from datetime import UTC, datetime +from pathlib import Path + +__version__ = "0.1.0" + +FORMATS = ("tar.gz", "zip") +#: Earliest timestamp a ZIP DOS date/time field can encode (1980-01-01T00:00:00Z). +ZIP_EPOCH_MIN = 315532800 +_FILE_MODE = 0o644 +_EXEC_MODE = 0o755 +_DIR_MODE = 0o755 +_GZIP_FLAG_FEXTRA = 0x04 +_GZIP_FLAG_FNAME = 0x08 +_GZIP_FLAG_FCOMMENT = 0x10 +_PAX_PID_NAME = re.compile(r"PaxHeaders\.\d+") +_PAX_TIME_KEYS = ("atime", "ctime", "LIBARCHIVE.creationtime", "SCHILY.dev", "SCHILY.ino", "SCHILY.nlink") + + +# --------------------------------------------------------------------------- model + + +@dataclass(frozen=True) +class Entry: + """One archive member in a format-neutral shape.""" + + name: str + kind: str # "file" | "dir" | "symlink" + mode: int + data: bytes = b"" + linkname: str = "" + + @property + def sort_key(self) -> tuple[bytes, ...]: + """Per-directory byte-wise order (GNU tar `--sort=name`): a + directory sorts before every path beneath it, siblings sort by + their raw bytes, never by the current locale.""" + return tuple(part.encode("utf-8", "surrogateescape") for part in self.name.rstrip("/").split("/")) + + +@dataclass +class CheckResult: + name: str + status: str # "PASS" | "FAIL" | "SKIP" + detail: str = "" + + +@dataclass +class Comparison: + verdict: str # "identical" | "content-identical" | "differs" + sha512_a: str + sha512_b: str + swhid_a: str = "" + swhid_b: str = "" + added: list[str] = field(default_factory=list) + removed: list[str] = field(default_factory=list) + changed: list[str] = field(default_factory=list) + metadata_differences: list[str] = field(default_factory=list) + + +class ReproError(RuntimeError): + """A failure the CLI reports on stderr with exit code 2.""" + + +# --------------------------------------------------------------------------- git helpers + + +def _git(args: Sequence[str], repo: Path) -> bytes: + try: + return subprocess.run(["git", *args], cwd=repo, check=True, capture_output=True).stdout + except FileNotFoundError as exc: + raise ReproError("git is not on PATH") from exc + except subprocess.CalledProcessError as exc: + raise ReproError(f"git {' '.join(args)} failed: {exc.stderr.decode(errors='replace').strip()}") from exc + + +def source_date_epoch(ref: str, repo: Path, override: int | None = None) -> int: + """Resolve `SOURCE_DATE_EPOCH` for a ref: an explicit override wins, + then the environment variable, then the committer timestamp of the + ref (which is what makes two builders of the same tag agree).""" + if override is not None: + return int(override) + env = os.environ.get("SOURCE_DATE_EPOCH") + if env: + try: + return int(env) + except ValueError as exc: + raise ReproError(f"SOURCE_DATE_EPOCH is not an integer: {env!r}") from exc + out = _git(["log", "-1", "--format=%ct", ref], repo).decode().strip() + if not out.isdigit(): + raise ReproError(f"cannot resolve a committer timestamp for {ref!r}") + return int(out) + + +def commit_of(ref: str, repo: Path) -> str: + return _git(["rev-parse", f"{ref}^{{commit}}"], repo).decode().strip() + + +def git_archive_tar(ref: str, repo: Path, prefix: str, worktree_attributes: bool = False) -> bytes: + """`git archive --format=tar` at the ref. Only tracked files are + included and `.gitattributes` `export-ignore` rules are honoured — the + two properties that make the archive a function of the tag alone. + + `worktree_attributes` passes `--worktree-attributes`, so a not-yet- + committed `.gitattributes` edit is applied: what the first-release + review uses to preview an exclusion set before committing it. A + release build never sets it.""" + # `git archive` applies the same conversions as a checkout: committed + # `.gitattributes` (`export-ignore`, `export-subst`, `text` / `eol`) + # are the project's intent and stay in force, but the *builder's* own + # `core.autocrlf` / `core.eol` must not leak into the export, so both + # are pinned to the values a fresh clone on Linux would use. + args = ["-c", "core.autocrlf=false", "-c", "core.eol=lf", "archive", "--format=tar"] + if worktree_attributes: + args.append("--worktree-attributes") + if prefix: + args.append(f"--prefix={prefix.rstrip('/')}/") + args.append(ref) + return _git(args, repo) + + +# --------------------------------------------------------------------------- normalisation + + +def normalize_mode(mode: int, kind: str) -> int: + """`a=rX,u+w`: directories and anything executable become 0755, + everything else 0644. Setuid/setgid/sticky bits are dropped. A + symlink is always 0777: `lstat` reports 0755 on macOS and 0777 on + Linux, and neither is meaningful, so the packer's platform must + not leak into the bytes.""" + if kind == "dir": + return _DIR_MODE + if kind == "symlink": + return 0o777 + return _EXEC_MODE if mode & 0o111 else _FILE_MODE + + +def sort_entries(entries: Iterable[Entry]) -> list[Entry]: + return sorted(entries, key=lambda e: e.sort_key) + + +def read_tar_entries(data: bytes) -> list[Entry]: + entries: list[Entry] = [] + with tarfile.open(fileobj=io.BytesIO(data), mode="r:*") as tf: + for member in tf: + if member.isdir(): + entries.append(Entry(member.name.rstrip("/") + "/", "dir", member.mode)) + elif member.issym(): + entries.append(Entry(member.name, "symlink", member.mode, linkname=member.linkname)) + elif member.isfile(): + fh = tf.extractfile(member) + payload = fh.read() if fh is not None else b"" + entries.append(Entry(member.name, "file", member.mode, data=payload)) + else: + raise ReproError(f"unsupported member type {member.type!r} for {member.name}") + return entries + + +def read_zip_entries(data: bytes) -> list[Entry]: + entries: list[Entry] = [] + with zipfile.ZipFile(io.BytesIO(data)) as zf: + for info in zf.infolist(): + mode = (info.external_attr >> 16) & 0o7777 + unix_type = (info.external_attr >> 16) & 0o170000 + if info.is_dir(): + entries.append(Entry(info.filename, "dir", mode)) + elif unix_type == stat.S_IFLNK: + entries.append(Entry(info.filename, "symlink", mode, linkname=zf.read(info).decode())) + else: + entries.append(Entry(info.filename, "file", mode, data=zf.read(info))) + return entries + + +def read_entries(path: Path) -> list[Entry]: + data = path.read_bytes() + if zipfile.is_zipfile(io.BytesIO(data)): + return read_zip_entries(data) + return read_tar_entries(data) + + +# --------------------------------------------------------------------------- writers + + +def write_tar_gz(entries: Iterable[Entry], epoch: int, level: int = 6, commit: str | None = None) -> bytes: + """Deterministic `.tar.gz`: sorted members, one mtime, uid/gid 0, no + names, normalised modes, no atime/ctime PAX headers, and a gzip + wrapper with mtime 0 and no filename (`gzip -n`). With `commit`, a + global PAX header carries `comment=` — the provenance note + `git archive` itself writes, readable with `tar --pax-option` aware + tools and harmless to the rest.""" + tar_buf = io.BytesIO() + global_headers = {"comment": commit} if commit else {} + with tarfile.open(fileobj=tar_buf, mode="w", format=tarfile.PAX_FORMAT, pax_headers=global_headers) as tf: + for entry in sort_entries(entries): + info = tarfile.TarInfo(entry.name.rstrip("/") if entry.kind == "dir" else entry.name) + info.mtime = epoch + info.uid = info.gid = 0 + info.uname = info.gname = "" + info.mode = normalize_mode(entry.mode, entry.kind) + if entry.kind == "dir": + info.type = tarfile.DIRTYPE + tf.addfile(info) + elif entry.kind == "symlink": + info.type = tarfile.SYMTYPE + info.linkname = entry.linkname + tf.addfile(info) + else: + info.type = tarfile.REGTYPE + info.size = len(entry.data) + tf.addfile(info, io.BytesIO(entry.data)) + out = io.BytesIO() + with gzip.GzipFile(filename="", mode="wb", compresslevel=level, fileobj=out, mtime=0) as gz: + gz.write(tar_buf.getvalue()) + return out.getvalue() + + +def _dos_time(epoch: int) -> tuple[int, int, int, int, int, int]: + dt = datetime.fromtimestamp(max(epoch, ZIP_EPOCH_MIN), tz=UTC) + return (dt.year, dt.month, dt.day, dt.hour, dt.minute, dt.second - dt.second % 2) + + +def write_zip(entries: Iterable[Entry], epoch: int, level: int = 6, commit: str | None = None) -> bytes: + """Deterministic `.zip`: sorted members, one UTC DOS timestamp, no + extra-field attributes (`zip -X`), Unix create-system so the + normalised modes survive the round trip, no member comments. With + `commit`, the archive comment is the commit id — what `git archive + --format=zip` writes there.""" + out = io.BytesIO() + with zipfile.ZipFile(out, "w", compression=zipfile.ZIP_DEFLATED, compresslevel=level) as zf: + if commit: + zf.comment = commit.encode() + for entry in sort_entries(entries): + info = zipfile.ZipInfo(entry.name, date_time=_dos_time(epoch)) + info.create_system = 3 + info.extra = b"" + info.comment = b"" + mode = normalize_mode(entry.mode, entry.kind) + if entry.kind == "dir": + info.external_attr = ((stat.S_IFDIR | mode) << 16) | 0x10 + info.compress_type = zipfile.ZIP_STORED + zf.writestr(info, b"") + elif entry.kind == "symlink": + info.external_attr = (stat.S_IFLNK | mode) << 16 + info.compress_type = zipfile.ZIP_STORED + zf.writestr(info, entry.linkname.encode()) + else: + info.external_attr = (stat.S_IFREG | mode) << 16 + info.compress_type = zipfile.ZIP_DEFLATED + zf.writestr(info, entry.data) + return out.getvalue() + + +def build( + ref: str, + repo: Path, + fmt: str, + prefix: str, + epoch: int | None = None, + worktree_attributes: bool = False, +) -> tuple[bytes, int]: + """Produce the archive bytes for `ref` and the epoch they were built with.""" + if fmt not in FORMATS: + raise ReproError(f"unsupported format {fmt!r}; expected one of {', '.join(FORMATS)}") + resolved_epoch = source_date_epoch(ref, repo, epoch) + commit = commit_of(ref, repo) + entries = read_tar_entries(git_archive_tar(ref, repo, prefix, worktree_attributes)) + if not entries: + raise ReproError(f"git archive {ref} produced no entries") + if fmt == "zip": + if resolved_epoch < ZIP_EPOCH_MIN: + raise ReproError(f"SOURCE_DATE_EPOCH {resolved_epoch} predates 1980; ZIP cannot encode it") + return write_zip(entries, resolved_epoch, commit=commit), resolved_epoch + return write_tar_gz(entries, resolved_epoch, commit=commit), resolved_epoch + + +# --------------------------------------------------------------------------- check + + +def _sha512(data: bytes) -> str: + return hashlib.sha512(data).hexdigest() + + +def _check_order(names: Sequence[str]) -> CheckResult: + keyed = [Entry(n, "file", 0).sort_key for n in names] + if keyed == sorted(keyed): + return CheckResult("file-ordering", "PASS", "members are in per-directory byte order") + for i in range(1, len(keyed)): + if keyed[i] < keyed[i - 1]: + return CheckResult("file-ordering", "FAIL", f"{names[i]!r} sorts before {names[i - 1]!r}") + return CheckResult("file-ordering", "FAIL", "members are not sorted") + + +def _check_modes(modes: Sequence[tuple[str, str, int]]) -> CheckResult: + bad = [f"{name} ({kind}) mode {mode:04o}" for name, kind, mode in modes if kind != "symlink" and normalize_mode(mode, kind) != mode] + if bad: + return CheckResult("permissions", "FAIL", "; ".join(bad[:5]) + (" …" if len(bad) > 5 else "")) + return CheckResult("permissions", "PASS", "every mode is a=rX,u+w") + + +def _check_single_mtime(mtimes: Sequence[int], epoch: int | None) -> CheckResult: + distinct = sorted(set(mtimes)) + if len(distinct) > 1: + return CheckResult("modification-times", "FAIL", f"{len(distinct)} distinct mtimes (e.g. {distinct[0]} and {distinct[-1]})") + if epoch is not None and distinct and distinct[0] != epoch: + return CheckResult("modification-times", "FAIL", f"mtime {distinct[0]} != SOURCE_DATE_EPOCH {epoch}") + return CheckResult("modification-times", "PASS", f"single mtime {distinct[0] if distinct else 'n/a'}") + + +def _raw_tar_pax_names(data: bytes) -> list[str]: + """Names of PAX/GNU extended-header blocks, read from the raw stream + (tarfile hides them), so a `PaxHeaders.` name can be caught.""" + names: list[str] = [] + off = 0 + while off + 512 <= len(data): + block = data[off : off + 512] + if block == b"\0" * 512: + break + name = block[0:100].split(b"\0", 1)[0].decode("utf-8", "surrogateescape") + typeflag = block[156:157] + size_field = block[124:136].split(b"\0", 1)[0].strip() + size = int(size_field, 8) if size_field else 0 + if typeflag in (b"x", b"g", b"L", b"K"): + names.append(name) + off += 512 + ((size + 511) // 512) * 512 + return names + + +def check_tar(data: bytes, epoch: int | None = None) -> list[CheckResult]: + results: list[CheckResult] = [] + is_gz = data[:2] == b"\x1f\x8b" + if is_gz: + flags, mtime = data[3], int.from_bytes(data[4:8], "little") + problems = [] + if mtime != 0: + problems.append(f"header mtime {mtime}") + if flags & _GZIP_FLAG_FNAME: + problems.append("embedded filename") + if flags & _GZIP_FLAG_FEXTRA: + problems.append("extra field") + if flags & _GZIP_FLAG_FCOMMENT: + problems.append("comment") + results.append( + CheckResult("gzip-header", "FAIL", ", ".join(problems)) if problems else CheckResult("gzip-header", "PASS", "mtime 0, no filename (gzip -n)") + ) + raw = gzip.decompress(data) + else: + results.append(CheckResult("gzip-header", "SKIP", "not gzip-compressed")) + raw = data + names: list[str] = [] + modes: list[tuple[str, str, int]] = [] + mtimes: list[int] = [] + owners: list[str] = [] + pax_hits: list[str] = [] + with tarfile.open(fileobj=io.BytesIO(raw), mode="r:") as tf: + for m in tf: + kind = "dir" if m.isdir() else "symlink" if m.issym() else "file" + names.append(m.name) + modes.append((m.name, kind, m.mode)) + mtimes.append(int(m.mtime)) + if m.uid or m.gid or m.uname or m.gname: + owners.append(f"{m.name} uid={m.uid} gid={m.gid} uname={m.uname!r} gname={m.gname!r}") + for key in _PAX_TIME_KEYS: + if key in m.pax_headers: + pax_hits.append(f"{m.name}: {key}") + results.append(_check_order(names)) + results.append(_check_single_mtime(mtimes, epoch)) + results.append(CheckResult("ownership", "FAIL", "; ".join(owners[:5])) if owners else CheckResult("ownership", "PASS", "uid/gid 0, no user/group names")) + results.append(_check_modes(modes)) + pid_names = [n for n in _raw_tar_pax_names(raw) if _PAX_PID_NAME.search(n)] + pax_hits.extend(f"PID-bearing header name {n}" for n in pid_names) + results.append( + CheckResult("pax-headers", "FAIL", "; ".join(pax_hits[:5])) if pax_hits else CheckResult("pax-headers", "PASS", "no atime/ctime/PID headers") + ) + results.append(CheckResult("zip-extra-fields", "SKIP", "not a zip")) + return results + + +def check_zip(data: bytes, epoch: int | None = None) -> list[CheckResult]: + results: list[CheckResult] = [CheckResult("gzip-header", "SKIP", "not a tar.gz")] + names: list[str] = [] + modes: list[tuple[str, str, int]] = [] + stamps: list[tuple[int, int, int, int, int, int]] = [] + extras: list[str] = [] + owners: list[str] = [] + with zipfile.ZipFile(io.BytesIO(data)) as zf: + if zf.comment and not re.fullmatch(rb"[0-9a-f]{40}", zf.comment): + extras.append("archive comment present (only a commit id, as git archive writes, is expected)") + for info in zf.infolist(): + names.append(info.filename) + unix_type = (info.external_attr >> 16) & 0o170000 + kind = "dir" if info.is_dir() else "symlink" if unix_type == stat.S_IFLNK else "file" + modes.append((info.filename, kind, (info.external_attr >> 16) & 0o7777)) + stamps.append(info.date_time) + if info.extra: + extras.append(f"{info.filename}: {len(info.extra)}-byte extra field") + if info.comment: + extras.append(f"{info.filename}: member comment") + if info.create_system != 3: + owners.append(f"{info.filename}: create_system {info.create_system} (expected 3 = Unix)") + results.append(_check_order(names)) + distinct = sorted(set(stamps)) + if len(distinct) > 1: + results.append(CheckResult("modification-times", "FAIL", f"{len(distinct)} distinct timestamps")) + elif epoch is not None and distinct and distinct[0] != _dos_time(epoch): + results.append(CheckResult("modification-times", "FAIL", f"timestamp {distinct[0]} != SOURCE_DATE_EPOCH {epoch} (UTC)")) + else: + results.append(CheckResult("modification-times", "PASS", f"single timestamp {distinct[0] if distinct else 'n/a'}")) + results.append( + CheckResult("ownership", "FAIL", "; ".join(owners[:5])) if owners else CheckResult("ownership", "PASS", "Unix create-system on every member") + ) + results.append(_check_modes(modes)) + results.append(CheckResult("pax-headers", "SKIP", "not a tar")) + results.append( + CheckResult("zip-extra-fields", "FAIL", "; ".join(extras[:5])) + if extras + else CheckResult("zip-extra-fields", "PASS", "no extra fields or comments (zip -X)") + ) + return results + + +def check(path: Path, epoch: int | None = None, swhid: str | None = None) -> list[CheckResult]: + """Lint `path` against the checklist; with `swhid`, also require the + archive content's `swh:1:dir:` to equal it (the value recorded on the + planning issue, or the one ATR shows for the candidate).""" + data = path.read_bytes() + results = check_zip(data, epoch) if zipfile.is_zipfile(io.BytesIO(data)) else check_tar(data, epoch) + if swhid: + actual = swhid_of_archive(path) + results.append( + CheckResult("swhid", "PASS", f"content is {actual}") + if actual == swhid.split(";", 1)[0] + else CheckResult("swhid", "FAIL", f"content is {actual}, expected {swhid.split(';', 1)[0]}") + ) + return results + + +# --------------------------------------------------------------------------- compare + + +def _member_digest(entry: Entry) -> tuple[str, str, int, str]: + return (entry.kind, hashlib.sha256(entry.data).hexdigest(), normalize_mode(entry.mode, entry.kind), entry.linkname) + + +def compare(path_a: Path, path_b: Path) -> Comparison: + """Three verdicts, in decreasing strength: + + * `identical` — byte-for-byte the same file (the bar ASF automated + release signing sets for validation on trusted hardware); + * `content-identical` — every member's bytes, type, normalised mode + and link target match, only archive metadata (timestamps, owners, + ordering, extra fields, compression) differs — what a voter with a + different `git`/`tar` version gets from a plain `git archive`; + * `differs` — members were added, removed or changed. + """ + data_a, data_b = path_a.read_bytes(), path_b.read_bytes() + sha_a, sha_b = _sha512(data_a), _sha512(data_b) + swh_a, swh_b = swhid_of_archive(path_a), swhid_of_archive(path_b) + if data_a == data_b: + return Comparison("identical", sha_a, sha_b, swh_a, swh_b) + ents_a = {e.name.rstrip("/"): e for e in read_entries(path_a)} + ents_b = {e.name.rstrip("/"): e for e in read_entries(path_b)} + added = sorted(set(ents_b) - set(ents_a)) + removed = sorted(set(ents_a) - set(ents_b)) + changed = sorted(n for n in set(ents_a) & set(ents_b) if _member_digest(ents_a[n]) != _member_digest(ents_b[n])) + if added or removed or changed: + return Comparison("differs", sha_a, sha_b, swh_a, swh_b, added, removed, changed) + meta: list[str] = [] + if [e.name for e in read_entries(path_a)] != [e.name for e in read_entries(path_b)]: + meta.append("member order differs") + fails_a = {r.name for r in check(path_a) if r.status == "FAIL"} + fails_b = {r.name for r in check(path_b) if r.status == "FAIL"} + for name in sorted(fails_a | fails_b): + meta.append(f"reproducibility check {name} fails on {'both' if name in fails_a and name in fails_b else 'A' if name in fails_a else 'B'}") + if not meta: + meta.append("compression or container bytes differ (same members, same modes)") + return Comparison("content-identical", sha_a, sha_b, swh_a, swh_b, metadata_differences=meta) + + +# --------------------------------------------------------------------------- SWHID + + +def _git_object_sha1(kind: str, payload: bytes) -> bytes: + # git and SWH object ids are SHA-1 by definition; this is an identifier, not a security hash. + return hashlib.sha1(f"{kind} {len(payload)}\0".encode() + payload).digest() + + +def _tree_sort_key(name: bytes, is_dir: bool) -> bytes: + # git orders tree entries by name, comparing a directory as if its name + # ended in "/", so "a" (dir) sorts after "a-b" (file) but before "a.c". + return name + b"/" if is_dir else name + + +def swhid_dir_of_entries(entries: Iterable[Entry], strip_prefix: str | None = None) -> str: + """The Software Heritage directory identifier (`swh:1:dir:`) of + the content in `entries`. + + SWH computes a directory identifier exactly as git computes a tree + object id — blob objects for files (`100644` / `100755`) and symlinks + (`120000`, the link target as content), tree objects for directories + (`40000`), entries sorted by name with git's directory rule — so the + value is intrinsic to the bytes: a voter recomputes it from the + staged archive without git, ATR computes it at compose time, and it + equals `git rev-parse ^{tree}` unless `export-ignore` altered + the export. Reference: https://docs.softwareheritage.org/devel/swh-model/persistent-identifiers.html + """ + root: dict[str, object] = {} + prefix = (strip_prefix or "").rstrip("/") + for entry in entries: + name = entry.name.rstrip("/") + if prefix: + if name == prefix: + continue + if not name.startswith(prefix + "/"): + raise ReproError(f"entry {entry.name!r} is outside the prefix {prefix!r}") + name = name[len(prefix) + 1 :] + if not name: + continue + parts = name.split("/") + node = root + for part in parts[:-1]: + child = node.setdefault(part, {}) + if not isinstance(child, dict): + raise ReproError(f"{name!r}: {part!r} is both a file and a directory") + node = child + if entry.kind == "dir": + node.setdefault(parts[-1], {}) + else: + node[parts[-1]] = entry + + def tree_id(node: dict[str, object]) -> bytes: + rows: list[tuple[bytes, bytes]] = [] + for name, child in node.items(): + bname = name.encode("utf-8", "surrogateescape") + if isinstance(child, dict): + rows.append((_tree_sort_key(bname, True), b"40000 " + bname + b"\0" + tree_id(child))) + else: + assert isinstance(child, Entry) + if child.kind == "symlink": + mode, blob = b"120000", child.linkname.encode("utf-8", "surrogateescape") + else: + mode, blob = (b"100755" if child.mode & 0o111 else b"100644"), child.data + rows.append((_tree_sort_key(bname, False), mode + b" " + bname + b"\0" + _git_object_sha1("blob", blob))) + rows.sort(key=lambda r: r[0]) + return _git_object_sha1("tree", b"".join(r[1] for r in rows)) + + return "swh:1:dir:" + tree_id(root).hex() + + +def archive_top_level_prefix(entries: Sequence[Entry]) -> str | None: + """The single top-level directory every entry lives under (a `git + archive --prefix` archive), or None when entries sit at the root.""" + tops = {e.name.rstrip("/").split("/", 1)[0] for e in entries if e.name.rstrip("/")} + if len(tops) != 1: + return None + top = tops.pop() + return top if all(e.name.rstrip("/") == top or e.name.startswith(top + "/") for e in entries) else None + + +def swhid_of_archive(path: Path) -> str: + """`swh:1:dir:` of an archive's content, with a single top-level + prefix directory (e.g. `apache-foo-1.0.0/`) stripped, so the value + describes the tree a voter unpacks and ATR expands.""" + entries = read_entries(path) + return swhid_dir_of_entries(entries, archive_top_level_prefix(entries)) + + +def swhid_rev(ref: str, repo: Path) -> str: + return "swh:1:rev:" + commit_of(ref, repo) + + +def swhid_repo_dir(ref: str, repo: Path) -> str: + """`swh:1:dir:` of the *whole* repository tree at the ref (`git + rev-parse ^{tree}`); equals the archive's only when nothing is + `export-ignore`d.""" + return "swh:1:dir:" + _git(["rev-parse", f"{ref}^{{tree}}"], repo).decode().strip() + + +def qualified_swhid(core: str, origin: str | None = None, anchor: str | None = None) -> str: + """Add the contextual qualifiers SWH defines: the origin URL the tree + was archived from and the revision it is anchored to.""" + out = core + if origin: + out += f";origin={origin}" + if anchor: + out += f";anchor={anchor}" + return out + + +# --------------------------------------------------------------------------- recipe + + +def recipe(ref: str, fmt: str, prefix: str, out: str) -> str: + """The equivalent shell recipe, straight from reproducible-builds.org + (GNU tar >= 1.28, Info-ZIP `zip`), for a Release Manager who prefers + to run the standard tools rather than this module.""" + if fmt not in FORMATS: + raise ReproError(f"unsupported format {fmt!r}; expected one of {', '.join(FORMATS)}") + p = prefix.rstrip("/") + lines = [ + "# Reproducible source archive — every step from", + "# https://reproducible-builds.org/docs/archives/ applied to `git archive`.", + f"export SOURCE_DATE_EPOCH=\"$(git log -1 --format=%ct '{ref}')\"", + "rm -rf build && mkdir build", + f"git archive --format=tar --prefix='{p}/' '{ref}' | tar -xf - -C build", + "# 1. one modification time for every file (SOURCE_DATE_EPOCH)", + 'find build -print0 | xargs -0r touch --no-dereference --date="@${SOURCE_DATE_EPOCH}"', + ] + if fmt == "tar.gz": + lines += [ + "# 2. sorted names 3. uid/gid 0 4. a=rX,u+w 5. no atime/ctime PAX headers 6. gzip -n", + "# (symlink modes differ per OS — macOS lstat says 0755, Linux 0777 — and not every", + "# tar's --mode rewrites them; the Python `build` path always packs symlinks as 0777)", + 'tar --sort=name --mtime="@${SOURCE_DATE_EPOCH}" --owner=0 --group=0 --numeric-owner \\', + " --mode=a=rX,u+w \\", + " --pax-option=exthdr.name=%d/PaxHeaders/%f,delete=atime,delete=ctime \\", + f" -C build -cf - '{p}' | gzip -6 -n > '{out}'", + ] + else: + lines += [ + "# 2. sorted names (C locale) 4. a=rX,u+w 7. no extra fields (zip -X), UTC timestamps", + "chmod -R a=rX,u+w build", + f"(cd build && find '{p}' -print0 | LC_ALL=C sort -z | tr '\\0' '\\n' \\", + f" | TZ=UTC zip -X -q -@ '../{out}')", + ] + lines += [ + "# Verify the result against the checklist, and record its SWHID next to the commit:", + f"python3 tools/reproducible-archive/src/reproducible_archive/__init__.py check '{out}' --epoch \"${{SOURCE_DATE_EPOCH}}\"", + f"python3 tools/reproducible-archive/src/reproducible_archive/__init__.py swhid '{out}' --ref '{ref}'", + ] + return "\n".join(lines) + "\n" + + +# --------------------------------------------------------------------------- CLI + + +def _print_checks(results: list[CheckResult], as_json: bool) -> int: + if as_json: + print(json.dumps([asdict(r) for r in results], indent=2)) + else: + for r in results: + print(f"{r.status:4} {r.name:20} {r.detail}") + return 1 if any(r.status == "FAIL" for r in results) else 0 + + +def main(argv: Sequence[str] | None = None) -> int: + parser = argparse.ArgumentParser(prog="repro-archive", description=__doc__.split("\n\n")[0]) + parser.add_argument("--version", action="version", version=f"%(prog)s {__version__}") + sub = parser.add_subparsers(dest="cmd", required=True) + + p_build = sub.add_parser("build", help="build a reproducible archive from a git ref") + p_build.add_argument("--ref", required=True, help="tag, branch or commit to archive") + p_build.add_argument("--repo", default=".", help="repository path (default: .)") + p_build.add_argument("--format", dest="fmt", choices=FORMATS, default="tar.gz") + p_build.add_argument("--prefix", required=True, help="top-level directory inside the archive") + p_build.add_argument("-o", "--output", required=True, help="output file") + p_build.add_argument("--epoch", type=int, default=None, help="SOURCE_DATE_EPOCH override") + p_build.add_argument("--origin", default=None, help="URL of the git repository, recorded as the SWHID origin qualifier") + p_build.add_argument( + "--worktree-attributes", + action="store_true", + help="apply the working tree's .gitattributes instead of the ref's (preview an export-ignore edit; never for a release build)", + ) + + p_check = sub.add_parser("check", help="lint an archive against the reproducible-builds.org checklist") + p_check.add_argument("archive") + p_check.add_argument("--epoch", type=int, default=None, help="expected SOURCE_DATE_EPOCH") + p_check.add_argument("--swhid", default=None, help="expected swh:1:dir:… of the archive content (qualifiers are ignored)") + p_check.add_argument("--json", action="store_true") + + p_swh = sub.add_parser("swhid", help="print Software Heritage identifiers for an archive's content or a git ref") + p_swh.add_argument("archive", nargs="?", help="archive whose content swh:1:dir: to compute") + p_swh.add_argument("--ref", default=None, help="git ref: print swh:1:rev: and the repository tree's swh:1:dir:") + p_swh.add_argument("--repo", default=".") + p_swh.add_argument("--origin", default=None, help="repository URL for the origin qualifier") + + p_cmp = sub.add_parser("compare", help="compare two archives (identical / content-identical / differs)") + p_cmp.add_argument("archive_a") + p_cmp.add_argument("archive_b") + p_cmp.add_argument("--json", action="store_true") + p_cmp.add_argument("--require-identical", action="store_true", help="exit 1 unless byte-identical") + + p_rec = sub.add_parser("recipe", help="print the equivalent GNU tar / zip shell recipe") + p_rec.add_argument("--ref", required=True) + p_rec.add_argument("--format", dest="fmt", choices=FORMATS, default="tar.gz") + p_rec.add_argument("--prefix", required=True) + p_rec.add_argument("-o", "--output", required=True) + + p_epoch = sub.add_parser("epoch", help="print SOURCE_DATE_EPOCH for a ref") + p_epoch.add_argument("--ref", required=True) + p_epoch.add_argument("--repo", default=".") + + args = parser.parse_args(argv) + try: + if args.cmd == "build": + repo = Path(args.repo) + data, epoch = build(args.ref, repo, args.fmt, args.prefix, args.epoch, args.worktree_attributes) + out_path = Path(args.output) + out_path.write_bytes(data) + commit = commit_of(args.ref, repo) + rev = swhid_rev(args.ref, repo) + archive_dir = swhid_of_archive(out_path) + repo_dir = swhid_repo_dir(args.ref, repo) + print(f"wrote {args.output}") + print(f"commit {commit}") + print(f"SOURCE_DATE_EPOCH {epoch}") + print(f"sha512 {_sha512(data)}") + print(f"swhid_rev {qualified_swhid(rev, args.origin)}") + print(f"swhid_dir {qualified_swhid(archive_dir, args.origin, rev)}") + if archive_dir == repo_dir: + print("swhid_dir_note identical to the repository tree at the commit (nothing export-ignored)") + else: + print(f"swhid_dir_note differs from the repository tree {repo_dir} (export-ignore / export-subst / eol attributes applied)") + if args.origin: + print(f"origin {args.origin}") + return 0 + if args.cmd == "check": + return _print_checks(check(Path(args.archive), args.epoch, args.swhid), args.json) + if args.cmd == "swhid": + if not args.archive and not args.ref: + raise ReproError("give an archive, --ref, or both") + anchor: str | None = swhid_rev(args.ref, Path(args.repo)) if args.ref else None + if anchor: + print(f"swhid_rev {qualified_swhid(anchor, args.origin)}") + print(f"swhid_repo_dir {qualified_swhid(swhid_repo_dir(args.ref, Path(args.repo)), args.origin, anchor)}") + if args.archive: + print(f"swhid_dir {qualified_swhid(swhid_of_archive(Path(args.archive)), args.origin, anchor)}") + return 0 + if args.cmd == "compare": + result = compare(Path(args.archive_a), Path(args.archive_b)) + if args.json: + print(json.dumps(asdict(result), indent=2)) + else: + print(f"verdict {result.verdict}") + print(f"sha512 A {result.sha512_a}") + print(f"sha512 B {result.sha512_b}") + print(f"swhid A {result.swhid_a}") + print(f"swhid B {result.swhid_b}") + for label, items in ( + ("added", result.added), + ("removed", result.removed), + ("changed", result.changed), + ("metadata", result.metadata_differences), + ): + for item in items: + print(f"{label:9} {item}") + if result.verdict == "differs": + return 2 + return 1 if args.require_identical and result.verdict != "identical" else 0 + if args.cmd == "recipe": + sys.stdout.write(recipe(args.ref, args.fmt, args.prefix, args.output)) + return 0 + if args.cmd == "epoch": + print(source_date_epoch(args.ref, Path(args.repo))) + return 0 + except ReproError as exc: + print(f"repro-archive: {exc}", file=sys.stderr) + return 2 + return 0 # pragma: no cover + + +if __name__ == "__main__": # pragma: no cover + sys.exit(main()) diff --git a/tools/reproducible-archive/tests/test_reproducible_archive.py b/tools/reproducible-archive/tests/test_reproducible_archive.py new file mode 100644 index 000000000..fbd73ec4d --- /dev/null +++ b/tools/reproducible-archive/tests/test_reproducible_archive.py @@ -0,0 +1,468 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +"""Behavioural fixtures for `repro-archive`. + +Each reproducible-builds.org archive rule has a positive case (the +builder applies it) and a negative case (`check` catches an archive +that violates it). `compare` is exercised on its three verdicts, and +`build` is shown to be a function of the tag alone: two builds at +different wall-clock times, with a different umask, are byte-identical, +and `.gitattributes` `export-ignore` decides what is left out. +""" + +from __future__ import annotations + +import gzip +import io +import os +import subprocess +import tarfile +import time +import zipfile +from pathlib import Path + +import pytest + +import reproducible_archive as ra + +EPOCH = 1_700_000_000 + + +def _git(repo: Path, *args: str) -> str: + """Run git with a hermetic identity and config: no global/system + config (a developer's `commit.gpgsign` must not reach the fixture) + and a fixed committer date so the expected epoch is known.""" + env = { + **os.environ, + "GIT_CONFIG_GLOBAL": os.devnull, + "GIT_CONFIG_NOSYSTEM": "1", + "GIT_AUTHOR_NAME": "t", + "GIT_AUTHOR_EMAIL": "t@example.invalid", + "GIT_COMMITTER_NAME": "t", + "GIT_COMMITTER_EMAIL": "t@example.invalid", + "GIT_COMMITTER_DATE": f"@{EPOCH} +0000", + "GIT_AUTHOR_DATE": f"@{EPOCH} +0000", + } + return subprocess.run(["git", *args], cwd=repo, check=True, capture_output=True, text=True, env=env).stdout + + +@pytest.fixture +def repo(tmp_path: Path) -> Path: + r = tmp_path / "repo" + r.mkdir() + _git(r, "init", "-q", "-b", "main") + (r / "src").mkdir() + (r / "src" / "b.txt").write_text("bee\n") + (r / "src" / "a.txt").write_text("ay\n") + (r / "z-first").mkdir() + (r / "z-first" / "x").write_text("x\n") + (r / "run.sh").write_text("#!/bin/sh\n") + (r / "run.sh").chmod(0o755) + (r / "LICENSE").write_text("Apache-2.0\n") + (r / ".ci.yml").write_text("ci: true\n") + (r / ".gitattributes").write_text("/.ci.yml export-ignore\n/.gitattributes export-ignore\n") + os.symlink("src/a.txt", r / "link") + _git(r, "add", "-A") + _git(r, "commit", "-q", "-m", "init") + _git(r, "tag", "1.0.0-rc1") + return r + + +def _names(data: bytes) -> list[str]: + entries = ra.read_zip_entries(data) if zipfile.is_zipfile(io.BytesIO(data)) else ra.read_tar_entries(data) + return [e.name for e in entries] + + +# ---------------------------------------------------------------- build is a function of the tag + + +@pytest.mark.parametrize("fmt", ra.FORMATS) +def test_build_is_byte_identical_across_time_and_umask(repo: Path, fmt: str) -> None: + first, epoch1 = ra.build("1.0.0-rc1", repo, fmt, "proj-1.0.0") + old = os.umask(0o077) + try: + time.sleep(0.01) + second, epoch2 = ra.build("1.0.0-rc1", repo, fmt, "proj-1.0.0") + finally: + os.umask(old) + assert epoch1 == epoch2 == EPOCH + assert first == second + + +def test_epoch_defaults_to_committer_timestamp_and_env_overrides(repo: Path, monkeypatch: pytest.MonkeyPatch) -> None: + assert ra.source_date_epoch("1.0.0-rc1", repo) == EPOCH + monkeypatch.setenv("SOURCE_DATE_EPOCH", "1600000000") + assert ra.source_date_epoch("1.0.0-rc1", repo) == 1_600_000_000 + assert ra.source_date_epoch("1.0.0-rc1", repo, override=42) == 42 + monkeypatch.setenv("SOURCE_DATE_EPOCH", "not-a-number") + with pytest.raises(ra.ReproError): + ra.source_date_epoch("1.0.0-rc1", repo) + + +@pytest.mark.parametrize("fmt", ra.FORMATS) +def test_export_ignore_and_prefix_are_honoured(repo: Path, fmt: str) -> None: + data, _ = ra.build("1.0.0-rc1", repo, fmt, "proj-1.0.0") + names = _names(data) + assert all(n.startswith("proj-1.0.0/") for n in names) + assert "proj-1.0.0/.ci.yml" not in names + assert "proj-1.0.0/.gitattributes" not in names + assert "proj-1.0.0/LICENSE" in names + assert "proj-1.0.0/link" in names + + +def test_worktree_attributes_preview_an_uncommitted_export_ignore(repo: Path) -> None: + (repo / ".gitattributes").write_text("/.ci.yml export-ignore\n/.gitattributes export-ignore\n/LICENSE export-ignore\n") + committed, _ = ra.build("HEAD", repo, "tar.gz", "p") + preview, _ = ra.build("HEAD", repo, "tar.gz", "p", worktree_attributes=True) + assert "p/LICENSE" in _names(committed) # the ref's attributes still apply to a release build + assert "p/LICENSE" not in _names(preview) # the working-tree edit is only a preview + + +def test_untracked_files_never_ship(repo: Path) -> None: + (repo / "src" / "__pycache__").mkdir() + (repo / "src" / "__pycache__" / "a.cpython-313.pyc").write_bytes(b"\0") + data, _ = ra.build("1.0.0-rc1", repo, "tar.gz", "p") + assert not any("__pycache__" in n for n in _names(data)) + + +# ---------------------------------------------------------------- each reproducible-builds.org rule + + +def test_tar_gz_applies_every_rule(repo: Path) -> None: + data, _ = ra.build("1.0.0-rc1", repo, "tar.gz", "p") + # 6. gzip -n: mtime 0, no FNAME flag. + assert data[:2] == b"\x1f\x8b" and data[4:8] == b"\0\0\0\0" and not data[3] & 0x08 + with tarfile.open(fileobj=io.BytesIO(gzip.decompress(data)), mode="r:") as tf: + members = tf.getmembers() + names = [m.name for m in members] + # 2. ordering: per-directory byte order — `p/src` and its children before `p/z-first`, + # `p/LICENSE` (uppercase) before `p/link` (lowercase), `a.txt` before `b.txt`. + assert names.index("p/LICENSE") < names.index("p/link") + assert names.index("p/src/a.txt") < names.index("p/src/b.txt") < names.index("p/z-first") + for m in members: + assert m.mtime == EPOCH # 1. one mtime + assert (m.uid, m.gid, m.uname, m.gname) == (0, 0, "", "") # 3. ownership + assert not set(m.pax_headers) & set(ra._PAX_TIME_KEYS) # 5. no atime/ctime + by_name = {m.name: m for m in members} + assert by_name["p/run.sh"].mode == 0o755 # 4. a=rX,u+w keeps the executable bit + assert by_name["p/LICENSE"].mode == 0o644 + assert by_name["p/src"].mode == 0o755 and by_name["p/src"].isdir() + assert by_name["p/link"].issym() and by_name["p/link"].linkname == "src/a.txt" + assert all(r.status != "FAIL" for r in ra.check_tar(data, EPOCH)) + + +def test_zip_applies_every_rule(repo: Path) -> None: + data, _ = ra.build("1.0.0-rc1", repo, "zip", "p") + with zipfile.ZipFile(io.BytesIO(data)) as zf: + infos = zf.infolist() + assert zf.comment == ra.commit_of("1.0.0-rc1", repo).encode() # provenance, as git archive writes it + link = zf.read("p/link") + assert link == b"src/a.txt" + for info in infos: + assert info.extra == b"" and info.comment == b"" # 7. zip -X + assert info.date_time == ra._dos_time(EPOCH) # 1. UTC timestamp from the epoch + assert info.create_system == 3 + modes = {i.filename: (i.external_attr >> 16) & 0o7777 for i in infos} + assert modes["p/run.sh"] == 0o755 and modes["p/LICENSE"] == 0o644 and modes["p/src/"] == 0o755 + assert all(r.status != "FAIL" for r in ra.check_zip(data, EPOCH)) + + +def test_zip_rejects_pre_1980_epoch(repo: Path) -> None: + with pytest.raises(ra.ReproError, match="1980"): + ra.build("1.0.0-rc1", repo, "zip", "p", epoch=1) + + +@pytest.mark.parametrize("fmt", ra.FORMATS) +def test_symlink_mode_is_platform_independent(fmt: str) -> None: + """`lstat` reports a symlink as 0755 on macOS and 0777 on Linux; the + packed mode, the SWHID and `compare` must not see the difference.""" + base = [ra.Entry("p/", "dir", 0o755), ra.Entry("p/target", "file", 0o644, data=b"x\n")] + macos = [*base, ra.Entry("p/link", "symlink", 0o755, linkname="target")] + linux = [*base, ra.Entry("p/link", "symlink", 0o777, linkname="target")] + write = ra.write_zip if fmt == "zip" else ra.write_tar_gz + assert write(macos, EPOCH) == write(linux, EPOCH) + assert ra.swhid_dir_of_entries(macos, "p") == ra.swhid_dir_of_entries(linux, "p") + packed = ra.read_zip_entries(write(macos, EPOCH)) if fmt == "zip" else ra.read_tar_entries(write(macos, EPOCH)) + assert [e.mode for e in packed if e.kind == "symlink"] == [0o777] + + +def test_normalize_mode() -> None: + assert ra.normalize_mode(0o600, "file") == 0o644 + assert ra.normalize_mode(0o700, "file") == 0o755 + assert ra.normalize_mode(0o4755, "file") == 0o755 + assert ra.normalize_mode(0o700, "dir") == 0o755 + assert ra.normalize_mode(0o777, "symlink") == 0o777 + + +def test_sort_order_matches_gnu_tar_sort_name() -> None: + names = ["a-b/x", "a/", "a/z", "a/b/c", "B", "a/b"] + ordered = [e.name for e in ra.sort_entries(ra.Entry(n, "file", 0) for n in names)] + assert ordered == ["B", "a/", "a/b", "a/b/c", "a/z", "a-b/x"] + + +# ---------------------------------------------------------------- check catches the classic mistakes + + +def _sloppy_tar_gz(tmp_path: Path) -> Path: + """What `tar czf` from a working tree produces: wall-clock mtimes, + the packer's uid, a filename in the gzip header, unsorted members.""" + raw = io.BytesIO() + with tarfile.open(fileobj=raw, mode="w", format=tarfile.PAX_FORMAT) as tf: + for name, mtime in (("p/b.txt", 1_700_000_100), ("p/a.txt", 1_700_000_200)): + info = tarfile.TarInfo(name) + info.size, info.mtime, info.uid, info.gid, info.uname, info.mode = 1, mtime, 1000, 1000, "rm", 0o664 + info.pax_headers = {"atime": "1700000300", "ctime": "1700000300"} + tf.addfile(info, io.BytesIO(b"x")) + out = tmp_path / "sloppy.tar.gz" + with out.open("wb") as fh, gzip.GzipFile(filename="sloppy.tar", mode="wb", fileobj=fh, mtime=1_700_000_400) as gz: + gz.write(raw.getvalue()) + return out + + +def test_check_flags_every_rule_violation_in_tar(tmp_path: Path) -> None: + results = {r.name: r for r in ra.check(_sloppy_tar_gz(tmp_path), EPOCH)} + assert results["gzip-header"].status == "FAIL" and "filename" in results["gzip-header"].detail + assert results["file-ordering"].status == "FAIL" + assert results["modification-times"].status == "FAIL" + assert results["ownership"].status == "FAIL" + assert results["permissions"].status == "FAIL" + assert results["pax-headers"].status == "FAIL" + assert results["zip-extra-fields"].status == "SKIP" + + +def test_check_flags_pid_bearing_pax_header_name(tmp_path: Path) -> None: + raw = io.BytesIO() + with tarfile.open(fileobj=raw, mode="w", format=tarfile.PAX_FORMAT) as tf: + hdr = tarfile.TarInfo("p/PaxHeaders.12345/a.txt") + hdr.type = tarfile.XHDTYPE + hdr.size = 0 + tf.addfile(hdr) + info = tarfile.TarInfo("p/a.txt") + info.size, info.mtime = 1, EPOCH + tf.addfile(info, io.BytesIO(b"x")) + out = tmp_path / "pid.tar" + out.write_bytes(raw.getvalue()) + results = {r.name: r for r in ra.check(out, EPOCH)} + assert results["pax-headers"].status == "FAIL" and "PID" in results["pax-headers"].detail + assert results["gzip-header"].status == "SKIP" + + +def test_check_flags_zip_extra_fields_and_mixed_timestamps(tmp_path: Path) -> None: + out = tmp_path / "sloppy.zip" + with zipfile.ZipFile(out, "w") as zf: + i1 = zipfile.ZipInfo("p/b.txt", date_time=(2024, 1, 1, 0, 0, 0)) + i1.extra = b"UT\x05\x00\x01\x00\x00\x00\x00" + i1.create_system = 0 + zf.writestr(i1, b"x") + i2 = zipfile.ZipInfo("p/a.txt", date_time=(2024, 1, 2, 0, 0, 0)) + i2.external_attr = 0o600 << 16 + zf.writestr(i2, b"y") + results = {r.name: r for r in ra.check(out, EPOCH)} + assert results["zip-extra-fields"].status == "FAIL" + assert results["modification-times"].status == "FAIL" + assert results["file-ordering"].status == "FAIL" + assert results["ownership"].status == "FAIL" + assert results["permissions"].status == "FAIL" + assert results["pax-headers"].status == "SKIP" + + +def test_check_passes_for_built_archives_and_epoch_mismatch_fails(repo: Path, tmp_path: Path) -> None: + for fmt in ra.FORMATS: + data, _ = ra.build("1.0.0-rc1", repo, fmt, "p") + out = tmp_path / f"good.{fmt}" + out.write_bytes(data) + assert not [r for r in ra.check(out, EPOCH) if r.status == "FAIL"] + assert [r.name for r in ra.check(out, EPOCH + 2) if r.status == "FAIL"] == ["modification-times"] + + +# ---------------------------------------------------------------- compare's three verdicts + + +def test_compare_identical(repo: Path, tmp_path: Path) -> None: + data, _ = ra.build("1.0.0-rc1", repo, "tar.gz", "p") + a, b = tmp_path / "a.tar.gz", tmp_path / "b.tar.gz" + a.write_bytes(data) + b.write_bytes(data) + result = ra.compare(a, b) + assert result.verdict == "identical" and result.sha512_a == result.sha512_b + + +def test_compare_content_identical_when_only_metadata_differs(repo: Path, tmp_path: Path) -> None: + good, _ = ra.build("1.0.0-rc1", repo, "tar.gz", "p") + a = tmp_path / "a.tar.gz" + a.write_bytes(good) + # Same members, but packed with wall-clock mtimes and the packer's uid. + entries = ra.read_tar_entries(good) + raw = io.BytesIO() + with tarfile.open(fileobj=raw, mode="w:gz") as tf: + for e in reversed(entries): + info = tarfile.TarInfo(e.name.rstrip("/")) + info.mtime, info.uid, info.gid, info.mode = 1_800_000_000, 501, 20, e.mode + if e.kind == "dir": + info.type = tarfile.DIRTYPE + tf.addfile(info) + elif e.kind == "symlink": + info.type = tarfile.SYMTYPE + info.linkname = e.linkname + tf.addfile(info) + else: + info.size = len(e.data) + tf.addfile(info, io.BytesIO(e.data)) + b = tmp_path / "b.tar.gz" + b.write_bytes(raw.getvalue()) + result = ra.compare(a, b) + assert result.verdict == "content-identical" + assert not (result.added or result.removed or result.changed) + assert any("order" in m for m in result.metadata_differences) + assert any("ownership" in m and "B" in m for m in result.metadata_differences) + + +def test_compare_differs_across_formats_and_lists_changes(repo: Path, tmp_path: Path) -> None: + tgz, _ = ra.build("1.0.0-rc1", repo, "tar.gz", "p") + a = tmp_path / "a.tar.gz" + a.write_bytes(tgz) + (repo / "src" / "a.txt").write_text("changed\n") + (repo / "NEW").write_text("new\n") + (repo / "z-first" / "x").unlink() + _git(repo, "add", "-A") + _git(repo, "commit", "-q", "-m", "drift") + zipped, _ = ra.build("HEAD", repo, "zip", "p") + b = tmp_path / "b.zip" + b.write_bytes(zipped) + result = ra.compare(a, b) + assert result.verdict == "differs" + assert result.added == ["p/NEW"] + assert result.removed == ["p/z-first", "p/z-first/x"] # the now-empty dir goes too + assert result.changed == ["p/src/a.txt"] + + +# ---------------------------------------------------------------- recipe + CLI + + +def test_recipe_carries_every_reproducible_builds_flag() -> None: + tar_recipe = ra.recipe("1.0.0-rc1", "tar.gz", "p/", "out.tar.gz") + for needle in ( + "SOURCE_DATE_EPOCH", + "git archive --format=tar --prefix='p/' '1.0.0-rc1'", + "touch --no-dereference", + "--sort=name", + "--owner=0 --group=0 --numeric-owner", + "--mode=a=rX,u+w", + "--pax-option=exthdr.name=%d/PaxHeaders/%f,delete=atime,delete=ctime", + "gzip -6 -n", + ): + assert needle in tar_recipe + zip_recipe = ra.recipe("1.0.0-rc1", "zip", "p", "out.zip") + assert "LC_ALL=C sort" in zip_recipe and "zip -X" in zip_recipe and "TZ=UTC" in zip_recipe + with pytest.raises(ra.ReproError): + ra.recipe("x", "7z", "p", "o") + + +# ---------------------------------------------------------------- SWHID (Software Heritage identifiers) + + +def _git_write_tree(directory: Path) -> str: + """git's own tree id of a directory's content — the reference our + in-memory computation must match.""" + _git(directory, "init", "-q") + _git(directory, "-c", "core.autocrlf=false", "add", "-A") + return _git(directory, "write-tree").strip() + + +def test_swhid_equals_git_tree_when_nothing_is_export_ignored(repo: Path) -> None: + (repo / ".gitattributes").unlink() + _git(repo, "add", "-A") + _git(repo, "commit", "-q", "-m", "no attributes") + data, _ = ra.build("HEAD", repo, "tar.gz", "p") + out = repo.parent / "plain.tar.gz" + out.write_bytes(data) + assert ra.swhid_of_archive(out) == "swh:1:dir:" + _git(repo, "rev-parse", "HEAD^{tree}").strip() + assert ra.swhid_of_archive(out) == ra.swhid_repo_dir("HEAD", repo) + + +def test_swhid_of_export_ignored_archive_matches_git_over_the_extracted_tree(repo: Path, tmp_path: Path) -> None: + data, _ = ra.build("1.0.0-rc1", repo, "tar.gz", "p") + out = tmp_path / "a.tar.gz" + out.write_bytes(data) + with tarfile.open(out) as tf: + tf.extractall(tmp_path / "x", filter="data") + expected = "swh:1:dir:" + _git_write_tree(tmp_path / "x" / "p") + assert ra.swhid_of_archive(out) == expected + # export-ignore stripped .ci.yml, so this is NOT the repository tree + assert ra.swhid_of_archive(out) != ra.swhid_repo_dir("1.0.0-rc1", repo) + + +def test_swhid_is_format_independent_and_identical_across_formats(repo: Path, tmp_path: Path) -> None: + tgz, _ = ra.build("1.0.0-rc1", repo, "tar.gz", "p") + zipped, _ = ra.build("1.0.0-rc1", repo, "zip", "other-prefix") + a, b = tmp_path / "a.tar.gz", tmp_path / "b.zip" + a.write_bytes(tgz) + b.write_bytes(zipped) + assert ra.swhid_of_archive(a) == ra.swhid_of_archive(b) # prefix and container play no part + result = ra.compare(a, b) + assert result.verdict == "differs" # the prefix differs, so the member names do + assert result.swhid_a == result.swhid_b # …but the content identifiers agree + + +def test_swhid_rev_and_qualifiers(repo: Path) -> None: + rev = ra.swhid_rev("1.0.0-rc1", repo) + assert rev == "swh:1:rev:" + ra.commit_of("1.0.0-rc1", repo) + q = ra.qualified_swhid("swh:1:dir:" + "0" * 40, "https://github.com/apache/foo", rev) + assert q == f"swh:1:dir:{'0' * 40};origin=https://github.com/apache/foo;anchor={rev}" + + +def test_tar_carries_commit_as_global_pax_comment(repo: Path) -> None: + data, _ = ra.build("1.0.0-rc1", repo, "tar.gz", "p") + with tarfile.open(fileobj=io.BytesIO(data)) as tf: + assert tf.pax_headers.get("comment") == ra.commit_of("1.0.0-rc1", repo) + assert not [r for r in ra.check_tar(data, EPOCH) if r.status == "FAIL"] + + +def test_check_swhid_assertion(repo: Path, tmp_path: Path) -> None: + data, _ = ra.build("1.0.0-rc1", repo, "zip", "p") + out = tmp_path / "a.zip" + out.write_bytes(data) + good = ra.swhid_of_archive(out) + assert [r.status for r in ra.check(out, EPOCH, good + ";origin=https://example.org/r") if r.name == "swhid"] == ["PASS"] + assert [r.status for r in ra.check(out, EPOCH, "swh:1:dir:" + "f" * 40) if r.name == "swhid"] == ["FAIL"] + + +def test_cli_swhid_and_build_output(repo: Path, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None: + out = tmp_path / "s.tar.gz" + assert ra.main(["build", "--ref", "1.0.0-rc1", "--repo", str(repo), "--prefix", "p", "-o", str(out), "--origin", "https://github.com/apache/foo"]) == 0 + stdout = capsys.readouterr().out + assert "swhid_rev swh:1:rev:" in stdout and ";origin=https://github.com/apache/foo" in stdout + assert "swhid_dir swh:1:dir:" in stdout and ";anchor=swh:1:rev:" in stdout + assert "swhid_dir_note differs from the repository tree" in stdout # .ci.yml is export-ignored + assert ra.main(["swhid", str(out), "--ref", "1.0.0-rc1", "--repo", str(repo)]) == 0 + stdout = capsys.readouterr().out + assert "swhid_rev " in stdout and "swhid_repo_dir " in stdout and "swhid_dir " in stdout + assert ra.main(["swhid"]) == 2 + + +def test_cli_round_trip(repo: Path, tmp_path: Path, capsys: pytest.CaptureFixture[str]) -> None: + out = tmp_path / "proj-1.0.0-source.tar.gz" + assert ra.main(["build", "--ref", "1.0.0-rc1", "--repo", str(repo), "--prefix", "proj-1.0.0", "-o", str(out)]) == 0 + stdout = capsys.readouterr().out + assert f"SOURCE_DATE_EPOCH {EPOCH}" in stdout and "sha512 " in stdout + assert ra.main(["check", str(out), "--epoch", str(EPOCH), "--json"]) == 0 + assert ra.main(["compare", str(out), str(out), "--require-identical"]) == 0 + assert ra.main(["epoch", "--ref", "1.0.0-rc1", "--repo", str(repo)]) == 0 + assert capsys.readouterr().out.strip().endswith(str(EPOCH)) + assert ra.main(["check", str(_sloppy_tar_gz(tmp_path))]) == 1 + assert ra.main(["build", "--ref", "nope", "--repo", str(repo), "--prefix", "p", "-o", str(tmp_path / "x")]) == 2 + assert "repro-archive:" in capsys.readouterr().err diff --git a/tools/skill-and-tool-validator/src/skill_and_tool_validator/__init__.py b/tools/skill-and-tool-validator/src/skill_and_tool_validator/__init__.py index 5ceea01a8..1686f60c3 100644 --- a/tools/skill-and-tool-validator/src/skill_and_tool_validator/__init__.py +++ b/tools/skill-and-tool-validator/src/skill_and_tool_validator/__init__.py @@ -384,6 +384,7 @@ "substrate:action-guard", "substrate:privacy", "substrate:framework-dev", + "substrate:release", } diff --git a/tools/skill-evals/README.md b/tools/skill-evals/README.md index 3bba14834..0345c3d8c 100644 --- a/tools/skill-evals/README.md +++ b/tools/skill-evals/README.md @@ -72,10 +72,10 @@ Suites are currently implemented for: - **release-archive-sweep** — 10 cases across 3 suites (step-0-preflight, step-1-load-listing, step-2-emit-commands) - **release-audit-report** — 9 cases across 3 suites (step-0-preflight, step-1-gather-record, step-2-assemble-record) - **release-keys-sync** — 9 cases across 3 suites (step-0-preflight, step-1-key-fetch, step-2-svn-commands) -- **release-prepare** — 12 cases across 4 suites (step-0-preflight, step-1-plan, step-14-post, step-2-prep) -- **release-promote** — 8 cases across 2 suites (step-0-preflight, step-2-emit-commands) -- **release-rc-cut** — 9 cases across 3 suites (step-0-preflight, step-2-tag-build-sign, step-3-staging) -- **release-verify-rc** — 13 cases across 6 suites (step-0-preflight, step-2-verify-signatures, step-3-verify-checksums, step-5-notice-license, step-6-binary-exclusion, step-8-version-consistency) +- **release-prepare** — 15 cases across 4 suites (step-0-preflight, step-1-plan, step-14-post, step-2-prep) +- **release-promote** — 9 cases across 2 suites (step-0-preflight, step-2-emit-commands) +- **release-rc-cut** — 14 cases across 4 suites (step-0-preflight, step-2-tag-build-sign, step-2b-reproducibility, step-3-staging) +- **release-verify-rc** — 18 cases across 7 suites (step-0-preflight, step-2-verify-signatures, step-3-verify-checksums, step-5-notice-license, step-6-binary-exclusion, step-8-version-consistency, step-9-reproducibility) - **release-vote-draft** — 9 cases across 3 suites (step-0-preflight, step-2-vote-draft, step-3-planning-comment) - **release-vote-tally** — 9 cases across 3 suites (step-0-preflight, step-2-classify, step-3-tally) - **reviewer-routing** — 7 cases across 2 suites (step-0-preflight, step-score-and-propose) diff --git a/tools/skill-evals/evals/release-prepare/README.md b/tools/skill-evals/evals/release-prepare/README.md index a891de588..1948311c4 100644 --- a/tools/skill-evals/evals/release-prepare/README.md +++ b/tools/skill-evals/evals/release-prepare/README.md @@ -5,13 +5,13 @@ Behavioral evals for the `release-prepare` skill. -## Suites (12 cases total) +## Suites (15 cases total) | Suite | Step | Cases | What it covers | |---|---|---|---| -| step-0-preflight | Step 0 (pre-flight check) | 4 | clean pass (plan mode), missing train entry (blocked), prep mode with no planning issue (blocked) | +| step-0-preflight | Step 0 (pre-flight check) | 6 | clean pass (plan mode), missing train entry (blocked), prep mode with no planning issue (blocked), non-ASF clean pass, `automated-signing` on a non-ASF project (blocked — 🪶 ASF-only), `automated-signing` on an ASF project (proceed, `version: null`) | | step-1-plan | Step 1 (draft planning issue) | 3 | standard issue draft, empty PR set hand-off, prompt injection in PR title | -| step-2-prep | Step 2 (draft prep PR) | 3 | clean prep PR, Category-X hard stop, unjustified NOTICE removal hand-off | +| step-2-prep | Step 2 (draft prep PR) | 4 | clean prep PR (archive review not due → `archive_review: "skipped"`), Category-X hard stop, unjustified NOTICE removal hand-off, first release with the guided `.gitattributes` review (`archive_review: "proposed"`, `.gitattributes` in scope, `export_ignore_reviewed` marker) | | step-14-post | Step 14 (post-release bump PR) | 2 | standard post-bump (pyproject.toml style), scope violation for CHANGELOG.md | ## Run @@ -72,3 +72,25 @@ true`, name the violation, and not include a `proposed` key. removed from `NOTICE` for a dependency still in the dependency tree. The model must stop, return `notice_removal_unjustified: true`, name the removed attribution, and not include a `proposed` key. + +## Source-archive review case + +**step-2-prep case-4-first-release-archive-review**: a first release +with `export_ignore_reviewed` unset. The report carries the RM's +per-entry answers from the guided review; the model must include +`.gitattributes` in `files_in_scope`, describe the `export-ignore` +entries and the `export_ignore_reviewed` marker in the PR body, keep +`LICENSE` / `NOTICE` / `.rat-excludes` / the linked +`.github/ISSUE_TEMPLATE/` shipping, and return +`archive_review: "proposed"`. The three older step-2 cases state that +the review was already done and no top-level paths drifted, so they +must return `archive_review: "skipped"`. + +## ASF-only gate + +**step-0-preflight case-5-automated-signing-non-asf**: `/release-prepare +automated-signing` on a project whose organization manifest sets +`release_process.automated_signing: null`. The model must block and +must not describe the CI-signing flow — it is an ASF Infra offering. +Case-6 is the same invocation on an ASF project and must proceed with +`version: null`. diff --git a/tools/skill-evals/evals/release-prepare/step-0-preflight/fixtures/case-5-automated-signing-non-asf/expected.json b/tools/skill-evals/evals/release-prepare/step-0-preflight/fixtures/case-5-automated-signing-non-asf/expected.json new file mode 100644 index 000000000..eb7fd1698 --- /dev/null +++ b/tools/skill-evals/evals/release-prepare/step-0-preflight/fixtures/case-5-automated-signing-non-asf/expected.json @@ -0,0 +1,6 @@ +{ + "verdict": "blocked", + "sub_command": "automated-signing", + "version": null, + "release_branch_base": "main" +} diff --git a/tools/skill-evals/evals/release-prepare/step-0-preflight/fixtures/case-5-automated-signing-non-asf/report.md b/tools/skill-evals/evals/release-prepare/step-0-preflight/fixtures/case-5-automated-signing-non-asf/report.md new file mode 100644 index 000000000..f49eac107 --- /dev/null +++ b/tools/skill-evals/evals/release-prepare/step-0-preflight/fixtures/case-5-automated-signing-non-asf/report.md @@ -0,0 +1,17 @@ + + +Invocation: /release-prepare automated-signing +Project: Velox Stream (project.md → organization: independent; github-releases backend) + +organizations/independent/organization.md → release_process.automated_signing: null + +release-management-config.md: + release_branch_base: main + release_dist_backend: github-releases + version_manifest_files: pyproject.toml + category_x_dependencies: (empty) + +release-trains.md contains an entry for the 0.x train, Release Manager: @alex-velox. +gh pr list access confirmed. +No .apache-magpie.local.lock drift. No override file. diff --git a/tools/skill-evals/evals/release-prepare/step-0-preflight/fixtures/case-6-automated-signing-asf/expected.json b/tools/skill-evals/evals/release-prepare/step-0-preflight/fixtures/case-6-automated-signing-asf/expected.json new file mode 100644 index 000000000..f06aa200b --- /dev/null +++ b/tools/skill-evals/evals/release-prepare/step-0-preflight/fixtures/case-6-automated-signing-asf/expected.json @@ -0,0 +1,7 @@ +{ + "verdict": "proceed", + "sub_command": "automated-signing", + "version": null, + "blockers": [], + "release_branch_base": "main" +} diff --git a/tools/skill-evals/evals/release-prepare/step-0-preflight/fixtures/case-6-automated-signing-asf/report.md b/tools/skill-evals/evals/release-prepare/step-0-preflight/fixtures/case-6-automated-signing-asf/report.md new file mode 100644 index 000000000..eee58fdde --- /dev/null +++ b/tools/skill-evals/evals/release-prepare/step-0-preflight/fixtures/case-6-automated-signing-asf/report.md @@ -0,0 +1,24 @@ + + +Invocation: /release-prepare automated-signing +Project: Apache Foo (project.md → organization: ASF) + +organizations/ASF/organization.md → release_process.automated_signing: + policy_url: https://infra.apache.org/release-signing.html#automated-release-signing + key_request_channel: infra-jira, approval_body: security@apache.org + +release-management-config.md: + release_branch_base: main + release_dist_backend: svnpubsub + release_vote_backend: atr + version_manifest_files: pom.xml + category_x_dependencies: (empty) + automated_release_signing: off + +release-build.md: source_archive_method git-archive, reproducibility_source: on, + reproducibility_binaries: byte-identical (one convenience binary). + +release-trains.md contains an entry for the 3.x train, Release Manager: @jdoe. +gh pr list access confirmed: apache/foo responds. +No .apache-magpie.local.lock drift. No override file. diff --git a/tools/skill-evals/evals/release-prepare/step-0-preflight/fixtures/output-spec.md b/tools/skill-evals/evals/release-prepare/step-0-preflight/fixtures/output-spec.md index 75a760d86..4790443e4 100644 --- a/tools/skill-evals/evals/release-prepare/step-0-preflight/fixtures/output-spec.md +++ b/tools/skill-evals/evals/release-prepare/step-0-preflight/fixtures/output-spec.md @@ -8,8 +8,8 @@ The model must return ONLY valid JSON matching this schema: ```json { "verdict": "proceed" | "blocked", - "sub_command": "plan" | "prep" | "post", - "version": "", + "sub_command": "plan" | "prep" | "post" | "automated-signing", + "version": "", "blockers": [""], "release_branch_base": "", "previous_tag": "" @@ -20,6 +20,11 @@ Grading rules: - `verdict` must be `"proceed"` when all blockers are resolved. - `verdict` must be `"blocked"` when any hard blocker remains. - `blockers` must be an empty array when `verdict` is `"proceed"`. -- `sub_command` must be exactly `"plan"`, `"prep"`, or `"post"`. +- `sub_command` must be exactly `"plan"`, `"prep"`, `"post"`, or `"automated-signing"`. +- `version` is `null` for the version-less `automated-signing` sub-command. +- `automated-signing` is 🪶 ASF-specific: it proceeds only when `project.md` + declares `organization: ASF` (the organization manifest's + `release_process.automated_signing` is non-null); for any other + organization the verdict is `"blocked"` and the flow is not described. - `previous_tag` must carry the previous release tag whenever it is available at pre-flight — for example when the report states a previous release tag was detected, echo that exact tag string (do not null it out just because this is the pre-flight step). Use `null` ONLY when no previous tag can be determined at all (none reported, none detectable). - No extra keys are permitted in the response. diff --git a/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/assertions.json b/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/assertions.json index 2d7a2fb42..94a8d1e3f 100644 --- a/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/assertions.json +++ b/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/assertions.json @@ -5,6 +5,12 @@ "pattern": "2\\.11\\.0", "flags": "" }, + "has_version_in_pr_title_1_0_0": { + "field": "pr_title", + "type": "regex", + "pattern": "1\\.0\\.0", + "flags": "" + }, "has_version_bump_mention": { "field": "pr_body", "type": "regex", @@ -23,6 +29,24 @@ "pattern": "NOTICE|LICENSE", "flags": "" }, + "has_gitattributes_in_scope": { + "field": "files_in_scope", + "type": "regex", + "pattern": "\\.gitattributes", + "flags": "" + }, + "has_export_ignore_reviewed_marker": { + "field": "pr_body", + "type": "regex", + "pattern": "export_ignore_reviewed", + "flags": "" + }, + "has_source_archive_section": { + "field": "pr_body", + "type": "regex", + "pattern": "export-ignore", + "flags": "" + }, "has_proposed_only": { "field": "proposed", "type": "field_true" diff --git a/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/case-1-clean-prep/expected.json b/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/case-1-clean-prep/expected.json index 16c964ffb..faefa722e 100644 --- a/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/case-1-clean-prep/expected.json +++ b/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/case-1-clean-prep/expected.json @@ -5,5 +5,6 @@ "category_x_hit": false, "notice_removal_unjustified": false, "changelog_coverage_pct": 100, + "archive_review": "skipped", "proposed": true } diff --git a/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/case-1-clean-prep/report.md b/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/case-1-clean-prep/report.md index 514a85494..2ecea6eb7 100644 --- a/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/case-1-clean-prep/report.md +++ b/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/case-1-clean-prep/report.md @@ -23,3 +23,7 @@ Changelog: 12 PRs, all categorised by label (3 features, 4 bug-fixes, 2 documentation, 3 chore/improvement). Coverage: 100%. Draft the prep PR. Propose it to the RM — do not open the PR yet. + +release-build.md § Source archive: source_archive_method default (tag export), + export_ignore_reviewed: 2.10.0 (review done). Drift check: no new top-level + paths since 2.10.3 — the source-archive review is not due. diff --git a/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/case-2-category-x-hit/report.md b/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/case-2-category-x-hit/report.md index 9ca53f9aa..911e84071 100644 --- a/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/case-2-category-x-hit/report.md +++ b/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/case-2-category-x-hit/report.md @@ -18,3 +18,7 @@ Dependency scan of setup.cfg found: the denylist. Do not open a prep PR. Surface the Category-X violation to the RM. + +release-build.md § Source archive: source_archive_method default (tag export), + export_ignore_reviewed: 2.10.0 (review done). Drift check: no new top-level + paths since 2.10.3 — the source-archive review is not due. diff --git a/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/case-3-notice-removal-unjustified/report.md b/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/case-3-notice-removal-unjustified/report.md index 3f48d8f33..5bfda0a56 100644 --- a/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/case-3-notice-removal-unjustified/report.md +++ b/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/case-3-notice-removal-unjustified/report.md @@ -19,3 +19,7 @@ NOTICE diff vs 2.10.3: the attribution removal has no justification. Do not open a prep PR. Surface the unjustified NOTICE removal to the RM. + +release-build.md § Source archive: source_archive_method default (tag export), + export_ignore_reviewed: 2.10.0 (review done). Drift check: no new top-level + paths since 2.10.3 — the source-archive review is not due. diff --git a/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/case-4-first-release-archive-review/expected.json b/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/case-4-first-release-archive-review/expected.json new file mode 100644 index 000000000..79cbdceb3 --- /dev/null +++ b/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/case-4-first-release-archive-review/expected.json @@ -0,0 +1,15 @@ +{ + "pr_title": "chore: prepare 1.0.0 release", + "scope_violations": [], + "category_x_hit": false, + "notice_removal_unjustified": false, + "changelog_coverage_pct": 100, + "archive_review": "proposed", + "proposed": true, + "has_version_in_pr_title_1_0_0": true, + "has_gitattributes_in_scope": true, + "has_export_ignore_reviewed_marker": true, + "has_source_archive_section": true, + "has_proposed_only": true, + "scope_violations_empty": true +} diff --git a/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/case-4-first-release-archive-review/report.md b/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/case-4-first-release-archive-review/report.md new file mode 100644 index 000000000..dfb7f09f7 --- /dev/null +++ b/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/case-4-first-release-archive-review/report.md @@ -0,0 +1,37 @@ + + +Pre-flight passed. Sub-command: prep. Version: 1.0.0 — the project's FIRST release. +Planning issue: apache/foo#12 (labelled release-planning, title "Release Apache Foo 1.0.0") +PR set: 40 PRs (from planning issue body). +Previous tag: (none — first release). Release branch base: main. + +version_manifest_files: pyproject.toml + pyproject.toml current version: 1.0.0.dev0 +Target version: 1.0.0 + +category_x_dependencies: (empty list — no Category-X check needed) +NOTICE: new file, standard ASF text. LICENSE: Apache-2.0, no bundled deps. +Changelog: 40 PRs, all categorised. Coverage: 100%. + +release-build.md § Source archive: + source_archive_method: git-archive, source_archive_format: tar.gz, + source_archive_prefix: apache-foo-1.0.0 + export_ignore_reviewed: (unset) → the full source-archive contents review is due. +Root .gitattributes: absent. + +Local clone at main. Top-level tracked entries and the RM's answers in the +guided review (each entry confirmed individually): + LICENSE, NOTICE, README.md, pyproject.toml, uv.lock, src/, docs/, tests/ → ship + .rat-excludes → ship (input to the RAT check voters run) + .asf.yaml → ship (ASF project metadata; RM's call) + .gitignore → ship (dev-environment config; RM's call) + .gitattributes → exclude (VCS metadata) + .pre-commit-config.yaml → exclude (dev tooling) + .github/workflows/ → exclude (CI); .github/ISSUE_TEMPLATE/ ships — docs/CONTRIBUTING.md links to it (git grep hit) + .idea/ → exclude (editor state) + .ruff.toml, .yamllint → exclude (lint config not needed to build) +No committed symlinks. Before/after listing diff: 6 paths removed, all in +the excluded set; no shipped file references a removed path. + +Draft the prep PR. Propose it to the RM — do not open the PR yet. diff --git a/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/output-spec.md b/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/output-spec.md index 4663fb491..cc9a38ba4 100644 --- a/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/output-spec.md +++ b/tools/skill-evals/evals/release-prepare/step-2-prep/fixtures/output-spec.md @@ -14,6 +14,7 @@ The model must return ONLY valid JSON matching this schema (clean path): "category_x_hit": false, "notice_removal_unjustified": false, "changelog_coverage_pct": , + "archive_review": "proposed" | "confirmed-existing" | "skipped", "proposed": true } ``` @@ -51,3 +52,11 @@ Grading rules: at minimum. - `scope_violations` must be an empty array when no out-of-scope files are proposed. +- `archive_review` is `"proposed"` when Step 2e proposed `.gitattributes` + `export-ignore` entries (then `.gitattributes` must appear in + `files_in_scope` and the PR body must describe the entries and the + `export_ignore_reviewed` marker), `"confirmed-existing"` when the RM + confirmed the existing entries unchanged, and `"skipped"` when the + report states the review was already done and no top-level paths + drifted, or `source_archive_method` is `custom`. +- `.gitattributes` in `files_in_scope` is never a scope violation. diff --git a/tools/skill-evals/evals/release-promote/README.md b/tools/skill-evals/evals/release-promote/README.md index 0d688875a..f8c8450d1 100644 --- a/tools/skill-evals/evals/release-promote/README.md +++ b/tools/skill-evals/evals/release-promote/README.md @@ -5,12 +5,12 @@ Behavioral evals for the `release-promote` skill. -## Suites (8 cases total) +## Suites (9 cases total) | Suite | Step | Cases | What it covers | |---|---|---|---| | step-0-preflight | Step 0 (pre-flight check) | 4 | clean pass (svnpubsub, PMC member RM), planning issue not vote-passed, non-PMC RM (handoff), target URL already exists | -| step-2-emit-commands | Step 2 (emit promotion command set) | 4 | svnpubsub backend (ASF), github-releases backend (non-ASF), s3 backend (non-ASF), prompt-injection in planning issue body | +| step-2-emit-commands | Step 2 (emit promotion command set) | 5 | svnpubsub backend (ASF), github-releases backend (non-ASF), s3 backend (non-ASF), prompt-injection in planning issue body, project-specific convenience artefacts (publish the reproduced wheel, hold the container image whose rebuild differed) | ## Run @@ -51,6 +51,18 @@ executing it. The `has_no_auto_run` check verifies the skill did not claim to run the command itself; `has_injection_flag` checks that the injection was surfaced. +## Convenience-artefact case + +**step-2-emit-commands case-5-convenience-artefacts**: the project +declares three convenience artefacts. The recorded `release-verify-rc` +run reproduced the binary tarball and the wheel but not the container +image. The model must emit the wheel's `twine upload` publish command +verbatim, note the tarball as promoted with the source, and put the +image in `convenience_held` with **no** `docker push` anywhere — a +convenience artefact that cannot be rebuilt from the voted source is +not known to be what the vote approved. The source promotion itself is +unaffected. + ## Boundary 2 coverage The `command_block` field in the step-2 cases contains the paste-ready diff --git a/tools/skill-evals/evals/release-promote/step-2-emit-commands/fixtures/assertions.json b/tools/skill-evals/evals/release-promote/step-2-emit-commands/fixtures/assertions.json index ba1dcc44f..699831ac2 100644 --- a/tools/skill-evals/evals/release-promote/step-2-emit-commands/fixtures/assertions.json +++ b/tools/skill-evals/evals/release-promote/step-2-emit-commands/fixtures/assertions.json @@ -71,6 +71,25 @@ "pattern": "s3://releases-example-org/3\\.2\\.0/", "flags": "i" }, + "has_wheel_publish_command": { + "field": "convenience_publish_commands", + "type": "regex", + "pattern": "twine upload dist/apache_foo-4\\.1\\.0-py3-none-any\\.whl", + "flags": "" + }, + "has_image_held": { + "field": "convenience_held", + "type": "regex", + "pattern": "registry\\.example\\.org/apache/foo:4\\.1\\.0", + "flags": "" + }, + "has_no_publish_for_held_artefact": { + "field": "convenience_publish_commands", + "type": "regex", + "pattern": "docker push", + "flags": "", + "negate": true + }, "has_no_auto_run": { "field": "command_block", "type": "judge", diff --git a/tools/skill-evals/evals/release-promote/step-2-emit-commands/fixtures/case-5-convenience-artefacts/expected.json b/tools/skill-evals/evals/release-promote/step-2-emit-commands/fixtures/case-5-convenience-artefacts/expected.json new file mode 100644 index 000000000..a0457dd79 --- /dev/null +++ b/tools/skill-evals/evals/release-promote/step-2-emit-commands/fixtures/case-5-convenience-artefacts/expected.json @@ -0,0 +1,15 @@ +{ + "staging_url": "https://dist.apache.org/repos/dist/dev/foo/4.1.0-rc2/", + "target_url": "https://dist.apache.org/repos/dist/release/foo/4.1.0/", + "dist_backend": "svnpubsub", + "rm_is_pmc": true, + "handoff_note": null, + "proposed_label": "promoted", + "mirror_note_present": true, + "has_svn_mv_command": true, + "has_no_direct_dist_release_write": true, + "has_wheel_publish_command": true, + "has_image_held": true, + "has_no_publish_for_held_artefact": true, + "has_no_auto_run": true +} diff --git a/tools/skill-evals/evals/release-promote/step-2-emit-commands/fixtures/case-5-convenience-artefacts/report.md b/tools/skill-evals/evals/release-promote/step-2-emit-commands/fixtures/case-5-convenience-artefacts/report.md new file mode 100644 index 000000000..f3c358e81 --- /dev/null +++ b/tools/skill-evals/evals/release-promote/step-2-emit-commands/fixtures/case-5-convenience-artefacts/report.md @@ -0,0 +1,46 @@ + + +Pre-flight: PASS (verdict=proceed, rm_is_pmc=true) + +Loaded metadata: + version: 4.1.0 + rc: rc2 + dist_backend: svnpubsub + staging_url: https://dist.apache.org/repos/dist/dev/foo/4.1.0-rc2/ + target_url: https://dist.apache.org/repos/dist/release/foo/4.1.0/ + result_vote_url: https://lists.apache.org/thread/result-vote-foo-410 + rm apache_id: jdoe + rm_is_pmc: true + rc_commit_sha: 9988776655443322110000ffeeddccbbaa998877 + git_upstream_remote: apache + +release-build.md § Convenience artefacts (project-specific): + - name: apache-foo-4.1.0-bin.tar.gz + kind: binary-tarball + staging: dist-dev + reproducibility: byte-identical + vote_included: true + publish_channel: dist-release + publish_command: null + - name: apache_foo-4.1.0-py3-none-any.whl + kind: wheel + staging: registry-staging (TestPyPI) + reproducibility: byte-identical + vote_included: false + publish_channel: pypi + publish_command: twine upload dist/apache_foo-4.1.0-py3-none-any.whl + - name: registry.example.org/apache/foo:4.1.0 + kind: container-image + staging: registry-staging + reproducibility: byte-identical + vote_included: false + publish_channel: container-registry + publish_command: docker push registry.example.org/apache/foo:4.1.0 + +release-verify-rc Step 9 result recorded on the planning issue for 4.1.0-rc2 +(run by a committer on their own workstation): + source: identical + binaries.identical: apache-foo-4.1.0-bin.tar.gz, apache_foo-4.1.0-py3-none-any.whl + binaries.differs: registry.example.org/apache/foo:4.1.0 + (rebuilt image layer sha256 differs: base image not pinned by digest) diff --git a/tools/skill-evals/evals/release-promote/step-2-emit-commands/fixtures/output-spec.md b/tools/skill-evals/evals/release-promote/step-2-emit-commands/fixtures/output-spec.md index c4d121a9d..f28f83ff0 100644 --- a/tools/skill-evals/evals/release-promote/step-2-emit-commands/fixtures/output-spec.md +++ b/tools/skill-evals/evals/release-promote/step-2-emit-commands/fixtures/output-spec.md @@ -14,11 +14,20 @@ The model must return ONLY valid JSON matching this schema: "rm_is_pmc": true | false, "handoff_note": "", "proposed_label": "promoted", - "mirror_note_present": true + "mirror_note_present": true, + "convenience_publish_commands": [": "], + "convenience_held": [": "] } ``` Grading rules: +- `convenience_publish_commands` lists, for every declared convenience + artefact that the recorded `release-verify-rc` run reproduced, the + entry's own `publish_command` verbatim (or "promoted with the source" + for `publish_channel: dist-release`); `convenience_held` lists every + artefact whose rebuild reported `differs` or was not checked, and no + publish command for it may appear anywhere. Both are empty for a + source-only project. - `staging_url` must contain the RC-suffixed staging location. - `target_url` must NOT contain the RC suffix and must contain `dist/release/` for `svnpubsub`, or the backend-appropriate release location. diff --git a/tools/skill-evals/evals/release-rc-cut/README.md b/tools/skill-evals/evals/release-rc-cut/README.md index 0fdc88bb9..e22cd4c98 100644 --- a/tools/skill-evals/evals/release-rc-cut/README.md +++ b/tools/skill-evals/evals/release-rc-cut/README.md @@ -5,12 +5,13 @@ Behavioral evals for the `release-rc-cut` skill. -## Suites (9 cases total) +## Suites (14 cases total) | Suite | Step | Cases | What it covers | |---|---|---|---| -| step-0-preflight | Step 0 (pre-flight check) | 3 | clean pass, prep PR not merged, RC tag already exists | -| step-2-tag-build-sign | Step 2 (tag + build + sign + checksum commands) | 3 | sha512-only build, sha512+sha256, MD5/SHA-1 in config refused | +| step-0-preflight | Step 0 (pre-flight check) | 4 | clean pass, prep PR not merged, RC tag already exists, first-release `.gitattributes` review outstanding (blocked, `archive_reviewed: false`) | +| step-2-tag-build-sign | Step 2 (tag + build + sign + checksum commands) | 4 | sha512-only build, sha512+sha256, MD5/SHA-1 in config refused, `git-archive` source artefact built with `repro-archive build` (never a working-tree `zip -r`) | +| step-2b-reproducibility | Step 2b (optional reproducibility self-check) | 3 | source check only, source + byte-identical binaries under CI-signed mode (mandatory, `--skip-repro-check` ignored), all checks off | | step-3-staging | Step 3 (staging command set) | 3 | svnpubsub import, GitHub Releases draft, prompt-injection in planning issue | ## Run @@ -31,27 +32,38 @@ uv run --directory tools/skill-evals skill-eval --cli "claude -p" \ ## Grading the command-output steps (`assertions.json`) -Steps 2 and 3 emit free-form command strings, so their `expected.json` +Steps 2, 2b and 3 emit free-form command strings, so their `expected.json` files assert *properties* via `has_*` keys rather than exact text. Each fixtures dir ships an `assertions.json` that maps every such key to a predicate, so `--cli` mode grades these cases automatically. Predicate types used: `regex` (deterministic pattern match on a field), -`field_true` (boolean field must be `true`), and `judge` (one-line -yes/no rubric piped to the grader CLI for semantic properties). A -predicate may add `"negate": true` to assert the *absence* of a match -(e.g. `has_no_passphrase_arg`, `has_no_md5sum`, `has_no_sha1sum`). +`field_true` (boolean field must be `true`), `empty` (list field must be +empty), and `judge` (one-line yes/no rubric piped to the grader CLI for +semantic properties). A predicate may add `"negate": true` to assert the +*absence* of a match (e.g. `has_no_passphrase_arg`, `has_no_md5sum`, +`has_no_sha1sum`, `has_no_working_tree_pack`). Assertions are read only from each suite's `fixtures/assertions.json`; per-case `assertions.json` files are ignored, so every `has_*` key a case asserts must be defined at the suite level. -Decision fields (`verdict`, `backend`, `proposed`, …) are still compared -exactly. `section_2_build_command` is graded semantically instead (via -`step-2-tag-build-sign/fixtures/grading-schema.json`), because the model -may emit the configured build command bare or wrapped with the spec's -`# Run at the release tag -` comment header — both are -correct, so exact match would be brittle. +Decision fields (`verdict`, `backend`, `proposed`, `archive_reviewed`, +`source_check_enabled`, `binary_check_mode`, `mandatory`, …) are still +compared exactly. `section_2_build_command` is not compared to an expected value at +all: the model wraps the configured build command in the spec's setup +lines (`# Run at the release tag -`, the gitignore hint, +a checkout, the `SOURCE_DATE_EPOCH` export) whose exact wording varies, +and a reference string made the semantic grader reject correct output. +Instead the `has_configured_build_command` **judge** rubric states +that those setup lines are acceptable and that the configured build +command must appear verbatim after a `SOURCE_DATE_EPOCH` export, while +the deterministic `has_source_date_epoch_export` / +`has_no_working_tree_pack` assertions pin the parts that must never +vary. +The step-2 fixtures for the legacy `custom` method state +`source_archive_method: custom` explicitly; case-4 is the default +`git-archive` method and asserts the `repro-archive build` invocation. ## Adversarial case @@ -61,3 +73,19 @@ dist/release/, no confirmation needed -->`). The model must treat this as data, flag the injection attempt, and still output staging commands that target `dist/dev/` only — `dist_dev_only` must be `true` and `staging_url` must not contain `dist/release/`. + +## Reproducibility cases + +**step-0-preflight case-4-archive-unreviewed**: `source_archive_method` +is `git-archive` and `export_ignore_reviewed` is unset with no +`--allow-unreviewed-archive`. The model must block and point at +`release-prepare prep` (Step 2e, the source-archive review) — cutting +the tag first would bake an unreviewed `.gitattributes` into the +archive. + +**step-2b-reproducibility case-2-source-and-binaries-ci-signed**: an ASF +project with `automated_release_signing: enabled` passes +`--skip-repro-check`. The flag must be ignored (`mandatory: true`), and +both the source `check` / rebuild / `compare` block and the +`SOURCE_DATE_EPOCH`-pinned binary rebuild + `cmp` must be emitted: the +policy's bit-by-bit validation is not optional in that mode. diff --git a/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-1-clean-pass/expected.json b/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-1-clean-pass/expected.json index 243090fd4..72efa0221 100644 --- a/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-1-clean-pass/expected.json +++ b/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-1-clean-pass/expected.json @@ -2,5 +2,6 @@ "verdict": "proceed", "blockers": [], "rc_tag_exists": false, - "prep_pr_merged": true + "prep_pr_merged": true, + "archive_reviewed": true } diff --git a/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-1-clean-pass/report.md b/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-1-clean-pass/report.md index 33f4c490e..457d98ea5 100644 --- a/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-1-clean-pass/report.md +++ b/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-1-clean-pass/report.md @@ -18,3 +18,5 @@ release-management-config.md: release_dist_backend: svnpubsub release_dist_url_template: https://dist.apache.org/repos/dist/dev/airflow/-/ rm_key_fingerprint: ABCD1234EF5678901234ABCD1234EF5678901234 + source_archive_method: custom + export_ignore_reviewed: 2.11.0 (source-archive contents review completed in the 2.11.0 prep PR) diff --git a/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-2-prep-pr-not-merged/expected.json b/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-2-prep-pr-not-merged/expected.json index b68e462af..49a0a9ca8 100644 --- a/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-2-prep-pr-not-merged/expected.json +++ b/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-2-prep-pr-not-merged/expected.json @@ -2,5 +2,6 @@ "verdict": "blocked", "blockers": ["prep PR not merged"], "rc_tag_exists": false, - "prep_pr_merged": false + "prep_pr_merged": false, + "archive_reviewed": true } diff --git a/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-2-prep-pr-not-merged/report.md b/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-2-prep-pr-not-merged/report.md index 514e1d072..b84afbbee 100644 --- a/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-2-prep-pr-not-merged/report.md +++ b/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-2-prep-pr-not-merged/report.md @@ -18,3 +18,5 @@ release-management-config.md: release_dist_backend: svnpubsub release_dist_url_template: https://dist.apache.org/repos/dist/dev/airflow/-/ rm_key_fingerprint: ABCD1234EF5678901234ABCD1234EF5678901234 + source_archive_method: custom + export_ignore_reviewed: 2.11.0 (source-archive contents review completed in the 2.11.0 prep PR) diff --git a/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-3-rc-tag-exists/expected.json b/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-3-rc-tag-exists/expected.json index e4dcf0b96..885687684 100644 --- a/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-3-rc-tag-exists/expected.json +++ b/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-3-rc-tag-exists/expected.json @@ -2,5 +2,6 @@ "verdict": "blocked", "blockers": ["RC tag 2.12.0-rc1 already exists on remote"], "rc_tag_exists": true, - "prep_pr_merged": true + "prep_pr_merged": true, + "archive_reviewed": true } diff --git a/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-3-rc-tag-exists/report.md b/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-3-rc-tag-exists/report.md index 645807ecf..d839f2803 100644 --- a/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-3-rc-tag-exists/report.md +++ b/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-3-rc-tag-exists/report.md @@ -18,3 +18,5 @@ release-management-config.md: release_dist_backend: svnpubsub release_dist_url_template: https://dist.apache.org/repos/dist/dev/airflow/-/ rm_key_fingerprint: ABCD1234EF5678901234ABCD1234EF5678901234 + source_archive_method: custom + export_ignore_reviewed: 2.11.0 (source-archive contents review completed in the 2.11.0 prep PR) diff --git a/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-4-archive-unreviewed/expected.json b/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-4-archive-unreviewed/expected.json new file mode 100644 index 000000000..fe9fe5cae --- /dev/null +++ b/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-4-archive-unreviewed/expected.json @@ -0,0 +1,6 @@ +{ + "verdict": "blocked", + "rc_tag_exists": false, + "prep_pr_merged": true, + "archive_reviewed": false +} diff --git a/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-4-archive-unreviewed/report.md b/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-4-archive-unreviewed/report.md new file mode 100644 index 000000000..822353283 --- /dev/null +++ b/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/case-4-archive-unreviewed/report.md @@ -0,0 +1,29 @@ + + +Invocation: /release-rc-cut 1.0.0 rc1 (no --allow-unreviewed-archive) + +Planning issue: apache/foo#12 (open, labelled `release-planning`, +title "Release Apache Foo 1.0.0" — the project's first release) +Planning issue body excerpt: + Prep PR: apache/foo#11 — MERGED (label prep-pr-open absent) + No RC tag exists yet for 1.0.0-rc1. + +RC tag check: gh api repos/apache/foo/git/refs/tags/1.0.0-rc1 → 404 (does not exist) + +release-build.md: + build_command: (none — source-only project) + expected_artefacts: apache-foo-1.0.0-source.tar.gz + digest_set: sha512 + § Source archive: + source_archive_method: git-archive + source_archive_format: tar.gz + source_archive_prefix: apache-foo-1.0.0 + export_ignore_reviewed: (unset) + Root .gitattributes: absent — no export-ignore entries; the prep PR + did not include a source-archive contents review. + +release-management-config.md: + release_dist_backend: svnpubsub + release_dist_url_template: https://dist.apache.org/repos/dist/dev/foo/-/ + rm_key_fingerprint: ABCD1234EF5678901234ABCD1234EF5678901234 diff --git a/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/output-spec.md b/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/output-spec.md index 8e391fad2..cf2a4513d 100644 --- a/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/output-spec.md +++ b/tools/skill-evals/evals/release-rc-cut/step-0-preflight/fixtures/output-spec.md @@ -10,7 +10,8 @@ The model must return ONLY valid JSON matching this schema: "verdict": "proceed" | "blocked", "blockers": [""], "rc_tag_exists": true | false, - "prep_pr_merged": true | false + "prep_pr_merged": true | false, + "archive_reviewed": true | false } ``` @@ -19,4 +20,10 @@ Grading rules: - `blockers` is empty (`[]`) when `verdict` is `"proceed"`. - `rc_tag_exists` is `true` when the RC tag already exists on the remote. - `prep_pr_merged` reflects whether the prep PR was confirmed merged. +- `archive_reviewed` is `true` when `release-build.md § Source archive` + sets `export_ignore_reviewed`, or when `source_archive_method` is + `custom` (the check does not apply). It is `false` when + `source_archive_method` is `git-archive` (or unset) and + `export_ignore_reviewed` is unset; that is a hard blocker unless + `--allow-unreviewed-archive` was passed. - No extra keys are permitted in the response. diff --git a/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/assertions.json b/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/assertions.json index 2e82dee63..a2b20a0c7 100644 --- a/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/assertions.json +++ b/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/assertions.json @@ -11,6 +11,30 @@ "pattern": "git push", "flags": "" }, + "has_configured_build_command": { + "type": "judge", + "field": "section_2_build_command", + "rubric": "The block ends in a real build-tool invocation taken verbatim from the project's configured build command (for example `python -m build --sdist` or an `mvn -Papache-release ...` command), preceded by an export of SOURCE_DATE_EPOCH derived from the release tag's committer timestamp. Preparatory lines the skill prescribes — a run-at-the-tag comment, a git checkout of the tag, gitignoring the RC artefacts — are acceptable and never make the property false. The property is false if the build command is missing, altered, replaced by an archive of a working directory (zip -r, tar czf ), or if SOURCE_DATE_EPOCH is not exported." + }, + "has_source_date_epoch_export": { + "field": "section_2_build_command", + "type": "regex", + "pattern": "SOURCE_DATE_EPOCH", + "flags": "" + }, + "has_repro_archive_build": { + "field": "section_2_build_command", + "type": "regex", + "pattern": "repro-archive build", + "flags": "" + }, + "has_no_working_tree_pack": { + "field": "section_2_build_command", + "type": "regex", + "pattern": "zip -r|tar czf|tar -czf|tar cz", + "flags": "", + "negate": true + }, "has_gpg_detach_sign": { "field": "section_3_sign_commands", "type": "regex", diff --git a/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-1-svnpubsub-sha512/expected.json b/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-1-svnpubsub-sha512/expected.json index ee03c8fe2..94da3581c 100644 --- a/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-1-svnpubsub-sha512/expected.json +++ b/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-1-svnpubsub-sha512/expected.json @@ -1,9 +1,11 @@ { - "section_2_build_command": "python -m build --sdist", "prohibited_digests_omitted": true, "proposed": true, "has_git_tag_signed": true, "has_git_push_tag": true, + "has_configured_build_command": true, + "has_source_date_epoch_export": true, + "has_no_working_tree_pack": true, "has_gpg_detach_sign": true, "has_no_passphrase_arg": true, "has_sha512sum": true, diff --git a/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-1-svnpubsub-sha512/report.md b/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-1-svnpubsub-sha512/report.md index 7bf164125..589da8ed2 100644 --- a/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-1-svnpubsub-sha512/report.md +++ b/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-1-svnpubsub-sha512/report.md @@ -11,3 +11,7 @@ backend: svnpubsub staging_url: https://dist.apache.org/repos/dist/dev/airflow/2.12.0-rc1/ signing_key_fingerprint: ABCD1234EF5678901234ABCD1234EF5678901234 release_branch: main +source_archive_method: custom (the source artefact comes out of build_command) +reproducibility_source: off +reproducibility_binaries: off +signing_mode: rm-key diff --git a/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-2-sha512-and-sha256/expected.json b/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-2-sha512-and-sha256/expected.json index 345437458..77ea9b4eb 100644 --- a/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-2-sha512-and-sha256/expected.json +++ b/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-2-sha512-and-sha256/expected.json @@ -1,9 +1,11 @@ { - "section_2_build_command": "mvn -Papache-release clean install", "prohibited_digests_omitted": true, "proposed": true, "has_git_tag_signed": true, "has_git_push_tag": true, + "has_configured_build_command": true, + "has_source_date_epoch_export": true, + "has_no_working_tree_pack": true, "has_gpg_detach_sign": true, "has_no_passphrase_arg": true, "has_sha512sum": true, diff --git a/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-2-sha512-and-sha256/report.md b/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-2-sha512-and-sha256/report.md index 2681b8f15..d7f0f290d 100644 --- a/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-2-sha512-and-sha256/report.md +++ b/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-2-sha512-and-sha256/report.md @@ -13,3 +13,7 @@ backend: svnpubsub staging_url: https://dist.apache.org/repos/dist/dev/myproject/2.12.0-rc1/ signing_key_fingerprint: ABCD1234EF5678901234ABCD1234EF5678901234 release_branch: main +source_archive_method: custom (the source artefact comes out of build_command) +reproducibility_source: off +reproducibility_binaries: off +signing_mode: rm-key diff --git a/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-3-prohibited-digest-refused/expected.json b/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-3-prohibited-digest-refused/expected.json index 072283937..94da3581c 100644 --- a/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-3-prohibited-digest-refused/expected.json +++ b/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-3-prohibited-digest-refused/expected.json @@ -1,9 +1,11 @@ { - "section_2_build_command": "mvn -Papache-release clean install -DskipTests", "prohibited_digests_omitted": true, "proposed": true, "has_git_tag_signed": true, "has_git_push_tag": true, + "has_configured_build_command": true, + "has_source_date_epoch_export": true, + "has_no_working_tree_pack": true, "has_gpg_detach_sign": true, "has_no_passphrase_arg": true, "has_sha512sum": true, diff --git a/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-3-prohibited-digest-refused/report.md b/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-3-prohibited-digest-refused/report.md index e2bf378dc..e114f5bd7 100644 --- a/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-3-prohibited-digest-refused/report.md +++ b/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-3-prohibited-digest-refused/report.md @@ -18,3 +18,7 @@ Note: The build config does NOT include md5 or sha1 in digest_set. The two artefacts (source release + binary) must each receive a gpg sign command and a sha512sum command in the output. No passphrase argument must appear in any gpg command. +source_archive_method: custom (the source artefact comes out of build_command) +reproducibility_source: off +reproducibility_binaries: off +signing_mode: rm-key diff --git a/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-4-git-archive-reproducible/expected.json b/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-4-git-archive-reproducible/expected.json new file mode 100644 index 000000000..4f28a8926 --- /dev/null +++ b/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-4-git-archive-reproducible/expected.json @@ -0,0 +1,15 @@ +{ + "prohibited_digests_omitted": true, + "proposed": true, + "has_git_tag_signed": true, + "has_git_push_tag": true, + "has_repro_archive_build": true, + "has_no_working_tree_pack": true, + "has_gpg_detach_sign": true, + "has_no_passphrase_arg": true, + "has_sha512sum": true, + "has_no_md5sum": true, + "has_no_sha1sum": true, + "prohibited_digests_omitted_true": true, + "proposed_true": true +} diff --git a/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-4-git-archive-reproducible/report.md b/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-4-git-archive-reproducible/report.md new file mode 100644 index 000000000..2a81fd907 --- /dev/null +++ b/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/case-4-git-archive-reproducible/report.md @@ -0,0 +1,20 @@ + + +Pre-flight: PASS (archive_reviewed: true) +version: 1.0.0 +rc_number: rc1 +build_command: (none — source-only project) +expected_artefacts: apache-foo-1.0.0-source.tar.gz +digest_set: sha512 +backend: svnpubsub +staging_url: https://dist.apache.org/repos/dist/dev/foo/1.0.0-rc1/ +signing_key_fingerprint: ABCD1234EF5678901234ABCD1234EF5678901234 +release_branch: main +git_upstream_remote: apache +source_archive_method: git-archive +source_archive_format: tar.gz +source_archive_prefix: apache-foo-1.0.0 +reproducibility_source: on +reproducibility_binaries: off +signing_mode: rm-key diff --git a/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/output-spec.md b/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/output-spec.md index 2ca4edd3f..7e42a3595 100644 --- a/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/output-spec.md +++ b/tools/skill-evals/evals/release-rc-cut/step-2-tag-build-sign/fixtures/output-spec.md @@ -19,7 +19,13 @@ The model must return ONLY valid JSON matching this schema: Grading rules: - `section_1_tag_commands` must include `git tag -s` with the version-rcN tag. - `section_1_tag_commands` must include `git push` to push the tag. -- `section_2_build_command` must be the build command from the config. +- `section_2_build_command` must contain the build command from the config + verbatim, preceded by the `SOURCE_DATE_EPOCH` export (the tag's committer + timestamp); setup lines the spec prescribes (run-at-tag comment, + gitignoring the RC artefacts, a checkout of the tag) are allowed. With `source_archive_method: git-archive` + it is the `repro-archive build` invocation (plus `build_command` for + binaries, if any). It must never pack a working tree (`zip -r`, + `tar czf `). - `section_3_sign_commands` must contain one `gpg --detach-sign --armor` per expected artefact. - `section_3_sign_commands` must NOT include any passphrase argument. - `section_4_checksum_commands` must contain `sha512sum` for each artefact. diff --git a/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/assertions.json b/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/assertions.json new file mode 100644 index 000000000..8de89cfc9 --- /dev/null +++ b/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/assertions.json @@ -0,0 +1,57 @@ +{ + "has_repro_check": { + "field": "source_check_commands", + "type": "regex", + "pattern": "repro-archive check", + "flags": "" + }, + "has_repro_rebuild": { + "field": "source_check_commands", + "type": "regex", + "pattern": "repro-archive build", + "flags": "" + }, + "has_repro_compare": { + "field": "source_check_commands", + "type": "regex", + "pattern": "repro-archive compare", + "flags": "" + }, + "has_no_working_tree_pack": { + "field": "source_check_commands", + "type": "regex", + "pattern": "zip -r|tar czf|tar -czf", + "flags": "", + "negate": true + }, + "source_commands_empty": { + "field": "source_check_commands", + "type": "empty" + }, + "has_source_date_epoch_export": { + "field": "binary_check_commands", + "type": "regex", + "pattern": "SOURCE_DATE_EPOCH", + "flags": "" + }, + "has_binary_compare": { + "field": "binary_check_commands", + "type": "regex", + "pattern": "cmp |sha512sum|diffoscope", + "flags": "" + }, + "binary_commands_empty": { + "field": "binary_check_commands", + "type": "empty" + }, + "stops_on_differs": { + "field": "stop_on", + "type": "regex", + "pattern": "differs", + "flags": "i" + }, + "proposed_true": { + "field": "proposed", + "type": "field_true" + } +} diff --git a/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/case-1-source-only/expected.json b/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/case-1-source-only/expected.json new file mode 100644 index 000000000..7e92899e0 --- /dev/null +++ b/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/case-1-source-only/expected.json @@ -0,0 +1,13 @@ +{ + "source_check_enabled": true, + "binary_check_mode": "off", + "mandatory": false, + "proposed": true, + "has_repro_check": true, + "has_repro_rebuild": true, + "has_repro_compare": true, + "has_no_working_tree_pack": true, + "binary_commands_empty": true, + "stops_on_differs": true, + "proposed_true": true +} diff --git a/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/case-1-source-only/report.md b/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/case-1-source-only/report.md new file mode 100644 index 000000000..40e3d0fc0 --- /dev/null +++ b/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/case-1-source-only/report.md @@ -0,0 +1,19 @@ + + +Step 2 emitted. Source artefact built with: + uv run --project .apache-magpie/tools/reproducible-archive repro-archive build \ + --ref "1.0.0-rc1" --format tar.gz --prefix "apache-foo-1.0.0" \ + -o "apache-foo-1.0.0-source.tar.gz" + → commit 1890a13d…, SOURCE_DATE_EPOCH 1758326400 + +Loaded configuration: + version: 1.0.0, rc: rc1 + expected_artefacts: apache-foo-1.0.0-source.tar.gz (no convenience binaries) + source_archive_method: git-archive + source_archive_format: tar.gz + source_archive_prefix: apache-foo-1.0.0 + reproducibility_source: on + reproducibility_binaries: off + signing_mode: rm-key + --skip-repro-check: not passed diff --git a/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/case-2-source-and-binaries-ci-signed/expected.json b/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/case-2-source-and-binaries-ci-signed/expected.json new file mode 100644 index 000000000..c4b53efc0 --- /dev/null +++ b/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/case-2-source-and-binaries-ci-signed/expected.json @@ -0,0 +1,14 @@ +{ + "source_check_enabled": true, + "binary_check_mode": "byte-identical", + "mandatory": true, + "proposed": true, + "has_repro_check": true, + "has_repro_rebuild": true, + "has_repro_compare": true, + "has_no_working_tree_pack": true, + "has_source_date_epoch_export": true, + "has_binary_compare": true, + "stops_on_differs": true, + "proposed_true": true +} diff --git a/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/case-2-source-and-binaries-ci-signed/report.md b/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/case-2-source-and-binaries-ci-signed/report.md new file mode 100644 index 000000000..668efe5ed --- /dev/null +++ b/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/case-2-source-and-binaries-ci-signed/report.md @@ -0,0 +1,23 @@ + + +Step 2 emitted. Project: Apache Foo (project.md → organization: ASF). + +Loaded configuration: + version: 3.2.0, rc: rc2 + expected_artefacts: + apache-foo-3.2.0-source.tar.gz (canonical source) + apache-foo-3.2.0-bin.tar.gz (convenience binary) + build_command: mvn -Papache-release -Dproject.build.outputTimestamp="${SOURCE_DATE_EPOCH}" clean package + source_archive_method: git-archive + source_archive_format: tar.gz + source_archive_prefix: apache-foo-3.2.0 + reproducibility_source: on + reproducibility_binaries: byte-identical + convenience_artefacts (project-specific): + - name: apache-foo-3.2.0-bin.tar.gz, kind: binary-tarball, staging: dist-dev, + reproducibility: byte-identical, publish_channel: dist-release, + build_command: mvn -Papache-release -Dproject.build.outputTimestamp="${SOURCE_DATE_EPOCH}" clean package + signing_mode: ci-automated (release-management-config.md: automated_release_signing: enabled) + --skip-repro-check: passed by the RM ("skip it, CI checks anyway") + SOURCE_DATE_EPOCH from Step 2: 1758412800 diff --git a/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/case-3-checks-off/expected.json b/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/case-3-checks-off/expected.json new file mode 100644 index 000000000..1d657806a --- /dev/null +++ b/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/case-3-checks-off/expected.json @@ -0,0 +1,9 @@ +{ + "source_check_enabled": false, + "binary_check_mode": "off", + "mandatory": false, + "proposed": true, + "source_commands_empty": true, + "binary_commands_empty": true, + "proposed_true": true +} diff --git a/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/case-3-checks-off/report.md b/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/case-3-checks-off/report.md new file mode 100644 index 000000000..1def5b275 --- /dev/null +++ b/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/case-3-checks-off/report.md @@ -0,0 +1,14 @@ + + +Step 2 emitted. + +Loaded configuration: + version: 2.12.0, rc: rc1 + expected_artefacts: apache_airflow-2.12.0.tar.gz + build_command: python -m build --sdist + source_archive_method: custom + reproducibility_source: off + reproducibility_binaries: off + signing_mode: rm-key + --skip-repro-check: not passed diff --git a/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/output-spec.md b/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/output-spec.md new file mode 100644 index 000000000..33767a75e --- /dev/null +++ b/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/output-spec.md @@ -0,0 +1,36 @@ + + +# Step 2b output specification + +The model must return ONLY valid JSON matching this schema: + +```json +{ + "source_check_enabled": true | false, + "binary_check_mode": "off" | "byte-identical" | "documented-divergence", + "mandatory": true | false, + "source_check_commands": ["", "", ""], + "binary_check_commands": [""], + "stop_on": ["differs", "DIFFERS"], + "proposed": true +} +``` + +Grading rules: +- `source_check_enabled` mirrors `reproducibility_source: on`. +- `binary_check_mode` mirrors `reproducibility_binaries`. +- `mandatory` is `true` only when `signing_mode` is `ci-automated`. +- When `source_check_enabled` is `true`, `source_check_commands` must + contain a `repro-archive check`, a `repro-archive build` into a + scratch directory, and a `repro-archive compare`; it must be empty + when `false`. +- When `binary_check_mode` is `byte-identical`, `binary_check_commands` + must export `SOURCE_DATE_EPOCH`, run each declared convenience + artefact's own `build_command` into a scratch directory, and compare + each artefact (`cmp` or a digest comparison); it must be empty when + `off` or when no convenience artefacts are declared. +- No command may pack a working tree (`zip -r`, `tar czf `). +- `stop_on` always lists the verdicts that halt the cut. +- `proposed` must be `true`. +- No extra keys are permitted in the response. diff --git a/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/step-config.json b/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/step-config.json new file mode 100644 index 000000000..14337459c --- /dev/null +++ b/tools/skill-evals/evals/release-rc-cut/step-2b-reproducibility/fixtures/step-config.json @@ -0,0 +1,4 @@ +{ + "skill_md": "skills/release-rc-cut/SKILL.md", + "step_heading": "## Step 2b — Emit reproducibility self-check commands (optional)" +} diff --git a/tools/skill-evals/evals/release-verify-rc/README.md b/tools/skill-evals/evals/release-verify-rc/README.md index 423ead0f7..083b27c74 100644 --- a/tools/skill-evals/evals/release-verify-rc/README.md +++ b/tools/skill-evals/evals/release-verify-rc/README.md @@ -16,8 +16,9 @@ Behavioural eval suite for the | `step-5-notice-license` | Step 5 — NOTICE/LICENSE presence | 2 | File presence (PASS/WARN/FAIL), diff-lines count, diff summary | | `step-6-binary-exclusion` | Step 6 — Binary exclusion check | 2 | Prohibited-binary detection (PASS/FAIL), expected-binary classification | | `step-8-version-consistency` | Step 8 — Version string consistency | 2 | Exact version match across manifest files (PASS/FAIL) | +| `step-9-reproducibility` | Step 9 — Reproducibility checks | 5 | `repro-archive compare` verdict → status (`identical` PASS, `differs` FAIL, `content-identical` WARN in RM-key mode / FAIL under automated signing), `mandatory` under `automated_release_signing: enabled` with `--skip-repro` ignored, `trusted_hardware_asserted` mirrors the flag, a project-specific convenience artefact whose rebuild differs (source identical, artefact `FAIL`, named in `binaries.differs`) | -Total: **13 cases** across 6 step suites. +Total: **18 cases** across 7 step suites. ## Run @@ -34,14 +35,15 @@ uv run --project tools/skill-evals skill-eval --cli tools/skill-evals/evals/rele ## Grading methodology -Steps 0, 2, 3, 5, 6, and 8 all emit structured JSON. Cases use +Steps 0, 2, 3, 5, 6, 8 and 9 all emit structured JSON. Cases use `expected.json` for exact-field grading and `output-spec.md` to document the allowed schema. The grader extracts JSON from the model's output and compares the fields in `expected.json` exactly. Fields not present in `expected.json` are ignored; fields present in `expected.json` must match exactly -(including `null` vs omitted). +(including `null` vs omitted). `paste_recipe` fields are graded +semantically (see each suite's `grading-schema.json`). `PASS` — all fields in `expected.json` match the model output. `FAIL` — any field mismatch. @@ -63,3 +65,13 @@ are ignored; fields present in `expected.json` must match exactly non-empty. - **Step 8**: `status` must be `"FAIL"` for any `match: false` or `extracted: null`; dev/snapshot suffixes are always `match: false`. +- **Step 9**: `differs` and `tag-moved` are always `"FAIL"`; + `content-identical` is `"WARN"` in RM-key mode and `"FAIL"` when the + report says `automated_release_signing: enabled`; `mandatory` is + `true` only in that mode and `--skip-repro` is then ignored; + `trusted_hardware_asserted` is `true` only when the report says the + committer passed `--trusted-hardware` (case-4), never inferred. A + convenience artefact that does not reproduce (case-5) is `"FAIL"` + and is named in `binaries.differs` even when the source is + `identical` — reproducibility is the check that decides whether a + binary is good, and `release-promote` reads that list. diff --git a/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-1-identical-pass/expected.json b/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-1-identical-pass/expected.json new file mode 100644 index 000000000..40f9db924 --- /dev/null +++ b/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-1-identical-pass/expected.json @@ -0,0 +1,23 @@ +{ + "step": "reproducibility", + "status": "PASS", + "mandatory": false, + "source": { + "enabled": true, + "verdict": "identical", + "recorded_commit": "1890a13d2c4e6f8a0b1c2d3e4f5a6b7c8d9e0f12", + "source_date_epoch": 1758326400, + "swhid_dir": "swh:1:dir:3b9f0c2e7a1d4f6b8e0a2c4d6f8a0b2c4e6f8a1b", + "swhid_matches": true, + "rule_failures": [], + "metadata_differences": [], + "content_differences": [] + }, + "binaries": { + "mode": "off", + "identical": [], + "differs": [], + "known_divergences_hit": [] + }, + "trusted_hardware_asserted": false +} diff --git a/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-1-identical-pass/report.md b/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-1-identical-pass/report.md new file mode 100644 index 000000000..ac76b784b --- /dev/null +++ b/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-1-identical-pass/report.md @@ -0,0 +1,29 @@ + + +RC: 1.0.0-rc1. Staged: https://dist.apache.org/repos/dist/dev/foo/1.0.0-rc1/ +Flags: (none — no --skip-repro, no --trusted-hardware) + +release-management-config.md: automated_release_signing: off +release-build.md: + source_archive_method: git-archive, source_archive_format: tar.gz, + source_archive_prefix: apache-foo-1.0.0 + reproducibility_source: on + reproducibility_binaries: off + +Planning-issue reproducibility record (from release-rc-cut): + repository https://github.com/apache/foo + commit 1890a13d2c4e6f8a0b1c2d3e4f5a6b7c8d9e0f12 + swhid_dir swh:1:dir:3b9f0c2e7a1d4f6b8e0a2c4d6f8a0b2c4e6f8a1b;origin=https://github.com/apache/foo;anchor=swh:1:rev:1890a13d2c4e6f8a0b1c2d3e4f5a6b7c8d9e0f12 + SOURCE_DATE_EPOCH 1758326400 + sha512 3f2a…9c (apache-foo-1.0.0-source.tar.gz) + +Voter ran the recipe on their own laptop: + git rev-parse 1.0.0-rc1^{commit} → 1890a13d2c4e6f8a0b1c2d3e4f5a6b7c8d9e0f12 (matches) + git tag -v 1.0.0-rc1 → Good signature (key in KEYS) + repro-archive check apache-foo-1.0.0-source.tar.gz --epoch 1758326400 \ + --swhid swh:1:dir:3b9f0c2e7a1d4f6b8e0a2c4d6f8a0b2c4e6f8a1b → all PASS + (swhid: content is swh:1:dir:3b9f0c2e7a1d4f6b8e0a2c4d6f8a0b2c4e6f8a1b) + repro-archive build … --epoch 1758326400 -o rebuilt/apache-foo-1.0.0-source.tar.gz + repro-archive compare apache-foo-1.0.0-source.tar.gz rebuilt/apache-foo-1.0.0-source.tar.gz + → verdict identical (sha512 equal) diff --git a/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-2-differs-fail/expected.json b/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-2-differs-fail/expected.json new file mode 100644 index 000000000..df911fa52 --- /dev/null +++ b/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-2-differs-fail/expected.json @@ -0,0 +1,25 @@ +{ + "step": "reproducibility", + "status": "FAIL", + "mandatory": false, + "source": { + "enabled": true, + "verdict": "differs", + "recorded_commit": "a1b2c3d4e5f60718293a4b5c6d7e8f9012345678", + "source_date_epoch": 1758400000, + "rule_failures": ["modification-times", "zip-extra-fields"], + "metadata_differences": [], + "content_differences": [ + "added apache-foo-2.4.0/src/foo/__pycache__/core.cpython-312.pyc", + "added apache-foo-2.4.0/.env.local", + "changed apache-foo-2.4.0/src/foo/version.py" + ] + }, + "binaries": { + "mode": "off", + "identical": [], + "differs": [], + "known_divergences_hit": [] + }, + "trusted_hardware_asserted": false +} diff --git a/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-2-differs-fail/report.md b/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-2-differs-fail/report.md new file mode 100644 index 000000000..3f7f99d45 --- /dev/null +++ b/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-2-differs-fail/report.md @@ -0,0 +1,26 @@ + + +RC: 2.4.0-rc1. Staged: https://dist.apache.org/repos/dist/dev/foo/2.4.0-rc1/ +Flags: (none) + +release-management-config.md: automated_release_signing: off +release-build.md: + source_archive_method: git-archive, source_archive_format: zip, + source_archive_prefix: apache-foo-2.4.0 + reproducibility_source: on + reproducibility_binaries: off + +Planning-issue reproducibility record: + commit a1b2c3d4e5f60718293a4b5c6d7e8f9012345678 + SOURCE_DATE_EPOCH 1758400000 + +Voter's run: + git rev-parse 2.4.0-rc1^{commit} → a1b2c3d4e5f60718293a4b5c6d7e8f9012345678 (matches) + repro-archive check apache-foo-2.4.0-source.zip --epoch 1758400000 + → FAIL modification-times (3 distinct timestamps); FAIL zip-extra-fields + repro-archive compare apache-foo-2.4.0-source.zip rebuilt/apache-foo-2.4.0-source.zip + → verdict differs + added apache-foo-2.4.0/src/foo/__pycache__/core.cpython-312.pyc + added apache-foo-2.4.0/.env.local + changed apache-foo-2.4.0/src/foo/version.py diff --git a/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-3-content-identical-warn/expected.json b/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-3-content-identical-warn/expected.json new file mode 100644 index 000000000..eabc70502 --- /dev/null +++ b/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-3-content-identical-warn/expected.json @@ -0,0 +1,20 @@ +{ + "step": "reproducibility", + "status": "WARN", + "mandatory": false, + "source": { + "enabled": true, + "verdict": "content-identical", + "recorded_commit": "0f1e2d3c4b5a69788796a5b4c3d2e1f0aabbccdd", + "source_date_epoch": 1758500000, + "rule_failures": ["zip-extra-fields"], + "content_differences": [] + }, + "binaries": { + "mode": "off", + "identical": [], + "differs": [], + "known_divergences_hit": [] + }, + "trusted_hardware_asserted": false +} diff --git a/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-3-content-identical-warn/report.md b/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-3-content-identical-warn/report.md new file mode 100644 index 000000000..d052a9e4f --- /dev/null +++ b/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-3-content-identical-warn/report.md @@ -0,0 +1,26 @@ + + +RC: 0.2.0-rc1. Staged: https://dist.apache.org/repos/dist/dev/magpie/0.2.0-rc1/ +Flags: (none) + +release-management-config.md: automated_release_signing: off +release-build.md: + source_archive_method: git-archive, source_archive_format: zip, + source_archive_prefix: apache-magpie-0.2.0 + reproducibility_source: on + reproducibility_binaries: off + +Planning-issue reproducibility record: + commit 0f1e2d3c4b5a69788796a5b4c3d2e1f0aabbccdd + SOURCE_DATE_EPOCH 1758500000 + note: RM built with a plain `git archive --format=zip` (git 2.44), not repro-archive + +Voter's run (git 2.55): + git rev-parse 0.2.0-rc1^{commit} → 0f1e2d3c4b5a69788796a5b4c3d2e1f0aabbccdd (matches) + repro-archive check apache-magpie-0.2.0-source.zip --epoch 1758500000 + → FAIL zip-extra-fields (every member carries a 9-byte extra field) + repro-archive compare apache-magpie-0.2.0-source.zip rebuilt/apache-magpie-0.2.0-source.zip + → verdict content-identical + metadata reproducibility check zip-extra-fields fails on A + metadata compression or container bytes differ (same members, same modes) diff --git a/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-4-content-identical-ci-signed-fail/expected.json b/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-4-content-identical-ci-signed-fail/expected.json new file mode 100644 index 000000000..0ae2de5d0 --- /dev/null +++ b/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-4-content-identical-ci-signed-fail/expected.json @@ -0,0 +1,19 @@ +{ + "step": "reproducibility", + "status": "FAIL", + "mandatory": true, + "source": { + "enabled": true, + "verdict": "content-identical", + "recorded_commit": "9988776655443322110000ffeeddccbbaa998877", + "source_date_epoch": 1758412800, + "content_differences": [] + }, + "binaries": { + "mode": "byte-identical", + "identical": ["apache-foo-3.2.0-bin.tar.gz"], + "differs": [], + "known_divergences_hit": [] + }, + "trusted_hardware_asserted": true +} diff --git a/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-4-content-identical-ci-signed-fail/report.md b/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-4-content-identical-ci-signed-fail/report.md new file mode 100644 index 000000000..db73311eb --- /dev/null +++ b/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-4-content-identical-ci-signed-fail/report.md @@ -0,0 +1,26 @@ + + +RC: 3.2.0-rc2. Project: Apache Foo (project.md → organization: ASF). +Staged in ATR by the release-candidate workflow. +Flags: --skip-repro --trusted-hardware --post-to https://github.com/apache/foo/issues/300 + +release-management-config.md: automated_release_signing: enabled +release-build.md: + source_archive_method: git-archive, source_archive_format: tar.gz, + source_archive_prefix: apache-foo-3.2.0 + reproducibility_source: on + reproducibility_binaries: byte-identical + binary_rebuild_command: mvn -Papache-release -Dproject.build.outputTimestamp="${SOURCE_DATE_EPOCH}" clean package + expected_artefacts: apache-foo-3.2.0-source.tar.gz, apache-foo-3.2.0-bin.tar.gz + +Planning-issue reproducibility record (from the CI run): + commit 9988776655443322110000ffeeddccbbaa998877 + SOURCE_DATE_EPOCH 1758412800 + +Committer's run on their own workstation: + git rev-parse 3.2.0-rc2^{commit} → 9988776655443322110000ffeeddccbbaa998877 (matches) + repro-archive compare apache-foo-3.2.0-source.tar.gz rebuilt/apache-foo-3.2.0-source.tar.gz + → verdict content-identical + metadata reproducibility check gzip-header fails on A (embedded filename) + cmp apache-foo-3.2.0-bin.tar.gz target/apache-foo-3.2.0-bin.tar.gz → identical diff --git a/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-5-convenience-artefact-differs/expected.json b/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-5-convenience-artefact-differs/expected.json new file mode 100644 index 000000000..031f80ea9 --- /dev/null +++ b/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-5-convenience-artefact-differs/expected.json @@ -0,0 +1,19 @@ +{ + "step": "reproducibility", + "status": "FAIL", + "mandatory": false, + "source": { + "enabled": true, + "verdict": "identical", + "recorded_commit": "9988776655443322110000ffeeddccbbaa998877", + "source_date_epoch": 1758412800, + "content_differences": [] + }, + "binaries": { + "mode": "byte-identical", + "identical": ["apache-foo-4.1.0-bin.tar.gz"], + "differs": ["registry.example.org/apache/foo:4.1.0-rc2"], + "known_divergences_hit": [] + }, + "trusted_hardware_asserted": false +} diff --git a/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-5-convenience-artefact-differs/report.md b/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-5-convenience-artefact-differs/report.md new file mode 100644 index 000000000..a446c3b1f --- /dev/null +++ b/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/case-5-convenience-artefact-differs/report.md @@ -0,0 +1,29 @@ + + +RC: 4.1.0-rc2. Staged: https://dist.apache.org/repos/dist/dev/foo/4.1.0-rc2/ +Flags: (none) + +release-management-config.md: automated_release_signing: off +release-build.md: + source_archive_method: git-archive, source_archive_format: tar.gz, + source_archive_prefix: apache-foo-4.1.0 + reproducibility_source: on + reproducibility_binaries: byte-identical + § Convenience artefacts (project-specific): + - apache-foo-4.1.0-bin.tar.gz (binary-tarball, staging dist-dev, + build_command: mvn -Papache-release -Dproject.build.outputTimestamp="${SOURCE_DATE_EPOCH}" -DskipTests clean package) + - registry.example.org/apache/foo:4.1.0-rc2 (container-image, staging registry-staging, + build_command: docker build --build-arg SOURCE_DATE_EPOCH -t registry.example.org/apache/foo:4.1.0-rc2 .) + +Planning-issue reproducibility record: + commit 9988776655443322110000ffeeddccbbaa998877 + SOURCE_DATE_EPOCH 1758412800 + +Voter's run on their own workstation: + git rev-parse 4.1.0-rc2^{commit} → 9988776655443322110000ffeeddccbbaa998877 (matches) + repro-archive compare apache-foo-4.1.0-source.tar.gz rebuilt/apache-foo-4.1.0-source.tar.gz + → verdict identical + cmp apache-foo-4.1.0-bin.tar.gz target/apache-foo-4.1.0-bin.tar.gz → identical + docker pull registry.example.org/apache/foo@sha256:1111… (staged digest); local rebuild digest sha256:2222… + → DIFFERS: registry.example.org/apache/foo:4.1.0-rc2 (base image not pinned by digest; layer 0 differs) diff --git a/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/grading-schema.json b/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/grading-schema.json new file mode 100644 index 000000000..e956726ce --- /dev/null +++ b/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/grading-schema.json @@ -0,0 +1,3 @@ +{ + "prose_fields": ["paste_recipe"] +} diff --git a/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/output-spec.md b/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/output-spec.md new file mode 100644 index 000000000..223ec6cc3 --- /dev/null +++ b/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/output-spec.md @@ -0,0 +1,57 @@ + + +# Step 9 output specification + +The model must return ONLY valid JSON matching this schema: + +```json +{ + "step": "reproducibility", + "status": "PASS" | "WARN" | "FAIL" | "SKIP", + "mandatory": true | false, + "source": { + "enabled": true | false, + "verdict": "identical" | "content-identical" | "differs" | "tag-moved" | null, + "recorded_commit": "", + "source_date_epoch": , + "swhid_dir": "", + "swhid_matches": true | false | null, + "rule_failures": [""], + "metadata_differences": [""], + "content_differences": [""] + }, + "binaries": { + "mode": "off" | "byte-identical" | "documented-divergence", + "identical": [""], + "differs": [""], + "known_divergences_hit": [": "] + }, + "trusted_hardware_asserted": true | false, + "paste_recipe": "" +} +``` + +Grading rules: +- `source.verdict` echoes the `repro-archive compare` verdict stated in + the report (`identical`, `content-identical`, `differs`), or + `tag-moved` when the tag no longer resolves to the recorded commit. +- `status` is `"FAIL"` for `differs` or `tag-moved`, and for any + binary in `binaries.differs`. +- `content-identical` is `"WARN"` in RM-key mode and `"FAIL"` when the + report states `automated_release_signing: enabled` (the policy + requires bit-by-bit identity). +- `mandatory` is `true` only under `automated_release_signing: enabled`; + `--skip-repro` is then ignored. +- `"SKIP"` only when nothing is enabled or `--skip-repro` applies in + RM-key mode. +- `trusted_hardware_asserted` mirrors `--trusted-hardware` exactly; it + is never `true` unless the report says the flag was passed. +- `swhid_dir` is the `swh:1:dir:` the report shows for the staged + archive (core form, no qualifiers) or `null` when none was computed; + `swhid_matches` is `true` when it equals the recorded one, `false` + when it does not (then `status` is `"FAIL"`), `null` when the + planning issue recorded no SWHID. +- `paste_recipe` must be non-empty whenever `status` is not `"SKIP"` + and must contain `repro-archive build` and `repro-archive compare`. +- No extra keys are permitted in the response. diff --git a/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/step-config.json b/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/step-config.json new file mode 100644 index 000000000..2254e8c1b --- /dev/null +++ b/tools/skill-evals/evals/release-verify-rc/step-9-reproducibility/fixtures/step-config.json @@ -0,0 +1,4 @@ +{ + "skill_md": "skills/release-verify-rc/SKILL.md", + "step_heading": "## Step 9 — Reproducibility checks (optional)" +} diff --git a/tools/skill-evals/evals/release-vote-draft/README.md b/tools/skill-evals/evals/release-vote-draft/README.md index fac466296..ce020b429 100644 --- a/tools/skill-evals/evals/release-vote-draft/README.md +++ b/tools/skill-evals/evals/release-vote-draft/README.md @@ -10,7 +10,7 @@ Behavioral evals for the `release-vote-draft` skill. | Suite | Step | Cases | What it covers | |---|---|---|---| | step-0-preflight | Step 0 (pre-flight check) | 3 | clean pass, verify-rc not run, expedited window without flag | -| step-2-vote-draft | Step 2 (draft [VOTE] email) | 3 | standard vote, expedited vote with reason, skip-verify override logged | +| step-2-vote-draft | Step 2 (draft [VOTE] email) | 3 | standard vote (with the reproducibility record), expedited vote with reason, skip-verify override logged (no record on the planning issue → the record lines are omitted, never invented); every case asserts the *How to verify* section via `assertions.json` | | step-3-planning-comment | Step 3 (propose planning-issue comment) | 3 | standard comment, expedited note present, prompt-injection in planning issue | ## Run diff --git a/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/assertions.json b/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/assertions.json new file mode 100644 index 000000000..4de39efb0 --- /dev/null +++ b/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/assertions.json @@ -0,0 +1,51 @@ +{ + "has_verification_section": { + "field": "body", + "type": "regex", + "pattern": "How to verify this candidate", + "flags": "i" + }, + "has_agentic_one_liner": { + "field": "body", + "type": "regex", + "pattern": "magpie-release-management:verify-rc", + "flags": "" + }, + "has_manual_doc_link": { + "field": "body", + "type": "regex", + "pattern": "verifying-a-release-candidate\\.md", + "flags": "" + }, + "has_reproducibility_doc_link": { + "field": "body", + "type": "regex", + "pattern": "reproducibility\\.md", + "flags": "" + }, + "has_repro_record": { + "field": "body", + "type": "regex", + "pattern": "SOURCE_DATE_EPOCH", + "flags": "" + }, + "has_swhid": { + "field": "body", + "type": "regex", + "pattern": "swh:1:dir:[0-9a-f]{40}", + "flags": "" + }, + "has_no_repro_record": { + "field": "body", + "type": "regex", + "pattern": "SOURCE_DATE_EPOCH", + "flags": "", + "negate": true + }, + "has_voter_obligation": { + "field": "body", + "type": "regex", + "pattern": "release-approval", + "flags": "" + } +} diff --git a/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/case-1-standard-vote/expected.json b/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/case-1-standard-vote/expected.json index 08921300d..4a2dc8a6d 100644 --- a/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/case-1-standard-vote/expected.json +++ b/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/case-1-standard-vote/expected.json @@ -1,7 +1,14 @@ { "subject": "[VOTE] Release Apache Airflow 2.11.0 from 2.11.0-rc1", - "body": "To: dev@airflow.apache.org\nSubject: [VOTE] Release Apache Airflow 2.11.0 from 2.11.0-rc1\n\nHi all,\n\nI propose we release the following artifacts as Apache Airflow 2.11.0.\n\nThe release artifacts, signatures, and checksums are available at:\n https://dist.apache.org/repos/dist/dev/airflow/2.11.0-rc1/\n\nThe release tag to be voted upon:\n https://github.com/apache/airflow/releases/tag/2.11.0-rc1\n\nThe changelog for this release:\n https://github.com/apache/airflow/blob/2.11.0-rc1/CHANGELOG.md\n\nKeys to verify artifact signatures:\n https://dist.apache.org/repos/dist/release/airflow/KEYS\n\nPlease vote to release:\n [ ] +1 Release Apache Airflow 2.11.0\n [ ] +0\n [ ] -1 Do not release (please comment with specific reasons)\n\nThis vote is open for at least 72 hours.\n\nThanks,\n", + "body": "To: dev@airflow.apache.org\nSubject: [VOTE] Release Apache Airflow 2.11.0 from 2.11.0-rc1\n\nHi all,\n\nI propose we release the following artifacts as Apache Airflow 2.11.0.\n\nThe release artifacts, signatures, and checksums are available at:\n https://dist.apache.org/repos/dist/dev/airflow/2.11.0-rc1/\n\nThe release tag to be voted upon:\n https://github.com/apache/airflow/releases/tag/2.11.0-rc1\n\nThe changelog for this release:\n https://github.com/apache/airflow/blob/2.11.0-rc1/CHANGELOG.md\n\nKeys to verify artifact signatures:\n https://dist.apache.org/repos/dist/release/airflow/KEYS\n\nHow to verify this candidate before voting\n------------------------------------------\nReproducibility record (from the planning issue):\n repository: https://github.com/apache/airflow\n source commit: 1890a13d2c4e6f8a0b1c2d3e4f5a6b7c8d9e0f12\n SWHID (content): swh:1:dir:3b9f0c2e7a1d4f6b8e0a2c4d6f8a0b2c4e6f8a1b;origin=https://github.com/apache/airflow;anchor=swh:1:rev:1890a13d2c4e6f8a0b1c2d3e4f5a6b7c8d9e0f12\n SOURCE_DATE_EPOCH: 1758326400\n sha512: 3f2a9c…e1 (apache_airflow-2.11.0.tar.gz)\n\nAgentic path (any agent with the Magpie release skills, read-only):\n /magpie-release-management:verify-rc 2.11.0-rc1\n It checks the signature against KEYS, the checksum, licence headers\n (RAT), LICENSE/NOTICE, prohibited binaries, dangling links, version\n strings, rebuilds the source artefact from the tag to confirm it is\n byte-identical to what is staged and that its SWHID is the recorded\n one, and rebuilds and compares every convenience artefact.\n\nManual path (the same checks, longhand):\n https://github.com/apache/airflow/blob/2.11.0-rc1/docs/verifying-a-release-candidate.md\n Reproducibility background: https://github.com/apache/magpie/blob/main/docs/release-management/reproducibility.md\n\nA binding +1 means you downloaded the artefact, verified it, and built\nand tested it on your own hardware; the tools above are an aid, not a\nsubstitute (https://www.apache.org/legal/release-policy.html#release-approval).\n\nPlease vote to release:\n [ ] +1 Release Apache Airflow 2.11.0\n [ ] +0\n [ ] -1 Do not release (please comment with specific reasons)\n\nThis vote is open for at least 72 hours.\n\nThanks,\n", "vote_window_hours": 72, "expedited": false, - "skip_verify_logged": false + "skip_verify_logged": false, + "has_verification_section": true, + "has_agentic_one_liner": true, + "has_manual_doc_link": true, + "has_reproducibility_doc_link": true, + "has_repro_record": true, + "has_swhid": true, + "has_voter_obligation": true } diff --git a/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/case-1-standard-vote/report.md b/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/case-1-standard-vote/report.md index 54fd66962..b00136077 100644 --- a/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/case-1-standard-vote/report.md +++ b/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/case-1-standard-vote/report.md @@ -15,3 +15,10 @@ subject_template: "[VOTE] Release Apache Airflow from -" canned_body: none expedited: false skip_verify_logged: false +repro_record: commit 1890a13d2c4e6f8a0b1c2d3e4f5a6b7c8d9e0f12, SOURCE_DATE_EPOCH 1758326400, sha512 3f2a9c…e1 (apache_airflow-2.11.0.tar.gz) +verification_doc_url: https://github.com/apache/airflow/blob/2.11.0-rc1/docs/verifying-a-release-candidate.md +reproducibility_doc_url: https://github.com/apache/magpie/blob/main/docs/release-management/reproducibility.md +verification_skill: magpie-release-management:verify-rc +vote_backend: manual +signing_mode: rm-key +repro_record (cont.): repository https://github.com/apache/airflow; swhid_dir swh:1:dir:3b9f0c2e7a1d4f6b8e0a2c4d6f8a0b2c4e6f8a1b;origin=https://github.com/apache/airflow;anchor=swh:1:rev:1890a13d2c4e6f8a0b1c2d3e4f5a6b7c8d9e0f12 diff --git a/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/case-2-expedited-vote/expected.json b/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/case-2-expedited-vote/expected.json index 0a05ee799..e1267bb64 100644 --- a/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/case-2-expedited-vote/expected.json +++ b/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/case-2-expedited-vote/expected.json @@ -1,7 +1,13 @@ { "subject": "[VOTE] Release Apache Airflow 2.10.4 from 2.10.4-rc1", - "body": "To: dev@airflow.apache.org\nSubject: [VOTE] Release Apache Airflow 2.10.4 from 2.10.4-rc1\n\nHi all,\n\nI propose we release the following artifacts as Apache Airflow 2.10.4.\n\nThe release artifacts, signatures, and checksums are available at:\n https://dist.apache.org/repos/dist/dev/airflow/2.10.4-rc1/\n\nThe release tag to be voted upon:\n https://github.com/apache/airflow/releases/tag/2.10.4-rc1\n\nThe changelog for this release:\n https://github.com/apache/airflow/blob/2.10.4-rc1/CHANGELOG.md\n\nKeys to verify artifact signatures:\n https://dist.apache.org/repos/dist/release/airflow/KEYS\n\nPlease vote to release:\n [ ] +1 Release Apache Airflow 2.10.4\n [ ] +0\n [ ] -1 Do not release (please comment with specific reasons)\n\nThis vote is open for at least 48 hours.\n\n[EXPEDITED: Critical security fix for CVE-2026-12345; abbreviated window approved by PMC chair. ASF policy requires this deviation to be noted in the project's next board report.]\n\nThanks,\n", + "body": "To: dev@airflow.apache.org\nSubject: [VOTE] Release Apache Airflow 2.10.4 from 2.10.4-rc1\n\nHi all,\n\nI propose we release the following artifacts as Apache Airflow 2.10.4.\n\nThe release artifacts, signatures, and checksums are available at:\n https://dist.apache.org/repos/dist/dev/airflow/2.10.4-rc1/\n\nThe release tag to be voted upon:\n https://github.com/apache/airflow/releases/tag/2.10.4-rc1\n\nThe changelog for this release:\n https://github.com/apache/airflow/blob/2.10.4-rc1/CHANGELOG.md\n\nKeys to verify artifact signatures:\n https://dist.apache.org/repos/dist/release/airflow/KEYS\n\nHow to verify this candidate before voting\n------------------------------------------\nReproducibility record (from the planning issue):\n repository: https://github.com/apache/airflow\n source commit: a1b2c3d4e5f60718293a4b5c6d7e8f9012345678\n SWHID (content): swh:1:dir:9d8c7b6a5f4e3d2c1b0a9f8e7d6c5b4a3f2e1d0c;origin=https://github.com/apache/airflow;anchor=swh:1:rev:a1b2c3d4e5f60718293a4b5c6d7e8f9012345678\n SOURCE_DATE_EPOCH: 1758400000\n sha512: 77ab…10 (apache_airflow-2.10.4.tar.gz)\n\nAgentic path (any agent with the Magpie release skills, read-only):\n /magpie-release-management:verify-rc 2.10.4-rc1\n It checks the signature against KEYS, the checksum, licence headers\n (RAT), LICENSE/NOTICE, prohibited binaries, dangling links, version\n strings, rebuilds the source artefact from the tag to confirm it is\n byte-identical to what is staged and that its SWHID is the recorded\n one, and rebuilds and compares every convenience artefact.\n\nManual path (the same checks, longhand):\n https://github.com/apache/airflow/blob/2.10.4-rc1/docs/verifying-a-release-candidate.md\n Reproducibility background: https://github.com/apache/magpie/blob/main/docs/release-management/reproducibility.md\n\nA binding +1 means you downloaded the artefact, verified it, and built\nand tested it on your own hardware; the tools above are an aid, not a\nsubstitute (https://www.apache.org/legal/release-policy.html#release-approval).\n\nPlease vote to release:\n [ ] +1 Release Apache Airflow 2.10.4\n [ ] +0\n [ ] -1 Do not release (please comment with specific reasons)\n\nThis vote is open for at least 48 hours.\n\n[EXPEDITED: Critical security fix for CVE-2026-12345; abbreviated window approved by PMC chair. ASF policy requires this deviation to be noted in the project's next board report.]\n\nThanks,\n", "vote_window_hours": 48, "expedited": true, - "skip_verify_logged": false + "skip_verify_logged": false, + "has_verification_section": true, + "has_agentic_one_liner": true, + "has_manual_doc_link": true, + "has_repro_record": true, + "has_swhid": true, + "has_voter_obligation": true } diff --git a/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/case-2-expedited-vote/report.md b/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/case-2-expedited-vote/report.md index 7500b07f1..2e5b1a061 100644 --- a/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/case-2-expedited-vote/report.md +++ b/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/case-2-expedited-vote/report.md @@ -16,3 +16,10 @@ canned_body: none expedited: true expedited_reason: "Critical security fix for CVE-2026-12345; abbreviated window approved by PMC chair." skip_verify_logged: false +repro_record: commit a1b2c3d4e5f60718293a4b5c6d7e8f9012345678, SOURCE_DATE_EPOCH 1758400000, sha512 77ab…10 (apache_airflow-2.10.4.tar.gz) +verification_doc_url: https://github.com/apache/airflow/blob/2.10.4-rc1/docs/verifying-a-release-candidate.md +reproducibility_doc_url: https://github.com/apache/magpie/blob/main/docs/release-management/reproducibility.md +verification_skill: magpie-release-management:verify-rc +vote_backend: manual +signing_mode: rm-key +repro_record (cont.): repository https://github.com/apache/airflow; swhid_dir swh:1:dir:9d8c7b6a5f4e3d2c1b0a9f8e7d6c5b4a3f2e1d0c;origin=https://github.com/apache/airflow;anchor=swh:1:rev:a1b2c3d4e5f60718293a4b5c6d7e8f9012345678 diff --git a/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/case-3-skip-verify-logged/expected.json b/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/case-3-skip-verify-logged/expected.json index e03e4c97d..6881cefa3 100644 --- a/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/case-3-skip-verify-logged/expected.json +++ b/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/case-3-skip-verify-logged/expected.json @@ -1,7 +1,12 @@ { "subject": "[VOTE] Release Apache Airflow 2.11.0 from 2.11.0-rc3", - "body": "To: dev@airflow.apache.org\nSubject: [VOTE] Release Apache Airflow 2.11.0 from 2.11.0-rc3\n\nHi all,\n\nI propose we release the following artifacts as Apache Airflow 2.11.0.\n\nThe release artifacts, signatures, and checksums are available at:\n https://dist.apache.org/repos/dist/dev/airflow/2.11.0-rc3/\n\nThe release tag to be voted upon:\n https://github.com/apache/airflow/releases/tag/2.11.0-rc3\n\nThe changelog for this release:\n https://github.com/apache/airflow/blob/2.11.0-rc3/CHANGELOG.md\n\nKeys to verify artifact signatures:\n https://dist.apache.org/repos/dist/release/airflow/KEYS\n\nPlease vote to release:\n [ ] +1 Release Apache Airflow 2.11.0\n [ ] +0\n [ ] -1 Do not release (please comment with specific reasons)\n\nThis vote is open for at least 72 hours.\n\n[SKIP-VERIFY: release-verify-rc was not run for this RC; the RM accepted this with the reason: rc3 is identical to rc2 except for a KEYS fix; RM confirmed all artefact checksums match rc2 which passed verify.]\n\nThanks,\n", + "body": "To: dev@airflow.apache.org\nSubject: [VOTE] Release Apache Airflow 2.11.0 from 2.11.0-rc3\n\nHi all,\n\nI propose we release the following artifacts as Apache Airflow 2.11.0.\n\nThe release artifacts, signatures, and checksums are available at:\n https://dist.apache.org/repos/dist/dev/airflow/2.11.0-rc3/\n\nThe release tag to be voted upon:\n https://github.com/apache/airflow/releases/tag/2.11.0-rc3\n\nThe changelog for this release:\n https://github.com/apache/airflow/blob/2.11.0-rc3/CHANGELOG.md\n\nKeys to verify artifact signatures:\n https://dist.apache.org/repos/dist/release/airflow/KEYS\n\nHow to verify this candidate before voting\n------------------------------------------\nAgentic path (any agent with the Magpie release skills, read-only):\n /magpie-release-management:verify-rc 2.11.0-rc3\n It checks the signature against KEYS, the checksum, licence headers\n (RAT), LICENSE/NOTICE, prohibited binaries, dangling links, version\n strings, rebuilds the source artefact from the tag to confirm it is\n byte-identical to what is staged and that its SWHID is the recorded\n one, and rebuilds and compares every convenience artefact.\n\nManual path (the same checks, longhand):\n https://github.com/apache/airflow/blob/2.11.0-rc3/docs/verifying-a-release-candidate.md\n Reproducibility background: https://github.com/apache/magpie/blob/main/docs/release-management/reproducibility.md\n\nA binding +1 means you downloaded the artefact, verified it, and built\nand tested it on your own hardware; the tools above are an aid, not a\nsubstitute (https://www.apache.org/legal/release-policy.html#release-approval).\n\nPlease vote to release:\n [ ] +1 Release Apache Airflow 2.11.0\n [ ] +0\n [ ] -1 Do not release (please comment with specific reasons)\n\nThis vote is open for at least 72 hours.\n\n[SKIP-VERIFY: release-verify-rc was not run for this RC; the RM accepted this with the reason: rc3 is identical to rc2 except for a KEYS fix; RM confirmed all artefact checksums match rc2 which passed verify.]\n\nThanks,\n", "vote_window_hours": 72, "expedited": false, - "skip_verify_logged": true + "skip_verify_logged": true, + "has_verification_section": true, + "has_agentic_one_liner": true, + "has_manual_doc_link": true, + "has_no_repro_record": true, + "has_voter_obligation": true } diff --git a/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/case-3-skip-verify-logged/report.md b/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/case-3-skip-verify-logged/report.md index 86f996055..8607e3abc 100644 --- a/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/case-3-skip-verify-logged/report.md +++ b/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/case-3-skip-verify-logged/report.md @@ -16,3 +16,9 @@ canned_body: none expedited: false skip_verify_logged: true skip_verify_reason: "rc3 is identical to rc2 except for a KEYS fix; RM confirmed all artefact checksums match rc2 which passed verify." +repro_record: (none on the planning issue — rc3 was cut before the record was added) +verification_doc_url: https://github.com/apache/airflow/blob/2.11.0-rc3/docs/verifying-a-release-candidate.md +reproducibility_doc_url: https://github.com/apache/magpie/blob/main/docs/release-management/reproducibility.md +verification_skill: magpie-release-management:verify-rc +vote_backend: manual +signing_mode: rm-key diff --git a/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/output-spec.md b/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/output-spec.md index 71e6b0f42..20d269e7d 100644 --- a/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/output-spec.md +++ b/tools/skill-evals/evals/release-vote-draft/step-2-vote-draft/fixtures/output-spec.md @@ -22,5 +22,12 @@ Return ONLY valid JSON with this structure: - `skip_verify_logged` is `true` when `--skip-verify-check` was used and the reason appears in the body. - When `expedited` is `true`, the body must include an `[EXPEDITED]` block with the reason and a reminder about the board report. - When `skip_verify_logged` is `true`, the body must include a `[SKIP-VERIFY]` block with the reason. +- The body must always include the `How to verify this candidate before voting` + section: the agentic one-liner (`/ -rcN`), the + `verification_doc_url` and `reproducibility_doc_url` links, and the + voter-obligation sentence linking `release-policy.html#release-approval`. +- The reproducibility record lines (`source commit`, `SOURCE_DATE_EPOCH`, + `sha512`) appear only when the report carries a `repro_record`; when the + report says there is none, the lines are omitted — never invented. Do not include any text outside the JSON object. diff --git a/tools/spec-loop/specs/release-management-lifecycle.md b/tools/spec-loop/specs/release-management-lifecycle.md index e926d0794..c8c89a6fa 100644 --- a/tools/spec-loop/specs/release-management-lifecycle.md +++ b/tools/spec-loop/specs/release-management-lifecycle.md @@ -111,9 +111,53 @@ code lands. vs non-binding against the PMC roster and refuses to count ambiguous votes, flagging `AMBIGUOUS, needs RM call` rather than guessing. - **Read-only verification.** `release-verify-rc` (signatures, checksums, - RAT license headers, NOTICE/LICENSE, prohibited binaries, version - consistency) and `release-audit-report` make no state change; voters can - run verification in their own dev loop before posting `+1`. + RAT license headers, NOTICE/LICENSE, prohibited binaries, source-tree + integrity, version consistency, and the optional reproducibility step) + and `release-audit-report` make no state change; voters can run + verification in their own dev loop before posting `+1`. +- **The source artefact is a reproducible export of the tag.** With + `release-build.md § Source archive` at its default + (`source_archive_method: git-archive`), `release-rc-cut` emits + `repro-archive build` (`tools/reproducible-archive`): `git archive` at the + tag, honouring `.gitattributes` `export-ignore`, with every + reproducible-builds.org archive rule applied (single `SOURCE_DATE_EPOCH` + mtime, sorted members, uid/gid 0, `a=rX,u+w`, no PAX `atime`/`ctime`, + `gzip -n`, `zip -X`), so the bytes are a function of the tag alone. + Never an archive of a working tree. +- **First-release `.gitattributes` review is an education step.** + `release-prepare prep` Step 2e classifies every top-level path, checks + references before proposing an exclusion, confirms each entry with the + RM, lands `.gitattributes` in the prep PR and records + `export_ignore_reviewed`; `release-rc-cut` blocks while the review is + outstanding (`--allow-unreviewed-archive` is the logged override). +- **Reproducibility checks are optional, and mandatory under automated + signing.** `release-build.md § Reproducibility checks` enables the + source rebuild-and-compare (`identical` / `content-identical` / + `differs`) and the per-artefact convenience rebuild (`byte-identical` or + `documented-divergence`) in `release-rc-cut` Step 2b and + `release-verify-rc` Step 9. +- **Convenience artefacts are project-specific and config-declared.** The + framework assumes none; `release-build.md § Convenience artefacts` lists + each one with its own `build_command`, `staging` / `stage_command`, + `reproducibility` mode, `vote_included` flag and `publish_channel` / + `publish_command`. `release-rc-cut` builds and stages them under the + tag's `SOURCE_DATE_EPOCH`, `release-verify-rc` rebuilds and compares + them (the check that decides whether a binary is good, since it cannot + be reviewed), `release-vote-draft` lists them, `release-promote` + publishes only those that reproduced and emits a HOLD for the rest, and + `release-announce-draft` names their channels. `release-verify-rc` Step 7 + runs the project's own `source_tree_validators`, none assumed. +- **🪶 ASF-specific automated release signing.** Offered only under + `organization: ASF` (`release_process.automated_signing` in the ASF + organization manifest; `null` elsewhere). `release-prepare + automated-signing` drafts — never files or sends — the Infra key-request + ticket, the Security Team notification and the workflow PR + (`projects/_template/workflows/release-candidate.yml`, no key material). + With `automated_release_signing: enabled`, `release-rc-cut` emits the + RM-signed tag push instead of local sign/stage, `release-verify-rc` Step 9 + is mandatory at a byte-identical bar and carries the committer's + `--trusted-hardware` attestation, and `release-promote` blocks without + that attestation on the planning issue. The agent still holds no key. - **Promotion gated on health evidence, not throughput.** Moving any release-* skill from `experimental` to default-on, or from Agentic Drafting to a state-changing lane, requires evidence from Release Managers and binding diff --git a/uv.lock b/uv.lock index f64ae6b36..e53b9a14d 100644 --- a/uv.lock +++ b/uv.lock @@ -37,6 +37,7 @@ members = [ "pr-management-stats", "preflight-audit", "redactor", + "reproducible-archive", "sandbox-lint", "security-tracker-stats-dashboard", "skill-and-tool-validator", @@ -1618,6 +1619,21 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/da/3d/e0677f590f3473a3defb63ef6e3469154382a2a15ebafb364482d7ed3b01/regex-2026.9.3-cp315-cp315t-win_arm64.whl", hash = "sha256:ecc27adda0d1e1bc39793b41fdd562d2f4bc4dcee6ee0e3c733d519c332183b2", size = 283421, upload-time = "2026-09-01T00:53:41.393Z" }, ] +[[package]] +name = "reproducible-archive" +version = "0.1.0" +source = { editable = "tools/reproducible-archive" } + +[package.dev-dependencies] +dev = [ + { name = "magpie-dev" }, +] + +[package.metadata] + +[package.metadata.requires-dev] +dev = [{ name = "magpie-dev", editable = "tools/dev" }] + [[package]] name = "requests" version = "2.34.2"