Skip to content

feat(release/auto-changelog): update release changelogs deterministically from a predecessor baseline - #303

Open
HowardBraham wants to merge 1 commit into
clean-room-proofreaderfrom
deterministic-updater
Open

HowardBraham wants to merge 1 commit into
clean-room-proofreaderfrom
deterministic-updater

Conversation

@HowardBraham

@HowardBraham HowardBraham commented Oct 26, 2026 •

Copy link
Copy Markdown
Contributor

References

Explanation

Current state

update-release-changelog.sh regenerates the release section on every push to a release branch, from a previous-version ref the caller has to supply. It overwrites edits a reviewer made in the open changelog PR, has no defined behavior when the AI step fails, and create-platform-release-pr.sh creates the same changelog PR a second time, racing the push workflow for the same branch.

Solution

update-release-changelog.mts replaces the shell script. On each run it does the following:

  • It picks a baseline: the target release branch once it has the section, otherwise the predecessor version on stable or on its release branch. It rebuilds release-changelog/<version> from the release tip with that baseline's CHANGELOG.md.
  • It generates the new section with auto-changelog (--preventBackfill), skips PRs already documented, and restores the generator's categories after proofreading. An entry's leading verb picks its category, and long PR links count as documented.
  • It keeps the section a reviewer edited in the open PR. Only new entries go to the clean-room proofreader, so a reviewer's wording, deletions and moves survive later pushes.
  • It validates the result, with validate --rc once proofreading succeeded, and checks for whitespace errors.
  • It pushes with a lease pinned to the branch tip it read, so a concurrent change fails the run. It skips the push when CHANGELOG.md is unchanged, so other release-branch commits don't replace the PR commit and dismiss approvals.
  • When the AI step fails, it keeps the deterministic section, titles the PR (AI automation failed, not proofread) and writes a report. The release is never failed by the model.

AI failure and retry

A failed run records Changelog-Proofread: failed in its commit message. When the PR is still untouched (its only commit is the updater's, with that marker), the next run proofreads the whole section again instead of only the new entries. Without the marker, a run that folded a reviewer's edits into its own commit would be mistaken for untouched and would overwrite them. A branch left without a PR (for example a failed gh pr create) is regenerated when only the updater committed to it, and otherwise refused.

Other changes

  • Release PR script: create-platform-release-pr.sh no longer creates the changelog PR. It only generates the test plan, and the push workflow owns the changelog. It also runs from the action's own checkout (GITHUB_TOOLS_DIR) instead of a second github-tools checkout, so the action and script come from the same revision.
  • Actions: update-release-changelog installs Node 24 and the github-tools dependencies and passes the AI key only to the proofreader step. The old update-release-changelog.sh is deleted.
  • Dependencies: @metamask/auto-changelog goes from ^6.2.0 to ^7.0.1.
  • Helpers: release-version-utils.mts holds branch parsing and version comparison. merge-previous-release-changelog.mts now uses its compareSemver.
  • Tests: update-release-changelog.test.mts, …-baseline.test.mts, …-git.test.mts (throwaway local origin and clone), release-section.test.mts and release-version-utils.test.mts.

Testing

  • Forks, with the action pinned to a SHA (2598a03):
    • consensys-test/metamask-mobile-howard:
      • Ran against real Mobile history, about 100 PRs for 8.18.0.
      • Covered a first run, new PRs after a run, and reviewer edits kept across runs, including deleted and moved entries.
      • Covered an invalid key, a timeout, and a failure followed by a retry.
      • Covered a failure followed by a reviewer edit and more PRs, where the edit survives.
      • A push that changes only another file leaves the changelog branch at the same commit.
    • consensys-test/metamask-extension-howard:
      • Ran against real Extension history, 96 PRs on top of v13.53.0, with stable carrying upstream's changelog.
      • Fresh run: 3 proofread chunks, no Uncategorized.
      • Reviewer edit, then 10 new PRs: only the new entries were proofread, and the edit survived.
      • Push that changes only another file: the changelog branch was not pushed.
      • Invalid key: the PR was titled "not proofread" and the commit carried the marker.
      • Valid key: the whole section was proofread again.

Note

Medium Risk
Changes core release automation (changelog generation, git pushes with force-with-lease, and PR creation), so regressions could affect every release branch push or duplicate/race changelog PRs; behavior is heavily tested but still high-impact CI.

Overview
Replaces release changelog updates with update-release-changelog.mts, which rebuilds release-changelog/<version> from a predecessor baseline (target branch, stable, or prior release branch), runs @metamask/auto-changelog v7 with --preventBackfill, and merges only new PR entries while keeping reviewer edits from the open changelog PR. Clean-room AI proofreading applies to new entries only; failures keep the deterministic section, mark the PR as not proofread, and upload a report artifact.

Release creation no longer opens a changelog PR or checks out github-tools separately—create-platform-release-pr.sh only builds the test plan; the push workflow owns the changelog. Composite actions run scripts from the pinned action checkout via GITHUB_TOOLS_DIR / github.action_path, with Node 24 and scoped LiteLLM credentials.

Adds release-section, baseline, and version-utils helpers (PR deduping, category restore, leased pushes, hotfix baseline trimming). The legacy update-release-changelog.sh forwards to the TypeScript entrypoint for older callers.

Reviewed by Cursor Bugbot for commit 9194aea. Bugbot is set up for automated code reviews on this repo. Configure here.

@HowardBraham HowardBraham self-assigned this Oct 26, 2026
@HowardBraham
HowardBraham added this pull request to stack #302 October 26, 2026 19:35
@socket-security

socket-security Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Updatedsemver@​7.7.2 ⏵ 7.8.5100 +1100100 +188100
Updated@​metamask/​auto-changelog@​6.2.0 ⏵ 7.0.198100100 +196 +2100

View full report

@socket-security

socket-security Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

All alerts resolved. Learn more about Socket for GitHub.

This PR previously contained dependency changes with security issues that have been resolved, removed, or ignored.

Ignoring alerts on:

  • get-stream@8.0.1
  • yargs@17.7.3

View full report

@HowardBraham

Copy link
Copy Markdown
Contributor Author

@SocketSecurity ignore npm/get-stream@8.0.1

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread .github/scripts/release-section.mts Outdated
@HowardBraham

Copy link
Copy Markdown
Contributor Author

@SocketSecurity ignore npm/yargs@17.7.3

@HowardBraham
HowardBraham force-pushed the deterministic-updater branch from 2598a03 to 45372b5 Compare October 26, 2026 21:20

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread .github/scripts/release-section.mts
@HowardBraham
HowardBraham force-pushed the deterministic-updater branch 2 times, most recently from c4d7e15 to f239d85 Compare October 26, 2026 21:53

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread .github/scripts/update-release-changelog.mts
@HowardBraham
HowardBraham force-pushed the deterministic-updater branch from f239d85 to 08630c6 Compare October 26, 2026 22:08

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread .github/scripts/update-release-changelog.mts
@HowardBraham
HowardBraham force-pushed the deterministic-updater branch from 08630c6 to 2168c0d Compare October 26, 2026 22:28

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread .github/scripts/release-section.mts Outdated
Comment thread .github/scripts/update-release-changelog-baseline.mts
@HowardBraham
HowardBraham force-pushed the deterministic-updater branch from 2168c0d to adaeded Compare October 26, 2026 22:55

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread .github/scripts/update-release-changelog.mts
@HowardBraham
HowardBraham force-pushed the deterministic-updater branch from adaeded to eb7fc7f Compare October 26, 2026 23:17

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit eb7fc7f. Configure here.

Comment thread .github/scripts/update-release-changelog.mts Outdated
@HowardBraham
HowardBraham force-pushed the deterministic-updater branch from eb7fc7f to 3cefe69 Compare October 26, 2026 23:41

@mcmire mcmire left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have not reviewed changes to scripts, but as far as the risk to other repos goes, this looks pretty safe to me. I don't believe other repos use the create-release-pr or update-release-changelog actions. (We do have a create-release-pr action which lives under http://github.com/MetaMask/action-create-release-pr, and we do use this in other repos, but that is completely separate from the one here.)

@HowardBraham
HowardBraham force-pushed the deterministic-updater branch from 3cefe69 to 9113097 Compare October 27, 2026 21:14
…ally from a predecessor baseline

Replace update-release-changelog.sh with a TypeScript updater that builds the release section from the predecessor release's changelog plus the new PRs, keeps reviewer edits on an open changelog PR, and sends only new entries to the clean-room proofreader.
@HowardBraham
HowardBraham force-pushed the deterministic-updater branch from 9113097 to 9194aea Compare October 27, 2026 23:12

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants