Repository navigation
GH-50492: [R] Add release process skill #50499
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
10 commits
Select commit
Hold shift + click to select a range
94c46b7
initial commit of skill
thisisnic 50cc811
Update release skill
thisisnic f31580e
Apply suggestion from @thisisnic
thisisnic ce89616
Add contributor stats script
thisisnic 5245400
Preserve ref to unsetting ARROW_HOME
thisisnic abeb666
Never push, always ask user
thisisnic 6ec153c
Check in re the release vote
thisisnic fcc12e6
Don't harcode version
thisisnic 8562573
Tweak contirbutor script
thisisnic c6624d0
Mkae portable
thisisnic File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,395 @@ | ||
| # R CRAN Release | ||
|
|
||
| Guide the R package maintainer through the CRAN release process for the Apache Arrow R package. Only use this skill when explicitly doing a CRAN release. | ||
|
|
||
| Use exactly this sequence of steps. Print a checklist of all steps at the start, and update it as you go. Do not skip ahead. After completing each step, ask the user to confirm before moving on. Once confirmed, update the corresponding checkbox on the tracking issue. Use exactly the commands and approaches specified in each step — do not improvise or substitute alternatives without checking with the user. | ||
|
|
||
| Never run `git push` yourself, under any circumstances. Whenever a step requires pushing, show the user the exact command to run and wait for them to confirm they have pushed before continuing. The `git push` commands in this file are for the user to run, not you. | ||
|
|
||
| If any earlier step reveals something that needs to be cherry-picked into the release branch, note it as a comment on the tracking issue. When you reach the cherry-pick step later, check the tracking issue comments for anything noted earlier. | ||
|
|
||
| ## 1. Create GitHub Tracking Issue | ||
|
|
||
| Ask the user for the release version number. | ||
|
|
||
| ```bash | ||
| gh issue create --repo apache/arrow \ | ||
| --title "[R] CRAN packaging checklist for version <VERSION>" \ | ||
| --body "$(cat r/PACKAGING.md | sed -n '/^- \[ \]/,$p')" | ||
| ``` | ||
|
|
||
| Track the issue number - update checkboxes as you complete each step. | ||
|
|
||
| ## 2. Create CRAN Release Branch | ||
|
|
||
| First check whether the final release tag exists: | ||
|
|
||
| ```bash | ||
| git fetch upstream --tags | ||
| git tag -l 'apache-arrow-<VERSION>*' | ||
| ``` | ||
|
|
||
| If `apache-arrow-<VERSION>` (no rc suffix) exists, the vote has already passed. Always branch from the final tag in this case — do not ask about RCs, as an earlier RC may be a different commit from the final release: | ||
|
|
||
| ```bash | ||
| git checkout apache-arrow-<VERSION> | ||
| ``` | ||
|
|
||
| Only if the final tag does not exist yet (vote still in progress), ask the user which RC number to use (e.g., rc1, rc2), then branch from it: | ||
|
|
||
| ```bash | ||
| git checkout apache-arrow-<VERSION>-rc<N> | ||
| ``` | ||
|
|
||
| Confirm with the user before creating the branch, then ask them to run the push command themselves: | ||
|
|
||
| ```bash | ||
| git checkout -b maint-<VERSION>-r | ||
| git push upstream maint-<VERSION>-r | ||
| ``` | ||
|
|
||
| All subsequent steps should be done on this branch. | ||
|
|
||
| ## 3. Remove Badges from README | ||
|
|
||
| In `r/README.md`, delete everything between `<!-- badges: start -->` and `<!-- badges: end -->` (inclusive): | ||
|
|
||
| ```bash | ||
| sed -i.bak '/<!-- badges: start -->/,/<!-- badges: end -->/d' r/README.md && rm r/README.md.bak | ||
| ``` | ||
|
|
||
| Commit this change to the `maint-<VERSION>-r` branch. | ||
|
|
||
| ## 4. Review Deprecated Functions | ||
|
|
||
| Find functions using `.Deprecated()` that may need to advance (deprecated -> defunct/removed): | ||
|
|
||
| ```bash | ||
| grep -rn "\.Deprecated" r/R/*.R | ||
| ``` | ||
|
|
||
| Review each match and decide if the deprecation should advance for this release (e.g., remove the function entirely or change to `.Defunct()`). | ||
|
|
||
| ## 5. Evaluate Nightly Build Status | ||
|
|
||
| Ask the user to check that R nightly builds were passing around RC time. They can check on Zulip or at https://crossbow.arrow-dev.org/ | ||
|
|
||
| ## 6. Check Current CRAN Check Results | ||
|
|
||
| Fetch https://cran.r-project.org/web/checks/check_results_arrow.html and extract the check results table showing platform, version, and status. Also check for any "Additional issues" section. | ||
|
|
||
| All platforms should show OK or NOTE status. NOTEs about package size (e.g., "installed size is 130+ Mb") are expected due to bundled Arrow C++ and can be ignored. Other NOTEs or any ERROR/WARN should be investigated. | ||
|
|
||
| ## 7. Ensure README is Accurate | ||
|
|
||
| Read `r/README.md` and verify: | ||
| - Installation instructions are current | ||
| - Feature descriptions match current functionality | ||
| - Version-specific notes (e.g., C++ version requirements) are correct | ||
| - No outdated information | ||
|
|
||
| Report any issues found. | ||
|
|
||
| ## 8. Run URL Checker | ||
|
|
||
| Confirm on the `maint-<VERSION>-r` branch: | ||
|
|
||
| ```bash | ||
| git branch --show-current | ||
| ``` | ||
|
|
||
| Then run: | ||
|
|
||
| ```bash | ||
| cd r && Rscript -e 'urlchecker::url_check()' | ||
| ``` | ||
|
|
||
| All URLs should pass (badges were already removed). Fix any broken links. | ||
|
|
||
| ## 9. Polish NEWS | ||
|
|
||
| Review `r/NEWS.md` and polish following tidyverse style (see https://style.tidyverse.org/news.html): | ||
|
|
||
| - Use present tense ("X now does Y", not "X did Y") | ||
| - Name contributors with `@username` if they're not a listed package author. Listed authors (do not credit): @nealrichardson, @ianmcook, @thisisnic, @paleolimbot, @romainfrancois, @jkeane, @brycemecum, @dragosmg, @jeroenooms, @assignUser | ||
| - Use categories: "New features", "Minor improvements and fixes", "Installation" (if relevant) | ||
| - Keep entries concise - match the style of previous releases | ||
| - Only include user-facing changes - no CI updates or internal refactoring | ||
|
|
||
| Find the previous version from NEWS.md: | ||
|
|
||
| ```bash | ||
| grep "^# arrow" r/NEWS.md | head -5 | ||
| ``` | ||
|
|
||
| Then find R commits since that version: | ||
|
|
||
| ```bash | ||
| git log --oneline apache-arrow-<PREVIOUS_VERSION>..HEAD | grep "\[R\]" | ||
| ``` | ||
|
|
||
| Do NOT update version numbers - this is done automatically later. | ||
|
|
||
| Open a GitHub issue for the NEWS updates, submit a PR to main from a branch on the fork (origin, not upstream), then cherry-pick into the `maint-<VERSION>-r` branch later. | ||
|
|
||
| ## 10. Cherry-pick Necessary Changes | ||
|
|
||
| Check if there are any fixes that need to be cherry-picked into the `maint-<VERSION>-r` branch: | ||
|
|
||
| 1. Check the comments on the release tracking issue for any noted cherry-picks | ||
| 2. Ask if there are any other fixes merged to main after the RC | ||
|
|
||
| Common reasons to cherry-pick: | ||
| - Fixes for CRAN check failures identified in earlier steps | ||
| - NEWS updates (from step 9) | ||
| - Critical bug fixes | ||
|
|
||
| For each PR noted, get the merge commit SHA: | ||
|
|
||
| ```bash | ||
| gh pr view <PR_NUMBER> --repo apache/arrow --json mergeCommit,title --jq '{sha: .mergeCommit.oid, title: .title}' | ||
| ``` | ||
|
|
||
| Present the list of commits and ask for confirmation before cherry-picking. | ||
|
|
||
| For each confirmed commit: | ||
|
|
||
| ```bash | ||
| git cherry-pick <commit-sha> | ||
| ``` | ||
|
|
||
| Ask the user to push: | ||
|
|
||
| ```bash | ||
| git push upstream maint-<VERSION>-r | ||
| ``` | ||
|
|
||
| ## 11. Create Crossbow Verification PR | ||
|
|
||
| Create a PR to run all R crossbow jobs against the CRAN release branch: | ||
|
|
||
| ```bash | ||
| gh pr create --repo apache/arrow \ | ||
| --base maint-<VERSION> \ | ||
| --head maint-<VERSION>-r \ | ||
| --title "WIP: [R] Verify CRAN release <VERSION>" \ | ||
| --body "Do not merge: Running R crossbow jobs against the CRAN release branch." \ | ||
| --draft | ||
| ``` | ||
|
|
||
| Then add a comment to trigger crossbow: | ||
|
|
||
| ```bash | ||
| gh pr comment <PR_NUMBER> --repo apache/arrow --body "@github-actions crossbow submit --group r" | ||
| ``` | ||
|
|
||
| Add a link to this PR in the tracking issue so progress can be monitored. | ||
|
|
||
| Before proceeding to CRAN submission, verify all crossbow jobs pass. | ||
|
|
||
| ## 12. Build and Check Package Locally | ||
|
|
||
| Ensure on the `maint-<VERSION>-r` branch with a clean working directory: | ||
|
|
||
| ```bash | ||
| git fetch upstream | ||
| git checkout maint-<VERSION>-r | ||
| git clean -f -d | ||
| ``` | ||
|
|
||
| Check if `ARROW_HOME` is set: | ||
|
|
||
| ```bash | ||
| echo "ARROW_HOME=${ARROW_HOME:-<not set>}" | ||
| ``` | ||
|
|
||
| If set, unset it so the build uses the vendored C++ version: | ||
|
|
||
| ```bash | ||
| unset ARROW_HOME | ||
| ``` | ||
|
|
||
| Run the build (this takes a while): | ||
|
|
||
| ```bash | ||
| cd r && make build | ||
| ``` | ||
|
|
||
| After the build, check for any generated doc changes that need to be committed: | ||
|
|
||
| ```bash | ||
| git status | ||
| ``` | ||
|
|
||
| If there are modified `.Rd` files or other doc changes, commit them: | ||
|
|
||
| ```bash | ||
| git add r/man/ r/inst/NOTICE.txt | ||
| git commit -m "[R] Update generated documentation" | ||
| ``` | ||
|
|
||
| Ask the user to push. | ||
|
|
||
| Then run the check: | ||
|
|
||
| ```r | ||
| devtools::check_built("arrow_<VERSION>.tar.gz") | ||
| ``` | ||
|
Comment on lines
+214
to
+237
|
||
|
|
||
| Fix any issues before proceeding. | ||
|
|
||
| ## 13. Wait for Release Vote | ||
|
|
||
| Check if the Apache Arrow release vote has passed before continuing. If the release branch was created from the final `apache-arrow-<VERSION>` tag in step 2, the vote has already passed and this step is a no-op. | ||
|
|
||
| ## 14. Update Checksums | ||
|
|
||
| Download checksums for pre-compiled binaries from ASF artifactory. Ask for the libarrow version (usually matches the release version): | ||
|
|
||
| ```bash | ||
| cd r && Rscript tools/update-checksums.R <LIBARROW_VERSION> | ||
| ``` | ||
|
|
||
| Then commit the checksums: | ||
|
|
||
| ```bash | ||
| git add -f tools/checksums/ | ||
| git commit -m "[CRAN] Add checksums" | ||
| ``` | ||
|
|
||
| Ask the user to push. | ||
|
|
||
| ## 15. Rebuild Package | ||
|
|
||
| Rebuild the tarball with checksums added: | ||
|
|
||
| ```bash | ||
| cd r && make build | ||
| ``` | ||
|
|
||
| Check for any new doc changes after rebuild: | ||
|
|
||
| ```bash | ||
| git status | ||
| ``` | ||
|
|
||
| Commit any changes and ask the user to push. | ||
|
|
||
| ## 16. Check Binary Distributions | ||
|
|
||
| Test that the package works with pre-compiled Arrow C++ binaries on different platforms. | ||
|
|
||
| ### 16.1 Windows (win-builder) | ||
|
|
||
| Upload `r/arrow_<VERSION>.tar.gz` to https://win-builder.r-project.org/upload.aspx (r-devel only). | ||
|
|
||
| Results are emailed to Jon (the package maintainer). Ping Jon and wait for confirmation that the check is clean - no ERRORs, WARNINGs, or unexpected NOTEs. NOTEs about package size are expected. | ||
|
|
||
| ### 16.2 macOS Builder | ||
|
|
||
| Upload `r/arrow_<VERSION>.tar.gz` to https://mac.r-project.org/macbuilder/submit.html | ||
|
|
||
| Check the results link for any issues. | ||
|
|
||
| ### 16.3 Ubuntu Binary Installation | ||
|
|
||
| Test on Ubuntu that hosted binaries are used: | ||
|
|
||
| ```r | ||
| install.packages("r/arrow_<VERSION>.tar.gz", repos = NULL) | ||
| ``` | ||
|
|
||
| The installation should download pre-compiled binaries rather than building from source. | ||
|
|
||
| ### 16.4 Final Local Check | ||
|
|
||
| Run one final local check: | ||
|
|
||
| ```r | ||
| devtools::check_built("r/arrow_<VERSION>.tar.gz") | ||
| ``` | ||
|
|
||
| ## 17. Submit to CRAN | ||
|
|
||
| Upload `r/arrow_<VERSION>.tar.gz` to https://xmpalantir.wu.ac.at/cransubmit/ | ||
|
|
||
| Ping Jon to confirm the submission email when it arrives. | ||
|
|
||
| ## 18. Tag for r-universe | ||
|
|
||
| After CRAN accepts the package, ask the user to run: | ||
|
|
||
| ```bash | ||
| git tag -f r-universe-release maint-<VERSION>-r | ||
| git push upstream r-universe-release --force | ||
| ``` | ||
|
|
||
| ## 19. Update Backwards Compatibility Matrix | ||
|
|
||
| Add a new line to `dev/tasks/r/github.linux.arrow.version.back.compat.yml` with the previous version. | ||
|
|
||
| Create a PR to main for this change. | ||
|
|
||
| ## 20. Wait for CRAN Binaries | ||
|
|
||
| Monitor https://cran.r-project.org/package=arrow until CRAN-hosted binaries reflect the new version. | ||
|
|
||
| ## 21. Prepare Social Media Content (Major Releases) | ||
|
|
||
| For major releases, prepare content for social media highlighting new features. | ||
|
|
||
| Format (thread of toots): | ||
| 1. Opening: "We're excited to announce the release of {arrow} <VERSION>. Here's a roundup of the new features and changes. Full details can be found at https://arrow.apache.org/docs/r/news/ #rstats" | ||
| 2. Feature highlights: One toot per major user-facing feature, with code screenshots from https://carbon.now.sh/ where appropriate. Skip minor fixes and CI/internal changes. | ||
| 3. Contributor stats: Total contributors, C++ only, R only, both, first-timers. | ||
|
|
||
| To generate contributor stats, run from the arrow repo root: | ||
|
|
||
| ```r | ||
| source("r/tools/contributor_stats.R") | ||
| release_contributor_stats("apache-arrow-<PREVIOUS_VERSION>", "apache-arrow-<VERSION>") | ||
| ``` | ||
|
|
||
| ## 22. Patch Releases Only: Update Version Numbers | ||
|
|
||
| For patch releases (e.g., X.Y.1), update the version in: | ||
| - `ci/scripts/PKGBUILD` | ||
| - `r/DESCRIPTION` | ||
| - `r/NEWS.md` | ||
|
|
||
| ## 23. CRAN-Only Releases: Update Documentation | ||
|
|
||
| For CRAN-only releases (no corresponding Arrow release): | ||
|
|
||
| Rebuild news page: | ||
|
|
||
| ```r | ||
| pkgdown::build_news("./r") | ||
| ``` | ||
|
thisisnic marked this conversation as resolved.
|
||
|
|
||
| Submit a PR to the `asf-site` branch of https://github.com/apache/arrow-site with contents of `arrow/r/docs/news/index.html` replacing `arrow-site/docs/r/news/index.html`. Ask the user to run the final push command. | ||
|
|
||
| ```bash | ||
| cd ~/arrow-site | ||
| git fetch upstream | ||
| git checkout upstream/asf-site | ||
| git checkout -b "r-<VERSION>" | ||
| cp ~/arrow/r/docs/news/index.html ./docs/r/news/index.html | ||
| git add docs/r/news/index.html | ||
| git commit -m "update R news page" | ||
| git push --set-upstream origin r-<VERSION> | ||
| ``` | ||
|
|
||
| If necessary, bump the version in `r/pkgdown/assets/versions.json` too. | ||
|
|
||
| ## 24. Check C++ Updates | ||
|
|
||
| Review C++ changes in this release and create GitHub issues for any items that need R bindings: | ||
|
|
||
| ```bash | ||
| git log --oneline apache-arrow-<PREVIOUS_VERSION>..apache-arrow-<VERSION> -- cpp/ | head -50 | ||
| ``` | ||
|
|
||
| ## 25. Review Checklist | ||
|
|
||
| Review this packaging checklist and update as needed based on any issues encountered during this release. | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Having an agent push kinda makes me sweat. What do you think about trying to harden the skill by instructing the agent never to push and instead prompt the user to do so.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1