auto-rebase: regression test — an approved PR stays approved/mergeable after an eligible update #929

Activity

1. don-petry

   added

   initiative

   on Aug 3, 2026
2. don-petry

   added a parent issue on Aug 3, 2026

   • Auto-rebase eligibility gate inverts the review-currency goal (reusable-side scope) #926
3. don-petry

   marked this as blocked on Aug 3, 2026

   • auto-rebase: default eligibility to all; remove dead review-ready code; lock merge-method invariant #927
4. don-petry

   marked this as blocking on Aug 3, 2026

   • auto-rebase: cut a release + promote channel for the eligibility fix #930
5. don-petry

   added

   dev-leadFor dev-lead agent pickupFor dev-lead agent pickup

   on Aug 3, 2026
6. don-petry commented on Aug 3, 2026

   

   don-petry

   on Aug 3, 2026

   ContributorAuthor

   More actions

   Dev-Lead Implementation Plan

   Issue: #929 — auto-rebase: regression test — an approved PR stays approved/mergeable after an eligible update

   Scope

   Add a bats integration test that drives the real auto-rebase-reusable.yml run-block against a stubbed gh, proving that a behind, conflict-free, non-draft, APPROVED PR gets updated (via update_method=merge) and remains APPROVED/mergeable afterward — the org's default ruleset behavior documented in standards/ci-standards.md ("the merge method is used (never rebase) so existing approvals are not invalidated").

   Implementation Checklist

   • test/workflows/auto-rebase/approval-survival.bats — new integration test file that:
     • extracts the reusable's "Update behind non-Dependabot PRs" run-block via yq
     • runs it against a gh stub modeling one behind/conflict-free/non-draft/APPROVED PR, tracking approval + mergeable state
     • the stub models GitHub's real behavior: a merge update preserves the approval; a rebase update (SHA rewrite) would dismiss it
   • Reuse existing test/workflows/auto-rebase/helpers/setup.bash (no new production code — this is test-only coverage)

   Tests to Write

   • Behind + APPROVED PR → workflow calls update-branch with update_method=merge, PR ends up-to-date (behind_by == 0)
   • After the eligible update the PR's review decision is still APPROVED and it stays mergeable
   • Teeth/control: driving the stub's update-branch with rebase dismisses the approval — proving the "stays APPROVED" assertion is non-vacuous and would catch a merge→rebase regression

   Test Command

   ./node_modules/.bin/bats test/workflows/auto-rebase/

   Notes

   This is a behavior-adding test (Phase 2 TDD applies). No production .sh/YAML changes are expected — the merge-method invariant already exists (issue #927/#931); #929 adds the end-to-end approval-survival regression guard requested in #926 AC8.
7. don-petry commented on Aug 3, 2026

   

   don-petry

   on Aug 3, 2026

   ContributorAuthor

   More actions

   Dev-Lead: Implementation Complete

   Plan Execution

   • Added test/workflows/auto-rebase/approval-survival.bats — an integration test driving the reusable workflow's real "Update behind non-Dependabot PRs" run-block (extracted verbatim via yq) against a stubbed gh.
   • Modeled the org's default-ruleset behavior in the stub: update_method=merge preserves the existing approval (no SHA rewrite); update_method=rebase dismisses it → REVIEW_REQUIRED.
   • No production .sh/YAML changes — the merge-not-rebase invariant already exists (issues auto-rebase: default eligibility to all; remove dead review-ready code; lock merge-method invariant #927/feat: implement issue #927 — auto-rebase: default eligibility to all; remove dead review-ready code; lock merge-method invariant #931). auto-rebase: regression test — an approved PR stays approved/mergeable after an eligible update #929 is the end-to-end approval-survival regression guard (part of Auto-rebase eligibility gate inverts the review-currency goal (reusable-side scope) #926, AC8).
   • Proved the test is non-vacuous (red phase): temporarily flipping the workflow to rebase turned the approval-preservation tests red; reverted byte-for-byte.

   Test Results

   $ ./node_modules/.bin/bats test/workflows/auto-rebase/
   1..17
   ok 1 approval-survival: an APPROVED behind PR is updated via merge
   ok 2 approval-survival: the PR ends up-to-date after the update
   ok 3 approval-survival: the PR remains APPROVED and mergeable afterward
   ok 4 approval-survival: a rebase update would dismiss the approval (control)
   ok 5 post_comment_best_effort: comment-cap failure is non-fatal (returns 0)
   ok 6 post_comment_best_effort: failure logs a continue warning naming the PR
   ok 7 post_comment_best_effort: success returns 0, posts the comment, no warning
   ok 8 post_comment_best_effort: a capped PR does not abort a bash -e loop over multiple PRs
   ok 9 pr_eligible all: every behind PR is eligible
   ok 10 pr_eligible: unknown mode errors out (exit 2)
   ok 11 pr_eligible: empty mode errors out (exit 2)
   ok 12 pr_eligible: the deprecated review-ready mode is treated as 'all' (exit 0)
   ok 13 auto_rebase_has_current_approval is no longer defined
   ok 14 auto_rebase_has_ready_label is no longer defined
   ok 15 merge-method: reusable workflow file exists
   ok 16 merge-method: update-branch is called with update_method=merge
   ok 17 merge-method: update-branch is never called with the rebase method
   
   $ shellcheck -x .github/scripts/auto-rebase/lib/eligibility.sh .github/scripts/auto-rebase/lib/comments.sh   # OK
   $ bash .dev-lead/scripts/dev-lead-lint.sh                                                                    # all checks passed
   

   Files Changed

   • test/workflows/auto-rebase/approval-survival.bats (new): 4 integration tests — the seeded scenario is a behind, conflict-free, non-draft, APPROVED, same-repo PR; asserts it is updated via merge, ends behind_by == 0, and stays APPROVED/mergeable. A control test drives the modeled endpoint with rebase to confirm the approval-survival assertions have teeth. Runs automatically under the existing test/workflows/auto-rebase/** CI trigger in auto-rebase-tests.yml.

   Notes

   • Because auto-rebase: regression test — an approved PR stays approved/mergeable after an eligible update #929 guards already-correct behavior, the suite is green from the start; the red-phase demonstration (merge→rebase, then revert) verifies it would catch a regression rather than passing trivially.
   • Follow-up (from the issue): if petry-projects/.github-private#1416's approval-survival spike (their issue #1437) ships a ruleset fix/bypass, extend this file with a second scenario covering the fixed/bypassed configuration. Not actionable yet — deferred until that lands.
8. don-petry

   added a commit that references this issue on Aug 3, 2026

   feat: implement issue #929 — auto-rebase: regression test — an approv…

   ...

   de298ab
9. don-petry

   closed this as completedin #933on Aug 3, 2026