Skip to content

fix(pr-validation): say which half of the gate failed, not just that it did - #946

Merged
bedatty merged 1 commit into
developfrom
fix/blocked-comment-names-the-stage
Oct 20, 2026
Merged

bedatty merged 1 commit into
developfrom
fix/blocked-comment-names-the-stage

Conversation

@bedatty

@bedatty bedatty commented Oct 20, 2026

Copy link
Copy Markdown
Contributor

Description

Found by running the gate for real. LerianStudio/ungoliant-controller#395 — the first breaking change in a repository that consumes go-pr-validation@tier-0 — produced this comment and nothing else:

💥 Breaking change gate could not run
The gate did not reach a verdict, so this check fails closed — an unknown is not the same as "no breaking changes". [Open the run] to see which step failed.

True, and no help. The actual cause was three steps away in the job log:

The 'breaking-approval' Environment does not exist in LerianStudio/ungoliant-controller
and no reviewers were given, so it cannot be created as a gate.

Nothing on the pull request pointed at it. The author has to open the run, find the red job among the green ones, and read the log.

The verdict already knew. It resolves DETECT_RESULT, REQUEST_RESULT and DETECTION_SUCCEEDED, and writes them to an ::error:: — to the log, and to nobody else.

It now emits the stage, the outcome forwards it as blocked-stage, and the comment names it:

  • detection — detection did not complete, so the gate does not know whether there is a break at all
  • request — the break was detected but the approval could not be requested. This one also names the cause that actually happens, since a reader cannot guess it: the Environment does not exist and could not be created, because either no reviewer could be resolved (no catch-all CODEOWNERS rule, no fallback configured) or the token cannot create Environments

Empty keeps the previous neutral wording, so a caller that does not pass it is no worse off.

@radagast-lerian offered exactly these two options on #926 — "Use neutral gate-failure wording or forward the failed stage". I took the cheaper one. This is the other, and the run above is why it was the right one.

Type of Change

  • feat: New workflow or new input/output/step in an existing workflow
  • fix: Bug fix in a workflow (incorrect behavior, broken step, wrong condition)
  • perf: Performance improvement (e.g. caching, parallelism, reduced steps)
  • refactor: Internal restructuring with no behavior change
  • docs: Documentation only (README, docs/, inline comments)
  • ci: Changes to self-CI (workflows under .github/workflows/ that run on this repo)
  • chore: Dependency bumps, config updates, maintenance
  • test: Adding or updating tests

Breaking Changes

None. blocked-stage is a new optional input on the reporter composite, defaulting to empty, and empty renders exactly the current text.

Testing

  • YAML syntax validated locally
  • Triggered a real workflow run on a caller repository using @this-branch or the beta tag
  • Verified all existing inputs still work with default values
  • Confirmed no secrets or tokens are printed in logs
  • Checked that unrelated workflows are not affected

actionlint clean, shellcheck clean on the reporter, typos clean. 115 workflow tests, 60 detector, 32 gate-scope, 28 commit-list, 22 CODEOWNERS.

test_a_blocked_gate_names_the_half_that_failed renders all three cases, checks each excludes the other's wording, and pins that the verdict resolves detection and request from the right inputs and leaves the stage empty when nothing is blocked. Verified by mutation: deleting the stage resolution fails it.

Caller repo / workflow run: https://github.com/LerianStudio/ungoliant-controller/actions/runs/39441483943 — the run that produced the unhelpful comment.

Related Issues

Follows the blocked state added in #935 and the option @radagast-lerian raised on #926.

…it did

The blocked comment said only that the gate did not reach a verdict and
linked the run. True, and no help: the author opens the run and hunts for
the red job.

Measured on ungoliant-controller, whose first breaking change produced
exactly that comment. The cause was three steps away in the log - no
catch-all CODEOWNERS rule, no fallback reviewer, so the Environment could
not be created - and nothing on the pull request pointed at any of it.

The verdict already knew which half failed; it was writing it to the log
and telling nobody else. It now emits the stage, the outcome forwards it,
and the comment names it. The request case also names the cause that
actually occurs, since it is the one a reader cannot guess.

Empty keeps the neutral wording, so a caller that does not pass it is no
worse off. @radagast-lerian offered both options on #926 and I took the
cheaper one; this is the other.
@bedatty
bedatty requested a review from a team as a code owner October 20, 2026 14:55
@coderabbitai

coderabbitai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

The included review limit has been reached and this organization has disabled usage-based review continuation. Wait for reviews to reset or ask a billing admin to change After included review limits.

  • Ask an admin to enable usage-based reviews

Open in CodeRabbit

Reviews can continue after your included limit without a manual trigger. An admin must approve usage-based billing.

Next included review available in 12 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available. Your 93 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Learn how review limits work.

Review configuration:

⚙️ Run configuration
  • Configuration used: Repository: LerianStudio/github-actions-shared-workflows/.coderabbit.yml
  • Review profile: ASSERTIVE
  • Plan: Essentials
  • Run ID: f7bb0d2e-8460-4a13-b89b-057f5553e5d1
📥 Commits

Reviewing files that changed from the base of the PR and between e61bb71 and 2d1c1f8.

📒 Files selected for processing: 4
  • .github/workflows/breaking-change-approval.yml
  • src/notify/breaking-change-approval-reporter/README.md
  • src/notify/breaking-change-approval-reporter/action.yml
  • src/validate/breaking-change-guard/test-workflow.py
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@lerian-studio lerian-studio added size/S PR changes 50–199 lines documentation Improvements or additions to documentation workflow Changes to one or more reusable workflow files notify Changes to notification composite actions (src/notify/) validate Changes to PR validation composite actions (src/validate/) composite Changes to any composite action manifest (src/**/*.yml) labels Oct 20, 2026
@lerian-studio

Copy link
Copy Markdown
Contributor

🔍 PR Validation Summary

✅ PR Mergeable — no blocking failures

Check Status Blocking
Source Branch ✅ success yes
PR Title ✅ success yes
PR Description ✅ success yes
Breaking Change Guard ✅ success yes
Commit Signatures ✅ success yes
PR Size ✅ success no
Auto Labels ✅ success no
PR Metadata ✅ success no

🔍 View workflow run

@lerian-studio

Copy link
Copy Markdown
Contributor

🔍 Lint Analysis

Check Files Scanned Status
YAML Lint 2 file(s) ✅ success
Action Lint 1 file(s) ✅ success
Pinned Actions 2 file(s) ✅ success
Markdown Link Check 1 file(s) ✅ success
Spelling Check 4 file(s) ✅ success
Shell Check 2 file(s) ✅ success
README Check 2 file(s) ✅ success
Composite Schema 1 file(s) ✅ success
Deployment Matrix no changes ⏭️ skipped

🔍 View full scan logs

@lerian-studio

Copy link
Copy Markdown
Contributor

🛡️ CodeQL Analysis Results

Languages analyzed: actions

✅ No security issues found.


🔍 View full scan logs | 🛡️ Security tab

@lerian-studio

Copy link
Copy Markdown
Contributor

@coderabbitai review

@lerian-studio lerian-studio added the review-ready Required checks passed — CodeRabbit is cleared to review label Oct 20, 2026
@coderabbitai

coderabbitai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@bedatty
bedatty merged commit 9583c80 into develop Oct 20, 2026
43 checks passed
@github-actions
github-actions Bot deleted the fix/blocked-comment-names-the-stage branch October 20, 2026 18:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

composite Changes to any composite action manifest (src/**/*.yml) documentation Improvements or additions to documentation notify Changes to notification composite actions (src/notify/) review-ready Required checks passed — CodeRabbit is cleared to review size/S PR changes 50–199 lines validate Changes to PR validation composite actions (src/validate/) workflow Changes to one or more reusable workflow files

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants