PR attention queue: treat pending-ci-rerun as a merge gate and dotnet-policy-service as a bot

[PureWeen]

PR attention queue: treat pending-ci-rerun as a merge gate and dotnet-policy-service as a bot

• You've read the Contributor Guide and Code of Conduct.
• You've included unit or integration tests for your change, where applicable.
• You've included inline docs for your change, where applicable.
• There's an open issue for the PR that you are making. If you'd like to propose a new feature or change, please open an issue to discuss the change or find an existing issue.

Treat pending-ci-rerun as a merge gate; recognize dotnet-policy-service as a bot

Description

Why

pending-ci-rerun is a merge gate, not review routing. The policy bot adds pending-ci-rerun to open, non-draft PRs after 7 days of inactivity and comments "please rerun validation before merging" (.github/policies/resourceManagement.yml:151-170). Pushes don't remove the label. Only an exact /azp run comment from a user with write access removes it (:678-690). Across the repo, 249 of 276 labeled open PRs are REVIEW_REQUIRED. The classifier sent every labeled PR to WaitingOnCI before it checked review state, so PRs that needed review or rescue were hidden there.

dotnet-policy-service was treated as a human. GraphQL returns the bare login dotnet-policy-service, without [bot], so Test-IsBotLogin missed it. Its boilerplate comments wrongly triggered non-author-discussion-requires-verification, which kept PRs out of the review digest. They could also count as a human response in the community inbox, which hid neglected PRs.

What changed

All changes are under .github/skills/pr-attention-queue/:

• scripts/PRAttentionQueue.psm1
  • Removed the elseif ($ciRerunPending) classifier branch that sent every labeled PR to WaitingOnCI.
  • Added a step that runs after classification. When the label is present, it adds ci-rerun-pending once as an informational reason. If the PR would otherwise be ReadyToMerge, it moves to WaitingOnCI, the next actor becomes CI/automation, and it gets the blocker CI must be rerun before merge. These PRs end up not-candidate for merge, because only ReadyToMerge items become candidates.
  • Because this step runs after all classification, ci-rerun-pending now also appears on Draft and Excluded (bot-authored) items.
  • The display text for ci-rerun-pending now describes the label as an inactivity marker that requires CI revalidation before merge. It does not mean CI is running.
  • Get-ResponseEvidence: a community PR whose only comments are from bots now gets no-response instead of unknown. This requires complete, untruncated evidence in which every collected comment is from a bot. Without this, adding the bot pattern alone left bot-only PRs at unknown. Incomplete or truncated evidence still gives unknown. Comments from other known bots, such as github-actions, also no longer prevent no-response.
• presets.json: added dotnet-policy-service to knownBotPatterns. pendingCiLabels is unchanged. Author comments such as @dotnet-policy-service agree stay author comments, because comments are attributed by the commenter's login, not by text in the body.
• SKILL.md: updated the precedence rules and the response-evidence section to match.
• Tests
  • Test-PRAttentionQueue.ps1:
    • Fixture PR 102 (approved, CLEAN, labeled) now asserts WaitingOnCI, the merge-gate blocker, the approved reason, CI/automation, not-candidate, and no merge verification.
    • PR 101 (* NO MERGE * plus the label) stays NeedsRescue and keeps the informational reason.
  • New fixtures/rerun-label-pull-requests.json (PRs 301-310). Each PR has the label plus one other condition: review requested, never reviewed, CHANGES_REQUESTED, failed CI, draft, CONFLICTING, approved + BEHIND, stale past the rescue threshold, bot-authored, or a design gate. Each keeps its normal bucket, next actor and reason, has exactly one ci-rerun-pending, and stays out of merge candidacy.
  • New fixtures/policy-bot-pull-requests.json (PRs 401-403):
    • A discussion whose only comment is from the policy bot is clear, stays in the digest, and gets no-response.
    • An author comment @dotnet-policy-service agree stays an author comment.
    • A truncated discussion with only bot comments stays unknown.
  • Test-MergeStateRefresh.ps1: a labeled PR whose merge state refreshes from UNKNOWN to CLEAN is still held by the merge gate.
  • Existing bot tests are unchanged.

Validation

With the fix:

Command	Result
pwsh .github/skills/pr-attention-queue/tests/Test-PRAttentionQueue.ps1	Passed: All PR attention queue tests passed.
pwsh .github/skills/pr-attention-queue/tests/Test-MergeEligibility.ps1	Passed: 63/63, All merge eligibility cases passed.
pwsh .github/skills/pr-attention-queue/tests/Test-MergeStateRefresh.ps1	Passed: Merge state refresh: 18/18 passed; 0 failed.

Without the fix (PRAttentionQueue.psm1 and presets.json reverted, new tests kept):

• Test-PRAttentionQueue.ps1 exits 1 at A merge-ready PR with a rerun label must explain the merge gate.
• A non-throwing run over all 556 assertions found 33 expected failures and no fixture or parse errors. The failing groups:
  • Every new rerun-label routing assertion (PRs 301-310).
  • The PR 102 merge-gate assertions and the PR 101 informational reason.
  • Every policy-bot assertion: discussion clear, automation attribution, digest membership, no-response, no recorded human response, and truncated evidence staying unknown.
• Test-MergeStateRefresh.ps1 exits 1 with FAIL regression pending-ci-rerun: Refreshed CLEAN state must preserve approval and the CI rerun merge gate.
• With only presets.json reverted, Test-PRAttentionQueue.ps1 exits 1 at A policy-bot-only discussion must be clear.

Live read-only Blazor run: pwsh .github/skills/pr-attention-queue/scripts/Get-PRAttentionQueue.ps1 -Preset blazor -OutputFormat Json -DisablePersonalInbox. It returned query.complete = true with 404/404 open PRs scanned and 139 matched.

Bucket	Before	After
ReviewNow	3	9
NeedsRescue	1	13
WaitingOnCI	62	39

• ReviewNow digest: Harden RendererSynchronizationContext restoration #69356, Avoid starting the response when navigating with exception #69174, Preserve modified state when InputBase parsing fails #68365, Replace custom reference resolution with explicit references and CPM #69631. Discussion is clear for all four. They were previously excluded because of the policy bot's comments.
• Improve Render Output Invalid Exception Message to Clarify Common Control Flow Causes #67695 (approved, CLEAN, labeled) is held by the merge gate: WaitingOnCI, CI/automation, not-candidate, with blocker CI must be rerun before merge.

Out of scope

• Bot detection based on GraphQL __typename, instead of login patterns, would be a possible follow-up.

Fixes: N/A (no tracking issue)

[Copilot]

Copilot review overview

🟢 Approval recommended

The implementation matches the documented policy and provides comprehensive regression coverage.

Review effort: Balanced
Findings: None

What changed in this PR

Updates PR attention routing so stale-CI labels gate merging without hiding review work, while recognizing policy automation correctly.

Changes:

• Applies pending-ci-rerun after normal classification.
• Treats policy-service comments as automation.
• Adds regression fixtures and coverage.

File	Description
scripts/​PRAttentionQueue.psm1	Revises routing and response evidence.
presets.json	Registers the policy bot.
SKILL.md	Documents updated semantics.
tests/​Test-PRAttentionQueue.ps1	Tests routing and bot handling.
tests/​Test-MergeStateRefresh.ps1	Tests refreshed merge-state gating.
tests/​fixtures/​rerun-label-pull-requests.json	Adds rerun-label scenarios.
tests/​fixtures/​policy-bot-pull-requests.json	Adds policy-bot scenarios.

---

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.