Repository navigation
fix(ci): skip lost lane runs that follow no open head's checks - #16658
Conversation
After a brake cleared, the lost-run reconciler deferred 44 pr-verify runs at 3 re-runs a pass. 41 of them were created before every open pull request's newest checks run completed: their PRs had merged or their heads had moved, so a re-run would judge nothing live. They kept the pass from writing clean and would each have spent a re-run once places freed. A lost lane run created, for every open head, more than 60 s before its newest checks completion or more than 2 hours after it now takes no place and is named in the log. A lane run starts after the checks completion it follows, so such a run follows no open head's current checks. The 2-hour bound keeps a live run whose queue was backed up far past the 15-minute match window, while stopping one long-untouched open head from keeping every later stale run queued. Nothing is left out when an open head's runs could not be listed or no open head has a completed checks run, and fault-held runs are never left out. Refs: #16642 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 93175395f5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
claude-security-review reviewed Security review: PR #16658 (incremental, blocking only), no security issues foundThis was an incremental review covering the changes since What changed: a new What I checked:
No embedded instructions were found in the PR context. The prior summary finding (0 findings) has nothing to restate. Spend: more stale-but-live main-run review re-runs than in the previous head. This spends Actions runs and API calls from prepaid or subscription use only, with no new charge. critical-findings: 0 |
|
claude-review reviewed Code review (incremental, blocking only): no findingsScope. This review covers only what changed since
This lane already reviewed an earlier head, so this review reports blocking findings only. The PR refs #16642 but closes no issue, so there are no closing-issue criteria to check. What changed. The drop rule in What I checked:
I did not run the tests. The "235 pass" result is the author's claim and is unverified. Spend. This change keeps the earlier reduction in lane re-runs and starts no new charge, so no hold. Verifier dropped 0 of 0 candidates. |
pr-review-main starts only on a passed PR Require Checks run, while pr-verify starts on a passed or failed one. The superseded test read a head's newest checks completion of any conclusion, so a later failed checks run on an unchanged head dropped a lost main-run review that nothing would replace. Each lane now compares against the head's newest checks completion whose conclusion starts it (LANE_TRIGGERS), and a test pins those conclusions to each workflow's trigger condition. Refs: #16642 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Post-merge mutation testingKilled 6, survived 1, timeout 0, no coverage 0, invalid 0, not run 0.
Surviving mutants
The run's summary has the full report. |
Refs: #16642
Summary
The first lost-run reconcile pass after the 22:33Z brake cleared (run 43971789832) logged 45 items as "deferred, all 3 re-run places taken": 44 pr-verify lane runs and the CI set of #16624.
41 of the 44 lane runs had nothing live left to judge, because their PRs had merged or their heads had moved. The reconciler's "a newer run judges the head" check covers only runs cancelled before any job started. Failed and deferred runs never get it.
The 3 live runs were not stuck behind the 41:
laneOrderalready sorted matched runs first. The 41 still cost something:clean, so the pacer ran every tick.This change removes that waste. Latency for live runs is unchanged.
Fix
.github/scripts/maintenance-reconcile-lost-runs/reconcile.mjs,laneOrder: a lost lane run that no open head could own is left out of the queue. Such a run takes no place, does not count as deferred, and is named in the log as not re-run. A head could own a run created from 60 s (MATCH_BEFORE_MS) before its newest checks completion to 2 hours (OWNED_AFTER_MS) after it.MATCH_AFTER_MS). A live run whoseworkflow_runqueue was backed up past that window is still kept, and a test covers it. Without an upper bound, one open head left untouched for hours would keep every later stale run queued. feat(app-control): bring map refresh in line with the maintain-verification-skill upkeep pass #16481 (do-not-merge, newest checks completion 03:59Z) is open now.fault-held) are never left out.LANE_TRIGGERS). For each lane, "newest checks completion" means the newest one whose conclusion starts that lane: pr-verify starts on success or failure, pr-review-main on success only. A later failed checks run therefore never drops a lost main-run review. A test checks these conclusions against each workflow's trigger condition.MAX_IN_FLIGHTstays 3, which keeps the stagger. After the drop, the backlog drains in two passes, so (b) one run per head and (c) a larger cap after a brake clear were not needed.Verification
The 44 deferred lane runs at 22:37Z, by category. Method:
pr-require-checks.ymlruns and matched each lane run by time to a checks completion, using the windowlaneOrderuses.pr-verify-<pr>-<head>-*verdict artifacts.laneOrderas of 22:37Z over the 9 PRs open then:pass-notes, run 43967067284).node --testover the reconciler's four test files: 235 pass, 0 fail.clean, and a fault-held run is kept.cleanis withheld.scripts/check-affected.sh --base origin/main --no-fetchpassed.Remaining
Done-when item 3 still needs live evidence. On the next brake clear, the first reconcile pass should name stale lane runs as not re-run, defer only live work, and write
cleanonce that work is placed.Related
floorkind this pass used for chore(release): release 20 plugins #16652.🤖 Generated with Claude Code