From 4cbbb16aa840f6106feefac739ab15b2efd6fe38 Mon Sep 17 00:00:00 2001 From: Amr AbuSair Date: Tue, 15 Sep 2026 11:07:07 -0500 Subject: [PATCH] fix(compliance): stop the gate job's name from satisfying its own required check MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The org ruleset requires a status check named "Check CLA/DCO". Two different things carried that exact name: the commit status policy_selector.py posts after actually verifying the contributor, and this workflow's job, which carries it merely by finishing. A required status check is satisfied by EITHER a commit status or a check run of that name. Verified experimentally rather than assumed — a repo-level ruleset on a throwaway branch requiring a context that only a job produced gave mergeStateStatus=CLEAN with zero commit statuses on the head SHA. So "the workflow ran" was indistinguishable from "the CLA was verified". That was survivable only because the script always posts a status before exiting. It does not always succeed at it: set_commit_status ignores github_api's return value, and github_api returns None on every failure, so a failed status POST leaves the job green, the check run green, and the PR mergeable having verified nothing. Fail-open, silently. Renaming the job means only a real commit status can satisfy the gate. No verification, no pass. Blast radius measured, not estimated: * Of 305 open PRs on gated default branches, 99 are satisfied by both a status and the check run, 2 by a status alone, and ZERO by the check run alone — so no open PR loses its passing check. (204 have neither and are already blocked, unaffected.) * No repo-level ruleset and no classic branch protection on any of the 84 gated repos requires this context; the org ruleset is the sole enforcer. * The ruleset's workflows rule pins the workflow FILE path, not the job name, so required-workflow enforcement is unaffected. * The only references to the string anywhere are STATUS_CONTEXT and this job name; nothing queries the check by name. Known cost: this cannot be tested before merging. The ruleset pins this workflow at refs/heads/main, so the usual trick of pointing a branch's sweeper checkout at itself does not reach the gate path. First exercise is a live PR event. Rollback is this one line reverted, effective on the next PR event rather than the next cron. The comment added at STATUS_CONTEXT is the other half of the guard: collapsing these two names back together — from either side — reopens the hole. Co-Authored-By: Claude Opus 5 (1M context) --- .github/workflows/required-compliance.yml | 15 ++++++++++++++- scripts/policy_selector.py | 3 +++ 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/.github/workflows/required-compliance.yml b/.github/workflows/required-compliance.yml index c8a786a..cc578fd 100644 --- a/.github/workflows/required-compliance.yml +++ b/.github/workflows/required-compliance.yml @@ -8,7 +8,20 @@ on: jobs: policy-enforcement: - name: Check CLA/DCO + # This name MUST NOT match STATUS_CONTEXT in scripts/policy_selector.py + # (currently "Check CLA/DCO"), which is also the context the org ruleset + # requires. A required status check is satisfied by EITHER a commit status + # or a check run of that name — verified experimentally: a green check run + # alone yields mergeStateStatus=CLEAN with zero commit statuses present. + # + # While the names matched, this job merely *finishing* satisfied the gate. + # That was survivable only because the script always posted a status before + # exiting; any path that returns without posting one (a failed status POST, + # or a deliberate bail on unreadable data) silently turned the gate from + # fail-closed into fail-open, merging a PR with no verification at all. + # + # Keeping the names distinct means only a real status can satisfy the gate. + name: Compliance Gate runs-on: ubuntu-latest permissions: actions: write diff --git a/scripts/policy_selector.py b/scripts/policy_selector.py index 42ad28d..607b514 100644 --- a/scripts/policy_selector.py +++ b/scripts/policy_selector.py @@ -27,6 +27,9 @@ def requires_CLA(repo, token=None, licenses_data=None, permissive_data=None, all # --- [INTEGRATION END] ------------------------------------- # --- CONFIGURATION --- +# The context the org ruleset requires. MUST NOT match the job name in +# .github/workflows/required-compliance.yml — see the comment there for why +# letting them collide makes the gate fail open. STATUS_CONTEXT = "Check CLA/DCO" BOT_ALLOWLIST = ["dependabot[bot]", "github-actions[bot]", "renovate[bot]"]