fix(compliance): stop the gate job's name from satisfying its own required check

[aabusair]

The org ruleset requires a status check named Check CLA/DCO. Two different things carried that exact name:

1. The commit status policy_selector.py posts after actually verifying the contributor
2. This workflow's job, which carried it merely by finishing

A required status check is satisfied by either a commit status or a check run of that name. So "the workflow ran" was indistinguishable from "the CLA was verified."

Verified, not assumed

I didn't want to act on inference, so I tested the semantics in isolation — a repo-level ruleset on a throwaway branch of a disposable fixture repo, requiring a context that only a job produced, no commit status ever posted:

CheckRun  ZZZ-Probe-Context -> SUCCESS
commit statuses: []  total_count=0
required context: ZZZ-Probe-Context

mergeable=MERGEABLE   mergeStateStatus=CLEAN

A green check run alone satisfies the requirement. Probe fully cleaned up afterwards.

Why it matters today

This was survivable only because the script always posts a status before exiting. It does not always succeed at it: set_commit_status ignores github_api's return value, and github_api returns None on every failure mode. So a failed status POST leaves the job green, the check run green, and the PR mergeable having verified nothing. Fail-open, silently, with no signal.

Renaming the job means only a real commit status can satisfy the gate. No verification, no pass.

Blast radius — measured

Check	Result
Open PRs (305 governed) passing via check run alone	0 — 99 have both, 2 status-only, 204 neither
Repo-level rulesets on the 84 gated repos requiring this context	none
Classic branch protections requiring it	none
Ruleset's workflows rule	Pins the file path, not the job name — required-workflow enforcement unaffected
References to the string anywhere in the repo	Only STATUS_CONTEXT and this job name; nothing queries the check by name

No open PR loses a passing check. Merged PRs are never re-evaluated. Existing check runs are immutable, so nothing already green turns red.

Test plan

• Semantics proven experimentally in isolation (above), then cleaned up
• Blast radius measured against every gated repo and all 305 governed open PRs
• 57 tests still green; workflow YAML validated
• Functional change is one line — the scripts/policy_selector.py diff is comment-only
• Cannot be tested before merge. See below.

Honest caveats

No pre-merge verification path. The ruleset pins this workflow at refs/heads/main, so the usual trick — pointing a branch's sweeper checkout at itself and dispatching — does not reach the gate path. First real exercise is a live PR event. This is the only change in this series without a pre-merge test, and that's a genuine cost rather than a formality.

Rollback is this one line reverted, effective on the next PR event rather than the next cron, so exposure is minutes.

Cosmetic churn. Old head SHAs keep their Check CLA/DCO check run; new ones get Compliance Gate. Both appear in the checks list for a while. Harmless, untidy.

What this does not fix. The gate still exits 0 on any path that writes no status — the PR is now correctly blocked rather than wrongly mergeable, but nobody is notified. That's tracked separately.

The comment added at STATUS_CONTEXT is the other half of the guard: collapsing these two names back together, from either side, reopens the hole.

🤖 Generated with Claude Code