[pjhartlieb]

Promotes dev to stable as v1.3.0 (version in pyproject.toml).

• CLA: save signatures on CLA's signatures branch; exemption that can't be claimed by email (1.3.0) #18: cla-reusable.yml saves signatures on CLA's bot-only signatures branch (CLA main has refused the bot's push since 10-27, so none have saved). The job and its App token run only for pull request events or a pull request comment that is recheck or contains the sign phrase. On a pull request opened by a member or named bot, other members and named bots are exempt; on one opened by anyone else, the opener and every commit author sign, members included. More than 100 commits, or an outside opener with no commit, fails. The ICLA link is pinned to a commit.

Callers keep their pinned version until they repin to v1.3.0.

stable has nothing dev lacks (only merge commits), so no merge-back afterwards.

Merge with a merge commit. On merge, release.yml waits twice at the release environment. Each approval needs a bbot-leads member other than whoever merged.

Validation

CI at d41dd9b:

• Tests 43147336827: python / test 3.11 to 3.15, python / lint (ruff), actionlint, python / passed: all success.
• dev-security 43147187905 on CLA: save signatures on CLA's signatures branch; exemption that can't be claimed by email (1.3.0) #18's head 489287d (same tree): workflow-lint (actionlint, zizmor), dependency-review: success.

Locally, a fresh clone at d41dd9b:

$ pytest -q (pytest 9.0.3, ruff 0.15.10, as pinned in pyproject.toml)
60 passed in 3.44s
$ ruff check .
All checks passed!
$ ruff format --check .
5 files already formatted
$ zizmor 1.30.1 (online) .
No findings to report. Good job! (2 ignored, 33 suppressed)
$ actionlint 1.7.12
(no output)

tests/test_cla_check.py (14 cases, one per exemption rule) fails 3 cases when members are made exempt on an outside contributor's pull request, and 1 when the opener check is removed.

[en0f]

Reviewed at d41dd9b.

🔴 A failed commits read still lets a member's pull request skip the CLA

cla-reusable.yml:87 reads the commits in a process substitution, whose exit status set -euo pipefail never sees. If that read fails, the loop runs zero times and a pull request opened by a member or named bot comes out all_exempt=true. The skip step then posts success. 1.3.0 already narrows the 1.2.0 hole, which failed open for every opener, but these lines now read as fail-closed (46, 52). Fix: COMMIT_AUTHORS=$(gh api ...), then done <<< "$COMMIT_AUTHORS", and exit 1 unless the line count equals N_COMMITS. The count check is needed because an empty here-string still yields one empty line. Add a test where the commits read fails.

🔴 A commit with no author login can pass under its committer's name

The step marks such a commit "CLA required" (line 78) but still runs the action. The pinned action credits it to its committer (graphql.ts, extractUserFromCommit), so on a member's pull request an outside commit with an unlinked email passes when its committer is on the allowlist or has ever signed. This predates 1.3.0. The test at tests/test_cla_check.py:74 only covers a lone commit with no login. Fix: exit 1 on any null author login, as line 94 does for an outside opener with no commit. Moving the committer into REQUIRED is not enough, because one past signature would cover it. Add the [null, m1] case.

🟢 On an outside contributor's pull request no one is exempt by commit email

Exemption now follows who opened the pull request, which GitHub attests, rather than a commit email anyone can set. The tests pin each rule: making members exempt again fails 3 cases.

🟢 The 100-commit guard matches the pinned action

The action queries commits(first: 100) once without paging. Refusing more than 100 commits beats passing on a partial list.

🟢 No signature is lost, and the ICLA text is unchanged

signatures carries all 27 signatures on CLA main (the excluded entry is not a signature), and the pinned ICLA commit is byte-identical to main's.

🟢 The App token is minted only when it can matter

The job-level if stops ordinary comments from minting the token, and event values moved from inline ${{ }} into env:.

📝 Test artifact owed: a signature saved on the signatures branch

The release depends on the CLA App bypassing signatures-bot-only. I can't see its bypass list, and the branch's only commit is the hand-seeded one. Attach a run from a test caller pinned to d41dd9b that saves a signature, or the bypass list. Not blocking during the QA trial.

Reproduce this review

# save the script below as verify-19.sh, and the control patch as control-19.patch, then:
bash verify-19.sh blacklanternsecurity/release-workflows d41dd9b5f3cf6a131fa8711387bd4e2ebef00811
# expected: SUMMARY 10 PASS, 2 FAIL, 0 SKIP, 0 BROKEN   (V06 and V12 FAIL: the two findings above)
# control:  bash verify-19.sh blacklanternsecurity/release-workflows d41dd9b5f3cf6a131fa8711387bd4e2ebef00811 "$PWD/control-19.patch"
#           -> V01, V03, V04 FAIL (members made exempt on an outside contributor's pull request)

--- a/.github/workflows/cla-reusable.yml
+++ b/.github/workflows/cla-reusable.yml
@@ -82,7 +82,7 @@
             fi
             AUTHORS="$AUTHORS$AUTHOR "
             OK=false
-            if [ "$OPENER_EXEMPT" = "true" ] && { is_named_bot "$AUTHOR" || is_member "$AUTHOR"; }; then OK=true; fi
+            if { is_named_bot "$AUTHOR" || is_member "$AUTHOR"; }; then OK=true; fi
             if [ "$OK" = "true" ]; then EXEMPT="$EXEMPT$AUTHOR "; else REQUIRED="$REQUIRED$AUTHOR "; fi
           done < <(GH_TOKEN="$PR_TOKEN" gh api "repos/$REPO/pulls/$PR_NUM/commits?per_page=100" \
             --jq '.[] | .author.login // "null"')

#!/usr/bin/env bash
# verify-19.sh: reproducible checks for blacklanternsecurity/release-workflows#19 (Release 1.3.0)
# Usage: bash verify-19.sh <owner/repo> <sha> [<control-patch>]
# Changes nothing outside a mktemp directory and nothing on GitHub (PR-VF-02).
#
# CONTROL: patch control-19.patch
# CONTROL-EXPECT: V01=FAIL V03=FAIL V04=FAIL
#   The author has admin on the repository (context.json author_permission), so the
#   test suite runs on the reviewer's workstation, inside $WORK, with no tokens exported
#   to it (qa.md § Rules, PR-VF-17). The control patch makes org members exempt on a
#   pull request opened by an outside contributor, the hole #18 closes.
set -uo pipefail

REPO="${1:?owner/repo}"; SHA="${2:?sha}"; PATCH="${3:-}"
PASS=0; FAIL=0; SKIP=0; BROKEN=0
pass()   { echo "PASS $1 $2"; PASS=$((PASS+1)); }
fail()   { echo "FAIL $1 $2 -- $3"; FAIL=$((FAIL+1)); }
skip()   { echo "SKIP $1 $2 -- $3"; SKIP=$((SKIP+1)); }
broken() { echo "BROKEN $1 $2 -- $3"; BROKEN=$((BROKEN+1)); }
summary() { echo "SUMMARY $PASS PASS, $FAIL FAIL, $SKIP SKIP, $BROKEN BROKEN"; [ "$FAIL" -eq 0 ] && [ "$BROKEN" -eq 0 ]; }

echo "script_sha256 $(shasum -a 256 "$0" | cut -d' ' -f1)"
echo "git $(git --version | cut -d' ' -f3)"
echo "uv $(uv --version 2>/dev/null | cut -d' ' -f2 || echo missing)"
echo "jq $(jq --version 2>/dev/null || echo missing)"
echo "bash $BASH_VERSION"

WORK="$(mktemp -d)"; trap 'rm -rf "$WORK"' EXIT
export GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1 GIT_TERMINAL_PROMPT=0
GIT=(git -c credential.helper= -c 'credential.helper=!gh auth git-credential')
"${GIT[@]}" clone --quiet --filter=blob:none --no-checkout "https://github.com/$REPO.git" "$WORK/src" || { broken V00 "clone" "clone failed"; summary; exit 1; }
cd "$WORK/src"
"${GIT[@]}" cat-file -e "$SHA^{commit}" 2>/dev/null || "${GIT[@]}" fetch --quiet origin "+refs/pull/*/head:refs/remotes/pr/*" || true
"${GIT[@]}" checkout --quiet --detach "$SHA" || { broken V00 "checkout $SHA" "checkout failed"; summary; exit 1; }
[ "$(git rev-parse HEAD)" = "$SHA" ] || { broken V00 "HEAD is $SHA" "HEAD mismatch"; summary; exit 1; }
if [ -n "$PATCH" ]; then git apply "$PATCH" || { broken V00 "apply control patch" "git apply failed"; summary; exit 1; }; fi

# ---- V01-V02: the repository's own suite and lint, from the lock (hash-checked by uv) ----
export UV_CACHE_DIR="$WORK/uv-cache" UV_PROJECT_ENVIRONMENT="$WORK/venv" UV_NO_CONFIG=1 RUFF_CACHE_DIR="$WORK/ruff-cache"
if ! command -v uv >/dev/null; then
  skip V01 "full pytest suite passes" "uv not installed"; skip V02 "ruff check and ruff format --check pass" "uv not installed"
elif ! env -u GH_TOKEN -u GITHUB_TOKEN uv sync --frozen --quiet > "$WORK/sync.log" 2>&1; then
  skip V01 "full pytest suite passes" "uv sync --frozen failed: $(tail -1 "$WORK/sync.log")"; skip V02 "ruff check and ruff format --check pass" "uv sync failed"
else
  OUT=$(env -u GH_TOKEN -u GITHUB_TOKEN uv run --frozen --no-sync pytest -q -p no:cacheprovider 2>&1 | tail -1)
  if echo "$OUT" | grep -Eq '^[0-9]+ passed' && ! echo "$OUT" | grep -q failed; then pass V01 "full pytest suite passes ($OUT)"; else fail V01 "full pytest suite passes" "$OUT"; fi
  if env -u GH_TOKEN -u GITHUB_TOKEN uv run --frozen --no-sync ruff check -q . >/dev/null 2>&1 && env -u GH_TOKEN -u GITHUB_TOKEN uv run --frozen --no-sync ruff format --check -q . >/dev/null 2>&1; then
    pass V02 "ruff check and ruff format --check pass"; else fail V02 "ruff check and ruff format --check pass" "ruff reported findings"; fi
fi

# ---- V03-V05: the exemption step, run with the reviewer's own stub gh (independent of tests/) ----
WF=.github/workflows/cla-reusable.yml
uv run --frozen --no-sync python -c '
import sys, yaml
steps = yaml.safe_load(open(sys.argv[1]))["jobs"]["CLAAssistant"]["steps"]
print(next(s for s in steps if s.get("id") == "cla-check")["run"])' "$WF" > "$WORK/step.sh" 2>/dev/null
mkdir -p "$WORK/bin"
cat > "$WORK/bin/gh" <<'STUB'
#!/bin/bash
case "$2" in
  orgs/blacklanternsecurity/members/*) case " $MEMBERS " in *" ${2##*/} "*) exit 0 ;; esac; exit 1 ;;
  repos/o/r/pulls/7) printf '{"user":{"login":"%s"},"commits":%s}\n' "$OPENER" "$NC" ;;
  repos/o/r/pulls/7/commits*) [ "$COMMITS_FAIL" = 1 ] && { echo "HTTP 502" >&2; exit 1; }; tr ' ' '\n' <<< "$COMMITS" ;;
  *) echo "unexpected gh $*" >&2; exit 2 ;;
esac
STUB
chmod +x "$WORK/bin/gh"
probe() { # opener members "commit authors" commits_fail -> prints exit code and outputs
  local o="$WORK/out.$RANDOM"; : > "$o"
  PATH="$WORK/bin:$PATH" GITHUB_OUTPUT="$o" REPO=o/r PR_NUM=7 GH_TOKEN=x PR_TOKEN=y OPENER="$1" MEMBERS="$2" COMMITS="$3" NC=$(wc -w <<< "$3" | tr -d ' ') COMMITS_FAIL="${4:-0}" \
    bash -c "$(cat "$WORK/step.sh")" >/dev/null 2>&1; echo "rc=$? $(tr '\n' ' ' < "$o")"
}
if [ ! -s "$WORK/step.sh" ]; then
  broken V03 "exemption step extractable" "could not read the cla-check step"
else
  R=$(probe ext "m1" "ext m1")
  if [ "$R" = "rc=0 all_exempt=false allowlist= " ]; then pass V03 "outside opener: a member's commit is not exempt (allowlist empty)"; else fail V03 "outside opener: a member's commit is not exempt (allowlist empty)" "$R"; fi
  R=$(probe ext "m1" "ext dependabot[bot]")
  if [ "$R" = "rc=0 all_exempt=false allowlist= " ]; then pass V04 "outside opener: a commit claiming dependabot[bot] is not exempt"; else fail V04 "outside opener: a commit claiming dependabot[bot] is not exempt" "$R"; fi
  R=$(probe m1 "m1 m2" "m1 m2")
  if [ "$R" = "rc=0 all_exempt=true allowlist=m1,m2 " ]; then pass V05 "member opener with member commits: all exempt"; else fail V05 "member opener with member commits: all exempt" "$R"; fi
  R=$(probe m1 "m1" "" 1)
  case "$R" in rc=0\ all_exempt=true*) fail V06 "a failed commits read fails closed on a member's pull request" "$R";; *) pass V06 "a failed commits read fails closed on a member's pull request";; esac
fi

# ---- V07-V09: state outside the tree that the change depends on (GET only) ----
ICLA_PIN=$(sed -n 's|.*CLA/blob/\([0-9a-f]\{40\}\)/ICLA.md.*|\1|p' "$WF")
if [ -z "$ICLA_PIN" ]; then fail V07 "ICLA link is pinned to a commit whose text equals CLA main's ICLA" "no pinned commit in $WF"
else
  A=$(gh api --method GET -H 'Accept: application/vnd.github.raw+json' "repos/blacklanternsecurity/CLA/contents/ICLA.md?ref=$ICLA_PIN" 2>/dev/null | shasum -a 256 | cut -c1-64)
  B=$(gh api --method GET -H 'Accept: application/vnd.github.raw+json' "repos/blacklanternsecurity/CLA/contents/ICLA.md?ref=main" 2>/dev/null | shasum -a 256 | cut -c1-64)
  if [ "$A" = "$B" ] && [ "$A" != "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" ]; then pass V07 "ICLA link is pinned to ${ICLA_PIN:0:8}, whose text equals CLA main's ICLA"; else fail V07 "ICLA link is pinned to a commit whose text equals CLA main's ICLA" "pinned $A main $B"; fi
fi
SIGS=signatures/version1/cla.json
gh api --method GET -H 'Accept: application/vnd.github.raw+json' "repos/blacklanternsecurity/CLA/contents/$SIGS?ref=main" > "$WORK/main.json" 2>/dev/null
gh api --method GET -H 'Accept: application/vnd.github.raw+json' "repos/blacklanternsecurity/CLA/contents/$SIGS?ref=signatures" > "$WORK/sig.json" 2>/dev/null
if ! jq -e .signedContributors "$WORK/main.json" "$WORK/sig.json" >/dev/null 2>&1; then skip V08 "signatures branch carries every signature on CLA main" "could not read $SIGS on both branches"
else
  MISSING=$(jq -r --slurpfile s "$WORK/sig.json" '.signedContributors[] | select(.comment_id != -1) | select([.id] | inside([$s[0].signedContributors[].id]) | not) | .name' "$WORK/main.json")
  if [ -z "$MISSING" ]; then pass V08 "signatures branch carries every signature on CLA main ($(jq '.signedContributors|length' "$WORK/sig.json") entries)"; else fail V08 "signatures branch carries every signature on CLA main" "missing: $MISSING"; fi
fi
ACTION_SHA=$(sed -n 's|.*contributor-assistant/github-action@\([0-9a-f]\{40\}\).*|\1|p' "$WF")
GQL=$(gh api --method GET -H 'Accept: application/vnd.github.raw+json' "repos/contributor-assistant/github-action/contents/src/graphql.ts?ref=$ACTION_SHA" 2>/dev/null)
if [ -z "$GQL" ]; then skip V09 "the pinned action reads at most 100 commits, matching the step's guard" "could not read graphql.ts"
elif grep -q 'commits(first: 100, after: \$cursor)' <<< "$GQL" && ! grep -q 'hasNextPage *)' <<< "$GQL" && grep -q '"$N_COMMITS" -gt 100' "$WF"; then pass V09 "the pinned action reads at most 100 commits, and the step refuses more than 100"
else fail V09 "the pinned action reads at most 100 commits, and the step refuses more than 100" "graphql.ts or the guard differs"; fi
# V12 (challenges C14, r2 C4/C5): the action credits a commit with no author login to its committer, who may be
# allowlisted or may already have signed, so the step must stop such a pull request for a maintainer (non-zero exit).
if [ ! -s "$WORK/step.sh" ] || [ -z "$GQL" ]; then skip V12 "a commit with no author login stops the step for manual handling" "step or graphql.ts unavailable"
else
  R1=$(probe m1 "m1" "null m1"); R2=$(probe m1 "m1 m2" "null m2")
  if grep -q "commit.author.user || commit.committer.user || commit.author" <<< "$GQL" && { [ "${R1%% *}" = "rc=0" ] || [ "${R2%% *}" = "rc=0" ]; }; then
    fail V12 "a commit with no author login stops the step for manual handling" "[null,m1]: $R1| [null,m2]: $R2; the action credits the commit to its committer"
  else pass V12 "a commit with no author login stops the step for manual handling"; fi
fi

# ---- V10-V11: release bookkeeping and CI at the head ----
PV=$(sed -n 's/^version = "\(.*\)"/\1/p' pyproject.toml | head -1)
LV=$(awk '/^name = "release-workflows"/{f=1;next} f&&/^version/{gsub(/[" ]/,"",$0);sub("version=","");print;exit}' uv.lock)
if [ "$PV" = "1.3.0" ] && [ "$LV" = "$PV" ] && ! gh api --method GET "repos/$REPO/git/ref/tags/v$PV" >/dev/null 2>&1; then pass V10 "pyproject and uv.lock agree on 1.3.0, and tag v1.3.0 does not exist yet"
else fail V10 "pyproject and uv.lock agree on 1.3.0, and tag v1.3.0 does not exist yet" "pyproject=$PV lock=$LV"; fi
CR=$(gh api --method GET "repos/$REPO/commits/$SHA/check-runs?per_page=100" --jq '[.check_runs[] | select(.name=="actionlint" or .name=="workflow-lint" or (.name|startswith("python / test"))) | .conclusion] | (length|tostring) + " " + (map(select(.!="success"))|length|tostring)' 2>/dev/null)
if [ -z "$CR" ]; then skip V11 "actionlint, workflow-lint and the python test matrix succeeded in CI at the head" "check-runs read failed"
elif [ "${CR% *}" -ge 7 ] && [ "${CR#* }" = 0 ]; then pass V11 "actionlint, workflow-lint and the python test matrix succeeded in CI at the head (${CR% *} runs)"
else fail V11 "actionlint, workflow-lint and the python test matrix succeeded in CI at the head" "runs/non-success: $CR"; fi

summary

[en0f]

🔴 set -euo pipefail can't see a failure inside < <(...). If this read fails, a pull request opened by a member or named bot comes out all exempt. Read the list into a variable first, then check the line count against N_COMMITS.

[en0f]

🔴 The action credits a commit with no author login to its committer, who may be allowlisted or may already have signed. Exit 1 here for a maintainer to handle, as line 94 does.