Skip to content

feat(pr-pipeline): accept an upstream-drift issue as the spec of its own re-audit - #16561

Merged
kyle-sexton merged 4 commits into
mainfrom
fix/pr-verify-drift-issue-spec
Nov 16, 2026
Merged

kyle-sexton merged 4 commits into
mainfrom
fix/pr-verify-drift-issue-spec

Conversation

@kyle-sexton

@kyle-sexton kyle-sexton commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

No related issue: owner decision, 2026-11-16, made twice: "go w/ recommendations" on linking a drift re-audit with Closes to its drift issue and having the verifier accept that issue as the spec, then "Go with recommendations." on this change's trust rule as built. No human-written issue exists for it.

Summary

A pull request that re-audits one upstream's docs/upstream page can now close the issue the weekly upstream-drift workflow filed for that upstream, and the intent verifier and the merge gate accept that issue as its spec. Before, every Bot-written issue had no spec, so such a pull request failed with verify:untrusted-spec and needed a person to restate the issue.

Fix

  • .github/scripts/pr-verify/resolve-pr.mjs: upstreamPages, driftSpec and compareFiles, wired into linkedIssues and resolvePull. A matching issue yields a spec entry with source: "upstream-drift".
  • plugins/source-control/skills/babysit-prs/scripts/babysit_verify_evidence.py: the merge gate's twin (drift_issue, page_pin, drift_scope), checked again inside _trusted_spec against the compare and each page at the default-branch tip. Each github-actions issue is read once, shared with the recurring-schedule rule.
  • docs/conventions/pr-pipeline.yaml: a new hold-floor row, docs/upstream/**, class enforcement, because the resolver now reads those pages inside a floored module and every file a hold path loads must be on the floor. The floor copies are regenerated (merge-holds.mjs and three workflows), both carry-refuse lists gain the glob, and the pinned floor count and digests move to 144 rows. A pull request that touches docs/upstream/** now rates as a review-path hold that a clean security-lane review bound to its head clears, with no person needed.
  • prompt.md, docs/conventions/pr-pipeline/README.md and docs/contracts/upstream-drift/README.md state the rule. Two changelog fragments (pr-pipeline minor, source-control patch).

Trust change

Before: an issue a Bot or App wrote has no spec, apart from the recurring-schedule issue and the size-split sub-issue.

After: an issue is the spec only when all of these hold.

  • Its author is the github-actions Bot (id 41898282).
  • It was never edited, or last edited by that same Bot.
  • Its title matches ^Upstream drift: ([A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+)$ exactly.
  • Its first body line equals the filer's hex-pin lead for that id. The vendored pin=unknown lead does not count.
  • The pull request closes it with a closing keyword; Refs: does not count.
  • Every file changed between the pull request's base and head has status modified and is a docs/upstream/*.md page whose **Last audited upstream state:** marker pins that id; at least one file and fewer than 300.

What is read from the issue: author type and id, last editor and edit time, the title, and the first body line by exact equality. The spec text is a fixed template naming only that id and page paths taken from main's own checkout. No row of the issue body, where names and paths from the outside repository live, is read, so text an upstream controls cannot reach the spec, and the file scope is checked in code.

What it does not cover: a re-audit that applies takes to plugin files (it changes files outside the pinned page), and callers that pass no changed-file read. The merge gate's hold on a closing issue that carries needs-human is unchanged.

Accepted risk: another workflow in this repository that wrote that exact title and lead under the github-actions identity could only point a spec at another already-pinned page.

Verification

  • resolve-pr.test.mjs: 7 new tests, 181 of 181 pass. They cover the accepted case, the Bot's own weekly edit, look-alikes (the same title from a person, an outsider or the lanes App; no lead line; the vendored lead; a different title; title and lead naming different ids; a body someone else last edited; an id no page pins), scope refusals (another upstream's page, an added unrelated file, a rename, a removal, an empty diff, the 300-file cap), a Refs: link, a failed compare read, and a caller with no compare.
  • tests/test_babysit_drift_spec_trust.py: 10 new tests for the gate twin, all pass; the *_trust.py set 32 OK; test_babysit_verify_evidence.py 441 OK; test_verify_rules_sync.py 19 OK.
  • pr-verify, pr-merge-risk and pr-unblock node suites: 1852 of 1853 in the build worktree; the one failure needs ajv, which that worktree had not installed.
  • The first head failed Test Python (1), Test Bash (5) and Check Skills on one cause: test_merge_holds_sync.py found docs/upstream/ loaded by a hold path and not on the floor. With the floor row added: test_merge_holds_sync, test_floor_paths, test_babysit_risk and both spec-trust modules, 206 tests OK (11 skipped); resolve-pr.test.mjs 181 of 181; sync-hold-floor --check clean. Two tests changed to keep asserting the same rule: python_closure walks only stand-ins that exist as files, and the per-glob risk test scores a concrete path each glob matches instead of the glob's literal text.
  • scripts/check-affected.sh --base origin/main --no-fetch: 18 passed, 1 skipped; the pre-push run passed again.

Related

🤖 Generated with Claude Code

https://claude.ai/code/session_01MWmASMXyUAaNx5GqGM3h6c

…own re-audit

pr-verify's resolve-pr.mjs takes an issue the upstream-drift workflow filed
(github-actions[bot], nobody else last editing it, title
`Upstream drift: <id>`, first body line the hex-pin lead
scripts/upstream-drift-issues.sh writes for that id) as the spec of a pull
request that closes it and only modifies the docs/upstream pages pinned to
<id> on main. The spec is built from the id and those page paths; no other
line of the issue is read. The babysit merge gate's _trusted_spec accepts the
same issue under the same changed-file scope, reading the compare from base
to head and each page at the default branch's tip, and reads each
github-actions issue once for this rule and the recurring-schedule rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MWmASMXyUAaNx5GqGM3h6c
@kyle-sexton
kyle-sexton marked this pull request as ready for review November 16, 2026 16:33
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-16T16:36:13.963332Z f525d75 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-refine/fix-docs
Model rung 0
Leg handoff
Run job success
Activity step success
Commits pushed none (read lane)
Run 43908945494

Failed GitHub API calls

Method Path Status
GET /repos/{repo}/git/ref/tags/* 404

@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

claude-review reviewed 243e259aaaff7b6c6f61f0a4af70a344926e3640: read 19 of 19 changed file(s); 0 finding(s), 0 as inline comments.

Code review: no blocking findings (blocking only)

This lane already reviewed an earlier head (f525d759), so this pass looked only for blocking findings. Advisory findings were dropped.

I read every hunk of all 19 changed files. Since the last head, the PR adds docs/upstream/** to the hold floor and regenerates the floor copies, and it updates the pinned floor tests.

The new floor row is wired the same way in every copy:

  • docs/conventions/pr-pipeline.yaml has a new enforcement row.
  • HOLD_PATH_CLASSES in merge-holds.mjs has the matching row.
  • The hold_patterns list in pr-require-checks.yml gains docs/upstream/*. In that bash pattern * crosses /, so it covers nested pages.
  • The trusted-path lists in pr-review-hosted.yml and pr-review-security-hosted.yml gain the glob.
  • Both carry-refuse lists gain it: CARRY_REFUSES in merge-holds.mjs and in babysit_merge_policy.py. Each sits right after .github/recurring-schedule.json, so the two lists stay in the same order.

The pinned test counts agree with each other:

  • test_babysit_risk.py counts 62 enforcement globs and 62 in FLOOR.review.
  • test_floor_paths.py counts 144 rows in total and 144 − 32 rows in the config.

Test changes:

  • test_merge_holds_sync.py: import_closure now records a directory it loads (docs/upstream/) as a stand-in file under it, so the floor check matches it against docs/upstream/**. python_closure now skips loaded names that are not files.
  • test_babysit_risk.py: the per-glob test now scores a concrete path each glob matches, not the glob's literal text, and asserts the match first.

Neither change weakens what the tests assert.

Unchanged since the last review: the resolver/gate trust rule (driftSpec / drift_issue / drift_scope / page_pin). The one known difference between the two copies, a page missing on main, still refuses the merge in both (gives no spec in the resolver, verify:unreadable in the gate).

Verification: I ran nothing. The PR body's test counts and its sync-hold-floor --check result are the author's claims and unverified. That includes the four new floor digests in test_floor_paths.py.

Spend (not a hold): each pull request that closes a drift-shaped issue costs one extra GitHub compare call, plus one contents read per changed page. A pull request touching docs/upstream/** now also needs a clean security-lane review bound to its head before it merges. Both draw on API rate limits and existing lane runs, not on billing.

Scope: whether github-actions[bot] authorship is strong enough to trust is left to the security lane.

Verifier dropped 0 of 0 candidates.

@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

claude-security-review reviewed 243e259aaaff7b6c6f61f0a4af70a344926e3640: read 19 of 19 changed file(s); 0 finding(s), 0 as inline comments.

Security review: PR #16561 (an upstream-drift issue as the spec of its own re-audit)

No security issues found. I read all 19 changed files in full in the diff. This was a blocking-only review, because the lane already reviewed head f525d759 (no findings there). Only CRITICAL or IMPORTANT findings would be reported here.

What changed since the last review

Nine files are new to this review. They add docs/upstream/** to the hold floor and to the carry-refuses list. These files carry the change:

  • merge-holds.mjs HOLD_PATH_CLASSES and CARRY_REFUSES
  • babysit_merge_policy.py CARRY_REFUSES
  • pr-pipeline.yaml merge floor row
  • the generated hold_patterns in pr-require-checks.yml
  • both review callers' hold-paths

The remaining new files are test pin updates. Each of these edits adds a path, so a pull request touching docs/upstream pages now holds as review-path: and stops an approval from carrying. That narrows the holds and lets nothing new through. It is also the right companion change: the resolver now reads those pages as trust input (resolve-pr.mjs#L116-L137).

The test-helper changes in test_merge_holds_sync.py (skipping directories and files that don't exist) only change what the test walks. No gate reads them.

The trust rule (untrusted-spec hold, a kept security control), checked again

  • Who the issue can come from. The author must be github-actions[bot] (databaseId === ACTIONS_BOT_ID). If the body was ever edited, the last editor must be that same Bot. The title must match exactly, and the first body line must equal the hex-pin lead exactly (resolve-pr.mjs#L147-L160; the gate's twin drift_issue is at babysit_verify_evidence.py#L614-L640).
  • What reaches the verifier. The spec is a fixed template. It names only the regex-limited upstream id and page paths taken from main's own checkout. No row of the issue body is read.
  • Which files are allowed. Every changed file must have status modified and be a page that pins that id on main. There must be at least one file and fewer than 300.
    • In the resolver, filenames must be in the own set built from main's checkout.
    • In the gate, drift_scope runs the path regex and the status check before _page_pin reads the page at the default-branch tip (babysit_verify_evidence.py#L654-L669, #L750-L784).
    • A crafted filename (for example, one carrying ?ref=) would also have to exist already on main to have status modified. A pull request author cannot plant one.
  • Failure handling. It fails closed: a compare that can't be read, or a page missing on main, raises incomplete / _Unreadable and never passes. A drift issue linked only by Refs: gets no spec. The needs-human hold on a closing issue is unchanged.
  • Who can use it. The verifier still judges only pull requests from the lane or the owner. Even when the rule applies, the change stays limited to pinned docs pages that already exist. That reaches no leak, compromise or charge.

Other notes

  • No standing instruction is deleted or narrowed. The prompt.md and docs edits only describe the new rule, so the protected-class register was not needed.
  • The PR context contains no embedded instructions.
  • Spend: one extra compare call and a few page reads per drift-linked pull request, all within the existing lane runs.

critical-findings: 0

kyle-sexton and others added 3 commits November 16, 2026 12:20
resolve-pr.mjs upstreamPages reads docs/upstream from its checkout, so the
pages join the floor as an enforcement row and CARRY_REFUSES in both
mirrors, as .github/recurring-schedule.json does. The hold-floor copies are
regenerated, the floor pins follow the new row, the import walk treats a
directory load as a file in it, and the review-clear test scores a path
each enforcement glob matches.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tream row

The floor gained one enforcement row, docs/upstream/**, because the
drift spec rule reads those pages inside a floored module. The owner
applied the three pinned values (the row count and two digests) by hand.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MWmASMXyUAaNx5GqGM3h6c
@kyle-sexton
kyle-sexton added this pull request to the merge queue Nov 16, 2026
Merged via the queue into main with commit 156fc1e Nov 16, 2026
81 checks passed
@kyle-sexton
kyle-sexton deleted the fix/pr-verify-drift-issue-spec branch November 16, 2026 18:08
@github-actions github-actions Bot added the merge-path: owner-agent Post-merge accounting record (README.md, Merge paths) label Nov 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-path: owner-agent Post-merge accounting record (README.md, Merge paths)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant