Skip to content

policy-sign does not cover the rest of .github/agent/ rule data or the tier runner itself #3095

Open

Description

@HugoXK

Implementation: FlyBot (ours) — PR follows within 24h. Do not pick up without a pickup-ok comment on this issue.

Written-by: FlyBot Lead
Claimed-by: FlyBot Lead
Claim-Id: cc02a85fb29b407693d723d7dcb3ea5f
Branch: dev/1-policy-sign-agent-rules
Base: 3e901fa
At: 2026-11-15T01:08:23Z

Bot: FlyBot Dev-1
Tier: T3 (.github/** paths; the PR also edits policy/gates.json, which is itself a gate file)

Dedupe (live main 3e901fab + closed issues): gateFiles → #3091/#3092 (ours, in flight, entry-level: .github/agent/risk-rules.json + .github/scripts/risk-gate.test.mjs), #2731 (a composite action test.yml uses), #2124 (deleting the signature step), #2671 (a parse gap), #2093/#2095 (closed, gate scripts) — none covers the directory or the tier runner. controller.json → only closed #105. risk-tier.yml → 17 hits, all closed, none about making it a gate file. "review-policy" → #2544/#2547 (PASS writer case) and #2837/#767/#627 (roster/writers semantics), none about gating the data. .github/agent in bodies → #3091/#3092 only. No open issue or PR covers this.

Current state

.github/agent/ holds four JSON files, all rule data the gates read: risk-rules.json (risk tiers), review-policy.json (who may record a PASS, and the writers roster), controller.json (maintainers — the only people whose edits keep an issue agent-ready — plus agentBranchPrefixes, agentAuthors, claimTtlHours), and merge-gate-ruleset.json (the ruleset gate-ruleset.mjs --check compares against).

Two of those files carry the sentence "Editing this file is itself T3". That sentence does not lock anything: the risk tier decides which gate a change is judged by, not whether it needs a human signature. policy-sign reads policy/gates.json gateFiles, and none of the four is on that list — gateFiles() returns [] for each of them and judge() answers ok: true — no gate file changed. #3092 is the live example: it moved new Function\( / \beval\( between the two test-boundary lists in risk-rules.json and policy-sign stayed green.

The same holds for .github/workflows/risk-tier.yml: the tier runner itself, report only, not in tests-ok's needs. It is the job that writes the risk-tier: … line a review quotes, and a pull request can rewrite it with no signature.

Fix

Two entries in policy/gates.json gateFiles, inserted in place (no reordering, policy/ not re-added):

  1. .github/agent/ — a directory prefix entry, so it covers the four files above and any rule data added later, by construction rather than by remembering to add a line. "A rule that does not cover its own switch is advice, not a gate" applies to the directory as a whole: the rules that decide what counts as ready, what counts as a PASS, and what counts as T3 live there.
  2. .github/workflows/risk-tier.yml — the one unsigned tier evidence producer: report-only, out of tests-ok's needs, and the source of the risk-tier: … line reviews cite. Naming the file only; no workflow content changes.

Nothing else: policy-sign.mjs, risk-gate.mjs, the four rule JSONs, the workflow body and every other file class stay untouched.

Directory semantics (binding, from the lead)

.github/agent/ holds rule data only — no documentation, no temporary files, no test fixtures, no notes. Anything that needs explaining goes in the issue, the PR body, or docs/. Reason, recorded so it is not rediscovered: whatever lands in that directory is now behind a human signature; a README would make a one-line doc change cost a signature, and the predictable next step after that is a proposal to remove the whole rule.

Red first

At base 3e901fab, with the source untouched and the tests added, node --test policy/gates.test.mjs must be red, twice, on:

  1. gateFiles(['.github/agent/risk-rules.json']) → [] today;
  2. gateFiles(['.github/agent/review-policy.json']) → [];
  3. gateFiles(['.github/workflows/risk-tier.yml']) → [];
  4. the judge() counterexample: a body that changes reviewers in .github/agent/review-policy.json and carries no Policy-sign: line is accepted today (ok: true, "no gate file changed").

With the two entries added: 1–3 return those paths, and 4 is refused. The same command is green twice.

Acceptance

  • The typo guard, as a test and not an eyeball: the directory entry must match real files — gateFiles(['.github/agent/risk-rules.json']) and gateFiles(['.github/agent/review-policy.json']) are both non-empty. A path entry misspelled is the one failure mode that never reports itself: it keeps returning [] and nobody notices until the hole is used.
  • Counterexample through judge(), not just gateFiles(): a pull request whose body changes review-policy.json's reviewers (or controller.json's maintainers) and carries no Policy-sign: line is refused, and the same change with a line naming a signable comment that names the tip is accepted.
  • Negative control: an ordinary client file (plugins/social/client.js) is not a gate change; policy/ still covers a new file under it; no entry is duplicated.
  • TZ=UTC npm run verify exit 0 at the tip; the diff is policy/gates.json + policy/gates.test.mjs only; no content change to any of the four rule JSONs or to risk-tier.yml.

Cost, accepted by the lead

From this landing, changing reviewers/writers in review-policy.json, or maintainers in controller.json, needs a human signature like any other gate change. That is deliberate: those are privilege rosters, and the seat list is exactly what should not move unsigned. When a new Dev seat is activated, the roster entry goes in the same gated change and is announced first, so it never becomes a last-minute blocker for a new Bot's first PR.

Landing note

T3, so the reviewer's Verdict has to name the tip; and because the PR edits policy/gates.json itself, its own policy-sign stays red until a person writes Policy-sign: <comment id> naming a comment that names this tip. That is the intended shape — the owner keeps the veto on widening the signature surface. Bot accounts cannot sign, so no one on the team writes that line.

Sequencing: cut the branch from live main after #3091's PR merges — both edit policy/gates.json and policy/gates.test.mjs. #3091's .github/agent/risk-rules.json entry is subsumed by this directory entry (it will sit inside it); its .github/scripts/risk-gate.test.mjs entry is not (that path is outside this directory), so it stays. This issue therefore lists neither of them.

Activity

  1. HugoXK commented on Nov 15, 2026

    @HugoXK
    ContributorAuthor

    Heartbeat

    Written-by: FlyBot Lead
    Still-working: cc02a85fb29b407693d723d7dcb3ea5f
    At: 2026-11-15T02:46:26Z

  2. HugoXK commented on Nov 15, 2026

    @HugoXK
    ContributorAuthor

    Heartbeat

    Written-by: FlyBot Lead
    Still-working: cc02a85fb29b407693d723d7dcb3ea5f
    At: 2026-11-15T04:45:30Z

  3. HugoXK commented on Nov 15, 2026

    @HugoXK
    ContributorAuthor

    Heartbeat

    Written-by: FlyBot Lead
    Still-working: cc02a85fb29b407693d723d7dcb3ea5f
    At: 2026-11-15T06:22:01Z

  4. HugoXK commented on Nov 15, 2026

    @HugoXK
    ContributorAuthor

    Heartbeat

    Written-by: FlyBot Lead
    Still-working: cc02a85fb29b407693d723d7dcb3ea5f
    At: 2026-11-15T06:44:58Z

  5. HugoXK commented on Nov 15, 2026

    @HugoXK
    ContributorAuthor

    Heartbeat

    Written-by: FlyBot Lead
    Still-working: cc02a85fb29b407693d723d7dcb3ea5f
    At: 2026-11-15T08:45:51Z

  6. HugoXK commented on Nov 15, 2026

    @HugoXK
    ContributorAuthor

    Heartbeat

    Written-by: FlyBot Lead
    Still-working: cc02a85fb29b407693d723d7dcb3ea5f
    At: 2026-11-15T12:45:09Z

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions