Skip to content

feat(runtime): audit publication invariants after every publishing stage (#261) - #365

Merged
datacas merged 1 commit into
mainfrom
feat/261-publication-invariants
Oct 29, 2026
Merged

datacas merged 1 commit into
mainfrom
feat/261-publication-invariants

Conversation

@datacas

@datacas datacas commented Oct 29, 2026

Copy link
Copy Markdown
Owner

Work item: #261 — security(runtime): verify publication invariants after each publishing stage.

Closes #261

Summary

After every publishing stage (implement, review, resolve, rereview), whatever status the stage reported, run_cycle.py compares the audited state with its state before the stage. It uses the new scripts/publication_audit.py:

  1. Change requests: every change request of the cycle that was open is still open, and any that the stage created is open. A merge or a close is a breach.
  2. Cycle branch: it did not move, or it moved fast-forward. A non-fast-forward move or a deletion is a breach. implement may create the branch.
  3. Default and protected branches: any move (even a fast-forward) or deletion is a breach.
  • The audited refs are the cycle branch, the default branch and the protected branches the host reports. They are read on origin only, once before the stage, and the same set is compared after it. A push to an unrelated branch passes (refactor(F0): limit the read-only review guard to the refs of the change under review #197).
  • A breach stops the cycle with the new publication_violation, whoever caused it. The stop line says "changed outside the stage?". The audit runs before the stage's report is read, so a breach wins over BLOCKED/stage_not_completed, the order fix(runtime): reconcile a publishing stage that reported BLOCKED after a verified publish #364 depends on. No later stage runs.
  • Evidence: CycleReport.publication_audit holds the before/after SHA per ref, the before/after change-request state, and each breach's kind with up to 20 added and removed commits. The cycle journal gets one publication_audit row per stage and one failure row per breach.
  • When the state cannot be read, the cycle stops with the new publication_unverified. That covers a failed protected-branch query, a failed change-request read, and missing commits that leave a move uncomparable. Nothing is read as "none". If the read fails before the stage, the stage is not dispatched.
  • Bitbucket and other hosts: the cycle branch and the default branch are still audited through Git. Change-request state and protected branches are reported unverified, never as passed, until Make provider capabilities first-class and close 0.6.0 workflow gaps #238.
  • No --disallowedTools or other adapter-level denies were added. Those belong to Recommended host permissions for Claude/Codex and per-project generalized approvals #142.
  • Telemetry schema 24 adds the two stop reasons. They are payload-only.

Diagnosis

Work units

Unit Behaviour Files Tests Verification Rollback
WU-1 Post-stage publication audit, its stop reasons, telemetry schema 24 and docs scripts/publication_audit.py, scripts/run_cycle.py, scripts/telemetry.py, scripts/runtime.manifest, scripts/windows-test-shards.json, docs/role-workspace-policy.md, docs/instrumentation.md, docs/review-cycle.md, CHANGELOG.md, tests/test_publication_audit.py, tests/test_work_item_outcomes.py tests/test_publication_audit.py (25 tests: local bare remotes plus a fake gh) see below independent. Rows written with schema 24 stay readable, because the stop reasons are payload-only.

Verification (agent-reported)

Command Exit Result
python3 -m unittest tests.test_publication_audit tests.test_installed_cycle 0 43 tests OK
python3 -m unittest discover -s tests 0 2049 tests OK
python3 scripts/validate-package.py 0 VALID
python3 -m unittest 0 ran 0 tests: from the root it discovers nothing; discover -s tests above is the real suite

The acceptance criteria are covered by tests:

  • A fake executor's stage merges the PR, force-pushes the cycle branch, pushes to the default branch or deletes a ref. The cycle stops with publication_violation and its evidence, and no further stage runs.
  • A resolve that pushes fast-forward commits passes, and so does a push to an unrelated branch.
  • A BLOCKED report with a merge stops with publication_violation.
  • A host with no reader is reported unverified, and its refs are still audited.

Environment the local run shares with CI: the tests use temporary bare repositories with their own Git identity and an in-memory fake gh. They need no global Git config, credentials or network. The local run is Linux only, so the Windows shards were not run locally. The new module is listed in windows-4.

Residual risk

  • The audit reads the state twice, before and after the stage. A ref that is changed and then restored within one stage goes unseen.
  • Each publishing stage adds a few gh api and git ls-remote calls.

…age (#261)

After every implement, review, resolve and rereview dispatch, whatever the
stage reported, the runtime compares the audited state on origin with its
state before the stage: the cycle's change requests stay open, the cycle
branch only moves fast-forward, and the default and protected branches
neither move nor disappear. A breach stops the cycle with
publication_violation and before/after evidence; an unreadable state stops
it with publication_unverified. Hosts other than GitHub keep their refs
audited and report the rest as unverified. Telemetry schema 24.

Closes #261
@datacas

datacas commented Oct 29, 2026

Copy link
Copy Markdown
Owner Author

Implementation summary for #261: post-stage publication audit (scripts/publication_audit.py), wired into every publishing stage in run_cycle.py. Two new stop reasons, publication_violation and publication_unverified, and telemetry schema 24.

Pushed head: 097a6c56d3d81de16f6cfb684f76ac3a8d994ed1. The upload went through publication.py push (HTTPS), and the remote ref and the PR head both match this SHA.

Local verification (agent-reported):

  • python3 -m unittest discover -s tests: 2049 tests, OK.
  • python3 scripts/validate-package.py: VALID.
  • python3 -m unittest: discovers 0 tests from the root.

Checks of the pushed head: none reported yet when they were read once after publication (CI policy parallel). The cycle's CI gate decides.

Routing: profile deep_coder, requested anthropic/claude-opus-5-5, resolved ?, effort high.

@datacas

datacas commented Oct 29, 2026

Copy link
Copy Markdown
Owner Author

Cycle note (not a review): the implement stage ended BLOCKED after publishing this change request. Two incidental commands were refused by the stage permissions (gh pr comment 365 and a read of run_cycle.diff); the implementation itself was not reported as failed.

Before resuming, the local HEAD, origin/feat/261-publication-invariants and the PR head were all 097a6c56d3d81de16f6cfb684f76ac3a8d994ed1. SHA agreement does not show the change is correct, so the cycle resumes at review (--from review --pr 365), as the owner decided. This is the case #364 describes.

@datacas

datacas commented Oct 29, 2026

Copy link
Copy Markdown
Owner Author

Initial review — PR #365 (work item #261)

[CCR-20261029-001] · senior_reviewer · anthropic/claude-opus-5-5→? · high · schema:2

Verdict: REVIEW_PASSED_AWAITING_CI for head 097a6c56d3d81de16f6cfb684f76ac3a8d994ed1. The review is complete for this head and found no finding that blocks approval. One required check is still pending, so this is not an approval. The cycle's CI gate decides.

Summary and scope

I reviewed the whole accumulated diff from merge base 6ade80932ce5a5ae91d83b1680f8cbcbefbc942c to 097a6c5: 11 files, +980/−9, 1198 diff lines. The diff file supplied by the runtime matched its stated line count. I read it in full.

  • scripts/publication_audit.py (new): PublicationAuditor.capture/verify and GitHubHostReader, which uses gh api --paginate --slurp.
  • scripts/run_cycle.py: advance() brackets each implement/review/resolve/rereview dispatch. It runs audit_publication before unfinished() reads the report, adds the CycleReport.publication_audit evidence, and wires the auditor in main() for every run except --local-only.
  • scripts/telemetry.py: schema 24 with two stop reasons. runtime.manifest and the Windows shards were updated as well.
  • Docs: docs/role-workspace-policy.md (Publication audit), docs/instrumentation.md and docs/review-cycle.md, plus CHANGELOG.md. The documentation rule in AGENTS.md is met.

I checked these points and they hold:

  • Every publishing dispatch goes through advance(). Its callers are at run_cycle.py:3939/3996/4003/4053/4142/4165, plus the recursive recovery path at :3335. review_issue() runs the non-publishing issue_review role and is correctly left out.
  • args.cwd is always set to the selected workspace (run_cycle.py:5199) before the auditor is built, so normal runs are audited.
  • A breach is checked before unfinished(), so it wins over BLOCKED/stage_not_completed. publication_violation stops with HUMAN_INTERVENTION, which matches the Exit table in review-cycle.md. publication_unverified stops with BLOCKED.
  • Unreadable state fails closed. A failed ls-remote, missing symref, gh error, malformed JSON or uncomparable commit raises AuditUnreadable, and that error is never read as empty.
  • Only origin is read, and only the named refs, so a push to an unrelated branch passes (refactor(F0): limit the read-only review guard to the refs of the change under review #197).

Findings

[REV-001] · medium · open · - · blocks:no — A cycle branch the host reports as protected cannot move forward

scripts/publication_audit.py:481-492

verify() treats a ref as strict, meaning any move is a breach, when name == before.default_branch or name in before.protected. It does not exclude before.cycle_branch. protected_branches() uses GET repos/{repo}/branches?protected=true. GitHub documents that this returns branches protected by branch protection or rulesets. A repository with a ruleset or protection pattern that covers every branch would therefore report the cycle branch as protected, for example a ~ALL ruleset that blocks force pushes or requires signed commits. Each ordinary fast-forward push by resolve would then stop the cycle with publication_violation/protected_ref_moved, and the stop line would report a breached invariant that did not happen. The documented rule for the cycle branch, "may move fast-forward", conflicts here with the rule for protected branches, "any move is a breach", and neither the code nor docs/role-workspace-policy.md says which one wins. No test covers a cycle branch that is also protected.

  • Evidence: code inspection. A runtime reproduction would use the PR's own FakeGh with protected = ["main", BRANCH] plus Remote.push_forward(). I could not run it because this review stage cannot write helper scripts. I also could not check whether this repository's own ?protected=true listing includes feature branches, because the stage policy does not allow that gh api call.
  • Why it does not block: the failure is fail-closed (the cycle stops; it never passes a real breach), and it occurs only under that host configuration.
  • Suggested direction: apply the cycle-branch rule to before.cycle_branch even when it is also listed as protected. That rule is: a fast-forward passes, a non-fast-forward move or a deletion is a breach. Document the precedence and add a test. If strictness is preferred, say so in the docs, since such repositories cannot run the cycle.

Security triage

The deterministic rule did not fire. .code-cycle.yml declares no security_review.always_when, so the defaults applied, and no changed path matches the mandatory minimum: no CI or automation files, no secrets or keys, no agent or toolkit configuration.

The reviewer's judgement fired the audit. The change is security tooling: it is the runtime's last line of defence against a stage merging, force-pushing or deleting refs. cc-security-review ran degraded, as an equivalent manual pass inside this review. Result: no confirmed vulnerability.

  • Every subprocess call uses an argv list, never a shell.
  • The head= query is URL-encoded.
  • change_request_id is normalized to a number by _pull_number before it is interpolated into the gh api path.
  • Remote branch names reach Git only as refs/heads/<name> refspecs.
  • ls-remote output is parsed, never executed.
  • Error messages carry exit codes and paths only, never tokens or bodies.
  • The audit fetches into FETCH_HEAD only and leaves no ref behind.

The fail-open paths I looked for all fail closed instead: a missing default branch on a non-empty remote, malformed JSON, and missing commits.

Commands executed

Command Exit Result
python3 -m unittest discover -s tests 0 Ran 2049 tests in 182.048s, OK. This includes the 25 new tests in tests/test_publication_audit.py.
python3 scripts/validate-package.py 0 VALID: 16 skills (7 cycle, 9 supporting), shared sections in sync, cross-agent manifests aligned
python3 -m unittest 0 Ran 0 tests. Discovery from the repository root finds nothing, so this configured command checks nothing. The suite is the discover -s tests row above.

Unverified

  • REV-001 is backed by code inspection only. The reasons are given in the finding.
  • The Windows shards and the PowerShell installer did not run locally. The run was on Linux only, and CI covers them.
  • gh api --paginate --slurp needs a gh version that supports --slurp. I did not check this against a minimum gh version.

CI (head 097a6c5, policy parallel, read once)

Check State
Skills, manifests, and Bash installer pass
Windows test shard (windows-1) pass
Windows test shard (windows-2) pass
Windows test shard (windows-3) pass
Windows test shard (windows-4) pass
PowerShell installer pending

The cycle's CI gate waits for the pending check and decides. This review neither waited for it nor reads it as success.

Residual risks

  • The before and after snapshots cannot see a ref that is changed and then restored within one stage. The PR body states this.
  • During implement, only the checkout's branch is audited as the cycle branch. If the implementer pushes a branch under another name, that branch is first audited from the next stage on, through the change request's head_ref. The new change request's open state is still checked.
  • Under the documented design, any move of the default branch during a stage stops the cycle, including a legitimate merge of another pull request. Long stages in busy repositories may stop more often. This follows the decision taken on security(runtime): verify publication invariants after each publishing stage #261.

@datacas
datacas merged commit ccc2f9d into main Oct 29, 2026
6 checks passed
@datacas
datacas deleted the feat/261-publication-invariants branch October 29, 2026 14:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

security(runtime): verify publication invariants after each publishing stage

1 participant