Skip to content

Stop an empty preflight plan from granting merge and completion (#610) - #925

Open
pengfei-threemoonslab wants to merge 1 commit into
mainfrom
claude/issue-610-planning-complete
Open

pengfei-threemoonslab wants to merge 1 commit into
mainfrom
claude/issue-610-planning-complete

Conversation

@pengfei-threemoonslab

@pengfei-threemoonslab pengfei-threemoonslab commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Why

A coding agent that asked preflight about a plan naming nothing — {"changed_files": []}, no flags at all, or shipgate.preflight with no arguments — got back the shared complete state, and that state's permission vector grants merge and report_complete, with no verifier run and no current-control identity behind it. Leaving files out of a plan therefore read as merge authority. The published 0.5 schema also rejected that very payload, because it pinned update_pr to false.

Reproduced on this branch's base (0e98f41d, contract 41):

$ agents-shipgate preflight --workspace . --plan plan.json --json   # {"changed_files": []}
"state": "complete", "completion_allowed": true,
"permissions": {"edit": true, "commit": true, "push": true, "update_pr": true, "merge": true, "report_complete": true}

A README-only plan already returned agent_action_required with an exact verify route and every permission false; the defect was only the empty-plan route.

What changes

The owner chose planning-only completion with no publication or merge authority (#610 comment, docs/research/application-days1-5/README.md). This PR implements that across the model, schema, projections and consumers:

  • Preflight 0.6 (PreflightResultV6, docs/preflight-schema.v0.6.json) gives control its own union: planning_complete (new), agent_action_required, human_review_required. complete and review_publishable cannot appear.
  • planning_complete means only that planning finished: no next_action, no commands, completion_allowed/must_stop/verify_required false, and a required, all-false permissions. Its reason says it is not a verification and authorizes nothing.
  • Every permission is false on every preflight route, enforced by a model validator and an allOf rule in the generated schema (which also requires the vector to be present). The schema mirrors each route's legacy projection, so the runtime payload and the published grammar agree for empty, verify-required and human-review plans.
  • The shared AgentControl union is untouched (it is embedded by six durable schemas). minimum_control_contract_version stays 21: a reader that does not know planning_complete cannot mistake it for complete, and the vector beside it denies everything.
  • One version ladder. The CLI's --base-preflight reader kept its own copy of the version dispatch, which would have refused a saved 0.6 answer. Both readers now go through parse_preflight_result. A stored 0.5 answer still reads as 0.5 (complete included); relabelled 0.6, it is refused.
  • Consumers.
    • The install-hooks script accepts preflight 0.5 and 0.6. A hook written before this change accepts only 0.5, so its instruction-structure check fails closed until install-hooks --write is re-run. This is documented in the migration note.
    • The adoption scorer reads planning_complete as itself, rather than as an unexecutable agent_action_required obligation.
    • The CLI text adds Authorizes: nothing (only verify can authorize merge or completion).
  • Contract 41 → 42. Preflight 0.5 is frozen and pinned by digest in FROZEN_CONTROL_SCHEMAS. Docs updated: STABILITY migration note, CHANGELOG, agent-contract-current.md, .well-known, INDEX, llms.txt/llms-full.txt, MCP/protocol docs and the per-agent guides.
  • AGENTS.md gains one planning_complete sentence in the preflight paragraph, and its schema-table row moves to 0.6. Owner approved both edits in conversation on 2026-10-01, as preflight routes AGENTS.md to human review.
  • Pilot ledger. The Route H dry run was re-run on contract 42 beside main (contract 41). Cells are identical apart from the contract number; preflight is not one of that route's cells.

Acceptance (from #610)

  • An empty or docs-only plan cannot be used as evidence that a change was verified or is mergeable. Seven empty-plan input forms (CLI, core, MCP) return planning_complete with every permission false; the docs-only plan still owes verify.
  • Runtime payload and published schema agree for empty, verify-required and human-review plans. The positive validation runs first, then 12 negative controls, each refused by both the model and the schema: any permission granted, the pre-0.6 complete shape, a missing vector, a planning result that owes verify, a publishing verify route, a human route granting merge, and review_publishable.
  • Consumers have one explicit way to tell planning completion from a verifier-bound vector: the planning_complete state, beside an all-false vector.
  • Required-verify and human-review routes are not cleared by omitting a plan or by replaying an old planning response. Host-grant drift and trust-root drift still stop an empty plan. A replayed 0.5 complete answer used as a base cannot clear a protected-surface, docs-only or empty plan.

Validation

  • tests/test_preflight_planning_only.py (new, 30 tests) and a harness scorer test (tests/harness/test_detectors.py). The scorer test fails without the scorer change, which was checked.
  • ruff check ., git diff --check and scripts/generate_schemas.py --check all pass.
  • Full suite (-n auto -m "not perf" --ignore=tests/test_adapter_static_only.py): everything passes except two test_check_unmodelled_host_config_keys.py[local_settings_enabled_plugins-*] cases. Those fail only on this machine, whose global gitignore hides .claude/settings.local.json from the fixture's commit, and they pass with XDG_CONFIG_HOME=<empty dir>. The separately run CI files (test_adapter_static_only, both P0 canary files, test_agent_boundary) pass. The self-review fixes after the full run are a refusal of a non-object base and a wording change; the affected files were re-run.
  • Self-review, before the PR: a base preflight that is not a JSON object is now refused by name rather than passed through, and the planning-only reason says "no changed file", which also covers a diff that names no file.

Closes #610

🤖 Generated with Claude Code

A preflight plan that named nothing to route returned the shared `complete`
state, whose permission vector grants `merge` and `report_complete`, with no
verifier run or current-control identity behind it. The published 0.5 schema
also refused that payload, because it pinned `update_pr` to false.

Preflight 0.6 gives `control` its own union: the new `planning_complete`,
`agent_action_required` and `human_review_required`. `planning_complete` owes
no action and authorizes nothing. `complete` and `review_publishable` cannot
appear, and every permission is false on every route, in the model and the
generated schema alike. The shared AgentControl union is unchanged, so
minimum_control_contract_version stays 21.

- Both base-preflight readers now parse through one version ladder
  (`parse_preflight_result`). A stored 0.5 answer still reads as 0.5.
- The hook script accepts preflight 0.5 and 0.6.
- The adoption scorer reads `planning_complete` as itself.
- The CLI text says the result authorizes nothing.
- Contract 41 -> 42. Preflight 0.5 is frozen and pinned by digest. Docs,
  STABILITY migration note and CHANGELOG are updated. The Route H dry run was
  re-run on contract 42.
- AGENTS.md edits were approved by the owner in conversation on 2026-10-01.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Preflight: empty plan exposes merge/completion permissions without verifier identity

1 participant