Add an exercised host-only advisory PR recipe with no manifest or saved baseline

[pengfei-threemoonslab]

Observed

The 2026-09-14 review at e2ab00078b2ca5b9e73405109a6f09b820f5fc9e found a gap between the shipped host-review engine and the integration a first team is told to copy:

• examples/github-actions/README.md:3 describes its recipes as requiring a root shipgate.yaml; the recommended first recipe is presented as adding a gate.
• docs/integrations.md:22–31 starts with config: shipgate.yaml, although the released Route H measurement in docs/design-partner-pilot-results.md:95–97 records successful manifest-free verify and six advisory change rows.
• The host-grant recipe is scheduled baseline/drift, a different job from reviewing the current PR against Git history. Local diff already needs neither a manifest nor a committed baseline.

The product has a manifest-free per-change path, but a team that got value locally has no equally clear, exercised first PR recipe. A GitHub App is not required to fix that gap.

Why it matters

Repeated use should arrive on the next relevant PR without a developer remembering another audit command. Requiring declarations or baseline maintenance before that happens creates adoption work unrelated to the selected host-review job. The first integration must show what the current PR changed, explain unread coverage and let the team's reviewer decide.

Proposed

Add one copyable host-only advisory PR recipe to the existing GitHub Actions examples, using the current composite Action and existing manifest-free verifier/PR projection. Pin the Action source ref and package version from the tested released provenance rather than maintaining another handwritten version constant. Do not introduce a new renderer, Action wrapper, hosted service or gate vocabulary.

Make the base/head checkout explicit and available locally (fetch-depth: 0 or an equivalently exercised bounded fetch). The engine never fetches. Exercise the actual composite Action, not a script that merely reconstructs its expected CLI arguments. A manifest or pre-existing host baseline must be absent in the tested repository.

Default to advisory output, with no required check, fail-on-verdict policy or generated host instruction. Show one human PR summary naming changed capabilities, evidence, coverage limits and next action. Keep detailed machine output in the existing artifacts. Explain the minimum contents/PR-comment permissions, and preserve the workflow-summary/artifact fallback when a fork or token cannot post a comment. Do not ask for broad write permissions or use an untrusted-code execution path to make a fork appear supported.

Let the existing in-job trigger/comparison decide applicability; do not add a separate path allowlist that can silently miss the engine's supported cases. A correct no-change run stays quiet or gives a compact no-change status. An incomplete comparison, missing base or execution error is visible and distinguishable from no change, even though findings do not make this default advisory integration a merge requirement.

Acceptance

• The new complete recipe runs in a repository with supported host configuration and no shipgate.yaml or committed baseline. README/quickstart link it as the next step after local host-diff value, while the existing builder/strict recipes retain their own scope.
• A real GitHub-hosted run exercises the pinned public Action and package on exact base/head refs. Preserve the run URL, version/ref, outputs and test inputs; a source-only unit test cannot stand in for this observation.
• Changed permission/MCP examples agree with the same-version local CLI on subject, direction, expansion semantics and coverage; include a narrowing case so every row cannot become a widening. No-change and malformed known input have distinct outcomes.
• A second update to the PR produces the current result without accumulating duplicate actionable comments. An unrelated or covered no-change update does not create a new warning; missing history and unread inputs never disappear as quiet success.
• Advisory defaults do not install a blocking policy or require the check. Execution/setup failure remains explicit and the documentation distinguishes it from a review finding or merge decision.
• The recipe uses the minimum demonstrated permissions and records the tested same-repository/fork behavior. With comment permission absent, a reviewer can still find the same evidence and the limitation is explicit.
• No user code, agent or declared MCP server is executed to analyze the PR. Installation uses the pinned trusted release, not package code selected from an untrusted PR checkout.
• Run the published advisory host-diff pilot: first reviewer value and the next eligible change #571 records actual setup time, assistance, continued enablement and second-reviewer use when a participant opts into this recipe. The hosted test proves integration behavior, not external adoption.

Sequencing

The same hosted run should satisfy #570's still-open literal post-publication tag/Action replay and be linked there; record tag resolution as well as the generated immutable-SHA default. No second harness is needed.

P1 in #778, alongside #779 and #781 in the first week. #779 owns the first local route; this issue owns the concrete team-workflow handoff. #653/#571 can begin human local observations before this recipe is finished. Reuse #684/#662's delivered host comparison and PR projection; do not reopen their completed implementations unless this exercised recipe reveals a bounded defect.

#664 remains conditional on useful repeat use and an observed obstacle that survives this working Action route. #690 stays paused. Baseline governance, authenticated approval (#337/#555), required branch checks and organization policy are separate choices, not default setup for this recipe.

Journey delivery - 2026-09-16

Program: #778. Delivery milestone: Adoption Phase 2 - Decide and verify the correction. Accountable role: GitHub integration/release engineering.

The recipe and source repair in #786 are delivered. Finish only remaining evidence against the exact distributed Action/package: actual hosted PR output, fork/minimal-permission fallback, second-push sticky update, artifact/summary agreement and safe install imports. Reuse this hosted observation for #570 rather than building a second harness. The owner's existing hosted-run deferral is retained until an authorized execution is selected; a milestone is not a completed hosted run. #811's local journey and human attempts continue independently. Offer CI through #796 after first value, not as a prerequisite.

Planning checkpoints preserve #778's original 2026-09-14 start: Oct 14 / Nov 13 / Dec 13. They are evidence-review dates, not promises about external replies. Phases overlap; a later milestone never delays supported research, an already-possible correction or an individual four-week opportunity window. A named accountable owner is required before execution; the role here is not an assignment to an uninvolved person.

[pengfei-threemoonslab]

Status — 2026-09-14. #786 adds the recipe, docs and a test that runs the recipe's inputs through action.yml's own step on manifest-free PR branches, plus agents-shipgate==1.0.0 runs of the same verify invocation. The owner decided to skip the GitHub-hosted run for now, so the hosted acceptance items (run URL and outputs on exact refs, second-push comment behaviour, fork/permission observation) remain open and unclaimed here. A defect found while isolating the route is filed separately as #785.

[pengfei-threemoonslab]

PR #786 was squash-merged as 3131c9c30c47e98d5eebde9e08f8038f209c4afa after three independent review/address cycles and green required checks.

Delivered

• examples/github-actions/14-host-only-advisory-pr.yml: no manifest or baseline, ci_mode: advisory, diff_base: target, sticky PR comment.
• A README section covering the comment shapes, what fails the job, trust, and SHA pinning.
• Hand-offs from the README, quickstart and integrations pages.
• The composite Action now runs its Python steps with python -P, so a PR cannot plant pip/ or agents_shipgate/ modules in the workspace and have them run before the engine installs. Non-GitHub CI recipes run python -m pip from the untrusted checkout, so a PR-supplied pip/ package can execute before install #787 tracks the other recipes.
• Tests run the Action's steps locally: the narrowing, not-compared, missing-history and invalid-manifest cases, plus parity with diff.

Still open (as of 2026-09-14)

• The GitHub-hosted run of the recipe on a real pull request (the fork/403 comment fallback and the sticky-comment update). The owner deferred it, so this issue stays open for those items.
• The Complete the published v1.0.0 tag/Action replay after the ordinary CI pin repair #570 hosted replay, which stays open as well.

[pengfei-threemoonslab]

Hosted run of the host-only advisory PR recipe — published v1.1.0, 2026-09-22

The owner authorized one run on 2026-09-22, against the published 1.1.0 Action and package only. v1.0.0 was not run. Nothing in this repository was changed: contents, tags, releases and settings are as they were.

• Fixture (public, synthetic): https://github.com/ThreeMoonsLab/agents-shipgate-hosted-fixture (main 6412a5529ed7a60256d77e061c58322d6674ee0e). It holds README.md, .claude/settings.json, .mcp.json and the workflow. It has no shipgate.yaml and no saved baseline.
• Fork, for the fork-PR case: https://github.com/pengfei-threemoonslab/agents-shipgate-hosted-fixture
• Workflow: examples/github-actions/14-host-only-advisory-pr.yml as published at tag v1.1.0, with only its two pins changed: @v1.0.0 → @v1.1.0 and shipgate_version: '1.0.0' → '1.1.0'. The recipe at the tag still pins 1.0.0; the pin move is Move the published-release pins to v1.1.0 #853 (open).

Release under test

Item	Value
Action ref	ThreeMoonsLab/agents-shipgate@v1.1.0 in 9 runs; @e3c6cb0c7657d9c53d4e29b2061d04dcf99a4e9b in run 35774401670
Resolved SHA	The runner resolved both forms to e3c6cb0c7657d9c53d4e29b2061d04dcf99a4e9b (log line: Download action repository … (SHA:e3c6cb0c…))
Tag	refs/tags/v1.1.0 → annotated tag 39514a0a4e2aa8c3bab7c41c46db0997b980c8a1 → e3c6cb0c7657d9c53d4e29b2061d04dcf99a4e9b
Package	python -P -m pip install agents-shipgate==1.1.0 from PyPI (Downloading agents_shipgate-1.1.0-py3-none-any.whl (2.6 MB), not cached); tool.version 1.1.0; runner CPython 3.12.14
Contract	40 (verifier schema 0.20)
Wheel digest	sha256 038bdb4650d45d9c81996f60d33781b5671bfb57f006233f80e74db8a7377d33. The run does not record it (#855). The value comes from PyPI's only 1.1.0 file, the matching v1.1.0 GitHub Release asset, and a local download whose 609 installed files match its RECORD

Scenarios

Every job ended success. "Local 1.1.0" is agents-shipgate diff --workspace <clone at the run's merge commit> --base <the run's base> --json from the PyPI 1.1.0 wheel (local CPython 3.13.12). It was compared as a whole ordered object with verifier.json host_comparison: status, reasons, base, unchanged limits, rows, coverage and review.

#	Scenario	PR	Run	Result	Local 1.1.0
1	Bash(npm test *) → Bash(npm *); Bash(git push*) deny → allow	1	35773922210	One sticky comment: ⚠ medium / moved, ⚠ medium / widened, coverage block (settings compared, 4 rows; workflow and .mcp.json unchanged), question "these 2 declared capability changes (from 4 rows)", Compared/Reproduce/Advisory	equal
2	Second push to PR 1 (adds Read(src/**))	1	35774315551	The same comment (5782650637) was edited once, in place. The PR still has 1 comment, 0 reviews and 0 review comments. The new body has 5 rows / 4 changes (pairing: #858)	equal
3	Narrowing Bash(npm:*) → Bash(npm test:*)	2	35774377665	medium / narrowed, "the new rule matches only what the old rule matched", no ⚠ anywhere, widenings 0	equal
4	Remote MCP server with a URL path	3	35774380269	⚠ high / added, url https://mcp.example.com/<redacted-path>; header keys X-Fixture-Key. The path and header value appear in no log, artifact, comment or local output	equal
5	README-only	4	35774385937	"No static host-grant changes detected in the covered comparison. No verdict is implied."; 3 sources compared; no question; no Shipgate annotation or warning	equal (comparable, 0 rows)
6	Invalid JSON in .mcp.json	5	35774387791	"Host capability comparison unavailable: head_inventory_incomplete" names .mcp.json parse_failed; Inputs: line, not Compared:; no question; job not failed	equal (incomparable)
7	fetch-depth: 1 plus a widening	6	35774393068	"Host capability comparison unavailable: shallow_history", next action fetch_base; job not failed	Differs, as expected. A full-history clone gives comparable, 4 rows. A depth-1 clone makes diff --base origin/main refuse with exit 2 config_error (JSON on stderr)
8	Fork PR from the fork above	9	35774429059	Token PullRequests: read; comment POST returned 403; warning annotation raised; PR has 0 comments. The same review is in the job summary and the artifact (pr-comment.md, verifier.json); see #854, #856	equal
9	PR adds root pip/__init__.py and agents_shipgate/__init__.py (each prints a marker only)	7	35774396881	Marker occurs 0 times in the full log, the per-step log archive and the artifacts; both files are in changed_files; no MCP server or npx ran	equal. Negative control on the same tree: python -m pip and python - without -P print both markers
10	Full-SHA uses: ref, same settings change as 1	8	35774401670	Resolves to e3c6cb0c. Settings rows equal scenario 1's. The extra high / changed workflow row says "adds no scope" and has no ⚠	equal

Downloaded logs, artifacts, comment edit history and local CLI outputs are kept with the run evidence.

Acceptance

1. Met, with a caveat. The recipe ran on host configuration with no shipgate.yaml and no baseline, and README/quickstart/integrations link it. The caveat: at tag v1.1.0 the published recipe still pins 1.0.0 and says the hosted run is outstanding, so the run used it with the pins moved. That pin move is Move the published-release pins to v1.1.0 #853, still open.
2. Met for v1.1.0, by the owner's decision; the literal 1.0.0 pin was not run. Ten hosted runs are recorded above with exact merge-head and base SHAs, the Action ref and resolved SHA, the PyPI package and contract 40.
3. Met. Local 1.1.0 is whole-object equal for scenarios 1–6 and 8–10, and scenario 7 differs as expected. Scenario 3 is the narrowing case. No-change (5) and malformed input (6) have distinct outcomes.
4. Partly met. A second push updates the same comment in place (2). Missing history (7) and an unreadable changed input (6) are visible. Not exercised: a no-change push to a PR that already has a comment (5 is a fresh PR), and the "Not compared" case of an unread, unchanged input.
5. Partly met. All 10 jobs succeeded despite widenings, a high MCP row and incomparable results; no required check or fail policy was installed. Not exercised in a hosted run: a setup or execution failure (install failure, CLI exit 2). Also, the job summary prints an empty Status: and Critical: 0 - High: 0 - Medium: 0 on every run (Host-only Action runs print an empty status, zero severity counts and nonexistent artifact paths in the job summary #854).
6. Met, with Host-only Action runs print an empty status, zero severity counts and nonexistent artifact paths in the job summary #854 and Fork-fallback job summary renders the limitation as raw text and prints '## Agents Shipgate' literally #856. The permissions were contents: read and pull-requests: write. Same-repo PRs got a sticky comment. For the fork PR, see scenario 8. Not exercised: the first-time-contributor approval path, because the fork belongs to the owner's account (the run started with no approval wait).
7. Met. No marker reached any log or artifact, and the install used python -P -m pip install agents-shipgate==1.1.0 from PyPI. As documented, the pull_request workflow file itself is PR-controlled.
8. Moved to Run the published advisory host-diff pilot: first reviewer value and the next eligible change #571. This hosted run is not evidence of setup time, assistance or second-reviewer use.

Defects filed from this run (milestone 11)

• Host-only Action runs print an empty status, zero severity counts and nonexistent artifact paths in the job summary #854 — On the host-only route, the job summary prints an empty status, zero severity counts and six nonexistent artifact paths.
• A shipgate_version Action run records no digest of the engine it installed #855 — A shipgate_version run records no wheel digest.
• Fork-fallback job summary renders the limitation as raw text and prints '## Agents Shipgate' literally #856 — The fork-fallback job summary renders the limitation as raw text and prints ## Agents Shipgate literally.
• control.reason counts rows but calls them changes ('4 change(s)' beside '2 changes (from 4 rows)') #857 — control.reason counts rows but calls them changes.
• An unrelated added allow rule splits a widened replacement into 'added' plus a misleading 'removed' #858 — An unrelated added allow rule splits a widened replacement into added plus a misleading removed.
• Workflow capability rows open with an unlabelled aggregate access value ('write, ...') #859 — Workflow rows open with an unlabelled aggregate access value.

#780 stays open. Items 4 and 5 still need the three unexercised cases above: a no-change push to a PR that already has a comment, the "Not compared" case, and a hosted setup or execution failure. Item 1's caveat closes when #853 merges. Item 8 is recorded as moved to #571 and does not block this issue. The same run is recorded against #570's acceptance on #570.