Skip to content

fix(autopilot): align worker and PR body contracts - #3419

Open
Sergio Sisternes (sergio-sisternes-epam) wants to merge 3 commits into
mainfrom
sergio-sisternes-epam-autopilot-review-fixes
Open

Sergio Sisternes (sergio-sisternes-epam) wants to merge 3 commits into
mainfrom
sergio-sisternes-epam-autopilot-review-fixes

Conversation

@sergio-sisternes-epam

@sergio-sisternes-epam Sergio Sisternes (sergio-sisternes-epam) commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

fix(autopilot): align worker and PR body contracts

Description

TL;DR

This PR fixes five source-backed inconsistencies in Autopilot worker packages and the PR-description skill. Human-owned deferral labels remain advisory-only, workers stop when active companion personas are missing, Mermaid validation is pinned, diagram requirements are conditional, and the review schema no longer points to an undeployed rubric path.

Problem (WHY)

The same review found a floating Mermaid CLI invocation and an unsupported rubric path in the panelist schema; the Approach and Implementation sections identify those exact surfaces.

Approach (WHAT)

# Fix Principle Source
1 Keep status/deferred advisory-only in PR triage. Governance: “Favor small, chainable primitives over monolithic frameworks.” PR triage worker
2 Declare consumer-provided agent prerequisites and fail closed with every missing role listed. DevX: “Add what the agent lacks, omit what it knows” Issue triage, PR review
3 Pin Mermaid CLI to @mermaid-js/mermaid-cli@12.0.0. DevX: “Grounding outputs in deterministic tool execution transforms probabilistic generation into verifiable action.” PR-description skill
4 Make diagrams optional for simple, docs-only, and mechanical PRs; retain validation for non-trivial structure or control flow. OSS: “agents pattern-match well against concrete structures” PR-description skill and rubric
5 Remove the undeployed rubric path while preserving the schema’s principle enum. OSS: “When you find yourself covering every edge case, consider whether most are better handled by the agent's own judgment.” Panelist return schema

Implementation (HOW)

Surface Change
PR triage package Removed automated deferral writes; aligned the bundled worker guidance with its human-decision label contract. Updated the Autopilot README and Starlight development guide to state that deferral remains a human action.
Issue triage and PR review packages Replaced unsupported relative persona paths with consumer .github/agents/ companions. Added a deterministic preflight script to each standalone package; missing active roles are reported before advice or fan-out. Updated both manifests, deployed skill copies, and the lockfile.
PR-description package Pinned Mermaid CLI, made diagram inclusion conditional across the skill, template, rubric, and manifest description, and refreshed its managed deployment.
PR-review schema Removed the undeclared rubric location and kept the principle taxonomy available in the schema.
Regression coverage Added source-contract assertions and subprocess checks for missing-role failure and complete-role readiness in tests/unit/test_workflow_actor_contract.py.
Changelog Added a concise Unreleased entry for this repair PR.

Diagrams

This flow shows the new preflight boundary: all active roles are checked before a worker can continue.

flowchart LR
  Start[Resolve active roles] --> Check{All role files exist?}
  Check -- Yes --> Continue[Read personas and run full panel]
  Check -- No --> Missing[Report every missing role]
  Missing --> Stop[Stop before advice or fan-out]
Loading

This flow distinguishes an advisory deferral recommendation from the human-owned label decision.

flowchart LR
  PR[No accepted linked scope] --> Advice[Recommend needs-issue]
  Advice --> Comment[Explain human review is required]
  Comment --> Decision{Human chooses deferral?}
  Decision -- Yes --> Apply[Human applies status/deferred]
  Decision -- No --> Leave[Leave the label unchanged]
Loading

Trade-offs

  • Keep specialist personas as consumer-provided companions instead of adding transitive dependencies; the workers remain standalone, while preflight makes the prerequisite explicit and fail-closed.
  • Keep one small checker per standalone worker rather than introducing a shared package dependency that would couple their installation.
  • Pin the Mermaid CLI package version while retaining the existing one-shot npx workflow; no global installation or unrelated dependency-management system is added.
  • Remove only the stale schema path, not the principle vocabulary used by panelists.

Benefits

  1. PR triage no longer writes status/deferred; the human decision boundary matches the shared label contract.
  2. Both worker packages report all absent active roles before partial advice or review can occur.
  3. Mermaid validation names an exact CLI version, and simple PRs can omit diagrams consistently.
  4. The panelist schema retains its principle enum without directing consumers to a nonexistent file.

Validation

uv run --extra dev pytest tests/unit/test_workflow_actor_contract.py -q

..................                                                       [100%]
18 passed in 0.87s

uv run --extra dev ruff check src/ tests/ scripts/lint_architecture_boundaries.py scripts/architecture_linter/ && uv run --extra dev ruff format --check src/ tests/ scripts/lint_architecture_boundaries.py scripts/architecture_linter/ && uv run --extra dev python -m pylint --disable=all --enable=R0801 --min-similarity-lines=10 --fail-on=R0801 src/apm_cli/ scripts/lint_architecture_boundaries.py scripts/architecture_linter/ && bash scripts/lint-auth-signals.sh

All checks passed!
2091 files already formatted
Your code has been rated at 10.00/10
[+] auth-signal lint clean

uv run apm audit --ci

[*] All 10 check(s) passed

The environment did not provide the optional microsoft/.github-private policy, so organization policy enforcement was skipped by the audit.

uv run apm install --refresh --target agent-skills,copilot --force

[*] Installed 12 APM dependencies in 6.3s.

cd docs && npm run build

19:20:07 [build] 126 page(s) built in 9.14s
[+] Checked 1111 relative link(s) across generated pages. No broken relative links found.

Scenario Evidence

# Scenario (user promise) Principle(s) Test(s) proving it Type
1 I can request PR triage without the worker applying the human-owned status/deferred label. Governed by policy tests/unit/test_workflow_actor_contract.py::test_pr_triage_never_writes_human_deferred_label (regression-trap for #3418) unit
2 A consumer missing an active persona gets every missing role reported and no partial worker result. DevX, OSS / community-driven tests/unit/test_workflow_actor_contract.py::test_external_persona_preflight_fails_closed_for_clean_consumers (regression-trap for #3418) integration
3 Simple PRs may omit diagrams, while non-trivial diagrams use the pinned renderer. DevX tests/unit/test_workflow_actor_contract.py::test_pr_description_diagrams_are_conditional_and_cli_is_pinned (regression-trap for #3418) unit
4 The review schema retains its principle choices without requiring a rubric file that is not deployed. OSS / community-driven tests/unit/test_workflow_actor_contract.py::test_pr_review_schema_has_no_undeclared_rubric_path (regression-trap for #3418) unit

How to test

  • Run uv run --extra dev pytest tests/unit/test_workflow_actor_contract.py -q; confirm all 18 contract tests pass.
  • Run uv run apm audit --ci; confirm all 10 deployment-integrity checks pass.
  • Run cd docs && npm run build; confirm the Starlight build and relative-link check pass.
  • Install the packages in a clean consumer, omit one active .github/agents/<slug>.agent.md, and confirm the worker stops with the missing role named.
  • Restore all active persona files and confirm the preflight reports ready: true.

Issue and approved scope

Issue: Fixes #3418

Human scope-approval comment: apm-scope:v1 approval

Does this PR complete the issue, or what remains? This PR completes the five bounded repairs and their regression coverage.

Type of change

  • Bug fix
  • New feature
  • Documentation
  • Maintenance / refactor

Testing

  • Tested locally
  • All existing tests pass
  • Added tests for new functionality (if applicable)

Spec conformance (OpenAPM v0.1)

If this PR changes behaviour that an OpenAPM v0.1 req-XXX covers,
confirm the three-step ritual in the
development guide:

  • Spec edit: docs/src/content/docs/specs/openapm-v0.1.md updated
    (new/changed <a id="req-XXX"></a> anchor + prose + Appendix C
    row).
  • Manifest edit: docs/src/content/docs/specs/manifests/openapm-v0.1.requirements.yml
    updated.
  • Test edit: a @pytest.mark.req("req-XXX") test under
    tests/spec_conformance/ added or extended.
  • CONFORMANCE.{md,json} regenerated via
    uv run --extra dev python -m tests.spec_conformance.gen_statement
    and committed.
  • N/A -- this PR does not change OpenAPM-observable behaviour.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 30, 2026 02:31
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The review worker can pass preflight but fail to load its mandatory synthesizer due to mismatched paths, while the triage template can falsely claim a human-owned label is present.

2 open issues
What changed in this PR

This PR addresses the five package-contract inconsistencies in #3418, aligning APM’s Autopilot guidance with human-owned decisions, explicit persona prerequisites, and conditional PR diagrams.

Changes:

  • Removes automated deferral-label instructions and adds missing-persona preflight checks.
  • Pins Mermaid CLI, makes diagram requirements conditional, and removes the stale review-schema rubric path.
  • Adds regression coverage and refreshes documentation, managed skill copies, and deployment records.
File Description
tests/​unit/​test_workflow_actor_contract.py Adds contract assertions and persona-check subprocess coverage.
packages/​pr-description-skill/​SKILL.md Pins Mermaid validation and makes diagrams conditional.
packages/​pr-description-skill/​assets/​section-rubric.md Updates diagram acceptance criteria.
packages/​pr-description-skill/​assets/​pr-body-template.md Allows omission of optional diagrams.
packages/​pr-description-skill/​apm.yml Updates the package description.
packages/​autopilot/​README.md Documents human-owned deferral and persona prerequisites.
packages/​autopilot/​autopilot-pr-triage-worker/​.apm/​skills/​autopilot-pr-triage-worker/​SKILL.md Prohibits automated deferral-label writes.
packages/​autopilot/​autopilot-pr-review-worker/​SKILL.md Declares companion personas and preflight requirements.
packages/​autopilot/​autopilot-pr-review-worker/​scripts/​check_required_agents.py Reports available and missing review roles.
packages/​autopilot/​autopilot-pr-review-worker/​assets/​panelist-return-schema.json Removes the stale rubric path while preserving the enum.
packages/​autopilot/​autopilot-pr-review-worker/​apm.yml Documents companion prerequisites in comments.
packages/​autopilot/​autopilot-issue-triage-worker/​SKILL.md Normalizes persona paths and requires preflight checks.
packages/​autopilot/​autopilot-issue-triage-worker/​scripts/​check_required_agents.py Reports available and missing triage roles.
packages/​autopilot/​autopilot-issue-triage-worker/​apm.yml Documents companion prerequisites in comments.
docs/​src/​content/​docs/​contributing/​development-guide.md Explains deferral ownership and missing-persona stops.
CHANGELOG.md Records the contract repairs under Unreleased.
apm.lock.yaml Refreshes hashes, checker records, and deployment targets.
.agents/​skills/​pr-description-skill/​SKILL.md Refreshes deployed diagram guidance.
.agents/​skills/​pr-description-skill/​assets/​section-rubric.md Refreshes deployed acceptance criteria.
.agents/​skills/​pr-description-skill/​assets/​pr-body-template.md Refreshes the deployed optional-diagram template.
.agents/​skills/​pr-description-skill/​apm.yml Refreshes the deployed package description.
.agents/​skills/​autopilot-pr-triage-worker/​SKILL.md Refreshes deployed deferral-label restrictions.
.agents/​skills/​autopilot-pr-review-worker/​SKILL.md Refreshes deployed persona and preflight guidance.
.agents/​skills/​autopilot-pr-review-worker/​scripts/​check_required_agents.py Deploys the review-role checker.
.agents/​skills/​autopilot-pr-review-worker/​assets/​panelist-return-schema.json Refreshes deployed schema descriptions.
.agents/​skills/​autopilot-pr-review-worker/​apm.yml Refreshes prerequisite comments.
.agents/​skills/​autopilot-issue-triage-worker/​SKILL.md Refreshes deployed persona paths and preflight guidance.
.agents/​skills/​autopilot-issue-triage-worker/​scripts/​check_required_agents.py Deploys the triage-role checker.
.agents/​skills/​autopilot-issue-triage-worker/​apm.yml Refreshes prerequisite comments.

Review effort: Balanced

Address review feedback by aligning the synthesizer path with preflight, removing the false deferred-label template claim, and retaining explicit preflight helper coverage.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Align five Autopilot and PR-description package contracts

2 participants