Skip to content

Add the formal-verification specify skill's refresh action #15775

Description

@kyle-sexton

Decision (owner, 2026-11-12)

Slice 3 of 3 of the formal-verification plugin. The whole design, the directory layout, the decided names and the design rules are on slice 1: Add the formal-verification plugin's chart skill, setup and consumer contract, proven on the loop-lane claim protocol.

Source: her maintain-formal-verification-skill (cursor/plugins pstack/skills/, MIT), text kept as far as possible. The dangerous case she names is a pass on a stale description, "because it looks like a proof".

Depends on

This item

/formal-verification:specify refresh: bring a project's formal verification back in line with the code after the code changes, and say whether each failure is drift in the verification or a bug in the code.

Every step of hers is kept:

  1. Find the checks by what they do, not by name. Collect the property, covered code, bounds, command and sync commit; with no sync commit recorded, use the last commit that changed the verification files and say so.
  2. Run each check unchanged first.
  3. Read every change to covered code since the sync commit, including moved or renamed code and new code touching guarded state. Classify each: no effect; the verification no longer describes the code; the promise changed on purpose (with cited evidence); the promise may be broken.
  4. "Update from the code, not from the result." Change a property only for an intended promise change, recording the old wording, the new wording and the evidence; report any reduced bound. A property change needs a person's yes even with cited evidence. Attended: propose old wording, new wording and evidence, change on yes. Unattended: report the proposal and leave the check blocked, sync commit not moved.
  5. Rerun and judge. On a pass, break the new code and confirm the check catches it. On a failure, replay the counterexample against the real code step by step: an impossible step means the verification is wrong; otherwise it is a bug, with the introducing commit named.
  6. Update the consumer's formal-<area> skill and move the sync commit forward, never for a blocked check. "Then follow the verify skill once as the next agent would, so that you know its steps still run."

Each check ends in sync, updated, bug or blocked; a check can be both updated and a bug. Edits stay inside the consumer's formal-<area> folder; verification updates go in their own change, separate from any product fix.

  • Report: her five per-check bullets as prose, each check opening with a fixed outcome line (a check can be both updated and bug). No machine-readable summary until a routine exists to read one.
  • Bugs: a bug is filed, through /bugs:write then /work-items:track add with no needs-human label, only when its counterexample was replayed step by step against the real code: attended on the person's yes, unattended automatically. An unreplayed counterexample stays in the report.
  • Unattended: refresh runs every step but writes only for checks that end in sync or updated with a seeded break caught, and reruns checks only when the toolchain is already installed. It makes no verification edit for a property change, a bug or a blocked outcome; those go in the report.
  • Branch rule, as /app-control:map refresh: on a ticket branch, a commit there; otherwise its own upkeep branch and one pull request; never in the same commit as a product fix. Only the formal-<area> folder, the paths listed in sync.yaml and a named conversion path are staged. A pass on a detached HEAD that is not the default branch's tip commits nothing.
  • A verification skill outside the contract: attended refresh lists it and converts it only when the person names it; unattended refresh runs her pass on it report-only, files nothing and converts nothing.
  • CI: a pull request that touches a path listed in any sync.yaml gets an advisory line from the check script in CI, naming the stale check and refresh; advisory by default, following the per-check on_stale setting.
  • Scheduling: the autonomy plugin's routines catalog gains a formal-verification-upkeep row: output a report, a work item and one pull request confined to the formal-<area> folder and listed paths; join: proven recurring manual pattern; stated as distinct from formal-logic-modeling. No routine file in this slice.

Before build

The interview ran and the owner confirmed it on 2026-11-14 ("Go with recommendations, whatever the research says, and whatever is the best fit in our repo", "Go with recommendations.", then "Confirmed."). The unattended behavior, the routine and the report's shape are stated above.

Left to planning: the exact form of the outcome line and what the refresh pull request body is built from; the argument that names a conversion path.

Dogfood

A deliberate change to code covered by this repository's first formal-<area> skill, classified and refreshed by the skill. It counts as one manual run and schedules nothing.

Owner step

If the dogfood's classification proposes a property change, it waits for a person's yes before the property is changed.

Done when

  • /formal-verification:specify refresh finds checks by what they do, not by name, and collects each one's property, covered code, bounds, command and sync commit.
  • With no sync commit recorded, /formal-verification:specify refresh uses the last commit that changed the verification files and says so in its report.
  • /formal-verification:specify refresh runs each check unchanged before any edit.
  • /formal-verification:specify refresh classifies each change to covered code since the sync commit, moved, renamed and new code touching guarded state included, into her four classes.
  • /formal-verification:specify refresh changes a property only for an intended promise change and records the old wording, the new wording and the evidence.
  • /formal-verification:specify refresh reports any reduced bound.
  • Attended, /formal-verification:specify refresh proposes a property change with old wording, new wording and evidence, and changes it only on a person's yes.
  • Unattended, /formal-verification:specify refresh reports a proposed property change, leaves that check blocked, and does not move its sync commit.
  • On a pass, /formal-verification:specify refresh breaks the new code and confirms the check catches it.
  • On a failure, /formal-verification:specify refresh replays the counterexample step by step against the real code, calls an impossible step a verification error, and otherwise reports a bug naming the introducing commit.
  • /formal-verification:specify refresh updates the consumer's formal-<area> skill and moves its sync commit forward, never for a blocked check.
  • After updating, /formal-verification:specify refresh follows the formal-<area> skill once as the next agent would.
  • Each check /formal-verification:specify refresh handles ends in sync, updated, bug or blocked, and one check can be both updated and bug.
  • The /formal-verification:specify refresh report gives her five per-check bullets as prose, each check opening with a fixed outcome line, with no machine-readable summary.
  • Unattended, /formal-verification:specify refresh writes only for checks that end in sync or updated with a seeded break caught.
  • Unattended, /formal-verification:specify refresh reruns checks only when the toolchain is already installed.
  • Unattended, /formal-verification:specify refresh makes no verification edit for a property change, a bug or a blocked outcome, and reports each one.
  • Started on a ticket branch, /formal-verification:specify refresh commits there; otherwise it commits on its own upkeep branch and opens one pull request.
  • /formal-verification:specify refresh never puts a verification update in the same commit as a product fix.
  • /formal-verification:specify refresh stages only the formal-<area> folder, the paths listed in sync.yaml and a named conversion path.
  • A /formal-verification:specify refresh pass on a detached HEAD that is not the default branch's tip commits nothing.
  • /formal-verification:specify refresh hands a bug to /bugs:write then /work-items:track add, with no needs-human label, only when its counterexample was replayed step by step.
  • Attended, /formal-verification:specify refresh hands a replayed bug on only after the person's yes; unattended, it does so without asking.
  • /formal-verification:specify refresh keeps an unreplayed counterexample in the report and hands it to no tracker.
  • Attended, /formal-verification:specify refresh lists a verification skill outside the contract and converts it only when the person names it.
  • Unattended, /formal-verification:specify refresh runs her pass on a verification skill outside the contract report-only, and hands nothing to a tracker and converts nothing.
  • The formal-verification check script prints an advisory line in CI for a pull request touching a path listed in any sync.yaml, naming the stale check and refresh, following the per-check on_stale setting.
  • The autonomy plugin's routines catalog has a formal-verification-upkeep row: output a report, a work item and one confined pull request; join: proven recurring manual pattern; distinct from formal-logic-modeling.
  • No routine file schedules /formal-verification:specify refresh.
  • The formal-verification dogfood makes a deliberate change to code the first formal-<area> skill covers, and /formal-verification:specify refresh classifies and refreshes it.

Out of scope

  • A scheduled refresh routine: none until refresh has been run by hand repeatedly.
  • A machine-readable refresh summary: added when a routine exists to read it.
  • A spec-side mutation check.

Activity

  1. added
    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.
    on Nov 13, 2026
  2. github-actions commented on Nov 13, 2026

    @github-actions
    Contributor

    Size: Medium
    Complexity: Medium
    Uncertainty: Knowable
    Flags: intent
    Blocked by: #15648

    I understood this as slice 3 of the formal-verification plugin: a /formal-verification:specify refresh action that re-syncs a project's verification with changed code and ends each check as in sync, updated, bug or blocked. It depends on the design and consumer contract in #15648. #15648 is still open. Related sibling: the propose/create/check slice (not a duplicate).

    The issue says three things must be locked before build: the unattended behavior, whether an autonomy routine schedules it, and the report's shape. They are open.

    1. Does an autonomy routine schedule refresh? RECOMMENDED: no routine in this slice; run it on demand and mirror /app-control:map refresh for unattended runs. That keeps the slice to the action itself.
    2. What is the report's shape? RECOMMENDED: one entry per check with its status, the evidence, the old and new property wording, and any reduced bound. This matches the steps in the issue.

    To answer, post a comment that starts with this line:

    **Decision recorded.** 1) <answer> 2) <answer>
    

    Automated triage. A maintainer reviews it.

  3. added
    needs-triageNot yet classified. Floor until a type and one priority tier are set.
    on Nov 13, 2026
  4. melodic-automation-lanes commented on Nov 13, 2026

    @melodic-automation-lanes
    Contributor

    The triage verdict left this issue at needs-owner-decision:

    • flagged: intent
    • held by needs-owner-decision
    • held by needs-human
    • no lanes App verdict comment shows whether an earlier verdict was flagged

    Reason: intent

  5. kyle-sexton commented on Nov 14, 2026

    @kyle-sexton
    MemberAuthor

    Decision recorded. Interview complete, owner-confirmed 2026-11-14 ("Go with recommendations, whatever the research says, and whatever is the best fit in our repo", "Go with recommendations.", then "Confirmed."). The shared decisions are on issue #15648 and the check script's are on #15773. Decisions for this slice:

    • Every step of hers is kept, including the last sentence of her step 6, which the body dropped: "Then follow the verify skill once as the next agent would, so that you know its steps still run."
    • Unattended, refresh runs every step but writes only for checks that end in sync or updated with a seeded break caught, and reruns checks only when the toolchain is already installed. It makes no verification edit for a property change, a bug or a blocked outcome; those go in the report.
    • A property change needs a person's yes even with cited evidence. Attended: propose old wording, new wording and evidence, change on yes. Unattended: report the proposal and leave the check blocked, sync commit not moved.
    • Branch rule, as /app-control:map refresh: on a ticket branch, a commit there; otherwise its own upkeep branch and one pull request; never in the same commit as a product fix. Only the formal-<area> folder, the paths listed in sync.yaml and a named conversion path are staged. A pass on a detached HEAD that is not the default branch's tip commits nothing.
    • Report: her five per-check bullets as prose, each check opening with a fixed outcome line (a check can be both updated and bug). No machine-readable summary until a routine exists to read one.
    • A bug is filed, through /bugs:write then /work-items:track add with no needs-human label, only when its counterexample was replayed step by step against the real code: attended on the person's yes, unattended automatically. An unreplayed counterexample stays in the report.
    • A verification skill outside the contract: attended refresh lists it and converts it only when the person names it; unattended refresh runs her pass on it report-only, files nothing and converts nothing.
    • plugins/autonomy/reference/routines.md gains a formal-verification-upkeep row: output a report, a work item and one pull request confined to the formal-<area> folder and listed paths; join: proven recurring manual pattern; stated as distinct from formal-logic-modeling. No routine file in this slice; the dogfood counts as one manual run and schedules nothing.
    • A pull request that touches a path listed in any sync.yaml gets an advisory line from the check script in CI, naming the stale check and refresh; advisory by default, following the per-check on_stale setting.

    Left to planning: the exact form of the outcome line and what the refresh pull request body is built from; the argument that names a conversion path.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    needs-humanHuman-in-the-loop required; autonomous sessions must not resolve items carrying this.needs-owner-decisionAwaiting a human or maintainer judgment call.needs-triageNot yet classified. Floor until a type and one priority tier are set.

    Fields

    Work status

    needs-human

    Priority

    medium

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions