Description
Decision (owner, 2026-11-12, revised the same day)
Source: the talk "All Code Is Slop (Except Mine)" (Lauren Tan, AI Engineer Code Summit, 2026-11-11), slides 27 and 32 and the transcript at 1:12:08 to 1:12:39 and 1:14:05: formal verification with TLA+ and Lean, and statecharts reviewed alongside videos and screenshots. "We've already found quite a few bugs with formal verification. But it's certainly not a silver bullet."
We absorb her three formal-verification skills (create-formal-verification-skill, maintain-formal-verification-skill, statechart in cursor/plugins pstack/skills/, MIT) as one plugin, keeping as much of her text and scripts as possible, placed, renamed and packaged to this marketplace's philosophy. Owner: "we keep as much of her stuff as possible. We just place it, rename it, and package it differently." This replaces the earlier throwaway proof of concept and its "ship a skill only if it finds a real defect" gate, and revises the parked "no TLA+ skill, written in our own words" decision.
The shape mirrors app-control, which absorbed her runtime verification pair the same way: a generator skill with actions over a consumer-owned skill under a contract, plus a skill that operates it.
plugins/formal-verification/
├── .claude-plugin/plugin.json options for the model checker and Java paths
├── README.md + MIT notice for her scripts
├── prerequisites.json script runtime, Java, model checker, renderer (chart only)
├── reference/formal-skill.md contract for the consumer's generated skill
└── skills/
├── specify/ bare = propose, writes nothing | create | check | refresh
│ └── reference/ create.md refresh.md choose-a-tool.md
├── chart/ model-check a TLA+ spec and chart its states
│ ├── reference/ view-module.md read-report.md render.md
│ └── scripts/ her scripts, install path removed
└── setup/ checks prerequisites; installs only on an interactive yes
<consumer repo>/.claude/skills/formal-<area>/
├── SKILL.md what is verified, tool and why, bounds, the run command
├── sync.yaml covers: <code>, commit: <last confirmed>
└── model/ spec, config, view file, harness
Names are decided: formal-verification, specify, chart, setup, formal-<area>, sync.yaml.
This item: slice 1 of 3
The chart skill, the setup skill and the consumer contract, proven on this repository's work-item claim protocol, defined in the work-item tracker contract's "Lease protocol" (one item claimed at once by a default claim, a lease-only claim and a triage-phase claim).
chartkeeps her statechart skill's text and scripts: check a TLA+ model, draw its states as a Mermaid chart, give the shortest path to each broken rule, for a person to judge. "Never weaken a rule to make it hold." A bug path is replayed against the real code before it counts: "If a bug does not occur, think that the model is wrong until you show that it is not."chartis the plugin's only TLA+-specific skill: its description and body declare "TLA+ with TLC", and it keeps her "Write the spec" rules.- Changed from hers: the scripts no longer install the model checker or Java while the skill runs; they take the jar, Java and bounds as arguments and never install. Each prerequisite is declared and
setupchecks it. Machine paths (jar, Java, script runtime) areuserConfigoptions, not environment variables. Numbers and pinned versions the skill text restates point at the script or carry an as-of date and a recheck trigger. setupinstalls the pinned, checksummed toolchain into the plugin data directory only on an interactive yes; unattended runs never install and report "blocked: toolchain missing". No CI job runs the model checker in this slice.- The contract (
reference/formal-skill.md) names what a consumer'sformal-<area>skill holds: frontmatter; the sections Property, Harm, Tool and why, Bounds and assumptions, Not covered, Run, Reading a result, Updating; onesync.yamlentry per check (id, covered paths, sync commit, command, recorded seeded break); and the view file the chart reads. Model files default toformal-<area>/model/; paths outside it are allowed only when listed insync.yaml, and unattended writes are limited to the folder plus those paths. It is the only surface the two skills share;specifynever copieschart's commands. - A human owns the invariants. Outside evidence is mixed: agents handle TLA+ mechanics but score under half on conformance and invariants unaided. The agent drafts them, each quoting the contract or code line it encodes; the owner approves the list before the model is checked.
- Both upstream records are updated in this slice's pull request: the parked TLA+ record gains the revised decision and attribution rows at the new pin, and the pstack record's three rows change from pending adoption to adopted.
Design rules for all three slices
- One responsibility per skill;
specify's create, check and refresh are actions because they read and write the same consumer skill and contract. specifynames no tool. Her sentence "TLA+ is the default" is replaced, recorded as a change (ADR 0101).- The tool is chosen by the question and the consumer's repository. A second tool is recorded only in the consumer's
formal-<area>skill ("Tool and why"); nospecify/reference/tools/<tool>.mdfile is added for any tool. A state-transition checker other than TLC adds a front end inchart/scripts. No tool interface until a second tool exists. - Skills reach each other only by slash reference with an "if installed" fallback:
/corrections:codify(the ladder's opt-informal-verificationrung, which Release enforcement ladder 3.0.0 with the verification tier and capability-named rungs added),/testing:writewhen no verifier exists for a language,/bugs:writefor a bug,/verification:confirmfor verdict evidence. - Nothing a consumer may need to change is hard-coded. Owner, 2026-11-12: "making things configurable and extensible, not shipping assumptions or hard-coded things that can't be changed by the downstream consumers. Every single plugin and skill must align with our plugin philosophy, our user config, and our extensibility and configurability requirements." Her scripts pin a model-checker version, a Java version, a script runtime, a renderer, and state and bound limits. Each becomes a setting with her value as the default, and a consumer can add a tool or a front end without editing shipped files. Machine paths (jar, Java, script runtime) are
userConfigoptions; version pins and default bounds are keys in the consumer'sdocs/conventions/formal-verification.yaml, her values as defaults; a per-model bound is an argument in that model's run command. The image renderer is a setting (default: her Pele 1.9.0), used only when an image is asked for. - Each
SKILL.mdholds routing and the quality bar; everything else loads on demand from reference files. - An unattended run installs nothing and writes nothing outside the consumer's
formal-<area>folder and the paths itssync.yamllists.
Before build
The interview ran and the owner confirmed it on 2026-11-14 ("Go with all recommendations.", then "Confirmed."). Its answers:
- States and actions the first model covers: one item claimed at once by a default claim, a lease-only claim and a triage-phase claim on the GitHub adapter, with lease expiry, release and the completed-run rule. Renew, reclaim and other adapters are left out. The race is not "two lanes": two issue-lane runs on one issue are already serialized.
- Who writes the invariants: the agent drafts them, each quoting the contract or code line it encodes; the owner approves the list before the model is checked.
- What counts as a real defect: a counterexample replayed step by step against the GitHub adapter's
claimscript. The tracker contract's three documented limits (clock skew, ttl-0, comment-id monotonicity) are reported as known limits. A defect is written up through/bugs:write. - First
formal-<area>skill or one-off: the model ships as this repository's firstformal-<area>skill under.claude/skills/.
The cost of carrying her scripts, raised on 2026-11-12:
- Who reads them in full before they ship.
chartruns about 3,900 lines of her TypeScript plus a large test folder. Before anything ships: a full read of her scripts by fresh-context agent reviewers, one per script group, and the plugin-acceptance security review recorded as a new section of ADR 0093. - How her updates are tracked. Copied scripts become ours to fix, including model-checker and Java version bumps. Once copied they diverge as ours; her changes are re-taken only through the pstack record's drift re-audit.
- Copy or call. Copied, not called: her skill is
disable-model-invocation: true. - Evidence. The chart's Mermaid text goes in the pull request's one evidence comment after the verifier passes.
Dogfood
This repository's work-item claim protocol (the work-item tracker contract's "Lease protocol"), charted by the new skill.
Owner step
The owner approves the agent-drafted invariants list before the claim-protocol model is checked. This is the one step in this slice that needs the owner.
Done when
-
/formal-verification:chartmodel-checks a TLA+ spec and writes a Mermaid chart of its states and a report giving the shortest path to each broken rule. - The
/formal-verification:chartdescription and body declare "TLA+ with TLC". - The
/formal-verification:chartbody keeps her statechart text, including "Never weaken a rule to make it hold." and her "Write the spec" rules. - The
/formal-verification:chartscripts take the model-checker jar, Java and bounds as arguments, and no script downloads or installs anything. - The formal-verification plugin declares the script runtime, Java, the model checker and the renderer as prerequisites, and
/formal-verification:setupchecks each one. -
/formal-verification:setupinstalls the pinned, checksummed toolchain into the plugin data directory only after an interactive yes. - An unattended formal-verification run with the toolchain absent installs nothing and reports "blocked: toolchain missing".
- The formal-verification plugin's jar, Java and script-runtime paths are
userConfigoptions, and no script reads them from environment variables. - The formal-verification plugin reads version pins and default bounds from keys in the consumer's
docs/conventions/formal-verification.yaml, with her values as the defaults. - A per-model bound for
/formal-verification:chartis an argument in that model's run command. - The formal-verification image renderer is a setting defaulting to Pele 1.9.0, used only when an image is asked for.
- Every number or pinned version the formal-verification skill text restates points at the script or carries an as-of date and a recheck trigger.
- The formal-verification consumer contract requires frontmatter and the sections Property, Harm, Tool and why, Bounds and assumptions, Not covered, Run, Reading a result, Updating.
- The formal-verification consumer contract requires one
sync.yamlentry per check holding id, covered paths, sync commit, command and recorded seeded break. - The formal-verification consumer contract names the view file
/formal-verification:chartreads. - The formal-verification consumer contract defaults model files to
formal-<area>/model/and allows a path outside it only whensync.yamllists it. - The formal-verification consumer contract limits unattended writes to the
formal-<area>folder and the paths itssync.yamllists. - The claim-protocol model covers one item claimed at once by a default claim, a lease-only claim and a triage-phase claim on the GitHub adapter.
- The claim-protocol model includes lease expiry, release and the completed-run rule, and leaves out renew, reclaim and other adapters.
- Each invariant in the claim-protocol model quotes the tracker contract or code line it encodes.
- The claim-protocol model ships as this repository's first
formal-<area>skill under.claude/skills/and meets the formal-verification consumer contract. - Every counterexample the claim-protocol model reports was replayed step by step against the GitHub adapter's
claimscript, and the replay result is in the pull request. - The tracker contract's three documented limits (clock skew, ttl-0, comment-id monotonicity) appear in the claim-protocol findings as known limits, not defects.
- If a claim-protocol counterexample reproduces against the
claimscript, it is written up through/bugs:write. - No CI job runs the model checker.
- Fresh-context agent reviewers, one per script group, each read every line of her copied scripts, and the pull request names each group and its reviewer's result.
- ADR 0093 has a new section recording the plugin-acceptance security review of the formal-verification plugin.
- The formal-verification plugin README carries the MIT notice for her scripts.
- The cursor pstack upstream record's rows for her three formal-verification skills read adopted, with the pin and the note that copied scripts diverge and are re-taken only through the drift re-audit.
- The parked TLA+ upstream record carries the revised decision and attribution rows at the new pin.
- The pull request's one evidence comment holds the claim-protocol chart's Mermaid text, added after the verifier passes for the head.
Out of scope
- A CI job that runs the model checker: revisit after the first model proves its value.
- A tool interface for a second checker: not until a second tool exists.
- Renew, reclaim and adapters other than GitHub in the first model.
specify(slices 2 and 3).
Metadata
Metadata
Assignees
Labels
Type
Fields
Work status
Priority
Activity
github-actions commentedon Nov 12, 2026
Size: Large
Complexity: High
Uncertainty: Known
Flags: none
Blocked by: none
Understood: this adds the formal-verification plugin's
chartandsetupskills and the consumer contract. It is proven on the work-item claim protocol as this repository's firstformal-<area>skill.The newest recorded decision (2026-11-16) is:
#15648 (comment)
That decision settles the open questions, so none are asked again. #16458 is named as the change that closes this issue; I did not find it in the recent merged pull requests, so it looks unmerged. The earlier
patch-protected-pathstop and thehost-runlabel are unchanged by this triage.Automated triage. A maintainer reviews it.
melodic-automation-lanes commentedon Nov 12, 2026
Not admitted to the issue lane: it carries host-run. Admission takes the issue once each of those is met, when intake triages it again (
.github/actions/resolve-config/admission-rules.json).melodic-automation-lanes commentedon Nov 12, 2026
Refinement. A read-only run explored this repository, searched its issues and pull requests, and read the allowed docs to draft what follows. On an issue a listed human filed, it counts as the issue's spec once posted, while unedited; it is still model text, so check it.
Intended outcome: A bounded TLA+ model of the claim and lease protocol in plugins/work-items/tools/work-item-tracker/CONTRACT.md, with owner-written invariants; counterexamples are replayed against the real scripts, and a skill ships only if a real defect is confirmed.
Related work
Open questions
RECOMMENDED: Cover the lease-only claim (issue lane) racing the default claim (local session): lease comment, assignee, earlier-live-lease arbitration, back-off, TTL expiry and run-completed supersession. Leave out renew, phase and the local-markdown adapter. Basis: plugins/work-items/tools/work-item-tracker/CONTRACT.md:637
RECOMMENDED: The owner writes them before the model is built and the agent writes only the TLA+ mechanics. A first candidate is that at most one live, non-superseded lease holder owns an item at any time. Record them in the spec header. Basis: judgment
RECOMMENDED: A counterexample trace that the real scripts reproduce, such as two lanes both proceeding on the same item. A model-only violation that does not reproduce is a model bug and does not count. Basis: issue body, Decision section: replay every counterexample against the real scripts
RECOMMENDED: Put the spec and its TLC config under docs/ or a research directory, with a short findings note stating the outcome of the TLC run and of each replay. Do not add a CI check. Basis: judgment
Sources read
To answer, post a comment that starts with this line:
Automated refinement by the lanes App. Answer with a comment whose first line is
**Decision recorded.**.[-]Model the loop-lane claim protocol in TLA+ as one bounded proof of concept[/-][+]Add the formal-verification plugin's chart skill, setup and consumer contract, proven on the loop-lane claim protocol[/+]kyle-sexton commentedon Nov 14, 2026
Decision recorded. Interview complete, owner-confirmed 2026-11-14 ("Go with all recommendations.", then "Confirmed."). Decisions for this slice and the ones all three slices share:
chartis the plugin's only TLA+-specific skill; its description and body declare "TLA+ with TLC" and it keeps her "Write the spec" rules.specifynames no tool. Her sentence "TLA+ is the default" is replaced, recorded as a change (ADR 0101).disable-model-invocation: true). They take the jar, Java and bounds as arguments and never install. Once copied they diverge as ours; her changes are re-taken only through the pstack record's drift re-audit.setupinstalls the pinned, checksummed toolchain into the plugin data directory only on an interactive yes; unattended runs never install and report "blocked: toolchain missing". No CI job runs the model checker in this slice.userConfigoptions; version pins and default bounds are keys in the consumer'sdocs/conventions/formal-verification.yaml, her values as defaults; a per-model bound is an argument in that model's run command. The image renderer is a setting (default: her Pele 1.9.0), used only when an image is asked for.reference/formal-skill.md): frontmatter; sections Property, Harm, Tool and why, Bounds and assumptions, Not covered, Run, Reading a result, Updating; onesync.yamlentry per check (id, covered paths, sync commit, command, recorded seeded break). Model files default toformal-<area>/model/; paths outside it are allowed only when listed insync.yaml, and unattended writes are limited to the folder plus those paths.claim.sh; the contract's three documented limits are reported as known limits; a defect is written up through/bugs:write.formal-<area>skill under.claude/skills/.Corrections to the body: the pstack record's three rows change from "Pending adoption" to adopted (they are not parked); the claim protocol is defined in
plugins/work-items/tools/work-item-tracker/CONTRACT.md"Lease protocol", not in the loop-lane README.9 remaining items
fix(formal-verification): guard the pid a chart test kills
fix(formal-verification): accept default labels and keep region condi…
fix(formal-verification): take download sources only from the shipped…
fix(formal-verification): stage the spec and bound each Java run
fix(formal-verification): fence and escape spec text in the report, d…
docs(formal-verification): state that a view file is code and report …
test(formal-verification): keep fixture rewrites out of the chart tes…
test(formal-verification): use US spelling in a guard test name
fix(formal-verification): escape only spec text in report sentences
kyle-sexton commentedon Nov 16, 2026
Decision recorded.
Owner, 2026-11-16, in session ("Go w/ recommendations"), posted by an agent session. Two done-when lines are accepted as #16458 delivers them:
claimscript": the earlier counterexamples were traced by reading the scripts; the two behind fix(work-items): hand back on an outranked release and judge hand-back leases at a clock read before the list #16515 are replayed as stub tests in that pull request, failing before the fix and passing after.#16458 closes this issue on merge.
fix(formal-verification): close the gaps the done-when check of #15648 …