Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ One orchestrator skill (`poteto-mode`) routes tasks to playbooks. Supporting ski
| `pstack:setup-pstack` | Configure the delegate model and reasoning budget. |
| `pstack:automate-me`, `pstack:recall`, `pstack:reflect`, `pstack:show-me-your-work`, `pstack:why` | Personalization, history, reflection, evidence, and intent. |
| `pstack:benchmark-checklist`, `pstack:blast-radius`, `pstack:bro`, `pstack:create-verification-skill`, `pstack:figure-it-out`, `pstack:maintain-verification-skill`, `pstack:no-comments`, `pstack:tdd`, `pstack:teach`, `pstack:technical-writing`, `pstack:typescript-best-practices` | Focused engineering workflows. |
| `pstack:ketchup`, `pstack:in-your-own-words`, `pstack:sharpen` | Catch up on what changed in a chat, restate the goal before starting, aim a task higher. |
| `pstack:arena`, `pstack:correct`, `pstack:create-formal-verification-skill`, `pstack:poteto-help` | Parallel candidates with a cross-judge, repeated-mistake enforcement, formal verification, and the help map. |
| `principle-*` (20 skills) | Engineering principles used by the orchestrator. |


Expand Down
2 changes: 1 addition & 1 deletion plugin.yaml
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
name: pstack
version: "0.2.0"
version: "0.3.0"
description: "Poteto's engineering rigor stack. Playbooks, principles, multi-model review."
author: poteto (adapted by Madeleine)
107 changes: 107 additions & 0 deletions skills/arena/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
---
name: arena
description: "Run N parallel candidates, cross-judge, graft, verify."
version: 0.1.0
author: poteto (Hermes adaptation)
license: MIT
platforms: [linux, macos, windows]
metadata:
hermes:
tags: [engineering, workflow, delegation]
related_skills: [swarm, poteto-mode]
---

# Arena

Spawn N candidates at the same task, cross-judge them, pick one base, graft the strongest parts of the losers into it, verify the result. Use when one attempt at a non-trivial artifact would lock in the wrong shape.

Open a todo list with one entry per phase (Frame, Fan out, Cross-judge, Pick, Graft, Verify) before launching anything.

## Phase A: Frame

Every candidate gets the same prompt, so the prompt is the contract.

1. State the artifact each candidate is producing.
2. Derive the rubric. 3-6 concrete gradeable criteria for what success looks like on this task. The rubric is the picker's tool in Phase D. Candidates only see the task, never the rubric.
3. Pick the runners (see Runners below).
4. Give each candidate its own output location: `/tmp/arena-<slug>/candidate-<n>/`. If the artifact is a change to an existing repo, candidates must not edit the same working tree. Either they write a patch into their own directory, or they return the full artifact in their result, per `pstack:principle-separate-before-serializing-shared-state`.

### Runners

Hermes resolves the delegate model from `delegation.model` and `delegation.provider`. There is no per-task model field, so one `delegate_task` call runs every task in it on the same model. Get family diversity by pinning the delegation model per batch and calling `delegate_task` once per family.

Default: two candidates on different model families, using the two strongest models available in the profile as configured in `pstack:setup-pstack`. Example slugs only, not requirements: `anthropic/claude-sonnet-4.5` and `openai/gpt-5.1`. Use whatever `hermes model` reports for this profile.

Record the current pin before touching anything:

```bash
hermes config get delegation
```

Then, per family:

```bash
hermes config set delegation.model <slug>
hermes config set delegation.provider <provider> # only when the slug needs a provider other than the parent's
```

The loader invalidates its config cache on file change, so the next `delegate_task` call in the same session reads the new pin. Restore the recorded values in Phase F.

- `inherit-parent` means clear the pin: `hermes config unset delegation.model` and `hermes config unset delegation.provider`. That batch then runs on the parent chat model.
- If the runtime rejects a configured slug, run that seat on its family default and say so in the synthesis note. Never invent a slug you have not confirmed.
- `inherit-parent` in the candidate set is legal. It just means the seats are not independent by model family, so say that in the note.
- Spawn more runners when the arena covers multiple design directions. Use the same model N times when the work is generation-bound rather than judgment-sensitive.
- Reasoning effort is global for delegates (`delegation.reasoning_effort`), set in `pstack:setup-pstack`. There is no per-seat reasoning level. Set the budget before the arena opens, generous enough for the hardest seat in it.
- If N exceeds `delegation.max_concurrent_children`, the tool refuses the call. Split into more batches, one per family per chunk.

## Phase B: Fan out

One `delegate_task` call per model family, `tasks=[...]` containing every candidate for that family. Run foreground (the default) so the call blocks and returns all results.

Each task brief is self-contained and carries:

- **Goal**: produce the artifact at its own output path.
- **Grounding**: the path to shared context every candidate needs, and nothing more, per `pstack:principle-guard-the-context-window`.
- **Output path**: its own `/tmp/arena-<slug>/candidate-<n>/` directory.
- **Rationale**: a short note naming the alternatives it considered and what it rejected, and why.

If a candidate produces no output, proceed with N-1 and note the dropout.

## Phase C: Cross-judge

Pick a judge from a model family different from the parent and from the candidate families. Same mechanism as Phase A: pin `delegation.model` to the judge slug, and restore the pin after.

The judge is read-only. Hermes has no tool-level read-only flag for delegates, so read-only is enforced by the brief: it gets the rubric and the candidate paths by label, is told to read candidates and write nothing, and returns per-criterion scores, a recommended base, and rationale in its final message. Do not let it write files into the candidate directories.

Spawn it only after every Phase B candidate finished. It runs in parallel with your own Phase D reading. Never spawn it while candidates are still writing.

## Phase D: Pick a base

Read every candidate end to end before picking. Score each against the rubric criterion by criterion, not on holistic feel, then compare with the cross-judge.

Agreement on the base confirms the pick. Disagreement means one of you is biased or the rubric was ambiguous. Read both rationales before deciding.

Pick the base a future maintainer can extend most easily without breaking invariants. When two feel tied, prefer the cleaner boundary or smaller API, per `pstack:principle-laziness-protocol`.

## Phase E: Graft

Walk each losing candidate once more for what is worth porting into the base. Usually one or two things per candidate, not most of it. Fold each graft in by hand, per `pstack:principle-redesign-from-first-principles`, so the result stays coherent under one mental model. Do not paste mechanically.

If the candidates converge on one shape, that is strong agreement. Note it and ship the consensus shape, no graft needed. If they wildly diverge, Phase A was under-specified. Reframe and re-run rather than averaging the divergence.

## Phase F: Verify

Verify the synthesized artifact like any other output, per `pstack:principle-prove-it-works`. Run it, test it, read the diff. If verification finds a problem the arena missed, either Phase A was wrong (reframe and re-run) or a candidate caught it and you skipped the graft (go back to Phase E). Do not paper over it.

Restore the delegation pin you recorded in Phase A, then run `hermes config get delegation` and confirm the values read back. Leaving a foreign model pinned silently reroutes every later delegate in the session.

## Outputs

One synthesized artifact, plus a short `arena-synthesis.md` beside it naming:

- the base and why it won
- the cross-judge's verdict, and any disagreement with your pick
- each graft with its source candidate
- what was rejected and why
- any convergence or dropouts
- the verification result and the evidence for it
43 changes: 43 additions & 0 deletions skills/correct/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
---
name: correct
description: "Turn repeated agent mistakes into enforced repo rules."
version: 0.1.0
author: poteto (Hermes adaptation)
license: MIT
platforms: [linux, macos, windows]
metadata:
hermes:
tags: [engineering, workflow, process]
related_skills: []
---

# Correct

The operator keeps correcting agents in this repo for the same mistakes. Change the repo so the next agent can't make them.

Assume every contributor is an agent that sees only the files it opened, copies the nearest example, and takes the shortest path that compiles. Design the repo so a change that looks right from one file is right for the whole repo.

## When to Use

When the operator has corrected the same thing more than once, after a revert or a review comment that names a recurring slip. Reach for `pstack:correct` instead of fixing the one instance and moving on.

## Find the mistake classes

Read recent commits, reverts, review comments, agent instruction files, and comments that explain workarounds. Use `terminal` for git history and `search_files` for the workaround comments and instruction files. Group the mistakes into classes. A class counts once it has happened twice.

## Fix each class at the highest level that works

1. **Eliminate it with architecture.** Give each piece of state one owner and each task one supported way. Hide internals so the wrong import fails. Replace hand-synced lists with one source of truth. Delete old ways and dead code an agent would copy.
2. **Enforce it with types so the bad state can't be written.** If bad code still compiles, add a lint or CI check whose error names the file, type, or function to use instead. If the pattern is already common, fail only when a change adds more.
3. **Test the behavior.** Fix or delete any test that still passes when the behavior it checks breaks, or that fails when only the internals change.
4. **Write docs or agent rules last, only for judgment calls.** Nothing fails when an agent skips them.

## Fix and prove

Fix the most frequent classes now, one commit each. Prove each new check fails on a real past mistake: check out the old code, or reconstruct the bad variant, and watch the check fail before it passes on the fix. Run the same command locally and in CI. Exceptions go on the offending line with a reason, an expiry date, and a human's approval.

## Keep the rule table

Keep a table in the agent instruction file (AGENTS.md or the repo's equivalent) that pairs each rule with what enforces it. When the operator corrects you, fix the mistake and add the rule. If the rule was already there and nothing enforces it, that's a repeat, so fix it at the highest level in the same change. Drop a rule once its mistake can't happen.

**Reply:** each class with its evidence, the level you picked, and why a higher level didn't work.
60 changes: 60 additions & 0 deletions skills/create-formal-verification-skill/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
---
name: create-formal-verification-skill
description: "Prove one invariant where tests reach the least."
version: 0.1.0
author: poteto (Hermes adaptation)
license: MIT
platforms: [linux, macos, windows]
metadata:
hermes:
tags: [engineering, verification, formal-methods]
related_skills: [create-verification-skill]
---

# Create a formal verification skill

A test checks the cases that someone thought to write. Formal verification checks every case in a stated scope: every order of steps in a model, every input up to a bound, or every input. This skill finds the part of the repo where that matters most, verifies one property of it, and leaves a project-local skill that lets the next agent check a change against that property. Write that skill for an agent who reads it cold, mid-change, and has never seen your work. The user must be able to trust the result and act on it without learning the verification language.

## When to Use

When concurrency, retries, state machines, parsers, or unsafe code need more than tests. Use it on `pstack:create-formal-verification-skill` requests such as "model-check this race" or "prove this invariant".

## Choose the target from the code

Answer from the codebase, and ask the user only what you cannot observe. Read with `search_files` and `read_file` before asking anything. If the user named an area or a bug, start there.

A good target is a promise the code makes, where a break harms users and tests rarely reach the case that breaks it. Choose the strongest such target in the repo, not the one that is easiest to verify.

## Let the question choose the tool

Pick the tool that answers the question at the lowest cost: a model checker, a proof assistant, an SMT-backed verifier, or another tool.

- Whether events can happen in a bad order usually suits a model checker over a small model of the design.
- Whether the code itself behaves for every input, as with memory safety, overflow, or a data-structure invariant, usually suits a verifier that runs on the code.
- A property that must hold at every size, where a bound is not enough, can need a proof.

For concurrent and distributed designs, TLA+ is often the most useful place to start. Prefer a tool that the repo's developers can install and run, and tell the user why you chose it.

## What makes it good

- **True to the code.** Every part of the model or proof points to code that a reviewer can open. What the verification treats as one indivisible step is one indivisible step in the code. Names come from this repo.
- **Able to fail.** A check that you have never seen fail proves nothing. Show each property failing on a broken variant of the code it guards, such as a removed check or a reordered step.
- **Findings stay findings.** When the current code breaks the property, report it with the counterexample in the code's terms. Do not weaken the property or the model until it passes, and do not change product code unless the user asks.
- **Honest about scope.** Say what is verified, under which bounds and assumptions, and what is left out. A pass covers the model or the bound, not the whole system.
- **Readable without the tool.** State the property and the harm in one plain sentence each. Tell a counterexample as a short sequence of steps in the code's words.
- **Cheap to rerun.** One command, a reproducible tool version, and a run short enough that the next agent will use it on every relevant change.
- **Kept in step with the code.** When verified code changes, something tells the next agent that the verification needs another look.

## Generate the verify skill

Write a project-local skill at `<repo>/.hermes/skills/verify-<area>/SKILL.md` with frontmatter that names the area and says when to use it. Without frontmatter the skill never registers, so give it `name` and `description` at minimum.

From the skill alone, the next agent must be able to tell whether a change touches the verified code, run the check, tell a real bug from a modeling error, and update the verification when a change is intended. Use this repo's real commands, paths, and nouns, and leave no placeholders.

## Prove it before you hand off

Follow the generated skill once, as the next agent would, on a scratch change that breaks the property in the real code. It must catch the break. Then drop the scratch change and confirm that the check is back where it started: passing, or failing only on the findings you will report. A verify skill that was never run is a draft. If the skill misses the break or a step does not run, fix the skill or the verification, not the scratch change.

## Hand off

Tell the user what you verified, which tool you used and why, the bounds, what the check found in plain words, how to run it, and what it does not cover. Suggest a second target only if you found a close one. Point at `pstack:create-verification-skill` when the gap is behavioral coverage rather than a provable property.
14 changes: 14 additions & 0 deletions skills/in-your-own-words/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
name: in-your-own-words
description: "Restate the user's goals and problem in your own words."
version: 0.1.0
author: poteto (Hermes adaptation)
license: MIT
platforms: [linux, macos, windows]
metadata:
hermes:
tags: [communication, workflow]
related_skills: [pstack:bro]
---

Restate in your own words what you think the user's goals are and what problem they're trying to solve.
Loading