Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
---
bump: minor
---

### Added

- `/testing:run-e2e map` drives every feature of the app's control skill one after another, with no base ref and no `affected` call. `features=` narrows the pass and `max_entries=` caps it, with each entry past the cap reported `not checked (cap)`. It records each failed stated result or proof step as a finding, records observed breakage as an `unstated` finding of kind error, crash, refusal, no-output or garbled-output, and reports a blocked entry `not checked` with its reason. It reports each entry's measured wall time and tokens, or null with the reason, plus the pass totals. It refreshes no control skill, edits no tracked file, and writes `findings.json` and `e2e-summary.json` at the top of its run directory.
- `run-e2e`'s `scripts/finding-id.sh` gives a finding its id: a hash of feature, part, kind and where, without the entry point, so one failure seen through several entry points is one finding.
- Settings `map_unit` (`feature` default, or `part`) and `map_entry_points` (`all` default, or `first`), as plugin options and as `docs/conventions/testing.yaml` keys, the repository value winning. `scripts/resolve-config.sh map` resolves them, and the test scan and `/testing:setup` leave them alone.
2 changes: 1 addition & 1 deletion docs/contracts/config-cascade/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -697,7 +697,7 @@ its conformance cell.
| `code-tidying` (`verification`) | team-only | team: `<home>/code-tidying.yaml` (`<home>` from the convention-home pointer line at the default branch, default `docs/conventions`; schema `plugins/code-tidying/schemas/code-tidying.schema.json`) | team only | the team file at the default branch; a skill entry over the plugin value; an invalid value resolves `standard`, no value `light` | per-key | conforms to the [location axis](#location-of-the-team-layer-a-docs-convention-file-or-claudename) ([ADR 0064](../../adr/0064-home-plugin-customization-in-docs-conventions-yaml.md)); no `userConfig` option, no `~/.claude` file and no personal override. `verification_skills.<skill>` beats `verification`. `lib/read-verification.sh` (the plugin's copy) reads the file at `origin/<default>` after a fetch, never the working tree, so a branch cannot lower its own level, and names a working-tree copy that gives another level. An invalid value or a file that does not parse is named and resolves `standard`; no value resolves `light`. Levels owned by [`docs/contracts/verification-level/README.md`](../verification-level/README.md); read by `/code-tidying:batch-simplify`. This row covers `verification` and `verification_skills` |
| `docs-naming` | team-only | team: `docs/conventions/docs-naming.yaml` (ADR 0064; schema `plugins/docs-naming/schemas/docs-naming.schema.json`); user-global `~/.claude/docs-naming.json`; personal override `docs/conventions/docs-naming.local.yaml` | all three | later layer; team on named policy-floor keys | per-key | conforms. The old team and personal override files, `.claude/docs-naming.json`, `.claude/docs-naming.local.json` and their retired `docs-hygiene` names, are not read; retirement records `docs-naming-r001` and `docs-naming-r002` are `report-only`, `/docs-naming:setup check` names each old file it finds, and while the YAML is absent the first `/docs-naming:setup apply` write copies the old team file's keys into it. The user-global `~/.claude/docs-hygiene.json` is still read, with a WARN, while `~/.claude/docs-naming.json` is absent; per-key override on `file_names.*`, with a policy-floor class on `tiers`, `generated`, `sweep_exclude`, `sweep_exclude_sites`, and the three `exempt_*` keys: a personal layer may add entries and never remove them, and `generated` is team-layer only. Those keys decide what `/docs-naming:realign-file-names` does to a tree (which files are frozen, which reference forms are rewritten, and which shell command runs after a move), so narrowing one from a single machine would weaken a team decision, while adding a scope root or an exemption weakens nothing and stays open. `rule`, `regex`, and `redirect_map` are nearest-wins. Keys owned by [`plugins/docs-naming/reference/config.md`](../../../plugins/docs-naming/reference/config.md), which also partitions them from plugin `userConfig` (this plugin declares none, so a layer naming one is an inert unknown key). Written by `/docs-naming:setup apply`, resolved by `plugins/docs-naming/scripts/resolve-config.sh` |
| `rendered-views` | no-floor | team `docs/conventions/rendered-views.yaml`, personal override `docs/conventions/rendered-views.local.yaml`, and each view-lane plugin's `medium` `userConfig` option; the retired `~/.claude/rendered-views.md` and `.claude/rendered-views*.md` give a notice, never a value | `userConfig`, team and personal override | the publish trust walk, not plain precedence: the argument decides alone; a `terminal` or `file` value from any layer keeps the view local; a publishing value counts only from the untracked, gitignored personal override, then `userConfig`; a team file never publishes; with no layer deciding, the lane's own default | per-key | declared deviation from later-layer-wins: `medium` decides whether a view leaves the machine, so a branch-writable layer can keep a view local but never publish it. An invalid value is reported and ignored without dropping another layer's value. Keys owned by [`rendered-views`](../rendered-views/README.md#the-rendered-views-settings). Resolved by `lib/view-settings.mjs`, which reads each layer with `readLayers` |
| `testing` (`run-e2e`) | no-floor | `.claude/testing/e2e.md` for `recording` / `browser_mode`; `docs/conventions/testing.yaml`, `~/.claude/testing.yaml` and `.claude/testing.local.yaml` for `e2e_driver` / `reuse_running_instance` / `browser_driver`, over the plugin's `userConfig` | all three | later layer | per-key | conforms; per-key override, keys owned by `/testing:run-e2e`. The YAML keys are resolved by `resolve-config.sh e2e`, which reports the supplying layer and resolves an unknown or refused value to the default without stopping |
| `testing` (`run-e2e`) | no-floor | `.claude/testing/e2e.md` for `recording` / `browser_mode`; `docs/conventions/testing.yaml`, `~/.claude/testing.yaml` and `.claude/testing.local.yaml` for `e2e_driver` / `reuse_running_instance` / `browser_driver` and the `map` scope's `map_unit` / `map_entry_points`, over the plugin's `userConfig` | all three | later layer | per-key | conforms; per-key override, keys owned by `/testing:run-e2e`. The YAML keys are resolved by `resolve-config.sh e2e` and `resolve-config.sh map`, which report the supplying layer and resolve an unknown or refused value to the default without stopping |
| `testing` (`write`) | no-floor | `docs/conventions/testing.yaml` and `.claude/testing.local.yaml` for `tdd_refactor_step` (`on` \| `off`, default `on`) and `first_test` (`user-flow` \| `unit`, default `user-flow`); no user-global file, no `userConfig` | team + personal override | later layer | per-key | declared deviation: repository settings, so the user-global layer is skipped with a warning. Keys owned by `/testing:write`; resolved by `resolve-config.sh write`, which reports the supplying layer and resolves an unknown value to the default without stopping |
| `testing` (`audit`, `test-weaken`) | no-floor | team: `docs/conventions/testing.yaml` (schema `plugins/testing/schemas/testing.schema.json`), else the `yaml config` block in `docs/conventions/testing.md` (read for one more release), else `.claude/testing.yaml`; user-global `~/.claude/testing.yaml`; personal override `.claude/testing.local.yaml` | all three | later layer; the first present of `testing.yaml`, the docs block and `.claude/testing.yaml` in the team layer | per-key (scalars override, lists concatenate) | conforms to [ADR 0064](../../adr/0064-home-plugin-customization-in-docs-conventions-yaml.md), except the declared stop on an unusable layer, which replaces rule 5's soft degrade and its inert unknown keys (see Declared): one warning names each shadowed team file. Keys and layer merge owned by `plugins/testing/scripts/resolve-config.sh`; written by `/testing:setup apply`, read by `/testing:audit` and the `test-weaken` hook, which skip the `run-e2e` and `write` keys the same file carries |
| `testing` (`check-visual-parity`) | lowest-value | team: `docs/conventions/testing.yaml` read from origin's default branch (the working tree only when the repository has no `origin`); user-global `~/.claude/testing.yaml`; personal override `.claude/testing.local.yaml`; per user: the plugin's `pixel_tolerance` `userConfig` option | all three, plus the user option | lowest value (policy floor) | per-key | declared deviation from [ADR 0064](../../adr/0064-home-plugin-customization-in-docs-conventions-yaml.md) Decision 5's floor wording, as the `implementation` row: the `pixel_tolerance` map keeps a tighten-only `userConfig` option. The team value, else the default 0, is the ceiling; a personal layer or the option can only lower it, and a raise is reported as ignored. An invalid layer is named with its file or option, key and value and dropped (Decision 7). Resolved by `visual-compare.sh config` in `/testing:check-visual-parity`, which reports the supplying layer; the test scan skips the map and `/testing:setup apply` keeps it |
Expand Down
2 changes: 1 addition & 1 deletion docs/skill-cheat-sheet.md
Original file line number Diff line number Diff line change
Expand Up @@ -181,7 +181,7 @@ From the stage definitions: A conditional stage runs only when its trigger holds
| [`/testing:cleanup`](../plugins/testing/skills/cleanup/SKILL.md) | `testing` | Rewrite, quarantine or delete low-value tests behind a mutation gate, or delete cannot-fail ones | | `<<folder>\|<repo root>> [--max <n>] [--flaky <test> ...] [--cannot-fail-only]` | | `/source-control:pull-request`, `/code-tidying:audit-dead-code` |
| [`/testing:diagnose`](../plugins/testing/skills/diagnose/SKILL.md) | `testing` | Root-cause failing tests, never retry blindly | | `[failure]` | [Pocock `diagnosing-bugs`](upstream/mattpocock-skills.md#attribution-table) | `/verification:confirm`, `/testing:write` |
| [`/testing:plan`](../plugins/testing/skills/plan/SKILL.md) | `testing` | Classify changes by required test type and coverage gaps | | `[range or scope]` | | `/testing:write` |
| [`/testing:run-e2e`](../plugins/testing/skills/run-e2e/SKILL.md) | `testing` | Start the app, drive real flows, capture evidence | | `[worst-case…] [--unattended] [app=<app>] [base=<ref>] [trusted-root=<dir>] [scenario\|<target>]` | [pstack `create-verification-skill`](upstream/cursor-pstack.md#attribution-table), [pstack `principle-experience-first`](upstream/cursor-pstack.md#attribution-table) | `/review:quality-gate`, `/verification:confirm`, `/testing:diagnose` |
| [`/testing:run-e2e`](../plugins/testing/skills/run-e2e/SKILL.md) | `testing` | Start the app, drive real flows, capture evidence | `map`, `worst-case` | `[map\|worst-case…] [--unattended] [app=<app>] [base=<ref>] [trusted-root=<dir>] [scenario\|<target>]` | [pstack `create-verification-skill`](upstream/cursor-pstack.md#attribution-table), [pstack `principle-experience-first`](upstream/cursor-pstack.md#attribution-table) | `/review:quality-gate`, `/verification:confirm`, `/testing:diagnose` |
| [`/testing:test-value`](../plugins/testing/skills/test-value/SKILL.md) | `testing` | Where expected values come from and which tests earn their keep | | | [pstack `principle-test-behavior-not-implementation`](upstream/cursor-pstack.md#attribution-table) | `/testing:audit`, `/testing:cleanup` |
| [`/testing:write`](../plugins/testing/skills/write/SKILL.md) | `testing` | Write and place tests with TDD cadence across ecosystems | | `[task]` | [pstack `tdd`](upstream/cursor-pstack.md#attribution-table), [pstack `principle-test-behavior-not-implementation`](upstream/cursor-pstack.md#attribution-table), [Pocock `tdd`](upstream/mattpocock-skills.md#attribution-table) | `/testing:audit` |

Expand Down
14 changes: 14 additions & 0 deletions plugins/testing/.claude-plugin/plugin.json
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,20 @@
"description": "The browser tool command (letters, digits, . _ -) a /testing:run-e2e playwright drive uses when run-e2e drives the app itself, not through /app-control:drive. Default playwright-cli. Missing from PATH, the run stops naming this setting; it never switches tools. testing.yaml wins.",
"default": "playwright-cli"
},
"map_unit": {
"type": "string",
"title": "Map pass unit",
"description": "What one entry of a /testing:run-e2e map pass is: feature (default) drives each feature of the control skill once; part drives each part of each feature that /app-control:drive resolves. testing.yaml wins.",
"default": "feature",
"options": ["feature", "part"]
},
"map_entry_points": {
"type": "string",
"title": "Map pass entry points",
"description": "Which entry points a /testing:run-e2e map pass checks per entry: all (default), or first, reporting the others not checked. testing.yaml wins.",
"default": "all",
"options": ["all", "first"]
},
"pixel_tolerance": {
"type": "number",
"title": "Visual parity pixel tolerance",
Expand Down
8 changes: 8 additions & 0 deletions plugins/testing/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,12 @@ checks before the first drive that the command is on PATH; if it is not, the run
setting, the value, where it came from, and the two ways forward (install the tool, or set
`browser_driver` to one you have). It never substitutes another tool or switches mid-run.

`/testing:run-e2e map` drives every feature of the app's control skill, with no base ref, and
writes `findings.json` and `e2e-summary.json` at the top of its run directory. `features=` narrows
it and `max_entries=` caps it. Two keys shape it, in the same layers as `e2e_driver`:
`map_unit` (`feature | part`, default `feature`) and `map_entry_points` (`all | first`, default
`all`), resolved by `scripts/resolve-config.sh map`. The pass is in `run-e2e/context/map.md`.

`pixel_tolerance` is a map in `docs/conventions/testing.yaml`: `pixels`, the differing pixels per
image `/testing:check-visual-parity` accepts (a whole number, default 0), and `reason`, required when
`pixels` is above 0. Quote a reason that holds a colon followed by a space. The team value is read
Expand Down Expand Up @@ -196,6 +202,8 @@ these and hands them to the plugin, see [User configuration](https://code.claude
| `e2e_driver` | string | `"auto"` | Who drives when /testing:run-e2e drives the app itself, not through /app-control:drive: auto (default) picks the repo's harness when a spec covers the flow, else run or playwright; harness, run, playwright, or chrome (attended only). docs/conventions/testing.yaml wins. |
| `reuse_running_instance` | string | `"auto"` | When run-e2e drives the app itself, not through /app-control:drive: true, or auto (default) when attended, reuses an app that already answers. With a Workspace environment entry, false and unattended auto start their own instance; without, unattended auto reuses and false stops. testing.yaml wins. |
| `browser_driver` | string | `"playwright-cli"` | The browser tool command (letters, digits, . _ -) a /testing:run-e2e playwright drive uses when run-e2e drives the app itself, not through /app-control:drive. Default playwright-cli. Missing from PATH, the run stops naming this setting; it never switches tools. testing.yaml wins. |
| `map_unit` | string | `"feature"` | What one entry of a /testing:run-e2e map pass is: feature (default) drives each feature of the control skill once; part drives each part of each feature that /app-control:drive resolves. testing.yaml wins. |
| `map_entry_points` | string | `"all"` | Which entry points a /testing:run-e2e map pass checks per entry: all (default), or first, reporting the others not checked. testing.yaml wins. |
| `pixel_tolerance` | number<br>*min 0* | `0` | Differing pixels per image /testing:check-visual-parity accepts. It can only lower the repository's pixel_tolerance in docs/conventions/testing.yaml, never raise it. 0 by default. |
| `stdin_read_timeout` | number<br>*min 1* | `2` | Idle bound on reading the hook payload from stdin: how long the pipe may go silent before the hook gives up and fails open |

Expand Down
10 changes: 10 additions & 0 deletions plugins/testing/schemas/testing.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,16 @@
"pattern": "^[A-Za-z0-9._-]+$",
"description": "The command name of the one browser tool a /testing:run-e2e playwright drive uses. When the driver chosen for the run is playwright (pinned or picked by auto) and this command is not on PATH, the run stops before driving, naming this key, the value and its source; it never switches tools. Any other value is named and the key takes its default. Applies only when run-e2e drives itself, not to a run through /app-control:drive."
},
"map_unit": {
"enum": ["feature", "part"],
"default": "feature",
"description": "What one entry of a /testing:run-e2e map pass is. feature drives each feature of the control skill once; part drives each <feature>/<part> that /app-control:drive resolves from the feature's ## Parts lines. Wins over the map_unit userConfig option."
},
"map_entry_points": {
"enum": ["all", "first"],
"default": "all",
"description": "Which entry points a /testing:run-e2e map pass checks for each entry. all checks every entry point the feature lists; first checks only the first, and reports the others not checked (map_entry_points: first). Wins over the map_entry_points userConfig option."
},
"tdd_refactor_step": {
"enum": ["on", "off"],
"default": "on",
Expand Down
Loading
Loading