Skip to content

feat(testing): add a whole-map scope to run-e2e with map_unit and map_entry_points settings - #16592

Merged
melodic-automation-lanes[bot] merged 3 commits into
mainfrom
work-resolve-issue/issue-16567-43924169748-1
Nov 16, 2026
Merged

melodic-automation-lanes[bot] merged 3 commits into
mainfrom
work-resolve-issue/issue-16567-43924169748-1

Conversation

@melodic-automation-lanes

Copy link
Copy Markdown
Contributor

Closes #16567

Summary

Opened by the work-resolve-issue lane. The run's own summary:

/testing:run-e2e map drives every feature of the app's control skill one after another on the drive route. It reads no base ref and never calls drive's affected. features= narrows the pass and max_entries= caps it. It finds the hub through drive's ready (not affected) and passes the resolved app to every drive call. A fixed table turns a failed stated result or proof step into a stated finding and observed breakage into an unstated finding of a closed kind, each with its severity. A new scripts/finding-id.sh hashes feature, part, kind and where into the id, without the entry point. The pass skips Step 4, edits no tracked file, writes findings.json and e2e-summary.json at the top of its run directory, and records measured wall time and tokens per dispatched entry, or null with a reason, plus pass totals. The settings map_unit (feature/part) and map_entry_points (all/first) are plugin options and testing.yaml keys, resolved by a new resolve-config.sh map mode in which the repository layers beat userConfig. The test-scan loader and /testing:setup skip and keep the two keys. Design choices I made, based on the control-skill contract and drive's report fields (evidence.md): the severity per kind, wall time from date +%s around each entry's subagent dispatch, tokens from the Agent result, and the build from drive ready's build, else git rev-parse HEAD.

Not verified

  • The changed test suites (plugins/testing/scripts/resolve-config.test.sh, skills/run-e2e/scripts/findings-compare.test.sh, skills/setup/scripts/setup.test.sh) were not executed: this lane runs no script outside its wrappers, and check-affected leaves *.test.sh to the test tier. Left to the PR's CI.
  • Tidy not run: the change is 17 files across two areas (plugins/testing and docs), over the 10-file and 1-area cutoffs.
  • finding-id.sh has no shebang (first line # shellcheck shell=bash) and is run as bash finding-id.sh, because this lane cannot set an exec bit and CI's exec-bit check flags a non-executable shebang file. Its tests went into the existing executable findings-compare.test.sh for the same reason.
  • The plugin options table in plugins/testing/README.md and the docs/skill-cheat-sheet.md row were edited by hand to match their generators (sync-plugin-options-docs.py, generate-cheatsheet.mjs), which this lane cannot run. check-affected's validate-plugins cheat-sheet drift check passes.
  • Untracked dotfiles in the checkout root (.bashrc, .bash_profile, .gitconfig, .idea, .mcp.json, .profile, .ripgreprc, .vscode, .zprofile, .zshrc) were not created by this run's edits. They look like sandbox artifacts and are not part of the change.
  • No live map pass was driven, since this lane has no app session. map.md's procedure is checked only by the review and the static checks.

Fix

  • plugins/testing/skills/run-e2e/context/map.md: new; the map scope: settings, route and hub through drive ready, entry listing (feature or part, features=, max_entries= cap), judging table and finding ids, measures, outputs
  • plugins/testing/skills/run-e2e/SKILL.md: map in argument-hint and Arguments; a Map scope section with the map resolve command and the finding-id command; Step 4 says a map pass skips it
  • plugins/testing/skills/run-e2e/scripts/finding-id.sh: new; prints the 16-hex-digit SHA-256 id of feature, part, kind and where, refusing a kind or where outside the findings schema
  • plugins/testing/skills/run-e2e/scripts/findings-compare.test.sh: added finding-id.sh cases (stable id, schema id pattern, each hashed field changes it, refusals)
  • plugins/testing/scripts/resolve-config.sh: new map mode resolving map_unit and map_entry_points with the e2e layer order; --user validated per mode; userConfig values held in an associative array
  • plugins/testing/scripts/resolve-config.test.sh: map mode cases (defaults, userConfig, layer order, unknown value, --user per mode, scan config ignores the keys)
  • plugins/testing/.claude-plugin/plugin.json: userConfig options map_unit and map_entry_points
  • plugins/testing/schemas/testing.schema.json: map_unit and map_entry_points keys
  • plugins/testing/skills/audit/scripts/adapter-load.awk: the scan config skips map_unit and map_entry_points
  • plugins/testing/skills/setup/scripts/setup.sh: apply keeps map_unit and map_entry_points lines in testing.yaml
  • plugins/testing/skills/setup/scripts/setup.test.sh: case for keeping the map keys
  • plugins/testing/skills/setup/SKILL.md: names the two keys among those testing.yaml holds and apply keeps
  • plugins/testing/skills/run-e2e/context/e2e-config.md: the map keys, their precedence and defaults, and the older-release note
  • plugins/testing/README.md: a map scope paragraph and the two options in the generated options table
  • docs/skill-cheat-sheet.md: run-e2e row regenerated by hand for the new argument-hint (actions map, worst-case)
  • docs/contracts/config-cascade/README.md: testing (run-e2e) row names the map keys and resolve-config.sh map
  • .changes/testing/work-resolve-issue-issue-16567-43924169748-1-e099423d.md: minor changelog fragment

Verification

The runner ran each check from the base copy against 1360288:

  • scripts/check-changelog-fragments.sh --check: passed
  • scripts/check-changed-skills.sh --merge-base: passed

The run reports:

  • check-changelog-fragments: all 4 fragments valid
  • check-changed-skills: audit, run-e2e and setup PASS, 0 errors (pre-existing size warnings only)
  • check-affected: 16 passed, 6 skipped over 17 changed files; an earlier run failed validate-plugins on an argument-hint … placement and a stale cheat-sheet row, and markdown on MD038 in map.md, all fixed and re-run green
  • fresh-context review subagent over the whole diff: no high findings; the medium ones (Mask hub for the Step 5 scan, hub discovery narrower than drive's, measures lost for blocked entries, what map_entry_points: first changes) and the low ones (argument grammar, passing the resolved app, entry-point…
  • The lane run: https://github.com/melodic-software/claude-code-plugins/actions/runs/43924169748

Left out: the items under Not verified.

Related

#16567

Lane-Activity: work-resolve-issue/resolve-issue
@github-actions

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-refine/fix-docs
Model rung 0
Leg handoff
Run job success
Activity step success
Commits pushed none (read lane)
Run 43928510751

Failed GitHub API calls

Method Path Status
GET /repos/{repo}/git/ref/tags/* 404

@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

claude-security-review reviewed cb1e9f0cc282ae287d7a7b0f07b33726b1066921: read 1 of 1 changed file(s); 0 finding(s), 0 as inline comments.

Security review: PR #16592, incremental since 0abce55c

No security issues in this push. I read the whole change in .claude-lane/incremental.diff: two hunks in one file, plugins/testing/skills/run-e2e/context/map.md. This lane already reviewed an earlier head (blocking only: yes), so this review reports only CRITICAL and IMPORTANT findings, and it found none.

critical-findings: 0

What changed

  • Hunk 1 (L57-58): this is a wording change. The rule for splitting a status from an entry point's text now spells out the separator as "the first : and space" instead of the inline code span `: `. That looks like a markdown lint fix (MD038). The parsing rule means the same thing as before.
  • Hunk 2 (L87): "mis-encoded" becomes "wrongly encoded" in a row of the judging table. The kind, part and severity values stay the same.

What I checked

  • Instruction surface: no standing instruction is deleted, narrowed or softened. Both hunks rephrase text and leave what it says the same.
  • Trusted paths and trust controls: the changed file matches no TRUSTED PATHS glob. The push touches no trust control, merge hold, token scope or kill switch.
  • Manifests: no plugin manifest changed in this push.
  • Prior findings: this lane's only earlier post is a summary comment with no line anchor that reported 0 findings, so nothing carries over.
  • Embedded text: the PR context file holds nothing that tries to steer the review.

Cost: none. This push changes only documentation prose for a skill action that a user runs.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-review: 1 inline finding(s) on 1360288e4af0d7de49ed004f2e687d3f516d641f; the summary is in this lane's pull request comment.

claude-lane: claude-review

@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

claude-review reviewed cb1e9f0cc282ae287d7a7b0f07b33726b1066921: read 1 of 1 changed file(s); 0 finding(s), 0 as inline comments.

Code review (incremental, blocking only): run-e2e map, wording-only fix

Scope. This pass covers only what changed since 0abce55c: two hunks in plugins/testing/skills/run-e2e/context/map.md. I read both in full. This lane has already reviewed earlier heads, so it reports blocking findings only.

What changed. The commit is labelled a CI fix, and it changes wording only:

  • Status-removal rule (L56-L58). The code spans `: ` and `not driven: ` had a space inside the backticks. They now read ": and space" and "not driven: and a space". A space inside a code span is what markdownlint MD038 flags. The rule itself is the same: on a line with no |, the status still starts at the first colon-and-space that is followed by driven at the end of the line or by not driven: and a space. The fix for the earlier colon-form finding, settled at 0abce55c, still holds.
  • Table row at L87. "mis-encoded" now reads "wrongly encoded". The finding kind (garbled-output), the stated result (unstated) and the severity (low) are unchanged.

No new blocking findings in this change.

I checked this by reading the diff only. I did not run markdownlint or any other command.

Spend: none beyond this review run.

Verifier dropped 0 of 0 candidates.

@github-actions

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-unblock/address-feedback
Model rung 0
Leg handoff
Run job success
Activity step success
Commits pushed none (read lane)
Run 43930880430

Failed GitHub API calls

Method Path Status
GET /repos/{repo}/git/ref/tags/* 404

@melodic-automation-lanes

Copy link
Copy Markdown
Contributor Author

Unattended merge: native auto-merge leaves this pull request unarmed at head 1360288e4af0d7de49ed004f2e687d3f516d641f. No person needs to act: a lane clears this reason, and the next scheduled run judges it again.

pr-merge's gate refuses its verifier verdict (verify:missing)

Lane-Activity: pr-unblock/address-feedback
Lane-Thread: PRRT_kwDOTCGFQM64EuGl
@github-actions

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-unblock/apply-feedback-patch
Model rung 0
Leg base
Run job success
Activity step success
Commits pushed 0abce55
Run 43930880430

Failed GitHub API calls

Method Path Status
GET /repos/{repo}/git/ref/tags/* 404

@melodic-automation-lanes

Copy link
Copy Markdown
Contributor Author

pr-unblock / address-feedback finished round 1 of max 4 on 0abce55.

@github-actions

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-unblock/publish-feedback
Model rung 0
Leg base
Run job success
Activity step success
Commits pushed none recorded
Run 43930880430

Failed GitHub API calls

Method Path Status
GET /repos/{repo}/git/ref/tags/* 404

@github-actions

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-unblock/fix-ci
Model rung 0
Leg handoff
Run job success
Activity step skipped
Commits pushed none (read lane)
Run 43932541102

Failed GitHub API calls

Method Path Status
GET /repos/{repo}/git/ref/tags/* 404

@github-actions

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-unblock/fix-ci
Model rung 0
Leg handoff
Run job success
Activity step success
Commits pushed none (read lane)
Run 43934312195

Failed GitHub API calls

Method Path Status
GET /repos/{repo}/git/ref/tags/* 404

@github-actions

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-unblock/apply-patch
Model rung 0
Leg base
Run job success
Activity step success
Commits pushed cb1e9f0
Run 43934312195

Failed GitHub API calls

Method Path Status
GET /repos/{repo}/git/ref/tags/* 404

@melodic-automation-lanes

Copy link
Copy Markdown
Contributor Author

The unattended merge path armed native auto-merge at head cb1e9f0cc282ae287d7a7b0f07b33726b1066921, the head its arming checks judged.

@melodic-automation-lanes

Copy link
Copy Markdown
Contributor Author

Verification evidence for cb1e9f0

The verifier passed this head. The capture artifact holds the capture summary, which says whether the capture ran, and any screenshot or video that passed the mask check; this comment uploads no file.

Capture artifact: pr-capture-16592-cb1e9f0cc282ae287d7a7b0f07b33726b1066921, expires on 2026-12-16.

One comment per pull request, updated in place for each new head.

@melodic-automation-lanes

Copy link
Copy Markdown
Contributor Author

Merge-Risk: max=HIGH demote=true head=cb1e9f0cc282ae287d7a7b0f07b33726b1066921 rubric=10.0.0 rater=claude-opus-5-5 checker=claude-opus-5-5

@melodic-automation-lanes
melodic-automation-lanes Bot added this pull request to the merge queue Nov 16, 2026
Merged via the queue into main with commit bfa8353 Nov 16, 2026
49 checks passed
@melodic-automation-lanes
melodic-automation-lanes Bot deleted the work-resolve-issue/issue-16567-43924169748-1 branch November 16, 2026 19:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Run a whole-map live verification pass on demand and report what is new since the last pass - part 2 of 6

0 participants