test: deduplicate explicit-skill-requests harness - #2159
Conversation
Extract shared helpers under tests/lib for fixtures, Claude runners, and stream-json assertions. Refactor single- and multi-turn scripts to use the shared library, add handoff/technique-recognition prompts, optional skill fixtures, docker-compose, and README. Co-authored-by: Cursor <cursoragent@cursor.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 83359b4423
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| run_case "handoff-with-manual" \ | ||
| "$SCRIPT_DIR/run-test.sh" "systematic-debugging" "$PROMPTS_DIR/handoff-with-manual.txt" | ||
|
|
||
| run_case "mid-conversation-technique-recognition" \ | ||
| "$SCRIPT_DIR/run-test.sh" "systematic-debugging" "$PROMPTS_DIR/mid-conversation-technique-recognition.txt" 4 |
There was a problem hiding this comment.
These cases judge whether Claude recognizes and invokes a skill from particular prompt phrasing, so they are skill-behavior evals rather than plugin-infrastructure tests. Adding them to this live-model shell suite puts nondeterministic behavior checks in the part of the repository reserved for infrastructure tests; move these scenarios to superpowers-evals instead of extending run-all.sh.
AGENTS.md reference: AGENTS.md:L102-L104
Useful? React with 👍 / 👎.
| volumes: | ||
| - ../..:/workspace:ro | ||
| - ${HOME}/.claude:/root/.claude:ro |
There was a problem hiding this comment.
When the documented docker compose run --rm explicit-skill-requests command is used, common.sh writes all diagnostics beneath /tmp/superpowers-tests inside the container, but these volumes mount only the repository and Claude configuration. The container is therefore removed along with the logs when the run finishes, making the advertised full logs unavailable precisely when a test fails; bind-mount the output directory or use a persistent named volume.
Useful? React with 👍 / 👎.
1 participant
Summary
tests/lib/(common.sh,assertions.sh,run-claude.sh,test-prompt.sh)run-test.shand multi-turn scripts to use the shared library instead of duplicating setup/assertion logic/handoffmanual skill invocation and mid-conversation technique recognitiontests/shared/optional-skills/README.md,.gitignore, anddocker-compose.ymlfor the explicit-skill-requests suiteTest plan
./tests/explicit-skill-requests/run-test.sh subagent-driven-development ./tests/explicit-skill-requests/prompts/subagent-driven-development-please.txt./tests/explicit-skill-requests/run-all.sh./tests/explicit-skill-requests/run-multiturn-test.sh./tests/explicit-skill-requests/run-extended-multiturn-test.shMade with Cursor