Repository navigation
feat(debugging): add a diagnose action and a Phase 1 surface pass - #16617
Conversation
/debugging:debug diagnose runs the loop and hypothesis phases, stops at the confirmed cause at file:line with the loop output, removes its instrumentation and changes no other code. The old diagnosis-only stop rule now points at the action. Phase 1 opens with a surface pass: /app-control:drive ready when the project has a control skill, otherwise manifests and launch config, then the surfaces the app type has. It never assumes a port, and turns each unreachable source into a request for scoped access, never for a secret. Phase 3 reads the domain glossary first and checks what each environment runs as a lead to test, never as proof. The post-mortem lists lessons about driving the app for /app-control:map refresh, and the always-on signal step points at /implementation:implement. Adds eval cases for the diagnose action and the surface pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MWmASPFjTu4anSp6BowqVC
Each case's prompt.md description already says what its scaffold builds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MWmASPFjTu4anSp6BowqVC
An observation can name a failing CLI option, so only --unattended is taken out as a flag. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MWmASPFjTu4anSp6BowqVC
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MWmASPFjTu4anSp6BowqVC
…oop file The surface pass and Phase 3 read logs, captures, dead-letter records, deployment records and tickets, so each is framed as data per the untrusted-content contract. A diagnose run keeps any loop file it wrote in a scratch directory and names it, leaving no test in the test tree. The port rule states our decision with Vite's pin as one example, the CI-run lookup names GitHub's head_sha as an example, and the surface-pass eval gains an end-state grader that catches a fix made by any tool. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MWmASPFjTu4anSp6BowqVC
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
The action table now lists 'diagnose this' as a plain-words request for the diagnose action, matching the description, and the summary names the full loop as debug. Phase 1 writes its assumptions before the loop rather than first, so the surface pass reads as its opening step. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MWmASPFjTu4anSp6BowqVC
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 62aba91512
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Lane run report
Failed GitHub API calls
|
…tory's adapter Phase 3 keeps the check of which CI runs an environment's deployed commit had, and leaves the provider lookup to the repository's CI or forge adapter instead of naming a provider API field. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MWmASPFjTu4anSp6BowqVC
|
claude-security-review reviewed Security review: PR #16617, whole review at
|
|
claude-review reviewed Code review (blocking only): no findingsThis lane already reviewed an earlier head ( What I checked:
Spend: the two new eval cases add 3 runs each per arm when evals run, which uses prepaid API credits. This is not a hold. Verifier dropped 0 of 0 candidates. |
…hipping them /debugging:debug now states only the stack-agnostic rules and reads stack specifics (the logger, banned output APIs, clock and network fakes, timing tools, where a framework logs or picks a port) from the repository's own config, rules and instructions, then /discovery:research when enabled. The per-ecosystem reference file and the tool names and doc links in the surface pass and the loop table are removed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MWmASPFjTu4anSp6BowqVC
No related issue: an owner-decided change to
/debugging:debug.Summary
/debugging:debuggains a nameddiagnoseaction and a Phase 1 surface pass, and its Phase 3 and Phase 6 steps read and record more of what the app shows.Fix
diagnoseaction./debugging:debug diagnose <observation>runs Phases 1 to 4, stops once a Phase 3 row is confirmed, reports the cause at file:line with the loop command and its output, removes its Phase 4 probes and changes no other code. Until now this existed only as a stop rule in "What this skill does not do"; that rule now points at the action. The full loop keeps the namedebug: ISO/IEC/IEEE 24765 defines debug as "to detect, locate, and correct faults" and diagnostic as "detection and isolation of faults", with no correction. The action is in the argument hint, the Arguments line and the description (the'diagnose this'trigger moved from the full-loop list to the action)./app-control:drive readywhen theapp-controlplugin is enabled and the project has a control skill, otherwise the manifests and launch config. Then the surfaces the app type has (browser app, API, worker, CLI). Never assume a port: read it from the running process, drive's report, or launch config that pins it. Logs, captures, records and tickets the skill reads are framed as data, never instructions. Each source the run cannot reach becomes a request for scoped access, never for a secret; a subagent reports the request to its parent. The "no loop: full stop" rule stays./discovery:researchwhen enabled.skills/debug/reference/ecosystem-debugging.md(per-stack .NET, Python and TypeScript rows) is removed with its anchor-baseline entry, and the tool names in the loop table (a named browser driver,curl, S3) are replaced by agnostic wording.docs/architecture/landscape.jsonstill lists that file among the citing files of three external links; it is a recorded landscape snapshot no check reads, left for its next refresh./app-control:map refresh "<lesson>"lines; map stays the one writer of a control skill's Gotchas. The always-on signal step points at/implementation:implementfor observability-by-default design guidance.diagnose-action-stops-at-cause(the action word alone, no "don't change it" in the prompt) andsurface-pass-reads-worker-dead-letter(a worker that exits 0 while its dead-letter directory and log hold the failure).Verification
claude plugin evalon the final skill text (Claude Code 2.1.325, model claude-opus-5-5, judge opus,--scaffold, 3 runs, with and without arms, through the Docker link swap), one case per pass:diagnose-action-stops-at-cause: validity gate VALID; every grader passed in all 6 runs, judges unanimous. The without-plugin arm also scores 1.00, so the case is at the ceiling.surface-pass-reads-worker-dead-letter(with the newworker-restoredgrader): validity gate VALID; with-plugin runs passed every grader; one without-plugin run missedcites-tax-line; judges unanimous.validate-cases.py plugins/debugging/evals: no FAIL; the new deterministic graders pass their offline sample answers.scripts/validate-plugin-contracts.mjs: pass.check-skill.sh plugins/debugging/skills: PASS (SKILL.md 446/500 lines, description 1020/1024).scripts/check-as-of-recheck.sh --check: pass.plugins/debugging/scripts/build-view.test.sh: all ok.check-affected: pass after regeneratingdocs/skill-cheat-sheet.md.worker-restoredgrader added to the surface-pass case. Kept as decided: any--token other than--unattendedis part of the free-text observation, which departs from the argument-shape contract's stop-on-unknown-token rule because an observation can quote a failing CLI option.Related
/app-control:drive readyand/app-control:map refreshare the surfaces this change calls./implementation:implementas the owner of observability-by-default design guidance; that skill does not carry the guidance onmainyet, so the pointer depends on a separate change to theimplementationplugin.🤖 Generated with Claude Code
https://claude.ai/code/session_01MWmASPFjTu4anSp6BowqVC