diff --git a/docs/guide/02-poteto-mode.md b/docs/guide/02-poteto-mode.md index 182b47c..8fad639 100644 --- a/docs/guide/02-poteto-mode.md +++ b/docs/guide/02-poteto-mode.md @@ -83,7 +83,7 @@ load pstack:poteto-mode im stepping away. keep going until the migration check r Work you'll review later routes through [`pstack:figure-it-out`](../../skills/figure-it-out/SKILL.md), which designs the run's phases and keeps a [`pstack:show-me-your-work`](../../skills/show-me-your-work/SKILL.md) decision log. [Run work while you sleep](./07-overnight.md) covers the full overnight contract. -**Pitfall:** don't enumerate skills in your prompt ("use pstack:how, then pstack:architect, then pstack:interrogate..."). The playbook already sequences them, and a hand-written sequence usually reorders or drops steps the playbook would have kept. Name a skill only when you want to override a specific choice. +**Pitfall:** don't enumerate skills in your prompt ("use pstack:architect, then pstack:interrogate, then pstack:unslop..."). The playbook already sequences them, and a hand-written sequence usually reorders or drops steps the playbook would have kept. Name a skill only when you want to override a specific choice. Read [`poteto-mode`](../../skills/poteto-mode/SKILL.md) itself for the full routing rules. diff --git a/docs/guide/03-understand.md b/docs/guide/03-understand.md index 375688c..fe02b69 100644 --- a/docs/guide/03-understand.md +++ b/docs/guide/03-understand.md @@ -10,7 +10,7 @@ Editing code you don't understand is how subtle regressions ship. pstack gives y load pstack:how do we dedupe notifications? is there an n+1 when we look up subscribers? ``` -Ask the question you actually have. [`pstack:how`](../../skills/how/SKILL.md) reads the code and answers at the level of a senior engineer onboarding you onto the subsystem, with the runtime flow, the key types, and the non-obvious parts. For a big subsystem it fans out two to four read-only explorers first. For a narrow question it just reads and explains. +Ask the question you actually have. [`pstack:how`](../../skills/how/SKILL.md) reads the code and answers at the level of a senior engineer onboarding you onto the subsystem, with the runtime flow, the key types, and the non-obvious parts. It starts one read-only `delegate_task` subagent that explores the code and writes the explanation. ## Dig up history with `pstack:why` @@ -18,7 +18,7 @@ Ask the question you actually have. [`pstack:how`](../../skills/how/SKILL.md) re load pstack:why was the retry limit set to five? does the reason still hold? ``` -[`pstack:why`](../../skills/why/SKILL.md) works like a detective on a cold case. It starts from source control, then queries whatever evidence categories your MCPs expose, such as the issue tracker, long-form docs, team chat, observability, error tracking, and analytics, all in parallel. The report cites everything, separates direct evidence from inference, and says "appears to" when the record is thin. A null result gets reported too, because "nobody wrote down why" is itself an answer. +[`pstack:why`](../../skills/why/SKILL.md) answers from the record, not the code. It reads the commits and PR discussions that introduced the code and each later change, and looks at other sources when that trail is thin. The answer cites what the authors wrote, marks the agent's own inferences as inferences, and says what the record does not answer, because "nobody wrote down why" is itself an answer. The two compose naturally. `do why first then how` is a perfectly good prompt when you suspect the history explains the mess. diff --git a/docs/guide/04-design.md b/docs/guide/04-design.md index 5e21d7a..1c23e9e 100644 --- a/docs/guide/04-design.md +++ b/docs/guide/04-design.md @@ -10,7 +10,7 @@ One attempt at a hard design locks in the first shape the model thought of. `pst load pstack:architect design the import pipeline before writing any code. i care most about how callers use it. ``` -[`pstack:architect`](../../skills/architect/SKILL.md) grounds itself first, running `pstack:how` over the code the design touches and `pstack:why` when it moves ownership or layers. Then it runs a competitive design round to produce competing design sketches, with the caller's usage written first in each, followed by types, signatures, and a module map. +[`pstack:architect`](../../skills/architect/SKILL.md) grounds itself first, tracing the code the design touches and running `pstack:why` when it moves ownership or layers. Then it runs a competitive design round to produce competing design sketches, with the caller's usage written first in each, followed by types, signatures, and a module map. By default it proceeds straight from the synthesized design into implementation. If you want to see the design first, say so: diff --git a/plugin.yaml b/plugin.yaml index 4d252a8..088efe4 100644 --- a/plugin.yaml +++ b/plugin.yaml @@ -1,4 +1,4 @@ name: pstack -version: "0.3.5" +version: "0.3.6" description: "Poteto's engineering rigor stack. Playbooks, principles, multi-model review." author: poteto (adapted by Madeleine) diff --git a/skills/architect/SKILL.md b/skills/architect/SKILL.md index d84ec1e..2ed066c 100644 --- a/skills/architect/SKILL.md +++ b/skills/architect/SKILL.md @@ -23,7 +23,7 @@ Before building a new subsystem, feature, or significant refactor. When the shap ### A. Ground the Problem -Run `pstack:how` over relevant existing subsystems. Understand what's already there before designing something new. Read the code the new work will touch and integrate with. +Trace the relevant existing subsystems. Naming a file isn't grounding. Read the code the new work will touch and integrate with, and write an architectural explanation of each. If the design redefines ownership or layering, also run `pstack:why` on the existing shape so the rationale becomes a constraint, not a guess. ### B. Sketch diff --git a/skills/blast-radius/SKILL.md b/skills/blast-radius/SKILL.md index f30b47a..b4e9873 100644 --- a/skills/blast-radius/SKILL.md +++ b/skills/blast-radius/SKILL.md @@ -37,10 +37,10 @@ Step 4 is usually one small script that imports the same library the app ships a ## Steps -1. Read the change. The diff, the symbols it adds, changes, and deletes, and what it now does differently, including the part the diff doesn't spell out. Use `why` step 2 to pull the PR and commits. +1. Read the change. The diff, the symbols it adds, changes, and deletes, and what it now does differently, including the part the diff doesn't spell out. Pull the PR and its commits. 2. Find the one fact it's safe because of. Most changes that look risky are safe because of a single fact, like "this call only drops already-dead cache entries and does nothing else". Find that fact. If it holds, most risky cases are cleared at once. Spend your time here, not on a long list of maybes. 3. Look where grep stops. Read the source of the library you call, and check its pinned version and any local patch. Work out when things run: microtasks, unmount and teardown, Solid versus React. Follow what a symbol search misses: the JSON an API returns, a DB column, a wire format, another language reading the same bytes, a feature flag, code three hops downstream. -4. Be honest about each risk. Give it a real chance of happening and a real cost if it does. Keep the risks you confirmed. List the ones you checked and cleared separately. Same rules as `why`. Cite a real `file:line`, a search that finds nothing is still an answer, and never make up a caller or an API. +4. Be honest about each risk. Give it a real chance of happening and a real cost if it does. Keep the risks you confirmed. List the ones you checked and cleared separately. Cite a real `file:line`, a search that finds nothing is still an answer, and never make up a caller or an API. 5. Prove the one fact. Write a script or test that runs the real code, run it, and paste what happened. 6. For a big or wide change, run it as an `arena`. Ask several models the same question and merge the answers. Different models catch different real bugs. diff --git a/skills/how/SKILL.md b/skills/how/SKILL.md index 057a1af..8961254 100644 --- a/skills/how/SKILL.md +++ b/skills/how/SKILL.md @@ -1,7 +1,7 @@ --- name: how description: "Use to explain how code works, with walkthroughs." -version: 0.1.0 +version: 0.2.0 author: poteto (Hermes adaptation) license: MIT platforms: [linux, macos, windows] @@ -13,58 +13,16 @@ metadata: # How -Answer "how does X work?" with clear architectural explanations grounded in actual code. +Explore the codebase to answer "how does X work?" questions. -## When to Use +If the scope is ambiguous, state your interpretation and explore. The user can redirect. -When the user asks how a subsystem, feature, or mechanism works. Also invoked by other pstack skills (e.g. `pstack:architect`) to ground design in existing code. +## 1. Explain -## Modes +Spawn one read-only `delegate_task` subagent (tools limited to `search_files` and `read_file`) that explores and explains in one pass. Give it the user's question and this brief: -### Explain (default) +> You are writing an architectural explanation for a senior engineer. Explore the codebase and write an explanation that a senior engineer unfamiliar with this area could read and walk away with a solid mental model. They should understand the architecture well enough to start working in it confidently. You have read-only access to the codebase. -Explore the codebase and produce an explanation. +## 2. Present -**Simple questions** (single module, one concept): do it in one pass - `search_files` to locate, `read_file` to understand, write the explanation. - -**Complex questions** (multi-file subsystem, cross-cutting concern): decompose into 2-4 exploration angles, delegate each as a read-only subagent via `delegate_task`, then synthesize results into a single explanation. - -### Critique - -Explain first (as above), then spawn multiple `delegate_task` subagents to independently identify architectural issues. Each critic works from the explanation plus direct code access. Synthesize their findings into a problems section appended to the explanation. - -## Exploration Technique - -Explorers use `search_files` and `read_file` to trace code. The approach: - -1. **Start broad** - glob for directories (`search_files` with `target="files"`), grep for key types/interfaces/exports. Understand the shape before the details. - -2. **Follow the thread** - from entry points, trace callers and callees. Follow data flow: where is it created, transformed, consumed? Read actual code at each hop. - -3. **Read actual code** - don't stop at file names or function signatures. Read the implementation. The interesting behavior lives in the body, not the type. - -4. **Note surprises** - anything that violates the expected pattern is worth calling out. Implicit dependencies, side effects, hidden state, fallback paths. - -## Output Format - -Structure the explanation as: - -- **Overview** - one paragraph, what this subsystem does and why it exists -- **Key Concepts** - types, abstractions, or domain terms needed to follow along -- **How It Works** - step-by-step walkthrough of the main flow, referencing files and functions -- **Where Things Live** - file/directory map of the relevant code -- **Gotchas** - surprises, implicit contracts, known quirks, easy mistakes - -## Delegation Pattern - -When decomposing for `delegate_task`: - -``` -Each subagent brief: -- Goal: "Trace how [specific aspect] works in [scope]" -- Constraint: read-only, no modifications -- Tools: search_files, read_file only -- Output: structured notes with file paths and line references -``` - -Synthesize by merging notes, resolving contradictions (re-read code if needed), and writing the unified explanation. +Present the subagent's output. Light edits for clarity or context from the conversation are fine. Do not substantially rewrite it. diff --git a/skills/no-comments/SKILL.md b/skills/no-comments/SKILL.md index df7e3dd..4379ee1 100644 --- a/skills/no-comments/SKILL.md +++ b/skills/no-comments/SKILL.md @@ -24,7 +24,7 @@ Use the caller's files or diff. Otherwise use the current diff against the base ## Steps 1. Spawn one `delegate_task` with the scope and Comment Sicko's review prompt. Pass the scope. Do not restate its rules. -2. Inspect its report and diff. Reject application-code edits, scope escapes, exception-protected deletions, misstated `MUST KILL` reasons, and flags that treat kept intentional code as guilty. Reshape flags on our-code surprises stay actionable. Do not restore those comments. A keep survives only with proof it is about something we cannot change. Audit missed scoped lint and TypeScript suppressions. Correctness or safety suppressions stay actionable `MUST KILL`s. Restore deletions only with exact exceptions and scoped proof. Before accepting thin `IMPORTANT` or `do not remove` kills or keeps, run `/how` or `/why` on their symbol. If a kill is ambiguous, do not restore. If a keep is refuted or still ambiguous, delete it. Revert and rerun one rejected report with the failure named. Reject a second, report it open, and fail `/no-comments`. +2. Inspect its report and diff. Reject application-code edits, scope escapes, exception-protected deletions, misstated `MUST KILL` reasons, and flags that treat kept intentional code as guilty. Reshape flags on our-code surprises stay actionable. Do not restore those comments. A keep survives only with proof it is about something we cannot change. Audit missed scoped lint and TypeScript suppressions. Correctness or safety suppressions stay actionable `MUST KILL`s. Restore deletions only with exact exceptions and scoped proof. Before accepting thin `IMPORTANT` or `do not remove` kills or keeps, trace their symbol or run `pstack:why` on it. If a kill is ambiguous, do not restore. If a keep is refuted or still ambiguous, delete it. Revert and rerun one rejected report with the failure named. Reject a second, report it open, and fail `/no-comments`. 3. Fix trivial accepted flags directly by deleting a dead path, dropping a parameter, or using the real API. If any fix needs a shape, run `/architect` once for the accepted set and surrounding code. Stop at the sketch. Architect shapes. Step 4 implements. 4. Implement the smallest root-cause fix in scope. Remove every named workaround. If the root cause is out of scope, land the smallest in-scope fix and report the rest open. The **principle-fix-root-causes** and **principle-redesign-from-first-principles** skills guide intent only. Neither authorizes widening the fence nor fixing instances outside it. Never bolt on symptom guards. 5. Constraint comments say `do not remove`, `do not change wording`, or `talk to X before changing`. Leave keeps about things we cannot change. Offer the cheapest in-scope type, runtime, test, or CI lint. Wait for interactive approval. Unattended and eval require caller pre-approval. If approved, encode then delete. Otherwise delete, report the constraint open, and sketch out-of-scope work. diff --git a/skills/no-comments/references/comment-sicko.md b/skills/no-comments/references/comment-sicko.md index 9a688b8..d18588c 100644 --- a/skills/no-comments/references/comment-sicko.md +++ b/skills/no-comments/references/comment-sicko.md @@ -23,7 +23,7 @@ That list is my only leash. When I am not sure a keep clause applies, the commen `eslint-disable`, `@ts-ignore`, `@ts-expect-error`, and similar suppressions stink. Look up the rule. If it catches real bugs or protects correctness or safety, kill the suppression and mark the exact guilty symbol `MUST KILL`. -`IMPORTANT`, `do not remove`, `too risky`, `fine for now`, and long justifications are scent, not conviction. Before judging, I read nearby code. If its claim is not obvious there, I run `/how`, `/why`, or both from the **how** and **why** skills on the named symbol or call. Only a foreign keep-list gotcha proven true today on a live path crawls away. Our-code surprises die with the reshape flag above. Doubt after the hunt is meat. +`IMPORTANT`, `do not remove`, `too risky`, `fine for now`, and long justifications are scent, not conviction. Before judging, I read nearby code. If its claim is not obvious there, I trace the named symbol or call, run the **why** skill on it, or both. Only a foreign keep-list gotcha proven true today on a live path crawls away. Our-code surprises die with the reshape flag above. Doubt after the hunt is meat. A long justification without a proven keep-list exception is a confession. Kill it. Never polish meat into a shorter alibi. Mark the exact guilty symbol `MUST KILL`. My kill ends there. I do not touch the code. diff --git a/skills/poteto-mode/SKILL.md b/skills/poteto-mode/SKILL.md index da4c314..c013edc 100644 --- a/skills/poteto-mode/SKILL.md +++ b/skills/poteto-mode/SKILL.md @@ -26,7 +26,6 @@ These triggers fire before you write anything. No exceptions. | Trigger | Action | |---------|--------| | Your first message on a task | Tell the user, in your own words, their goal, the problem they are solving, and, if the task ships a change, the smallest change you expect to meet it. Don't wait for confirmation. If the evidence rules out the change they asked for, show it and stop. Restate the goal in each subagent brief, in each PR's `## Why`, and at the top of each final report, except after a fully specified one-line edit. | -| Nontrivial change (new feature, refactor, multi-file edit) | `skill_view(name='pstack:how')` first. Plan before code. | | Any code produced | Name the data shape before writing logic. Types first, functions second. | | Code crossing a function/module boundary | `skill_view(name='pstack:architect')`. Design the seam. | | Parallel fan-out (3+ independent slices) | `skill_view(name='pstack:swarm')`. Coordinate subagents. | diff --git a/skills/poteto-mode/playbooks/bug-fix.md b/skills/poteto-mode/playbooks/bug-fix.md index 40f4a70..03fb6db 100644 --- a/skills/poteto-mode/playbooks/bug-fix.md +++ b/skills/poteto-mode/playbooks/bug-fix.md @@ -6,7 +6,7 @@ 1. **Reproduce it yourself.** Don't ask the user for more info yet - try to hit the bug with `terminal`, tests, or manual verification. If you can't reproduce, *then* ask. A bug you can't trigger is a bug you can't verify you fixed. -2. **Binary-search the cause.** Seed understanding with `pstack:how` over the affected subsystem. Then: +2. **Binary-search the cause.** Seed understanding by tracing the affected subsystem yourself (`search_files`, `read_file`). Then: - Form 2-3 hypotheses for root cause. - For each: find runtime evidence that confirms or eliminates it (`terminal` for logs/debugger, `search_files` for call sites, `read_file` to trace flow). - Eliminate until one survives. If none survive, your model is wrong - re-read. diff --git a/skills/poteto-mode/playbooks/feature.md b/skills/poteto-mode/playbooks/feature.md index 7d70649..ebc80d0 100644 --- a/skills/poteto-mode/playbooks/feature.md +++ b/skills/poteto-mode/playbooks/feature.md @@ -4,7 +4,7 @@ ## Steps -1. **Understand the subsystem.** Run `pstack:how` over every module the feature touches. Don't design until you know what's already there. +1. **Understand the subsystem.** Trace every module the feature touches. Don't design until you know what's already there. 2. **Design exploration.** Run `pstack:architect` - explore approaches, pick one, document why. Output: a short design doc with the chosen approach and its trade-offs. diff --git a/skills/poteto-mode/playbooks/hillclimb.md b/skills/poteto-mode/playbooks/hillclimb.md index c6441b9..a56f00e 100644 --- a/skills/poteto-mode/playbooks/hillclimb.md +++ b/skills/poteto-mode/playbooks/hillclimb.md @@ -4,12 +4,12 @@ Core discipline: one change, one measurement, keep or revert. Never stack untested changes, and never claim a win from code inspection (the **prove-it-works** principle skill). -1. Ground the workload and architecture before choosing the metric. Run the **how** skill over the target, name the realistic workload dimensions that can move the result (data size, history, state, concurrency), and select a case that reproduces the user's complaint. If no case reproduces it, fix the repro instead of hillclimbing. Then fix one metric, the direction that counts as better, and a checkable stop predicate that pairs a target with a floor on attempts so a lucky early win can't end the run (the example "at least 50% better than baseline and at least 10 iterations" is this shape). Use the user's numbers when given, otherwise agree them. +1. Ground the workload and architecture before choosing the metric. Trace the target, name the realistic workload dimensions that can move the result (data size, history, state, concurrency), and select a case that reproduces the user's complaint. If no case reproduces it, fix the repro instead of hillclimbing. Then fix one metric, the direction that counts as better, and a checkable stop predicate that pairs a target with a floor on attempts so a lucky early win can't end the run (the example "at least 50% better than baseline and at least 10 iterations" is this shape). Use the user's numbers when given, otherwise agree them. 2. Build the measurement harness, prove its sensitivity, then freeze it (the **build-the-lever** principle skill). Run contrasting realistic workloads and confirm the target case reproduces the symptom while easier cases separate as expected. If the harness cannot distinguish them, revise the workload or metric. Vet the harness with the **benchmark-checklist** skill before you freeze it, and make it print its error count and a count of the work done. Once frozen, one repeatable command emits the metric, sampled enough to clear the noise (median of N, not a single run). Record the baseline metric and a green run of the regression gate (the tests that must keep passing) before any change. 3. Open the decision log via the **show-me-your-work** skill. A `decision.tsv`, one row per attempt: id, hypothesis, change, before, after, delta, tests, verdict (kept or reverted), note. Read it before each attempt. Keep it out of the tree (gitignored). 4. Ground each hypothesis in the architecture model from step 1, so it names a specific mechanism ("defer X off the boot path because it blocks first paint"), not "try memoizing something". For a perf metric, order hypotheses by the performance mantras in step 2 of the Perf issue playbook (`playbooks/perf-issue.md`). Borrow only their order, not that step's stop rule. 5. Loop, one hypothesis per iteration: - - Hand the change to a subagent using your configured hillclimb model (default `grok-4.7-xhigh-fast`) with a tight scope. Supervise and review the diff rather than typing it (the **guard-the-context-window** principle skill). When several independent hypotheses are live, fan them to parallel subagents, each in its own worktree (the **separate-before-serializing-shared-state** principle skill). + - Hand the change to a `delegate_task` subagent (it runs on your configured `delegation.model`) with a tight scope. Supervise and review the diff rather than typing it (the **guard-the-context-window** principle skill). When several independent hypotheses are live, fan them to parallel subagents, each on its own branch or scratch copy (the **separate-before-serializing-shared-state** principle skill). - Measure before and after with the frozen harness, and run the regression gate. - Accept only when the metric moves past noise and the gate stays green. Otherwise revert the change in full. A tweak that "might help" is not kept. - One commit per accepted fix, staging only the files you changed (`git add `, never `-A`). Log the row either way, kept or reverted. diff --git a/skills/poteto-mode/playbooks/investigation.md b/skills/poteto-mode/playbooks/investigation.md index 6b2de4f..74dbf1f 100644 --- a/skills/poteto-mode/playbooks/investigation.md +++ b/skills/poteto-mode/playbooks/investigation.md @@ -6,9 +6,7 @@ No code changes. No PR. Read-only answer. ## Steps -1. **Route through `pstack:how`** - pick the right mode: - - *Explain* for "how does X work" / "where does Y live" - - *Critique* for "should we do X or Y" / "is this approach sound" +1. **Trace the code the question is about.** For motivation questions, also route through `pstack:why`. For a decision between alternatives, the deliverable is a recommendation with a tradeoffs table instead of the structure in step 3. 2. **Gather context** - `search_files` + `read_file` to trace the real flow end-to-end. Don't stop at the entry point; follow calls until the picture is complete. Use `web_search` / `web_extract` for external APIs or specs. diff --git a/skills/recall/SKILL.md b/skills/recall/SKILL.md index 871a002..ad563e5 100644 --- a/skills/recall/SKILL.md +++ b/skills/recall/SKILL.md @@ -17,14 +17,14 @@ metadata: Keep it tight and on-topic. Read only what the in-scope threads need, then stop. -Your context lives in two records. Your own chat history holds what you did and decided. The shared record holds everything that happened around the same code under other names: the symptoms users keep reporting, the fixes that shipped and got reverted, the errors still firing in prod. That second record is what the **why** skill searches, across source control, the issue tracker, chat and issue channels, long-form docs, and error tracking. A feature with a long bug tail keeps most of its story there, so don't reconstruct it from your transcripts alone. +Your context lives in two records. Your own chat history holds what you did and decided. The shared record holds everything that happened around the same code under other names: the symptoms users keep reporting, the fixes that shipped and got reverted, the errors still firing in prod. That second record lives in source control, the issue tracker, chat and issue channels, long-form docs, and error tracking. A feature with a long bug tail keeps most of its story there, so don't reconstruct it from your transcripts alone. Search your own chat history with the `session_search` tool: a query, or a `session_id` plus `around_message_id` to read context around a known message. It only reaches the current Hermes profile's session history, never another profile's. 1. Classify, then route. One specific prior chat to resume is the `session-pickup` playbook, not this. Turning habits into a durable skill is `automate-me`. A human-readable summary of your work is a different task. Recall loads working context across recent chats before you act. If the user already gave you a full state capsule (paths, branch, the change), use it and skip the mining. 2. Lock the scope before searching. Pin the window ("recent" is a real range, default the last 7 days), the topic if named, and the Hermes profile (default the active one. Never read another profile's sessions without being asked). State the scope back. Never quietly turn "all" into "recent N". 3. Fan out across your chat history. Spawn parallel subagents on a fast, cheap model, each taking a slice of the window. Tell every subagent to query `session_search` with the topic, order candidates by recency, and read only the matching sessions and their relevant regions (`around_message_id`), skipping the current chat plus obvious noise (subagent, eval, and test chats). Each returns the same schema, one block per chat: topic, the user's goal, decisions, open threads, struggles and corrections, and artifacts (PRs, tickets, branches), each citing the session id. For one or two chats, skip the fan-out and search directly. The raw transcripts stay in the subagents. The main thread gets only their findings. -4. Sweep the shared record whenever the topic names a feature, file, subsystem, area, or bug. This is the default, not a judgment call, and "my work on X" does not exempt it. Hand it to the **why** skill's source investigators, but steer their question from "why was this built this way" to "what's the current state, what's been tried and didn't hold, and what are users still reporting". Reuse its per-source playbooks, run the investigators in parallel with the chat-history mining, and inherit its posture: one investigator per source, null results are findings, skip an unavailable MCP and say so. Fold what comes back into the brief. Skip this step only for pure activity recall with no named target ("what did I do this week"), where your own history and live state are the entire answer. +4. Sweep the shared record whenever the topic names a feature, file, subsystem, area, or bug. This is the default, not a judgment call, and "my work on X" does not exempt it. Spawn one `delegate_task` investigator per source, in agent mode because read-only strips MCP access, and ask each "what's the current state, what's been tried and didn't hold, and what are users still reporting". Run them in parallel with the chat-history mining. Null results are findings. Skip an unavailable MCP and say so. Fold what comes back into the brief. Skip this step only for pure activity recall with no named target ("what did I do this week"), where your own history and live state are the entire answer. 5. Verify against live state. Take the PRs, branches, and tickets that the mining and the sweep surfaced and check them with `git` and `gh`. When the answer hinges on what an agent actually did (the tools it ran, files it read, errors it hit), read the full transcript, not just a trimmed local copy. 6. Write the brief to the contract below. Group by thread. Stay on the named topic. diff --git a/skills/reflect/references/synthesizer.md b/skills/reflect/references/synthesizer.md index d0e7819..bedf81c 100644 --- a/skills/reflect/references/synthesizer.md +++ b/skills/reflect/references/synthesizer.md @@ -25,7 +25,7 @@ Drop (implementation details that drift): - "linter at SHA `bd91aa7` uses chars/4 heuristic" - "`` has 175 tokens at limit 80" - "Bugbot flagged regex backtracking on May 2" -- "we renamed `gpt-4` to `gpt-4o` in `encodingForModel`" +- "we renamed `parseConfig` to `loadConfig` in `settings.ts`" Keep (durable patterns): - "closed regex enums for trigger detection are brittle. Prefer schema-validated structures" diff --git a/skills/sharpen/SKILL.md b/skills/sharpen/SKILL.md index 2e6c420..5f4756e 100644 --- a/skills/sharpen/SKILL.md +++ b/skills/sharpen/SKILL.md @@ -20,4 +20,4 @@ I want a deeper version of this task, not a bigger one. Before rewriting it, do Then rewrite my prompt to ask for exactly that. Return only the rewritten prompt. Do not do the task. -Leave choices that are mine to make open instead of deciding them. +Have the rewritten prompt ask the agent to prototype any choice an experiment can settle and recommend an option based on what it observed. Leave product or preference choices that no experiment can settle for me to make. diff --git a/skills/teach/SKILL.md b/skills/teach/SKILL.md index 91a2dae..5410d4a 100644 --- a/skills/teach/SKILL.md +++ b/skills/teach/SKILL.md @@ -18,7 +18,7 @@ metadata: Teach sits on top of `how` and `why`. Get your bearings on what the work is and what it touches, then run `how` for how it works and `why` for why it's that way. Those are real skill invocations that do their own digging. Blend what they find into one plain explanation, lead with what matters to the person, and go deeper when they ask. Reword freely for teaching, with one exception. Keep `why`'s confidence language intact (its hedges are findings, not style). 1. Decide the few things they should walk away understanding. Choose them from why they're asking (about to change it, reviewing it, debugging it, new to it) and what they already know, both read from the conversation, not quizzed out of them. Skip what they plainly already know. Put the depth where their question is. -2. Let `how` and `why` do the work, don't redo it. Read the code yourself to get oriented, then run `how` for how it works and `why` for why. Run them in parallel and combine the results. Match the size to the question. Run both for a subsystem, maybe one is enough for a small change. Keep `why` narrow by default since its full sweep is slow. Put the narrowing in the ask itself (a scoped question, git plus a source or two) so `why` records the skipped categories per its own contract, and widen it only when the reasons are the point. +2. Let `how` and `why` do the work, don't redo it. Read the code yourself to get oriented, then run `how` for how it works and `why` for why. Run them in parallel and combine the results. Match the size to the question. Run both for a subsystem, maybe one is enough for a small change. 3. Start with a plain definition. Name the thing and say what it is in general terms, the way a senior engineer would say it out loud, with its common name if it has one. Then tie it to the case in front of you ("in X, we use this to ...") and build from there: how it works, the deeper reasons, the edge cases. For each part, explain the idea so it clicks: the problem it solves and how it actually works. Walk through what happens as the person does the thing (opens a long chat, scrolls up) when that is what makes it land. Listing functions and constants is reference, not teaching. Don't print framing labels ("the one idea to hold onto", "the thing to walk away with", "the key insight", "at its core", "TL;DR"). Give the smallest complete answer first, a sentence or two, not a dense paragraph, then stop. Add layers when they ask. Never a wall of text. 4. Keep it a conversation, not a lecture or a performance. Offer to go deeper or move on, and follow their lead. No quizzes. No pacing theater. Don't print "Pause", don't ask them to say it back, don't announce "the sentence to nail", and don't flag a part as important or hard ("here is the part worth slowing down on", "this is the tricky part", "here is where it gets interesting"). Just say it. When you would pause, stop and let them respond. Running one-shot with no live human, deliver it cleanly and put any offer to go deeper at the end. 5. Show, don't only tell, and build the picture up diagram by diagram. Open the diff, the code, or the debugger when that is the fastest way to land it. Draw when a picture lands faster than words. For anything with three or more moving parts, do not draw one diagram with all of them at once. Draw a short series instead, where each diagram redraws the last and adds a single part, so the reader watches the system assemble. A single all-at-once diagram, especially one saved for the end, is a reference, not teaching. Concretely, to teach a flow from A to B to C, draw it three times. First A to B. Then redraw and add C. Then redraw and add the return edge or the next piece. Match the medium to the idea, and use both kinds when both help. A mermaid diagram fits a flow or structure where the labels carry the meaning. When the idea is spatial, like layout, overlap, scroll position, or a before and after, reach for the image-generation tool and draw it marker-on-whiteboard style with a few short labels, since image models garble long text. Generate that picture, don't settle for describing it in words. The build-up rule holds for generated images too. A single simple point needs no figure. diff --git a/skills/why/SKILL.md b/skills/why/SKILL.md index dcc00e6..f98c32a 100644 --- a/skills/why/SKILL.md +++ b/skills/why/SKILL.md @@ -1,7 +1,7 @@ --- name: why description: "Use for 'why does X work this way' or 'why did we pick Y'." -version: 0.1.0 +version: 0.2.0 author: poteto (Hermes adaptation) license: MIT platforms: [linux, macos, windows] @@ -13,151 +13,4 @@ metadata: # Why -Investigate the motivation and intent behind code. - -Companion to the `how` skill. `how` answers what the code does and how it works. `why` answers what forces led to its shape. - -Every investigator and the synthesizer below run on the Hermes-configured `delegation.model` (set by `pstack:setup-pstack`). `delegate_task` has no per-task model override, so model diversity is not available within one round. - -## Operating Posture - -Operate as a **careful, cautious, and precise investigator**. Be honest about what you know vs what you're inferring. Read `references/epistemics.md` for the full confidence framework and phrasing guide. The synthesizer must follow it. - -## Step 1. Understand the Target and the Question - -Parse what the user is asking. The **target** is usually a chunk of code, a pattern, a feature, or a named design decision. The **question** is usually a design rationale, a tradeoff, a motivating edge case, an external constraint, dead code, or a broad history sweep. - -If the target is vague ("why do we do it this way?" with no clear referent), make your best guess from conversation context (open files, recent edits, cursor location, what was just discussed). State your interpretation briefly so the user can redirect if you're off, then proceed. - -## Step 2. Establish the Code Anchor - -Before spawning investigators, anchor the investigation in concrete code. You need: - -- The relevant file path(s) and line range(s) -- The key symbols (function names, class names, constants) -- An initial commit list. The last few commits touching the target. -- PR numbers from merge commits (pattern `(#1234)` in the subject line) - -Build this inline. - -```bash -# Blame target lines for last-touch commits -git blame -L , - -# Full file history, with patches, through renames -git log --follow -p -- - -# Last N commits touching the file, PR numbers visible -git log --oneline -20 -- - -# Extract PR numbers from a commit message -git log -1 --format=%B -``` - -Pull PR bodies and discussion via `gh` for any substantive commits: - -```bash -gh pr view --json title,body,author,createdAt,mergedAt,labels,closingIssuesReferences,comments,reviews -``` - -Capture this as seed context (file paths, symbols, commits, PR numbers, linked ticket IDs). Pass it to the investigators. - -## Step 3. Spawn Parallel Investigators (default posture) - -**Default to the full parallel investigation.** - -### Discovery - -Before spawning investigators, list the available MCPs from the Hermes Agent environment. Use the available-tools map when present. Otherwise inspect the `mcps/` directory Hermes Agent exposes for enabled MCP servers. - -Map each available MCP to one evidence category: - -1. Source control history -2. Issue / ticket tracker -3. Long-form documents -4. Real-time team chat -5. Infrastructure observability -6. Error / exception tracking -7. Product analytics warehouse - -Source control is always available through git and `gh`. For the other six, classify using the MCP name, server instructions, tool names, and resource descriptors. If an MCP could fit more than one category, choose the one matching its primary evidence. Record ambiguous cases in the coverage map. - -Aim for a complete **coverage map**, not a minimal one. Document the null, don't skip the search. - -Launch all matching investigators in a single message so they run concurrently. Don't ask one agent to cover multiple MCPs. - -Subagent config (each): -- `toolsets`: `generalPurpose` -- agent mode, not read-only. **Do not use read-only mode.** It strips MCP access, which disables MCP-backed investigators entirely. Investigators still shouldn't write anything. - -Each investigator gets: -1. The base prompt from `references/investigator-prompt.md` -2. The category playbook `references/sources/.md` for the selected MCP, adapted from the examples in `references/source-playbook.md` -3. The cross-cutting `references/sources/incident-postmortem.md` **if the target code looks defensive** (null checks, retry logic, timeout handling, rate limiting, feature flags, egress guards, OOM handlers) -4. The code anchor from Step 2 (file paths, symbols, commit hashes, PR numbers, ticket IDs) -5. The user's original question - -### Investigator roster. One per available evidence category - -Spawn one investigator per category that has a matching MCP. Each owns exactly one tool or MCP. - -Each entry names the category and the kind of "why" it uniquely surfaces. Use it to know what to expect back, how to name a gap when a category returns empty, and (only in the rare provably-irrelevant case) to justify a skip. - -1. **Source control investigator**. Git history, `gh` for PRs, code comments, tests. Always spawn. The only guaranteed source. Best at surfacing *implementation-time rationale captured during review*. - -2. **Issue / ticket tracker investigator** (e.g. Linear, Jira, GitHub Issues, Plane, Shortcut MCP). Best at surfacing *the product or business forcing function*. Strongest when the why is external to engineering. - -3. **Long-form documents investigator** (e.g. Notion, Confluence, Google Docs, Coda MCP). Best at surfacing *long-form design rationale*. Where the why is written out before it becomes code. - -4. **Real-time team chat investigator** (e.g. Slack, Discord, Microsoft Teams, Mattermost MCP). Best at surfacing *real-time deliberation that never reached a doc*. Especially important when the source control, ticket, and doc paper trail is thin. - -5. **Infrastructure observability investigator** (e.g. Datadog, New Relic, Honeycomb, Grafana, Splunk MCP). Infra/runtime view. Best at surfacing *infrastructure and runtime reality that motivated the code*. Strongest when the target reacts to an infra signal (timeouts, retries, rate limits, circuit breakers). - -6. **Error / exception tracking investigator** (e.g. Sentry, Rollbar, Bugsnag, Airbrake MCP). Best at surfacing *the specific exceptions and error trajectories that motivated defensive or corrective code*. Strongest for catch blocks, null guards, type checks, retries, and other defenses. - -7. **Product analytics warehouse investigator** (e.g. Databricks, Snowflake, BigQuery, ClickHouse, dbt, Redshift MCP). Product/data view. Best at surfacing *product and data reality that shaped the code*. Strongest for flag-gated code, experiment-driven ships, data migrations, and "where did this number come from" questions. - -### When to skip an investigator - -Only skip with an **explicit, written justification** that goes in the final "Sources Consulted" section. Two valid reasons: - -- **No MCP is available for that category** in this environment. Flag this as a gap, not a choice. Example: "Real-time team chat skipped. No matching MCP available, so the conversational record was not searchable." -- **The source is provably irrelevant**, not just "probably irrelevant." A high bar. Example: "Error / exception tracking skipped. Target is a build-time script with no runtime code path." - -If your scope assessment suggests a single-commit trivial target where the PR description already contains the complete answer, you may answer inline **only after** confirming all seven available category searches would be redundant. Say so explicitly. This should be rare. - -## Step 4. Synthesize - -Spawn one synthesizer subagent: - -- `toolsets`: `generalPurpose` -- agent mode, not read-only. The synthesizer's quality check spot-verifies citations, which can require MCP access. Read-only mode strips MCPs and defeats that. - -The synthesizer gets: -1. The investigator findings, including any null results and any categories skipped with justification -2. The code anchor from Step 2 (file paths, symbols, commit hashes, PR numbers, ticket IDs) -3. The user's original question -4. The epistemics framework from `references/epistemics.md` -5. The synthesizer prompt template from `references/synthesizer-prompt.md` - -## Step 5. Present - -Take the synthesizer's output and present it to the user. You may lightly edit for clarity or add context from the conversation, but **do not rewrite the confidence language**. - -## Output Format - -The output structure is the one in `references/synthesizer-prompt.md`: The Question, The Code in Question, What We Found, What We Can Reasonably Infer, Competing Hypotheses, What We Don't Know, Sources Consulted, Confidence Summary. Adapt as needed, but keep the confidence separation intact, and keep Sources Consulted as one line per investigator, including the ones that returned nothing or were skipped, with the reason. - -After the Sources Consulted block, if the user's `why` question is a precursor to actually changing this code, convert the lineage findings into a Preserve / Change / Avoid / Risk constraint set suitable for planning the change. - -## Common Failure Modes to Avoid - -- **Recency bias**. Assuming the most recent commit is authoritative. The current shape is often the accretion of many earlier decisions. Trace back. - -## Reference Files - -- `references/epistemics.md`. Confidence tiers and phrasing guide. The synthesizer must follow it. -- `references/investigator-prompt.md`. Base prompt template for investigator subagents. -- `references/source-playbook.md`. Index pointing at the category playbooks below. -- `references/sources/*.md`. One self-contained example playbook per category, plus cross-cutting `incident-postmortem.md`. Give an investigator the single file that matches its category and adapt it to the available MCP. -- `references/synthesizer-prompt.md`. Prompt template for the synthesizer subagent, including the output format. +Answer from the record, not the code: read the commits and PR discussions that introduced the target and each later change (`git log --follow -p`, `git blame`, `gh pr view`), and other sources if the trail is thin. Cite what the authors wrote, mark your own inferences as inferences, and say what the record does not answer. diff --git a/skills/why/references/epistemics.md b/skills/why/references/epistemics.md deleted file mode 100644 index 3732bf6..0000000 --- a/skills/why/references/epistemics.md +++ /dev/null @@ -1,144 +0,0 @@ -# Epistemics - -How to reason about confidence when evidence is historical, fragmentary, and sometimes contradictory, and how to communicate it without flattening it into false certainty. - -Code doesn't carry its own motivation. You can read what code does. You can't read *why it exists*. That lives in commits, PRs, tickets, docs, and conversations, all incomplete, biased, and sometimes missing entirely. Pretending otherwise produces confident-sounding guesses that mislead the user. - -## Confidence Tiers - -Every claim in the final output must sit in one of these tiers. The tier determines which output section the claim goes in and how it's phrased. - -### 1. Direct - -An explicit, textual citation that answers the question. Not "the code does X so the author must have wanted X." Something an author actually *wrote* that says why. - -Examples: -- A PR description that says "this fixes the bug where users with >1000 items couldn't paginate" -- A ticket that says "we're adding this because customer Acme requested it in their security review" -- A code comment that says "// clamp to 100 because the upstream API rejects larger values" -- A design doc that says "we chose option A over option B because we need persistence across restarts" -- A chat message from the author saying "switching to this approach since the old one was flaky in tests" - -Phrasing: confident, present tense. "This exists because X." Cite the source. - -### 2. Supported - -Multiple pieces of indirect evidence converge. No single source states it explicitly, but the pattern across sources makes it likely. - -Examples: -- The PR title says "improve performance," the ticket is labeled "perf," and the surrounding commits all touch the same hot path -- Multiple tests were added alongside the change, all exercising edge cases with very large inputs -- The author's other PRs from the same week all mention the same incident in their descriptions - -Phrasing: confident but clearly derived. "The evidence points strongly to X: [the specific pieces]." Cite multiple sources. - -### 3. Inferred - -A reasonable reading of the context, but nothing explicitly supports it. The reader should understand this is *your interpretation*, not a fact from the record. - -Examples: -- The PR doesn't say why, but given the error was happening in production (per the incident channel timing) and the fix was rushed (merged the same day), it was likely a hotfix. -- The function name suggests retry logic. The retry count is 3. This matches the team's general convention of "3 retries" seen elsewhere in the codebase. - -Phrasing: hedged. "It appears", "likely", "suggests", "is consistent with", "one reading is". Make the inference chain explicit: "Given A and B, C seems likely because D." - -### 4. Speculative - -A plausible hypothesis, but the evidence is thin and other explanations fit equally well. Presenting these is valuable, but mark them clearly as guesses. - -Examples: -- "This might be a workaround for a browser bug that's since been fixed, but we found no contemporary evidence of that." -- "It's possible this threshold was chosen to match an SLA commitment, but no SLA doc references it." - -Phrasing: explicitly speculative. "One possibility is X, but we have no direct evidence." Usually lives in the "Competing Hypotheses" section alongside other possibilities. - -### 5. Unknown - -You looked and couldn't find out. A valid and important outcome. Document it. - -Phrasing: "We searched X, Y, and Z and found no evidence of why." Be specific about *what* you searched. "We couldn't find out" is less useful than "we searched the ticket tracker with keywords A and B, scanned the 6 PRs that touched this file since 2023, and grep'd the repo for string literals matching the threshold. None surfaced a rationale." - -## Phrasing Guide - -### Words that carry confidence. Use carefully - -These imply **Direct** or **Supported** confidence. Don't use them for inferences. - -- "because". Implies a causal claim with evidence -- "the reason is". Same -- "was designed to". Claims author intent -- "fixes", "addresses", "solves". Claims the change achieved its goal -- "the team decided". Claims a group decision happened - -If you're using these, you should have a citation immediately adjacent. - -### Words that hedge. Use for inferences - -- "appears to" -- "seems to" -- "likely" -- "suggests" -- "is consistent with" -- "one reading is" -- "plausibly" -- "may have been" -- "the evidence points toward" - -These signal that you're interpreting, not reporting. Use them liberally in the "What We Can Reasonably Infer" section. - -### Words to avoid - -- "obviously". If it were obvious, the user wouldn't be asking -- "clearly". Almost always precedes a claim that isn't clear -- "of course". Same -- "just" (as in "it's just X for performance"). Dismissive and usually hides uncertainty -- "I think" / "I believe". You're synthesizing evidence, not giving a personal opinion. Use "the evidence suggests" instead. - -### Avoid rationalization - -Code that "makes sense" today may have been written for reasons that no longer apply, or that were wrong when they were written. Don't retrofit a clean rationale onto messy history. - -Resist the urge to: -- Assume the author did the "right" thing and work backward to justify it -- Assume a consistent pattern across the codebase was intentional when it might be copy-paste -- Turn an absence of evidence into evidence of absence ("no one mentioned security concerns, so it must not have been a concern") - -## The Sycophancy Trap - -Users often phrase `why` questions with an embedded hypothesis: "Why do we do it this way, I assume it's for performance?" Don't simply confirm it. Treat it as one candidate among others and check the evidence independently. If the evidence supports it, say so with citations. If not, say so and present what the evidence *does* support. - -The user's guess is a prompt for investigation, not a conclusion to validate. - -## When Evidence Contradicts - -If two sources disagree (the PR description says one thing, the ticket says another), surface both. Don't pick the one that fits a tidier narrative. A typical pattern: - -- **The ticket says** "we need this for customer X's compliance requirement" -- **The PR says** "cleaning up tech debt in this area" - -Both may be true (the ticket motivated the work, the PR is the author's framing of it), or one may be wrong. Present both with their citations and let the user make the call. - -## When Evidence Is Missing - -An honest "we don't know" is one of the most valuable outputs this skill can produce. The user now knows: - -- The answer isn't in the obvious places -- They'll need to ask a human (the original author, the product owner, the team lead) to find out -- Or they can decide the question isn't worth pursuing further - -Failing to mark a gap and filling it with a confident guess actively harms the user. They'll act on the guess. - -When you hit a gap, name it concretely: -- What question you were trying to answer -- What sources you searched -- What you searched for in each -- What you found (nothing, or only tangentially related material) - -## Calibration Check Before Finalizing - -Before delivering the output, the synthesizer should review every claim in "What We Found" and "What We Can Reasonably Infer" and ask: - -1. Does this claim have a citation? If not, either add one or move it to "Inferred" / "Hypotheses". -2. Is the phrasing calibrated to the tier? (A Direct claim can use "because". An Inferred claim cannot.) -3. Am I treating the code itself as evidence for its own intent? If so, that's not evidence. Remove or reclassify. -4. Does the output include a "What We Don't Know" section? If no gaps are mentioned, that's suspicious. Either the evidence was unusually complete or something is being swept under the rug. diff --git a/skills/why/references/investigator-prompt.md b/skills/why/references/investigator-prompt.md deleted file mode 100644 index 3b56af4..0000000 --- a/skills/why/references/investigator-prompt.md +++ /dev/null @@ -1,103 +0,0 @@ -# Investigator Prompt Template - -Build each investigator's prompt from this template. Fill in the placeholders. Append the single category playbook `sources/.md` matching this investigator's evidence category (see `source-playbook.md` for the index). If the target code looks defensive (null checks, retry logic, timeout handling, rate limiting, feature flags, egress guards, OOM handlers), also append `sources/incident-postmortem.md` for the incident-flavored queries to run inside its own source. - ---- - -You are investigating the historical context and motivation behind a piece of code. A separate synthesizer combines your findings with other investigators' into a final answer, so gather evidence accurately rather than writing prose. - -Other investigators search different sources in parallel. Don't try to cover everything. Focus on your assigned source and go deep. - -## Operating Posture - -Work like a careful, cautious, precise investigator. Don't produce a narrative. Surface evidence and describe it accurately, including the parts that don't fit a tidy story. The more boring and exact your output, the more useful it is. A single verbatim quote with a precise citation beats a paragraph of plausible-sounding summary. - -- **Quote, don't paraphrase** when the exact wording matters. Citations should let the reader jump to the source and confirm the claim in seconds. -- **Go wide before going deep.** Cast a broad first net so you don't miss related context. Only then narrow in. -- **Track what you searched, not just what you found.** An absence is only useful if the reader knows what was looked for. Record queries verbatim. -- **Resist the story.** If three pieces of evidence line up neatly and a fourth contradicts them, the contradiction is the most interesting finding. Don't file it away. -- **Consider the counterfactual.** Before reporting a finding as strong, ask whether you would expect to find it if your current reading were wrong, and how the evidence would differ. -- **Never invent.** If you're tempted to round a partial finding up into a confident statement, stop and label it partial. The synthesizer is counting on your output being accurate. - -## The Question - -> {QUESTION} - -## The Code Anchor - -**Target files:** {FILES_WITH_LINE_RANGES} - -**Key symbols:** {SYMBOLS} - -**Initial commits touching this code (most recent first):** -{COMMIT_LIST} - -**PR numbers extracted from commit messages:** {PR_NUMBERS} - -**Ticket IDs mentioned in commits or PR bodies (if any):** {TICKET_IDS} - -## Your Assigned Source - -{SOURCE_NAME} - -{SOURCE_PLAYBOOK_SECTION} - -## Investigation Instructions - -Gather **evidence**. Don't answer the question directly. The synthesizer weighs the evidence and forms conclusions. Follow this loop: - -1. **Cast a wide net first.** Start broad so you don't miss related context, then narrow in on specific items. -2. **Read the whole thing.** Read any PR, ticket, doc, or thread fully, not just the title or summary. The key evidence is often buried in a comment, a subtask, or a follow-up. -3. **Follow links within your assigned source.** If a PR references another PR or commit, pull it. If a ticket links a parent or sibling, pull it. If a doc links another doc, pull it. Stay inside your assigned source. When you spot a cross-source reference, do NOT chase it yourself. Record it under "Additional Leads" so the investigator assigned to that source can pick it up. The one-investigator-per-category design depends on this. Chasing cross-source links duplicates work and confuses scope. -4. **Capture quotes verbatim** with their location (PR number, ticket ID, URL, commit hash, file:line). The synthesizer needs to cite this precisely. -5. **Note absences.** If you searched for something and came up empty, that's also a finding. Record what you searched for and what you didn't find. -6. **Watch for contradictions.** If two items in your source disagree, record both. Don't suppress the inconvenient one. - -Don't synthesize or form a final opinion on "the why." Collect the raw material honestly and completely. The synthesizer does the reasoning. - -## Epistemic Discipline - -- **Don't confuse mechanics with motivation.** A commit changing `limit = 50` to `limit = 100` shows the change, not necessarily why. Look for the explanation in the commit message, PR description, linked ticket, or review comments. -- **Don't infer intent from code style.** "The author chose a functional approach" is an observation about code, not evidence of intent. Claim intent only when the author stated it. -- **Preserve uncertainty.** If the evidence is ambiguous, say so. If one reading is more plausible but not certain, say that. Don't collapse ambiguity to look decisive. -- **No silent substitutions.** If the question is about feature X and you only find evidence about feature Y, don't present Y's evidence as if it answers X. - -## Output Format - -Return your findings in this structure. The synthesizer will read it directly. - -### Source -Which source you investigated (source control, issue / ticket tracker, long-form documents, real-time team chat, infrastructure observability, error / exception tracking, product analytics warehouse, code comments, etc.). - -### What I Searched -The queries you ran, the items you opened, the places you looked. Be specific. This tells the synthesizer how thorough the investigation was and what might still be unsearched. - -### Direct Evidence Found -For each piece that explicitly addresses the question: -- **What it says**: verbatim quote or accurate paraphrase -- **Where it's from**: PR #123, ticket ID, doc URL, chat permalink, commit hash, or file:line -- **Author and date** (if available) -- **Relevance**: one sentence on how it bears on the question - -### Indirect / Circumstantial Evidence -Items that don't explicitly answer the question but bear on it. For each: -- **What it is**: brief description -- **Where it's from**: location -- **What it suggests**: what a careful reader might infer, and why. Name the inference chain. -- **Alternative readings**: if the same evidence could support a different interpretation, note it - -### Contradictions -Two items that disagree with each other, with both citations. - -### Gaps -What you searched for and didn't find. Be specific: "Searched the issue tracker for [query] across [time range]. No matching issues." These absences are valuable data. - -### Additional Leads -Anything that suggests further investigation in a different source. For example, if a PR references a chat thread that wasn't in your source, note it so the real-time team chat investigator or a follow-up pass can pursue it. - -## What You're Not Doing - -- Writing the final answer. The synthesizer does that. -- Picking sides in contradictions. Surface them. -- Speculating beyond what the evidence supports. A hunch with no evidence isn't evidence. -- Reading the code itself to figure out intent. You may read the code to understand what the target *is*, but don't confuse "what the code does" with "why." diff --git a/skills/why/references/source-playbook.md b/skills/why/references/source-playbook.md deleted file mode 100644 index aa3d877..0000000 --- a/skills/why/references/source-playbook.md +++ /dev/null @@ -1,17 +0,0 @@ -# Source playbooks - -The why skill spawns one investigator per available evidence category, each reading a single source-specific playbook below. The playbooks are concrete examples for common MCPs. Adapt them for a different MCP in the same category. - -| Category | Playbook | Example MCP it documents | -|---|---|---| -| Source control history | [`code-archaeology.md`](./sources/code-archaeology.md) | git, `gh` | -| Issue / ticket tracker | [`linear.md`](./sources/linear.md) | Linear (adapt for Jira, GitHub Issues, Plane, Shortcut) | -| Long-form documents | [`notion.md`](./sources/notion.md) | Notion (adapt for Confluence, Google Docs, Coda) | -| Real-time team chat | [`slack.md`](./sources/slack.md) | Slack (adapt for Discord, Microsoft Teams, Mattermost) | -| Infrastructure observability | [`datadog.md`](./sources/datadog.md) | Datadog (adapt for New Relic, Honeycomb, Grafana, Splunk) | -| Error / exception tracking | [`sentry.md`](./sources/sentry.md) | Sentry (adapt for Rollbar, Bugsnag, Airbrake) | -| Product analytics warehouse | [`databricks.md`](./sources/databricks.md) | Databricks SQL (adapt for Snowflake, BigQuery, ClickHouse, dbt) | - -Cross-cutting: - -- [`incident-postmortem.md`](./sources/incident-postmortem.md). Add this if the target code looks defensive (null checks, retry, timeout, rate limit, feature flag, egress guard, OOM handler). diff --git a/skills/why/references/sources/code-archaeology.md b/skills/why/references/sources/code-archaeology.md deleted file mode 100644 index 32d5f97..0000000 --- a/skills/why/references/sources/code-archaeology.md +++ /dev/null @@ -1,88 +0,0 @@ -# Code Archaeology (git + in-repo) - -## What this source contains - -- Commit history (messages, dates, authors, diffs) -- PR descriptions, review comments, and discussion threads (via `gh`) -- Inline code comments, TODOs, FIXMEs, deprecation notes -- ADRs (architectural decision records) if the repo keeps them -- Tests. Names and assertions often encode the edge cases that motivated a change -- Related files modified in the same commits (co-change signal) -- CHANGELOG entries, release notes in the repo -- Issue/ticket IDs mentioned in commit messages and PR bodies - -The most trustworthy source, tied directly to the code, and the most complete. Everything that went through the repo should be here. - -## How to search it - -Expand the seed commit list: - -```bash -# Full history of the file through renames -git log --follow --oneline -- - -# Pickaxe: commits that added or removed this exact text -git log -S '' -- - -# Or for patterns: -git log -G '' -- - -# Who wrote each line and when -git blame -L , - -# The full diff of a specific commit -git show - -# Commits between two points affecting this file -git log .. -p -- -``` - -For each substantive commit, pull the PR context: - -```bash -# Find the PR number from the merge commit or branch -git log -1 --format=%B - -# Full PR context: body, review comments, linked issues -gh pr view --json title,body,author,createdAt,mergedAt,labels,closingIssuesReferences,comments,reviews,files - -# The --json reviews and comments fields are where the real signal is -``` - -Look for out-of-band docs: - -```bash -# ADRs often live in docs/adr/ or similar -rg -l -i 'architecture.decision' --glob '*.md' - -# TODOs and FIXMEs near the target -rg -n -C2 '(TODO|FIXME|HACK|XXX|NOTE)' - -# Related tests. Names often encode the "why" -rg -l '' --glob '*test*' -``` - -## What good evidence looks like here - -- A PR description that explains the problem being solved, not just the change ("This fixes the pagination bug that caused X") -- A long review thread where alternatives were debated -- An inline comment near the target line that explains a non-obvious constraint -- A test named `test_handles_edge_case_when_X` that reveals an edge case motivating the code -- A commit message that references a ticket or incident ID -- A CHANGELOG entry that summarizes the user-visible rationale - -## Common pitfalls - -- **Squash-merge flatlands.** If the repo squashes PRs, individual commits in the branch history are lost. Fall back to PR body and comments. -- **Misleading commit messages.** "Small refactor" sometimes hides an intentional behavior change. Look at the diff, not the message. -- **Cargo-culted patterns.** The author may have copied a pattern without understanding why. Check if the pattern originated earlier in the codebase and investigate *that* commit. -- **Bot commits and auto-merges.** Dependabot, Renovate, and automated backports usually don't carry motivation. Skip them when trying to find intent. -- **Treating code as evidence of intent.** The code itself isn't evidence for why it exists. Evidence comes from commit messages, PRs, comments, tests, docs. Don't cite "the function is named X" as evidence of intent. - -## What to return - -Every commit/PR/comment that bears on the question, with: -- The exact text (quoted) -- The hash / PR number / file:line -- Author and date -- Whether it's direct (explicitly addresses the question) or circumstantial diff --git a/skills/why/references/sources/databricks.md b/skills/why/references/sources/databricks.md deleted file mode 100644 index 2770763..0000000 --- a/skills/why/references/sources/databricks.md +++ /dev/null @@ -1,70 +0,0 @@ -# Databricks Analytics & System Tables - -## What this source contains - -Databricks is the product-analytics, data-pipeline, and warehouse-telemetry layer. It complements Datadog. Datadog is the *infra/runtime* view, Databricks is the *product/data* view (what users did, which experiments ran, how feature usage evolved, where a threshold constant came from). - -- **Product analytics events.** `your_warehouse.events.analytics_track_event` (raw) and typed, deduplicated per-event dbt models in `..`. User behavior: feature invocations, clicks, accepts/rejects, submissions, client-reported errors. -- **Usage & billing events.** `your_warehouse.events.usage_event` / `..stg_usage_events`, `your_warehouse.events.raw_model_event` / `..stg_raw_model_events`. For cost- or volume-driven decisions. -- **Experiment / feature-flag data.** Exposure and outcome tables. **Schema is company-specific.** Probe with `SHOW TABLES` before assuming names. -- **System tables.** `system.query.history`, `system.compute.warehouses`, `system.billing.*`, `system.access.audit`. Answer "was this query expensive?", "how often did anyone run this?", "when did warehouse load spike?" -- **dbt lineage.** Models in `.` reveal what pipelines depend on a table/field. Upstream changes frequently motivate consumer-code changes. -- **Databricks notebooks.** Exploratory analyses engineers wrote before code changes. **Not queryable via the SQL MCP.** If you suspect the rationale lives in a notebook, name it as a gap. - -## How to search it - -Use the Databricks SQL MCP. Primary tool: `execute_sql_read_only`. If it returns a `statement_id`, poll with `poll_sql_result` rather than re-running. - -**Orient before querying.** Schemas are company-specific. Probe before trusting a table name: - -```sql -SHOW TABLES IN . LIKE '**'; -DESCRIBE TABLE ..stg_; -``` - -**Time-bound every query.** These tables are huge and unconstrained scans time out. Filter on `_timestamp` (events) or `start_time` (`system.query.history`) with a window bracketing the ship date, typically ~30 days before and after, wider only for strong reason. - -**Prefer typed dbt models over the raw table.** `..
` is deduplicated, typed, and liquid-clustered. `your_warehouse.events.analytics_track_event` has duplicates and untyped `properties_json`. Model-name pattern: `stg__`, where `` is `app`, `backend`, `website`, or `cli`. Confirm the exact model name with `SHOW TABLES` when the pattern alone doesn't resolve it. Drop to the raw table only when there's no dbt model yet, or you need events from inside the dbt refresh lag. - -**Column conventions on the typed dbt models** (knowing these avoids a `DESCRIBE` round-trip): - -- `_timestamp`, `_id`, `_auth_id`, `_request_id`, `event_name`. Standard on every model -- `properties_`. Typed, underscore-cased event properties (`properties_entrypoint`, `properties_size_bytes`, …) -- `context_team_id`, `context_client_version`, `context_country`, `context_client_os`. Pre-extracted client context - -### Investigation patterns that tend to pay off - -Pick the table + column combination that matches the target: - -1. **Event usage trajectory.** Daily counts on the relevant `stg_*` model across a ±30d window around the PR merge. A step function from zero to steady volume within a day or two of the merge is strong circumstantial evidence the PR launched the feature. A decay to zero suggests a deprecation or deletion. -2. **Guard-rail / defensive-check origin.** Distribution (median / p99 / max) of the relevant `properties_` column in the 14 days *before* the PR. A p99 that matches the target's threshold constant suggests the number was chosen from data. -3. **Experiment / feature-flag lookup.** `SHOW TABLES ... LIKE '*experiment*'` to find the exposure table, then pull exposure counts by variant for the relevant flag key near the PR date. -4. **Query-history evidence for migrations, backfills, or perf rewrites.** `system.query.history` filtered by `statement_text ILIKE '%%'` with a tight `start_time` window surfaces the expensive queries that likely motivated the change (sort by `total_duration_ms` or aggregate `SUM(read_bytes)`, `COUNT(*)`). -5. **dbt lineage.** If the target reads from or writes into a `.` model, the model's own git history (in this repo) often carries the rationale. Hand that lead back to the git investigator rather than chasing it yourself. - -## What good evidence looks like here - -Beyond the pattern shapes above: - -- An error-classifying event's count drops to near zero in the days after a defensive-code PR. Suggests the PR resolved that error class -- An exposure table row names the target's feature-flag key with a "shipped" / "concluded" decision around the PR ship date - -## Common pitfalls - -- **Instrumented ≠ caused.** An event's existence means someone cared enough to log it, not that the target code exists *because* of it. Pair with a PR/commit citation from the git investigator before claiming causation. -- **Silent instrumentation changes.** A step function in event volume may mean a new event started being logged, not that user behavior changed. Check for instrumentation PRs in the same window before reading the ramp as a feature-launch signal. -- **Schema drift.** Event properties evolve. A column on the typed dbt model today may not have existed when the target was written. Older data may carry the property only inside raw `properties_json`. -- **dbt refresh lag.** `..*` is rebuilt on a schedule (often hourly/daily). For events from the last few hours, fall back to `your_warehouse.events.*` and deduplicate by `_id`. -- **Company-specific tables.** Experiment, feature-flag, billing, and usage tables vary. Reporting a result from a table whose existence you never confirmed is a classic failure mode. Probe with `SHOW TABLES` / `DESCRIBE TABLE` first. -- **Retention cliff.** If the relevant window predates the table's retention or the dbt model's creation date, that's a *gap*, not a null result. Name it explicitly so the synthesizer doesn't read "no results" as "no activity." -- **Notebooks aren't queryable.** The SQL MCP can't see Databricks notebooks. If you suspect the rationale lives in one, return a gap. - -## What to return - -For each relevant finding: -- Type (product event / experiment exposure / usage or billing event / system-table row / dbt model) -- Fully-qualified table name and the exact query you ran -- Time window queried -- Compact numeric summary (counts, percentiles, first/last-seen timestamps). **Don't dump raw rows.** -- Temporal correlation with the target's ship date (e.g., "first row 2024-08-15, PR #49074 merged 2024-08-14") -- Relevance + strength: direct / circumstantial / weak diff --git a/skills/why/references/sources/datadog.md b/skills/why/references/sources/datadog.md deleted file mode 100644 index d8363b1..0000000 --- a/skills/why/references/sources/datadog.md +++ /dev/null @@ -1,99 +0,0 @@ -# Datadog Telemetry - -## What this source contains - -Datadog holds the runtime record, what actually happened in production, as opposed to what was planned or discussed. - -- **Metrics.** Counters, gauges, histograms instrumented by the team. A metric's *presence* is itself evidence. Someone thought this number worth watching. -- **Monitors & alerts.** Conditions the team decided warranted waking someone up. A monitor firing on `rate_limit_hit > 10/min` is direct evidence the team worried about that threshold. -- **Dashboards.** Curated views. The charts tell you what the team considers important for a subsystem. -- **APM traces & spans.** Request-level runtime data. Useful for "why is this slow" / "why is there a timeout here" questions. -- **Logs.** High-volume event records. Often contain the error conditions that motivated defensive code. -- **Incidents.** Formal incident records with timelines and linked postmortems. -- **Notebooks.** Exploratory investigations. Often contain hypotheses and analyses. - -Datadog answers "what was the production reality around the time this code was written?", which often explains the code's shape. - -## How to search it - -Use the Datadog MCP. Start broad, then narrow. - -1. **Identify the owning service(s).** - - ``` - search_datadog_services (filter by name or team) - search_datadog_service_dependencies (see upstream/downstream) - ``` - -2. **Dashboards and monitors first. They tell you what the team cares about.** - - ``` - search_datadog_dashboards (query: feature name, service name, symbol) - search_datadog_monitors (same queries) - ``` - - When a dashboard or monitor covers the target, note its queries and watched thresholds. The threshold is frequently the answer to "why is this clamped at N?" - -3. **Metrics around the target.** - - ``` - search_datadog_metrics (by name pattern, e.g., the feature or symbol) - get_datadog_metric_context (metadata: description, units, tags) - get_datadog_metric (timeseries; "was there a spike around the PR date?") - ``` - - Correlating a metric's trajectory with the target's add/change date is strong supporting evidence: "the `payment_timeout` metric spiked 2023-11-03, and the retry logic merged 2023-11-06." - -4. **Logs. Narrow, don't dump.** - - ``` - search_datadog_logs (raw log patterns near the target, set use_log_patterns=true) - analyze_datadog_logs (SQL-style aggregations, only when you need counts) - ``` - - Search with symbols, error strings, or feature names. **Strongly prefer time-bounded queries** (e.g., 30 days before/after the change). Log volume is huge. Unconstrained searches waste time and may time out. - -5. **APM spans and traces.** - - ``` - aggregate_spans (stats: "how often does this endpoint fail?") - search_datadog_spans (inspect individual spans) - get_datadog_trace (a specific trace ID) - ``` - - Useful for timeouts, retries, slow paths, and cross-service behavior. - -6. **Incidents.** - - ``` - search_datadog_incidents (by title, team, date range) - get_datadog_incident (full detail for a specific incident) - ``` - - If the target looks defensive, search for incidents around the time it was added. An incident whose timeline includes "added defensive check for X" is near-direct evidence. - -## What good evidence looks like here - -- A monitor whose query and threshold match the constraint the code enforces (code clamps to 100, monitor alerts when requests exceed 100/min) -- A dashboard created by the target's author, with widgets that correspond to what the code measures or guards against -- A metric showing a production spike immediately before the code was merged, and stable values after -- An incident record referencing the target code, the same symbols, or the same error strings -- Logs showing a specific error pattern the defensive code would prevent, timestamped in the window before the change - -## Common pitfalls - -- **Correlation is not causation.** A spike before a PR and stabilization after is suggestive, not definitive. Other changes may have landed in the same window. Check neighboring PRs. -- **Overfitting to the chart you found.** Datadog visualizations are *made* by humans and reflect that human's framing. A chart named "retry success rate" is evidence the team cared about retry success, not that it's why a specific line of code exists. -- **Vanished telemetry.** Metrics can be renamed, deleted, or have short retention. If you can't find data from the relevant window, that's a gap, not a null result. -- **Noise at scale.** Searching logs for a common string returns thousands of matches. Narrow by service, tag, and time aggressively. Use `analyze_datadog_logs` to aggregate rather than dumping raw logs. -- **Instrumented != caused.** A metric's existence tells you someone cared enough to measure something, not that the code was added *because* of it. Cross-reference with commit/PR dates. - -## What to return - -For each relevant item: -- Type (dashboard / monitor / metric / log pattern / trace / incident / notebook) -- Title or name -- Link or identifier (dashboard ID, monitor ID, metric name, incident ID) -- Owner/author and created/modified date -- The specific condition, query, or quote that bears on the question (verbatim where possible) -- Relevance: what this suggests about the target code, and how strong the connection is diff --git a/skills/why/references/sources/incident-postmortem.md b/skills/why/references/sources/incident-postmortem.md deleted file mode 100644 index 450d6e5..0000000 --- a/skills/why/references/sources/incident-postmortem.md +++ /dev/null @@ -1,15 +0,0 @@ -# Incident & Postmortem Context - -Not a separate source, a **cross-cutting angle**. Incidents often motivate defensive code ("we added this check after the X outage"), so if the target looks defensive (null checks, retry logic, timeout handling, rate limiting, feature flags), specifically hunt for incident history across every available source: - -- **Notion**: search for postmortems mentioning the target file, feature, or error string -- **Linear**: look for tickets labeled `incident`, `sev-*`, `postmortem-action-item`, `reliability` -- **Slack**: search `#sev-*` and `#incident-*` channels around the dates the target code was added -- **Git**: commits with messages like "fix for incident", "add defensive check", "revert" followed by "re-apply with..." are strong signals -- **Datadog**: `search_datadog_incidents` for formal incident records with timelines, dashboards and monitors created as postmortem action items -- **Sentry**: issues whose first-seen/last-seen window aligns with the target's PR ship date, stack traces through the target -- **Databricks**: product-analytics events that classify an error condition (client-reported failures, user-visible retry events, etc.) often spike during an incident window. A drop in that event count after the target PR ships is circumstantial support that the target code resolved the user-visible symptom, even when Datadog/Sentry signal is noisy. - -If you find an incident link, fetch the full postmortem. Postmortems typically have an "Action Items" section that ties directly to code changes. When multiple sources corroborate (a Datadog incident ID appears in a Linear ticket, which appears in a Notion postmortem, which appears in a Slack thread that links to the target PR, and the Databricks error-event count drops after the fix), the evidence is especially strong. - -Worth spending time on when the code's defensive character makes an incident-driven origin plausible. Skip it for code that doesn't look defensive. diff --git a/skills/why/references/sources/linear.md b/skills/why/references/sources/linear.md deleted file mode 100644 index 899c643..0000000 --- a/skills/why/references/sources/linear.md +++ /dev/null @@ -1,48 +0,0 @@ -# Linear Tickets - -## What this source contains - -- Issues describing features, bugs, and their motivation -- Project docs attached to issues (often PRDs or specs) -- Parent/sub-issue relationships (broader initiative → specific tickets) -- Comments on issues (clarifications, scope changes, "why we're doing this" rationale) -- Labels (e.g., `compliance`, `customer-request`, `perf`) that signal the type of motivation -- Status updates that explain scope changes -- Attachments and linked GitHub PRs - -Linear is where the product/business context often lives: the "we're doing this because customer X asked" or "this is for the Q3 compliance initiative" layer. - -## How to search it - -Use the Linear MCP. - -1. **Start with linked tickets.** If the seed commits or PRs reference ticket IDs (e.g., `ENG-1234`, `[BUG-567]`), fetch those first with `get_issue`. Read the full issue including comments. -2. **List related issues by keyword.** Use `list_issues` with text search for the feature name, key symbol, or business term. Try multiple phrasings. -3. **Walk the issue tree.** If you land on a sub-issue, fetch its parent. Sub-issues are tactical. Parents often carry the "why." -4. **Read project docs.** If the issue belongs to a project, use `get_project` and check attached docs. Project-level documents are where specs and rationale are most often captured. -5. **Check labels and milestones.** Labels hint at the category of motivation (customer-request, incident-followup, compliance). Milestones tie work to deadlines, which often reveal motivation. - -## What good evidence looks like here - -- An issue description stating the business problem: "Customer Acme needs X because of their SOC2 audit" -- A comment recording a decision: "We decided to go with approach B because approach A would require touching the billing service" -- A parent issue titled like an initiative: "Q3 Enterprise Readiness" or "Reduce Payment Failures" -- An attached PRD or spec -- Labels like `customer:acme`, `incident-followup`, `compliance`, `perf-regression` - -## Common pitfalls - -- **Scope drift.** The ticket the PR references may have been closed and reopened with a different scope. Read the whole history. -- **Mechanical templates.** Some teams require "Why" sections but fill them with boilerplate. Generic text ("improve user experience") is probably not a real answer. -- **Stale tickets.** Old tickets often reflect a version of the plan that changed. Check dates and cross-reference with the code's ship date. -- **Closed-as-duplicate chains.** Follow the duplicate-of relationships back to the canonical ticket. -- **Private workspace content.** If you can't access an issue, note that as a gap rather than guessing. - -## What to return - -For each relevant ticket: -- Ticket ID and title -- The problem/motivation quoted from the description or comments (not paraphrased. The synthesizer needs the exact text to cite) -- Labels, parent issue, project -- Author, created date, closed date -- Link to the ticket if available diff --git a/skills/why/references/sources/notion.md b/skills/why/references/sources/notion.md deleted file mode 100644 index d16230a..0000000 --- a/skills/why/references/sources/notion.md +++ /dev/null @@ -1,55 +0,0 @@ -# Notion Docs - -## What this source contains - -- PRDs (product requirement documents) -- Technical specs and RFCs -- Architectural decision records (ADRs) -- Meeting notes from design reviews -- Team pages with domain context -- Postmortems from incidents -- Runbooks that may explain defensive code -- Strategy documents that set priorities - -Notion is where "why" often lives in long-form before it becomes code. A significant feature usually has a doc. - -## How to search it - -Use the Notion MCP. - -1. **Keyword searches with `notion-search`.** Try: - - The feature name - - Key symbols / class names from the target code - - Author handles (design docs are often authored before the code lands) - - Error strings or user-visible terms - - Time-bounded queries if you know when the code shipped -2. **Fetch candidate pages with `notion-fetch`.** Read the full content, not the preview. Rationale is often buried mid-document. -3. **Follow backlinks and child pages.** Design docs often have sub-pages for alternatives considered, appendices, or implementation notes. -4. **Check related databases.** `notion-query-data-sources` and `notion-query-meeting-notes` can surface meeting notes that discussed the decision. -5. **Search author-specific spaces.** If the PR author has a personal notebook (common at some companies), it may hold exploratory thinking that preceded the code. - -## What good evidence looks like here - -- A PRD with a "Problem statement" or "Motivation" section that matches the target code's purpose -- An "Alternatives considered" or "Rejected approaches" section -- A postmortem that names the target code as the fix for a specific incident -- Meeting notes that record "we decided X because Y" and tie to the same author/date range as the PR -- An ADR template filled out non-trivially (status, context, decision, consequences) - -## Common pitfalls - -- **Outdated docs.** Specs are often written before implementation and not updated. The doc may describe a plan that changed. Cross-check against the actual PR. -- **Doc vs. reality drift.** A spec may say "we'll do X" but the code actually does Y. Flag the divergence. The synthesizer will surface the contradiction. -- **Boilerplate templates.** Some orgs require a "Why" section that gets filled with fluff. Look for specificity. -- **Unlinked docs.** The most relevant doc may not be linked from anywhere. Broad keyword searches help. -- **Multiple drafts.** If a topic has multiple docs, find the one that was finalized or most recently updated. Check dates. -- **Access-restricted pages.** If you can't access a page, note it as a gap. - -## What to return - -For each relevant doc: -- Title and URL -- Authors and last-updated date -- The motivation text (verbatim quote), with page/section location -- Relevant linked pages (so the synthesizer can cite them) -- Whether the doc was finalized or draft diff --git a/skills/why/references/sources/sentry.md b/skills/why/references/sources/sentry.md deleted file mode 100644 index 2b7cf6f..0000000 --- a/skills/why/references/sources/sentry.md +++ /dev/null @@ -1,100 +0,0 @@ -# Sentry Error History - -## What this source contains - -Sentry is the archive of things that went wrong. For defensive, corrective, or error-handling code, it often holds the direct motivation: the specific exceptions, stack traces, and frequencies that pushed someone to add a check, catch, retry, or fallback. - -- **Issues.** Grouped errors with counts, first/last seen timestamps, affected releases, and comments -- **Events.** Individual error instances within an issue (stack traces, tags, user context) -- **Releases.** Deployment records with associated issues (useful for "which version fixed this?") -- **Replays.** Session recordings of user-facing errors (if enabled) -- **Profiles.** Performance profiling data (less useful for "why", more for "how slow") -- **Issue comments & assignments.** Sometimes contain engineer notes on root cause - -The most valuable thing Sentry provides is **temporal correlation**: "issue X was created 2024-01-02, peaked at 500 events/day, stopped appearing after release v2.14.0 on 2024-01-15, the release that shipped the defensive check." - -## How to search it - -Use the Sentry MCP. - -1. **Orient.** If you don't know the project slug and organization: - - ``` - find_organizations - find_projects - ``` - -2. **Search for issues related to the target.** - - ``` - search_issues (natural language, e.g., "errors in PaymentService timeout", "unhandled exceptions in uploadFile") - ``` - - Good query components: exception class names the target handles, the function or class name of the target, error message strings the target checks for, the file path of the target. - -3. **Narrow by release and time window.** - - ``` - search_issue_events (filter by release, time, environment, trace ID, tags) - get_issue_tag_values (for an issue, see distribution across versions, users, environments) - ``` - - For a suspected issue, check: - - **First seen.** When did the error start appearing? - - **Last seen.** When did it stop? Does it line up with the target's ship date? - - **Affected releases.** Which versions saw it? Which was the fix? - - **Frequency trajectory.** Did it spike, then get resolved? - -4. **Pull the full event for context.** - - ``` - get_sentry_resource (pass a Sentry URL or type+ID) - ``` - - Does the stack trace pass through the target code? Do the tags and breadcrumbs match the conditions the target defends against? - -5. **Check releases that landed near the target.** - - ``` - find_releases (around the commit date of the target) - ``` - - Cross-reference release version with the PR's merge date. - -6. **Use Seer sparingly.** - - ``` - analyze_issue_with_seer - ``` - - Seer produces AI root-cause analyses. Useful as a hypothesis generator, but treat them as inference, not authoritative. The actual events and stack traces are the primary evidence. Seer's narrative is secondary. - -## What good evidence looks like here - -- An issue whose **first seen** is shortly before the target's PR and **last seen** shortly after, suggesting the target addressed this error -- Stack traces that pass through or land on the target function, showing the exact failure mode being defended against -- A comment on the issue from the PR author describing the fix -- The target's PR description or commit message referencing a Sentry issue URL or ID -- An issue with high event counts that stops after the release containing the target - -## Common pitfalls - -- **Grouping drift.** Sentry groups errors by fingerprint. Refactors or renames can track the "same" error under a new issue ID. If an issue ends abruptly, the error may have just been regrouped. Check for new issues immediately after. -- **Release correlation is noisy.** A release contains many commits. An issue stopping at v2.14.0 doesn't prove the target fixed it. Another change in the same release might have. Cross-reference with the target's exact commit. -- **Silent fixes.** Sometimes the error stops because upstream changed, not because of the defensive code. The correlation suggests the fix. It doesn't prove authorship. -- **Resolved != fixed.** Issues can be marked "resolved" manually without any code change. Treat `resolved` as a human marker, not evidence that code fixed it. -- **Seer hallucinations.** Seer can generate confident-sounding explanations that aren't right. Fall back to the actual events, stack traces, and timestamps when making claims. -- **Sampling.** Some projects sample events aggressively. A low event count may just mean high sampling, not a rare error. If in doubt, note the gap. - -## What to return - -For each relevant issue: -- Issue ID and title -- Project and organization -- First seen / last seen timestamps -- Event count (and sampling rate if known) -- Affected releases -- A representative stack trace snippet showing relevance to the target (verbatim excerpt, not summary) -- First/last-seen correlation with the target's ship date -- Link to the issue -- Any author comments or resolution notes diff --git a/skills/why/references/sources/slack.md b/skills/why/references/sources/slack.md deleted file mode 100644 index 863a527..0000000 --- a/skills/why/references/sources/slack.md +++ /dev/null @@ -1,54 +0,0 @@ -# Slack Conversations - -## What this source contains - -- Real-time discussions of problems and decisions -- Incident channels where fire-drill decisions were made -- Design discussion threads where tradeoffs were debated -- Questions answered by senior engineers that didn't make it into docs -- Post-merge discussions that explain why something was revisited -- DMs (usually not searchable, scope accordingly) - -Slack is frequently where the *real* decisions got made, especially for smaller changes that didn't warrant a doc. It's also the most ephemeral source. Threads get deleted, channels get archived, and search quality degrades over time. - -## How to search it - -Slack MCP tools vary. Check which Slack MCP is available and inspect its tool schema first. It may require `mcp_auth`. If authentication fails, stop and report the gap. - -1. **Author-bounded search.** Messages from the PR author around the PR merge date. Limits scope dramatically and often hits gold. -2. **Keyword search for the feature name and key symbols.** Include misspellings and casual phrasings. -3. **PR URL search.** Slack often links PRs when they're reviewed or discussed. Search for the PR URL (or just `/pull/`). -4. **Error string search.** If the code handles a specific error, search for the error string. Incident threads often surface. -5. **Channel-scoped search.** Narrow to likely channels: - - `#eng-*`. Engineering discussions - - `#proj-*`. Project channels - - `#incident-*` / `#sev-*`. Incident channels - - Team-specific channels for the owning team - - Design review channels -6. **Thread traversal.** When you find a relevant message, fetch the whole thread. The decision often lives in the replies. - -## What good evidence looks like here - -- A thread where tradeoffs were explicitly debated ("I was going to use A but B is better because...") -- An incident channel message describing the bug the code prevents -- A question from a reviewer and an authoritative answer from the author or lead -- A reference to a meeting where a decision was made -- A message from a product manager or customer-facing engineer explaining a customer ask - -## Common pitfalls - -- **Channel archaeology limits.** Very old messages may be gone due to retention policies. If you can't find anything before a certain date, note the retention cliff. -- **Unsearched DMs.** Many decisions happen in DMs that aren't searchable. You'll miss them. That's a known limitation. -- **Speculative jokes as "decisions."** Slack is casual. "Lol just do the thing" isn't a decision, even if it preceded the commit. Look for considered discussion. -- **Context collapse in single messages.** Without the thread, a single message often reads differently than in context. Always fetch threads. -- **Auth failures.** If the MCP isn't authenticated, stop. Don't make up findings. Report that Slack wasn't searchable. - -## What to return - -For each relevant thread: -- Channel name -- Permalink or thread ID -- Participants -- Date range of the discussion -- The key quotes (verbatim) with attribution -- Context: what thread/incident/discussion this was part of diff --git a/skills/why/references/synthesizer-prompt.md b/skills/why/references/synthesizer-prompt.md deleted file mode 100644 index 0e218c6..0000000 --- a/skills/why/references/synthesizer-prompt.md +++ /dev/null @@ -1,135 +0,0 @@ -# Synthesizer Prompt Template - -Build the synthesizer's prompt from this template. Fill in the placeholders. - ---- - -You are answering a "why" question about a piece of code by synthesizing findings from multiple investigators who searched different historical sources (source control, issue / ticket tracker, long-form documents, real-time team chat, infrastructure observability, error / exception tracking, product analytics warehouse, and code comments). Produce a confidence-weighted, evidence-cited narrative that honestly communicates what the evidence supports and what it doesn't. - -## The Question - -> {QUESTION} - -## The Code Anchor - -**Target files:** {FILES_WITH_LINE_RANGES} - -**Key symbols:** {SYMBOLS} - -## Investigator Findings - -{ALL_INVESTIGATOR_FINDINGS} - -## Sources That Weren't Searched - -{SKIPPED_SOURCES_WITH_REASONS} - -## Epistemics Framework - -You MUST follow the framework in `references/epistemics.md`. Read it in full before writing the output. The key rules: - -1. Every claim sits in one of these tiers: **Direct**, **Supported**, **Inferred**, **Speculative**, **Unknown**. The tier determines what section the claim goes in and how it's phrased. -2. Every Direct/Supported claim must have a citation (PR #, ticket ID, doc URL, chat permalink, commit hash, or file:line). -3. Inferred and Speculative claims must use hedged language ("appears to", "likely", "suggests", "one possibility is"). -4. Never cite code as evidence for its own intent. -5. Gaps in the evidence must be documented. Don't fill them with plausible-sounding guesses. -6. If the user's question embedded a hypothesis, treat it as a candidate, not a conclusion. Check the evidence independently. - -## Instructions - -1. **Read all investigator findings.** They gathered raw evidence, not conclusions. You weigh it. -2. **Reconcile overlapping findings.** Multiple investigators may have cited the same PR, ticket, or doc. Merge into a single, authoritative reference. -3. **Identify contradictions.** If two items of evidence disagree, don't pick one. Surface both. -4. **Calibrate confidence.** For each claim, identify the evidence and the tier. State Direct claims plainly with a citation. Hedge Inferred claims and explain the inference. Mark Speculative claims explicitly. Put claims with no evidence in the gaps section. -5. **Verify citations by spot-checking.** You can read the codebase and call MCP tools to verify citations. Do not write files, commit, or modify external state. If you're uncertain a cited item exists or says what's claimed, check it. Don't propagate errors. -6. **Don't overreach.** The user will act on your output. Better to leave an open question open than to fill it with a confident-sounding guess. - -## Output Format - -Write the output for the user. Use this exact structure: - ---- - -### The Question - -Restate the user's question in one or two sentences so the answer is anchored. - -### The Code in Question - -File paths, line ranges, key symbols. Two or three lines to orient a reader who lands here cold. - -### What We Found - -**Claims with direct evidence**, one per bullet. Quote or paraphrase the source and cite precisely. Format each finding like: - -- **[Direct]** {Claim}. Source: PR #123 (link) / ticket ID / file:line. {Brief quote or paraphrase.} -- **[Supported]** {Claim}. Evidence: {list of items and what each contributes}. - -Use `[Direct]` for single-source, explicit evidence. Use `[Supported]` when multiple indirect items converge on a conclusion. - -### What We Can Reasonably Infer - -**Claims that aren't explicitly stated anywhere but are well-supported by indirect evidence.** Make the inference chain visible: "Given A and B, it's likely that C." Use hedged language ("appears to", "likely", "suggests", "is consistent with"). Format: - -- **[Inferred]** {Hedged claim}. Reasoning: {the specific evidence and the inference step}. - -If there's nothing to infer, skip this section. - -### Competing Hypotheses - -**If the evidence fits multiple stories, present them.** Don't force a winner when the record doesn't support one. For each hypothesis: - -- **Hypothesis:** {one-sentence statement} -- **Evidence for:** {specific items} -- **Evidence against or missing:** {what would need to be true but isn't, or what counter-signals exist} - -Skip this section if there's a single clear answer. - -### What We Don't Know - -**Explicit gaps.** Things the user asked that the evidence didn't answer. Sources searched that came up empty. Sources that weren't searchable at all, such as a missing real-time team chat MCP. - -Be specific. "We searched the issue tracker for [query1], [query2], [query3] and found no issue discussing the rate-limit threshold" is useful. "We don't know why" is not. Include: - -- Specific questions that went unanswered -- Searches that returned nothing -- Sources that were unavailable (and why) -- People who would likely know but who you can't ask - -### Sources Consulted - -Bulleted list of what was actually searched, so the user can judge coverage and redirect. Format: - -- **Source control history**: {file paths}, {number of commits reviewed}, PRs #{numbers}, and code comments searched. Or "Not searched. This should not happen because git and `gh` are always expected." -- **Issue / ticket tracker**: {ticket IDs and keyword searches}. Or "Not searched. No matching MCP available in this environment." -- **Long-form documents**: {page titles and search queries}. Or "Not searched. No matching MCP available in this environment." -- **Real-time team chat**: {channels searched, date ranges, queries}. Or "Not searched. No matching MCP available in this environment." -- **Infrastructure observability**: {dashboards, monitors, metrics, logs, traces, or incidents searched}. Or "Not searched. No matching MCP available in this environment." -- **Error / exception tracking**: {issues, events, or releases searched}. Or "Not searched. No matching MCP available in this environment." -- **Product analytics warehouse**: {fully-qualified tables queried, the time windows, and the numeric summaries (counts, percentiles, first/last-seen timestamps) that bore on the question}. Or "Not searched. No matching MCP available in this environment." - -### Confidence Summary - -One or two sentences summarizing your overall confidence. E.g.: - -> "The core rationale (A) is well-supported by direct PR and ticket evidence. The specific threshold value (100) is inferred from the surrounding context but not explicitly documented. The question of whether this was driven by a customer request could not be answered. No relevant issue tracker or long-form doc content surfaced, and real-time team chat search was unavailable." - ---- - -## Quality Check Before Returning - -Before finalizing, review your output against this checklist: - -1. Does every claim in "What We Found" have a citation? If not, add one or move the claim to "Inferred" or "Hypotheses." -2. Is the phrasing tier-appropriate? (Direct claims can use "because". Inferred claims cannot.) -3. Did you surface any contradictions you noticed, or did you quietly pick one? -4. Does the "What We Don't Know" section exist and name specific gaps? If it's empty or missing, be suspicious. Historical investigations almost always have gaps. -5. If the user embedded a hypothesis in their question, did you check it against the evidence rather than rubber-stamping it? -6. Did you cite any code as evidence for its own intent? Remove those. Code is mechanics, not motivation. -7. Is the overall tone calibrated? A confident-sounding answer with weak evidence is the exact failure mode this skill exists to prevent. - -If any item fails, revise before returning. - -## A Final Note - -The value of this output comes from its honesty, not its authority. A reader who takes your answer to the original author, an engineering lead, or a product manager should be well-positioned to ask the right follow-up questions. Be clear about what's known, what's inferred, and what's missing. Don't optimize for looking decisive. Optimize for being useful.