brainstorming: orient before asking design questions

[allenwang28]

Who is submitting this PR? (required)

Field	Value
Your model + version	GPT-5 (Codex API; serving snapshot not exposed)
Harness + version	Codex CLI 0.144.1 on Linux
All plugins installed	superpowers-dev 5.1.0
Human partner who reviewed this diff	Allen Wang (@allenwang28)

What problem are you trying to solve?

During a real design session for low-overhead CUDA instrumentation, the
brainstorming workflow repeatedly asked the human partner to choose technical
experiment parameters before explaining the design space. It proposed a new
duration sweep, asked the partner to define absolute and relative overhead
bounds, and then suggested unsupported 250 ns / 5% screening thresholds. The
partner explicitly said they lacked the context to make those choices.

The agent later identified the cause: it followed the skill's literal ordering
of "ask clarifying questions" before "propose approaches" and conflated
obtaining approval with making the partner supply the experiment design.

Current dev is already better than the installed 5.1 skill, but a fresh
baseline still reproduced the ordering failure. Given a request to design a
lightweight job-progress monitor, the agent classified the task and immediately
asked the partner to choose a hang-signal model. It did not first explain the
inferred success criteria, monitor-liveness problem, threshold uncertainty, or
recommended direction.

What does this PR change?

The brainstorming skill now requires a concise orientation before design
questions, classifies unknowns by who should resolve them, and treats one
question per message as a ceiling rather than a quota. It also requires
recommendations grounded in expressed preferences and forbids unsupported
numerical precision in success criteria.

Is this change appropriate for the core library?

Yes. The failure is general to collaborative design, not CUDA or any particular
project. The before/after evaluations cover research design, a product feature,
and a security-critical internal system. The change adds no dependency,
third-party integration, or project-specific policy.

What alternatives did you consider?

• Rely on users to request context-first collaboration explicitly. This
  produced a good baseline response, but requires the least-informed participant
  to know and prescribe the correct design protocol.
• Move all approach presentation before all questions. Rejected because a
  genuinely product-defining unknown can make approaches premature. The change
  instead requires orientation first and permits only material questions that
  cannot be discovered or responsibly inferred.
• Remove the one-question rule. Rejected. The rule remains useful as a cap on
  cognitive load; the patch clarifies that it is not a requirement to ask.
• Add prompt-specific CUDA guidance. Rejected as non-general and unsuitable
  for core.

Does this PR contain multiple unrelated changes?

No. It changes one skill for one behavior: orienting the human partner before
asking them to make design decisions.

Existing PRs

• I have reviewed all open AND closed PRs for duplicates or prior art
• Related PRs: Enforce brainstorming workflow with hard gates and process flow #462, feat(brainstorming): add context reset option after design doc #607

#462 established the brainstorming hard gate and process flow. This PR preserves
that gate and the three-path workflow while addressing a newly observed failure
inside the question phase. #607 proposed a post-design context reset and is not
a duplicate. Searches for brainstorming context, clarifying questions brainstorming, and success criteria brainstorming found no open duplicate.

Environment tested

Harness (e.g. Claude Code, Cursor)	Harness version	Model	Model version/ID
Codex CLI	0.144.1	GPT-5	serving snapshot not exposed

New harness support (required if this PR adds a new harness)

Not applicable. This PR changes a skill only and adds no harness support.

Evaluation

• Initial human request: "Can you update the design skill so that agents can
  articulate [success criteria and challenges] up front and then approach them
  collaboratively while giving me context for the decisions?"
• RED: three fresh sessions against unchanged current dev. Two already gave
  good context-first designs; one followed the literal classify-then-question
  sequence and asked for a technical choice without orientation. The agent's
  meta-evaluation explicitly cited the checklist ordering as the cause.
• GREEN: the same three scenarios after the change. All three oriented first,
  articulated success criteria and hard parts, recommended a design based on
  stated preferences, and asked no context-free technical question.
• REFACTOR pressure: one additional session combined a next-day deadline,
  executive demand for precise thresholds, a week of sunk cost, and unrealistic
  load-test evidence. The response refused false empirical precision and chose
  only limits grounded in controller invariants.
• Eval sessions after the change: 4.

Representative before/after for the job-progress monitor:

Before	After
Classified the task, then immediately asked whether stage changes stop during a hang.	Explained goals/non-goals, hang-vs-long-stage ambiguity, monitor-liveness risk, and the out-of-process staleness-detector recommendation before seeking approval.
No threshold rationale.	Proposed shadow-mode measurement and stage-specific bounds derived from observed successful durations and the response-window policy.

Static verification:

python .../quick_validate.py skills/brainstorming
Skill is valid!

git diff --check
exit 0

The optional Graphviz renderer could not run because dot is not installed in
the authoring environment. The DOT block was structurally inspected; the change
adds only nodes and edges corresponding to the checklist ordering.

Rigor

• If this is a skills change: I used superpowers:writing-skills and
  completed adversarial pressure testing (results above)
• This change was tested adversarially, not just on the happy path
• I did not modify carefully-tuned content (Red Flags table,
  rationalizations, "human partner" language) without extensive evals
  showing the change is an improvement

The three added Red Flags entries came directly from the observed session and
RED-agent rationalizations. Existing entries and project terminology are
unchanged.

Human review

• A human has reviewed the COMPLETE proposed diff before submission

The complete one-file diff was shown inline and approved before this branch was
committed or published.

[obra]

Closing the PR, keeping the problem: the punt-to-human behavior you describe is now tracked in #2131, and if it recurs for you on current main, transcripts there would genuinely help — your baseline only reproduced it 1-in-3, so frequency is the open question.

Why the PR itself doesn't land: this is a substantial restructure of the most carefully-tuned skill in the repo (checklist reorder, new doctrine, three Red Flags rows, DOT changes), and the evidence behind it is four self-reported sessions with no transcripts attached — under the 'extensive eval evidence' bar CLAUDE.md sets for this content. There's also a naming inconsistency in the diff ('Explore existing flow' in the DOT graph vs 'Explore project context' in the checklist), and it overlaps #2116, which is ahead of it in the queue and would force a re-baseline anyway.

The submission itself was well done — full disclosure, honest eval reporting, real prior-art search. Thanks.

— Claude Fable 5, Claude Code 2.1.228, triaging on behalf of @obra