Skip to content

feat: Group prompts under roles - #13

Merged
thykel merged 9 commits into
mainfrom
llm-roles
Sep 30, 2026
Merged

thykel merged 9 commits into
mainfrom
llm-roles

Conversation

@thykel

@thykel thykel commented Sep 30, 2026

Copy link
Copy Markdown
Collaborator

What this changes

Every LLM call now names a role: planner, advisor, implementer, pr_author, and 7 more. A role is two files side by side in lib/opilot/prompts/:

  • <role>.yml holds the tool grant (read/write), mcp, model (heavy/light), memory (session/none) and the role's charter.
  • <role>.rb is the role's prompt module (Prompts::Planner.plan, Prompts::PrAuthor.fix_ci, …).

Text and helpers that several roles use are in prompts/_shared.rb and prompts/_blocks/*.md.

Why

Before this PR, each call site chose its tool grant, its model and its session by hand. Each prompt also stated its grant by hand: READ_ONLY appeared in 10 prompts, and the write rules in 3. Nothing checked that the prompt text, the grant and the session agreed.

How it works

  • Helpers#llm(:role, prompt, …) is the only way to call the model. The role decides the tools and the model.

  • A memory: none role refuses a session file. The health check's independence from chat is now enforced by the code, not by a comment.

  • A builder returns a Prompts::Prompt that is tagged with its role. llm refuses a prompt sent under another role.

  • Every prompt that orients the model opens with Prompts.charter: the role's charter, then the rules of its grant. A read role gets READ_ONLY, and a write role gets WRITE_GRANT. These rules are derived from the grant, so a prompt cannot state a grant that its role does not have.

  • The runner sends X-Harness-Role. server.js loads the same YAML files and refuses these requests:

    • a request with no role, or with an unknown role;
    • a request with no tool grant, because pi's default tools include write;
    • a grant that the role does not allow.

    This check catches a bug in the runner. It does not stop a compromised runner, because the runner chooses both the role and the grant.

Prompt text changes

Only commit e1d7ff6 changes prompt text:

  • The "You are the WRITER" and "You are opilot, an AI code assistant…" openings are replaced by the role's charter and one context sentence.
  • The grant rules move to the top of each prompt. The no-commit rule for write roles had three wordings. It now has one.
  • commit_subject gains READ_ONLY. propose and propose_feedback gain the write rules, including the git rm/git clean note.

All other commits are moves. A snapshot of all 19 builders over 35 argument combinations renders byte-identical across each of them.

Before merge

  • Run a real ./opilot dev plan <id> on a known work package, and compare the plan with one made before this change. No live model run has checked the new prompt text.
  • The harness image must be rebuilt, because it now copies lib/opilot/prompts/*.yml. ./opilot rebuilds on every start.

Tests

  • roles_test.rb pins every role's tuple and checks every grant against ALLOWED_TOOL_GRANTS. It also fails when a call site bypasses llm.
  • prompts_test.rb renders every builder. It checks the charter, checks that the builder's own grant block appears exactly once and the other block never appears, and checks the file pairs.
  • test/js/roles_test.js checks the server's parser and its role check against the real role files.

🤖 Generated with Claude Code

thykel and others added 9 commits September 30, 2026 18:00
Every harness call now goes through Helpers#llm with a named role
(lib/opilot/roles.rb): base grant, whether the MCP tools join it,
model, and whether it is stateless. No behaviour change: each role
reproduces the exact (grant, model, session) tuple its call sites sent.

A stateless role (auditor, triager, scribe) raises when given
a session file, so health's independence from chat is now structural.
The model plumbing through FixRunner, #implement_plan and
#generate_pr_description is gone; the role carries the model.

Divergences kept as-is, now visible in one table:
- pr_author (gh-agent own PR) gets the MCP tools; pr_refresher
  (dev refresh) does not, for a similar job.
- wp_writer (create wp) gets no MCP tools, unlike advisor and planner.
- pr_advisor (upstream PRs) gets no MCP tools.

Guard tests: every role x flag combination resolves to a grant in
server.js's ALLOWED_TOOL_GRANTS, and no lib/ file outside helpers,
harness and roles calls @harness.run/capture or names Harness::TOOLS_*.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each role is now one markdown file: frontmatter for tools, mcp, model
and memory, a body saying what the role does and who calls it. The
table in lib/opilot/roles.rb is gone; the loader reads the files at
boot and rejects an unknown key or value. The body is not sent to the
model yet — it becomes the role's charter in a later step.

No behaviour change: roles_test pins every role to the tuple the old
table held. It also checks that every llm(:name) call site names a
role that exists, and that a malformed role file fails to load.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each role now has lib/opilot/prompts/<role>.rb holding the builders it
sends (Prompts::Planner.plan, Prompts::PrAuthor.fix_ci, ...). What more
than one role uses stays in prompts.rb: the constants at top level of
Prompts, the helper methods in Prompts::Sections, which every role
module and Prompts itself extend.

A builder now returns a Prompts::Prompt, a String tagged with its role.
Helpers#llm raises when a tagged prompt is sent under another role. A
bare follow-up message is a plain String and is not checked.

No prompt text changed: a snapshot of all 19 builders over 35 argument
combinations renders byte-identical before and after.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each role's body in roles/<name>.md is now its charter, and every
builder that orients the model opens with Prompts.charter: the charter,
then the rules the role's GRANT carries — READ_ONLY for a read role,
WRITE_GRANT for a write role. The grant rules were pasted by hand into
16 places; they are now derived, so a prompt cannot state a grant its
role does not hold. Follow-up turns (propose_revise, a chat's later
messages) carry no charter.

Shared prompt text moved to roles/_blocks/*.md. The reason for each
block stays on the Ruby constant that loads it; compositions such as
REPLY_CONTRACT + PLAIN_ENGLISH stay in Ruby.

Prompt text changes (from a snapshot of all builders):
- "You are the WRITER" / "You are opilot, an AI code assistant ..."
  openings are replaced by the charter plus a context sentence.
- The grant rules move to the top of each prompt.
- The no-commit rule for write roles is one text now (WRITE_GRANT);
  it was worded three ways. WRITE_RULES is PR_WRITE_RULES without it.
- scribe prompts (commit subject, PR description): commit_subject gains
  READ_ONLY, which it did not have.
- spec_writer (propose, propose_feedback) gains the write-grant rules,
  including the git rm / git clean note.

No live LLM run compared the output yet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The runner now sends X-Harness-Role with every call. server.js loads
the same roles/*.md files the runner loads (copied into the harness
image) and refuses:
- a missing role (400) or an unknown one (403);
- a request with no tool grant (403) — pi's default tools include write;
- a grant the role does not allow (403): a role without mcp accepts
  only its base grant, one with mcp also the op_query/gh_query variants.

The parser is strict, like the Ruby one, so a bad role file stops the
container at boot. The runner still chooses both role and grant: this
catches a runner bug, not a compromised runner.

Harness#run and #capture take a required role:. Test fakes accept it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
prompts.rb now only loads lib/opilot/prompts/:
- blocks.rb   — the text several roles share (from roles/_blocks/)
- prompt.rb   — Prompt, the role-tagged String, and Prompts.charter
- sections.rb — the helper methods several roles share

Names only one role uses moved into that role's module:
Planner::OPTIONS_CONTRACT/OPTIONS_SENTINEL and repos_section,
Auditor::HEALTH_*, Advisor::LENSES/TERMINAL_REPLY with lens and
artifact_block, PrAdvisor::SUGGESTION_CONTRACT. Callers updated.

No prompt text changed: the builder snapshot renders byte-identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A role is now two files side by side in lib/opilot/prompts/:
- <role>.yml — tools, mcp, model, memory and the charter, as plain
  YAML keys (was roles/<role>.md with frontmatter, a body and notes)
- <role>.rb  — the role's module, as before

The shared Ruby parts (blocks, Prompt and Prompts.charter, Sections)
merge into prompts/_shared.rb; the text blocks move to prompts/_blocks/.
roles/ is gone. The harness image copies prompts/*.yml to the same path,
and server.js parses the small YAML subset these files use.

No prompt text changed: the builder snapshot renders byte-identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The ROLE constant in each prompt module repeated what the layout already
fixes: Prompts::PrAdvisor sits in pr_advisor.rb beside pr_advisor.yml.
Sections#role derives :pr_advisor from the module name, and tagged and
charter use it. A module whose name maps to no role fails at its first
charter call, which the builder tests reach for every module.

No prompt text changed: the builder snapshot renders byte-identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@thykel thykel changed the title Name the role of every LLM call, and derive prompt rules from it feat: Group prompts under roles Sep 30, 2026
@thykel
thykel marked this pull request as ready for review September 30, 2026 16:56
@thykel
thykel merged commit 823b59f into main Sep 30, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant