Skip to content

spike(unic-archon-dlc): can an Archon node reach a plugin agent, command or skill? #628

Closed

Description

@orioltf

Problem

The maintainer wants /pr-review to find as much as pr-review-toolkit does, without vendoring it. There are two ways a Box could use the toolkit: spawn its agents by subagent_type, or invoke /pr-review-toolkit:review-pr from a node. #547's resolution, § Broadening /pr-review, not decided here compares the two.

Neither way works unless an Archon node can reach a plugin that is enabled in the repository, and nobody has measured that. On Archon v0.10.1 three facts are measured. A project skill reaches a node only through skills:. An MCP server reaches it only through mcp:. A node reads CLAUDE.md but not AGENTS.md. See docs/agents/agent-tool-traps.md § Archon, #521 and #425. Two runs before v0.9.0 called the toolkit's agents and recorded no result.

This spike measures reach only. It changes no Box and does not touch the reviewer, so it does not disturb #501's baseline or #502's seal.

Acceptance criteria

  • The probe runs in a scratch repository outside every clone, with pr-review-toolkit@claude-plugins-official enabled in that repository's .claude/settings.json, on the Archon version the Consumer runs. The result names that version and the Claude Code version the nodes ran.
  • The result states, for each of the following, whether it reached the node and how that was shown: a toolkit agent spawned by subagent_type from a node, /pr-review-toolkit:review-pr invoked from a node, and a plugin skill invoked from a node.
  • Each result that is "reached" carries a marker the reached component could only have produced. A result that is "not reached" shows that the node attempted the call. A probe that cannot fail scores as not run.
  • The result states whether any node field, such as skills: or a settings source, changes reach, and names each field that was tried.
  • The result is posted on this issue with the workflow YAML and the commands that produced it. Each finding that is a tool trap goes to docs/agents/agent-tool-traps.md § Archon.

Notes

  • /prototype is installed at the maintainer's user scope, not in this repository. A spike that uses it depends on one machine, and no Archon node or teammate has it by default. The probe needs no branch here.
  • The broadening itself is a separate issue, blocked by this one.

Activity

  1. added
    needs-triageMaintainer needs to evaluate
    spikeTime-boxed investigation; no production code committed
    p2Medium priority
    on Sep 29, 2026
  2. orioltf commented on Sep 29, 2026

    @orioltf
    MemberAuthor

    Trap for the command form of the probe

    /pr-review-toolkit:review-pr takes its scope from git diff --name-only and looks for the pull request with gh pr view (its commands/review-pr.md, step 3, in plugin cache fbe07fb6ce7d). Two things follow for this spike:

    • git diff --name-only lists uncommitted changes in the working tree, not a branch against its merge-base. A node in a scratch repository with a clean tree gives the command no files. That empty review looks like a command that never ran. Give the probe an uncommitted change, or pass the scope explicitly, and record which one it used. Otherwise a "not reached" result for the command form is not evidence.
    • gh pr view works on GitHub only. On an Azure DevOps repository the command finds no pull request. This does not change reach, but feat(unic-archon-dlc): /pr-review finds as much as pr-review-toolkit, without vendoring it #629 has to account for it.

    Reported by the e2e-suite seat's wayfinder and checked by unic-sdlc-orchestrator-4 against the command file, 2026-09-29. The #547 resolution records the same two facts.

  3. orioltf commented on Sep 29, 2026

    @orioltf
    MemberAuthor

    Opener for #628, written 2026-09-29 by unic-sdlc-orchestrator-4 at dispatch.

    Amended 2026-09-29, same day. The maintainer disabled pr-review-toolkit at user scope. ~/.claude/settings.json now sets it to false, and the scratch repository enables it at project scope. So the real-toolkit arm measures project scope only and answers #629 directly, and the marker plugin is a cross-check. The worker also found a lead: settingSources: [project] on a node may remove user scope. Its control uses code-review@claude-plugins-official, which is still enabled at user scope only. Label any run made before the change as "enabled at user and project scope". The maintainer re-enables the toolkit after the result is posted.

    What it produces

    A measured answer, posted on this issue, to one question. Can an Archon node reach three things from a plugin: an agent through subagent_type, a command such as /pr-review-toolkit:review-pr, and a skill? And does it matter whether the plugin is enabled at user scope or at project scope?

    You are already in /wayfinder 498 628. That resolves one ticket: post the answer, close #628, and add it to #498's Decisions so far. Do not start #629. It is blocked by this ticket, #501 and #502, and it is grilled later.

    Read order

    1. This issue's body, then the trap comment. The comment amends the command-form criterion.
    2. #547's resolution, section "Broadening /pr-review, not decided here". It says why the two forms matter. Read the table, not the history.
    3. docs/agents/agent-tool-traps.md § Archon, the entries from "A node gets CLAUDE.md and nothing else by default" to "A probe proves the items it called and nothing beside them". The probe builds on them, and they already name the setup traps.

    The one trap: this machine enables the toolkit at user scope

    ~/.claude/settings.json on this machine sets pr-review-toolkit@claude-plugins-official: true, and so do unic-archon-dlc, code-review and others. The measured trap says a subagent sees every skill the host has, plugin skills included. So a probe that finds the toolkit reachable may be measuring the maintainer's machine, not what a Consumer teammate gets from the repository's .claude/settings.json. That positive would be real and useless.

    Report every result with the scope that enabled it. A result that holds only while the plugin is enabled at user scope says nothing about a Consumer.

    The decision this ticket does not make

    How to separate user scope from project scope. Two competent implementers would differ here:

    • A marker plugin. A throwaway local plugin, enabled only in the scratch repository's .claude/settings.json and never at user scope. It holds one agent, one command and one skill, and each writes a marker nothing else could produce. This needs no change to any of the maintainer's files, and it is what I recommend. Whether a project-scope local marketplace installs in a headless node is part of what it measures.
    • Disabling the toolkit at user scope for the probe. This edits the maintainer's own settings file. Propose it to the maintainer through me. Never do it yourself.

    Run the real toolkit as well, and label its result "enabled at user and project scope".

    Other traps

    • The scratch repository lives outside every clone and has no remote. Pass --no-worktree, or add a dummy remote unique to this run.
    • git diff --name-only lists only uncommitted changes. Give the command form an uncommitted change, or pass the scope, and say which. An empty review looks like a command that never ran.
    • Never ask a node what it can see. Make it call the thing and write the tool result to a file.
    • A node that fails a Skill or an Agent call still finishes green. Judge by the marker, never by the run status.
    • Record the Archon version (0.10.1 here) and the Claude Code version the node ran. The host has 2.1.284, and the node may run another one.
    • pr-review-toolkit agents are expensive. Point them at a diff of a few lines.
    • The NDA hook refuses a client term in any git or gh command. The scratch repository needs no client name.

    The bar that proves it

    For each of the three forms, a positive result is a marker file the reached component wrote. A negative result is the node's own tool result, showing the attempted call and its error. A form with neither was not tested, so report it as not run, never as "not reached". The marker-plugin result is the one that answers #629. The real-toolkit result only confirms that the marker plugin behaves like a published one.

    On landing

    • Post the result on this issue, with the workflow YAML and every command. Then close it through /wayfinder.
    • Each result that is a tool trap goes to docs/agents/agent-tool-traps.md § Archon, in the entry style it already uses, dated, with the versions. Do it on a feature/unic-archon-dlc/628-plugin-reach-from-node branch, in a worktree outside the clone, created with git worktree add --no-track -b <branch> <path> origin/develop, then run pnpm install --frozen-lockfile there. Open the PR as a draft to develop with Closes #628 left out, because /wayfinder closes the ticket. Tell me the PR number. The maintainer merges.
    • Leave the scratch repository in place, and name its path in the result.

    Report-back clause

    BEFORE YOU FINISH, AND WHENEVER YOU LEARN SOMETHING THAT WOULD HAVE CHANGED THIS PROMPT,
    report to unic-sdlc-orchestrator-4 with four things and nothing else:
      1. what you expected
      2. what happened
      3. the command or file that proved it
      4. whether it changes only this ticket, or every session in this repo
    Send it when you learn it, not only at the end — a session that dies unreported
    takes its learnings with it.
    Talk only to unic-sdlc-orchestrator-4. Do not message another orchestrator or its
    sessions; tell unic-sdlc-orchestrator-4, who relays it.
    

    Review and handover

    This ticket ends in a posted result, not a merge. I read the result against the bar before #629 counts it. For the traps PR, run pnpm check and judge it by its exit code, then leave the PR in draft for me.

  4. self-assigned this
    on Sep 29, 2026
  5. orioltf commented on Sep 29, 2026

    @orioltf
    MemberAuthor

    Result: a node reaches a plugin agent, command and skill, with conditions

    Measured on 2026-09-29 with Archon CLI v0.10.1 (git commit 9820785c, darwin-arm64). Every node ran Claude Code 2.1.284: Archon logged "binaryPath":"/Users/oriol.torrent/.local/bin/claude","source":"env", and that symlink points at versions/2.1.284.

    The scratch repository is /Users/oriol.torrent/Sites/scratch-628-plugin-reach. It has one dummy remote, https://example.invalid/probe-628-3784ce0a.git. Every run used --no-worktree. The marker plugin reach-marker lives in a directory marketplace outside it, /Users/oriol.torrent/Sites/scratch-628-marker-marketplace. Its agent, its command and its skill each append a line with a random token to markers/<form>.txt, and no node prompt holds a token. The 362 run events in archon.db hold no Read, Grep or Glob call, and no event names the marketplace path or the plugin cache. So every marker below comes from the component itself. Every node finished green, including the ones that failed their call, so the scores below come from markers and tool results only.

    Answer for #629

    Once the plugin is installed on the machine at project scope, and enabled only there:

    Form Default node Node with settingSources: [project] Node with skills: [<plugin>:<name>]
    Agent tool, subagent_type: reach-marker:marker-agent Reached, marker AGENT-… agent-default Reached, marker AGENT-… agent-project-only not run
    Node prompt that is the command, /reach-marker:marker-command label=… Reached, marker COMMAND-… label=command-literal not run not run
    Skill tool, reach-marker:marker-command Not reached: Skill reach-marker:marker-command is not in this session's skills allowlist not run Reached, marker COMMAND-… label=command-declared
    Skill tool, reach-marker:marker-skill Not reached: Skill reach-marker:marker-skill is not in this session's skills allowlist Not reached: the same allowlist error Reached, marker SKILL-… skill-declared, and with settingSources: [project] too, marker SKILL-… skill-declared-project-only

    So:

    • An agent reaches a node by subagent_type with no declaration. That holds with user scope switched off.
    • A command reaches a node in two ways. The node's whole prompt can be the slash command, which Claude Code expands. Or the node declares the command in skills: and calls it with the Skill tool. Without the declaration, the Skill tool refuses it.
    • A plugin skill reaches a node only through skills:. This matches the measured rule for project skills.

    Before installation, nothing reaches a node

    Phase 1 ran with the repository's .claude/settings.json alone: extraKnownMarketplaces naming the directory marketplace, and enabledPlugins enabling reach-marker@reach-marker-628. Nothing had been installed on the machine. All five calls failed. The agent calls returned Agent type 'reach-marker:marker-agent' not found, the Skill calls returned Unknown skill: …, and the literal command prompt was not expanded. No marker file appeared. So a headless node does not install a marketplace or a plugin that the committed settings only declare. Somebody has to run claude plugin install <plugin>@<marketplace> --scope project on that machine first. It wrote no line to ~/.claude/settings.json. It added the marketplace to ~/.claude/plugins/known_marketplaces.json and the install to ~/.claude/plugins/installed_plugins.json, tagged "scope": "project" with this repository's path. Whether an interactive session installs from the same settings is not measured here.

    What settingSources: [project] changes

    On v0.10.1 the Claude provider takes settingSources from the node, then from assistants.claude.settingSources, then falls back to ["project","user"]. So by default a node loads user-scope plugins. Two controls show that settingSources: [project] on the node removes them:

    • In phase 1 the error of the default node listed the unic-pr-review:* agents, which only user settings enable. The error of the settingSources: [project] node did not list them, but still listed pr-review-toolkit:*, which the scratch repository enables.
    • code-review@claude-plugins-official is enabled only at user scope. A default node calling code-review:code-review got Skill code-review:code-review is not in this session's skills allowlist, so the skill was loaded and only refused. The same node with settingSources: [project] got Unknown skill: code-review:code-review, so the skill was not loaded.

    The two errors tell the cases apart. "Not in this session's skills allowlist" means the skill is loaded and the node did not declare it. "Unknown skill" means the skill is not loaded at all.

    Node fields tried: settingSources: [project] and skills:. Not tried: inline agents:, assistants.claude.settingSources in .archon/config.yaml, and settingSources: [user].

    The real toolkit, enabled at project scope only

    The maintainer set pr-review-toolkit@claude-plugins-official to false in ~/.claude/settings.json. The file's mtime is 2026-09-29T10:45:16Z. Phase 1 started at 10:45:03Z, so phase 1 ran with the toolkit enabled at user and project scope. Every toolkit run below started at 10:47:03Z, with the toolkit enabled at project scope only. The probe gave the command an uncommitted one-line change to app.js, which turns its comment into a false one. It passed no scope.

    Form Result
    Agent tool, subagent_type: pr-review-toolkit:comment-analyzer, default node Reached. The agent reported the false comment at app.js:1
    The same, with settingSources: [project] Reached, with the same finding
    Node prompt /pr-review-toolkit:review-pr comments Reached. The node printed the command's # PR review summary (comments) format, found the false comment, and reported no PR because the remote is not GitHub. It reviewed the one-line diff itself and launched no agent
    Skill tool, pr-review-toolkit:review-pr, not declared Not reached: Skill pr-review-toolkit:review-pr is not in this session's skills allowlist

    This matches the marker plugin form for form. The toolkit markers are weaker than the plugin's tokens, because they are review text. They only confirm that the marker plugin behaves like a published one.

    One probe defect, corrected

    The phase 2 skill-declared node loaded the skill (Launching skill: reach-marker:marker-skill) but wrote no marker. Its rules said "Do not run any marker command yourself", and the skill runs inside the node's own agent. I scored it as not run and ran it again in phase 3 with rules that let the node follow the skill. Those results are the ones in the first table.

    Commands

    S=/Users/oriol.torrent/Sites/scratch-628-plugin-reach
    M=/Users/oriol.torrent/Sites/scratch-628-marker-marketplace
    # phase 1: committed settings only
    cd $S && archon workflow run probe-628-marker --no-worktree "phase 1"
    # install at project scope, then phase 2 and 3
    cd $S && claude plugin marketplace add --scope project $M
    cd $S && claude plugin install reach-marker@reach-marker-628 --scope project
    cd $S && for w in probe-628-marker probe-628-declared probe-628-toolkit probe-628-control; do archon workflow run $w --no-worktree "phase 2"; done
    cd $S && archon workflow run probe-628-declared --no-worktree "phase 3"
    # the node's settingSources resolution, read from the binary
    strings -n 6 "$(readlink -f "$(command -v archon)")" | grep -oE '.{200}nodeConfig\?\.settingSources.{300}'

    The tool results and markers of every phase are in /Users/oriol.torrent/Sites/scratch-628-results/, with the logs next to it (scratch-628-phase{1,2,3}.log). Run ids: 2cdce522 (phase 1), 64c1901f, 403a6461, 6945b67c, 0c9599fd (phase 2) and 75465d8d (phase 3).

    probe-628-marker.yaml
    name: probe-628-marker
    description: Measure whether a node reaches a plugin agent, command and skill (#628)
    nodes:
      - id: agent-default
        prompt: |
          Rules. Make the call below exactly once. Then use the Write tool to write the full text of the tool result, verbatim, including any error, to the file named below. Do not read, grep or list any file outside the current directory. Do not run any marker command yourself. Do not retry with a different name.
          Call: the Agent tool with subagent_type "reach-marker:marker-agent" and prompt "label=agent-default".
          File: probe/agent-default.txt
      - id: agent-project-only
        settingSources: [project]
        prompt: |
          Rules. Make the call below exactly once. Then use the Write tool to write the full text of the tool result, verbatim, including any error, to the file named below. Do not read, grep or list any file outside the current directory. Do not run any marker command yourself. Do not retry with a different name.
          Call: the Agent tool with subagent_type "reach-marker:marker-agent" and prompt "label=agent-project-only".
          File: probe/agent-project-only.txt
      - id: command-literal
        prompt: "/reach-marker:marker-command label=command-literal"
      - id: command-skill-tool
        prompt: |
          Rules. Make the call below exactly once. Then use the Write tool to write the full text of the tool result, verbatim, including any error, to the file named below. Do not read, grep or list any file outside the current directory. Do not run any marker command yourself. Do not retry with a different name.
          Call: the Skill tool with skill "reach-marker:marker-command" and args "label=command-skill-tool".
          File: probe/command-skill-tool.txt
      - id: skill-default
        prompt: |
          Rules. Make the call below exactly once. Then use the Write tool to write the full text of the tool result, verbatim, including any error, to the file named below. Do not read, grep or list any file outside the current directory. Do not run any marker command yourself. Do not retry with a different name.
          Call: the Skill tool with skill "reach-marker:marker-skill" and args "label=skill-default".
          File: probe/skill-default.txt
      - id: skill-project-only
        settingSources: [project]
        prompt: |
          Rules. Make the call below exactly once. Then use the Write tool to write the full text of the tool result, verbatim, including any error, to the file named below. Do not read, grep or list any file outside the current directory. Do not run any marker command yourself. Do not retry with a different name.
          Call: the Skill tool with skill "reach-marker:marker-skill" and args "label=skill-project-only".
          File: probe/skill-project-only.txt
    probe-628-declared.yaml
    name: probe-628-declared
    description: Measure whether skills declares a plugin skill or command on a node (#628)
    nodes:
      - id: skill-declared
        skills: [reach-marker:marker-skill]
        prompt: |
          Rules. Make the call below exactly once, and follow the instructions it loads. Then use the Write tool to write the full text of the tool result of that call, verbatim, including any error, to the file named below. Do not read, grep or list any file outside the current directory. Do not retry with a different name.
          Call: the Skill tool with skill "reach-marker:marker-skill" and args "label=skill-declared".
          File: probe/skill-declared.txt
      - id: skill-declared-project-only
        skills: [reach-marker:marker-skill]
        settingSources: [project]
        prompt: |
          Rules. Make the call below exactly once, and follow the instructions it loads. Then use the Write tool to write the full text of the tool result of that call, verbatim, including any error, to the file named below. Do not read, grep or list any file outside the current directory. Do not retry with a different name.
          Call: the Skill tool with skill "reach-marker:marker-skill" and args "label=skill-declared-project-only".
          File: probe/skill-declared-project-only.txt
      - id: command-declared
        skills: [reach-marker:marker-command]
        prompt: |
          Rules. Make the call below exactly once, and follow the instructions it loads. Then use the Write tool to write the full text of the tool result of that call, verbatim, including any error, to the file named below. Do not read, grep or list any file outside the current directory. Do not retry with a different name.
          Call: the Skill tool with skill "reach-marker:marker-command" and args "label=command-declared".
          File: probe/command-declared.txt
    probe-628-toolkit.yaml
    name: probe-628-toolkit
    description: Measure whether a node reaches the real pr-review-toolkit (#628)
    nodes:
      - id: tk-agent-default
        prompt: |
          Rules. Make the call below exactly once. Then use the Write tool to write the full text of the tool result, verbatim, including any error, to the file named below. Do not read, grep or list any file outside the current directory. Do not run any marker command yourself. Do not retry with a different name.
          Call: the Agent tool with subagent_type "pr-review-toolkit:comment-analyzer" and prompt "Review only the uncommitted change in app.js, which `git diff app.js` shows. Reply in at most five lines."
          File: probe/tk-agent-default.txt
      - id: tk-agent-project-only
        settingSources: [project]
        prompt: |
          Rules. Make the call below exactly once. Then use the Write tool to write the full text of the tool result, verbatim, including any error, to the file named below. Do not read, grep or list any file outside the current directory. Do not run any marker command yourself. Do not retry with a different name.
          Call: the Agent tool with subagent_type "pr-review-toolkit:comment-analyzer" and prompt "Review only the uncommitted change in app.js, which `git diff app.js` shows. Reply in at most five lines."
          File: probe/tk-agent-project-only.txt
      - id: tk-command-literal
        prompt: "/pr-review-toolkit:review-pr comments"
      - id: tk-command-skill-tool
        prompt: |
          Rules. Make the call below exactly once. Then use the Write tool to write the full text of the tool result, verbatim, including any error, to the file named below. Do not read, grep or list any file outside the current directory. Do not run any marker command yourself. Do not retry with a different name.
          Call: the Skill tool with skill "pr-review-toolkit:review-pr" and args "comments". Follow what it loads to the end, then write the tool result of the Skill call itself, verbatim, to the file.
          File: probe/tk-command-skill-tool.txt
    probe-628-control.yaml
    name: probe-628-control
    description: Control for settingSources with a plugin enabled only at user scope (#628)
    nodes:
      - id: ctl-skill-default
        prompt: |
          Rules. Make the call below exactly once. Then use the Write tool to write the full text of the tool result, verbatim, including any error, to the file named below. Do not read, grep or list any file outside the current directory. Do not run any marker command yourself. Do not retry with a different name.
          Call: the Skill tool with skill "code-review:code-review". Do not follow the instructions it loads. Write only the tool result, then stop.
          File: probe/ctl-skill-default.txt
      - id: ctl-skill-project-only
        settingSources: [project]
        prompt: |
          Rules. Make the call below exactly once. Then use the Write tool to write the full text of the tool result, verbatim, including any error, to the file named below. Do not read, grep or list any file outside the current directory. Do not run any marker command yourself. Do not retry with a different name.
          Call: the Skill tool with skill "code-review:code-review". Do not follow the instructions it loads. Write only the tool result, then stop.
          File: probe/ctl-skill-project-only.txt
    Marker plugin

    agents/marker-agent.md

    ---
    name: marker-agent
    description: Probe agent. Writes a marker file and nothing else.
    tools: Bash
    ---
    Run this one Bash command, with LABEL replaced by the label your prompt gives you, and do nothing else:
    
    mkdir -p markers && echo "AGENT-<token> LABEL" >> markers/agent.txt
    
    Then reply with the single word done.

    commands/marker-command.md

    ---
    description: Probe command. Writes a marker file and nothing else.
    allowed-tools: Bash
    ---
    Run this one Bash command and do nothing else:
    
    mkdir -p markers && echo "COMMAND-<token> $ARGUMENTS" >> markers/command.txt
    
    Then reply with the single word done.

    skills/marker-skill/SKILL.md

    ---
    name: marker-skill
    description: Probe skill for the reach probe. Use only when explicitly asked to invoke marker-skill.
    ---
    Run this one Bash command, with LABEL replaced by the label you were given, and do nothing else:
    
    mkdir -p markers && echo "SKILL-<token> LABEL" >> markers/skill.txt
    
    Then reply with the single word done.

    The tokens are left out here. They are in the plugin files and in /Users/oriol.torrent/Sites/scratch-628-tokens.txt.

  6. added
    closedPR has been merged
    and removed
    needs-triageMaintainer needs to evaluate
    on Sep 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

app:unic-archon-dlcclosedPR has been mergedp2Medium priorityspikeTime-boxed investigation; no production code committed

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions