Skip to content

Let caller-offered computer tools pass through the headless middleware - #805

Open
charan-rathore wants to merge 1 commit into
CopilotKit:mainfrom
charan-rathore:fix-remote-bot-computer-passthrough
Open

charan-rathore wants to merge 1 commit into
CopilotKit:mainfrom
charan-rathore:fix-remote-bot-computer-passthrough

Conversation

@charan-rathore

Copy link
Copy Markdown
Contributor

What this changes

Fixes #804. In a chat with a remote AG-UI Bot, every computer_* call was refused with "The tool computer_navigate is unavailable for this headless turn." while the built-in General Assistant browsed fine on the same deployment.

HeadlessToolsMiddleware claims every computer_* tool call by name prefix, but its tool map only holds the Bot's server-side grants. A remote Bot has no server-side computer grant, so a call to the browser-offered computer_* tools was intercepted and refused instead of passing through to the caller that offered them. The middleware's own continuation logic already anticipates partially-owned pending sets (owned.length !== pending.length), so the fix is one condition: claim a computer_* call only when the caller did not offer the tool. The governed refusal still covers genuinely browserless turns (routines), where the call would otherwise die silently.

Where it runs

  • New state that outlives a request? None - one per-run Set of tool names already in the input.
  • What happens on the second replica? Nothing new: the check reads the current run's input only.
  • Anything serialised? No.
  • Anything fanned out to a browser? That is the point of the fix: the tool call now reaches the caller's browser through the existing AG-UI stream instead of being refused server-side.
  • New listener, port, or schedule? None.

Boundary and audit

  • Every acting call still goes through the gateway: resolve, decide, audit, then act. Server-side computer grants are still claimed and executed through the policy path as before; only caller-offered tools pass through, executed by the caller's own browser policy.
  • New refusals and new failures each write a row. No new refusals; the existing refusal for not-offered tools is unchanged.
  • Nothing new is trusted from the client that the server can resolve itself. The caller already controls which tools it offers in its own run input.

Changelog

  • A line in CHANGELOG.md under Unreleased.

Proof

New toolkit/tests/headless-tools.test.ts (bun test):

Existing server/tests/headless-tools.test.ts: all 8 tests pass unchanged, including "unknown computer tool calls become error results rather than commands".

HeadlessToolsMiddleware claimed every computer_* tool call by name prefix
but only holds the Bot's server-side grants, so in a chat with a remote
AG-UI Bot the browser-offered computer_* calls were intercepted and
refused as "unavailable for this headless turn" instead of reaching the
browser. Claim computer_* calls only when the caller did not offer the
tool; the governed refusal still covers genuinely browserless turns.

Fixes CopilotKit#804

Signed-off-by: Charan Rathore <180254320+charan-rathore@users.noreply.github.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-18T09:07:43.585937Z 767bfe0 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remote AG-UI Bots can't use the browser in chat: computer_* calls refused as "unavailable for this headless turn"

1 participant