Skip to content

feat(guards): inspect MCP tool invocations and record refusal telemetry - #3

Open
Pipe5linger wants to merge 3 commits into
dioptx:mainfrom
Pipe5linger:feat/mcp-tool-guards-and-refusal-logging
Open

Pipe5linger wants to merge 3 commits into
dioptx:mainfrom
Pipe5linger:feat/mcp-tool-guards-and-refusal-logging

Conversation

@Pipe5linger

Copy link
Copy Markdown

Motivation & Summary

Closes #2.

This PR addresses the two gaps identified in #2 when running agentic workflows with Claude Code:

  1. MCP Tool Guard Bypass: Tool calls executing shell commands through Model Context Protocol (MCP) servers (e.g. mcp__*, context-mode's ctx_execute, custom shell execution tools) previously evaded PreToolUse inspection because the matcher and parser were hardcoded solely to Bash, Edit, and Write.
  2. Refusal Telemetry: Blocked guard invocations previously left no trace in the agentry event stream (~/.local/state/agentry/events.jsonl), making it impossible to differentiate an agent that was stopped from one that never attempted the action.

What's Changed

  1. PreToolUse Hook Matcher (hooks/hooks.json):

    • Expanded the PreToolUse matcher pattern to include mcp__.*: "matcher": "Bash|Edit|Write|Agent|mcp__.*".
  2. MCP Command Extraction (core/guard_engine.py):

    • Implemented _extract_command_and_target(tool_name, tool_input) to extract commands or script bodies from MCP tools and custom executors matching command, cmd, script, query, eval, or code.
    • Also inspects file path mutations made through MCP tools (file_path).
    • Extended the pre-accept allowlist check to support approved MCP commands alongside standard Bash.
  3. Refusal Telemetry Logging (core/guard_engine.py):

    • Implemented _log_refusal(): when a tool call is blocked, writes an append-only refusal record to ~/.local/state/agentry/events.jsonl with:
      • event: "guard"
      • tool: tool name (e.g., Bash, mcp__...)
      • refused_input: the attempted command/path
      • rule: matching guard pattern
      • reason: refusal message
      • build_id & task: parsed from the AGENT_P_TAGS environment variable (e.g., build_id=xyz,task=abc).
  4. Dedicated Test Suite (tests/test_mcp_guards.py):

    • Added unit tests verifying:
      • MCP tool commands (e.g. mcp__context_mode__ctx_execute) are caught by step guards.
      • Alternate key arguments (cmd, script) are inspected.
      • Refusal telemetry is correctly written to ~/.local/state/agentry/events.jsonl with tags parsed from AGENT_P_TAGS.

Verification

$ pytest tests/test_mcp_guards.py -v
============================== 3 passed in 0.25s ==============================

All existing unit tests in tests/test_guard_engine.py continue to pass without regression.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

weft does not log what it blocks, and MCP tool calls bypass the guards

1 participant