Skip to content

Investigate output_budget_guard: unscoped firing and mistrusted deny messages #4284

Description

@Trecek

This issue is an investigation prompt. Do NOT implement fixes from this issue directly. The operator will run a dedicated, focused investigation session against it; that session must produce its own investigation report before any remediation is planned.

Operator context

output_budget_guard was put in place for the Codex backend — Claude Code sessions were fine and did not need it (see #4272, the motivating token-consumption issue). The guard's observed mechanism raises concerns: it fires in session types it was never needed for, its deny messages are being actively distrusted by agents, and its instructions push sessions toward behavior that conflicts with skill output discipline. A dedicated investigation report is required on why it behaves as observed.

Observed anomalies (2026-07-18, discovered incidentally during the #4283 investigation)

  1. Unscoped firing. The guard fires in interactive-session subagents. main() in src/autoskillit/hooks/guards/output_budget_guard.py has no session-scope gating: no AUTOSKILLIT_HEADLESS check (unlike write_guard.py / planner_result_naming_guard.py), no backend check, and _EXEMPT_SKILLS is an empty frozenset (line 34). Registered in hook_registry.py:206 (also listed at :426). If the intent was Codex/headless protection, nothing in the code expresses that scope.
  2. Deny messages mistaken for prompt injection. Three independent Explore subagents (sonnet) hit denials on commands they believed bounded and independently classified the deny text as a prompt-injection attempt embedded in tool results — then deliberately routed around it; one inadvertently wrote a scratch file as a byproduct of complying before recognizing the pattern. The deny reason (lines 329–333) does not self-identify as an AutoSkillit hook, which is why agents treat it as untrusted. An agent trained to resist injection will resist an anonymous guard — this failure mode gets worse with more capable models, not better.
  3. Temp-file steering. The deny message instructs: "redirect both descriptors under .autoskillit/temp/ then read slices." This actively steers sessions toward creating ad-hoc temp artifacts, in direct tension with skill output manifests — see companion issue rectify skill writes undeclared working files in interactive runs #4283, where the same PR's skill-prompt block ("Preserve full evidence in project temp and return a bounded summary plus the artifact path") is identified as a contributing cause of undeclared working files.
  4. Possible classifier false positives. Denied commands reported by subagents include a recursive grep piped to head -c (byte-capped) and ls | wc -l — both plausibly bounded. Disposition logic: R1–R3 enumerated shapes via classify_command_output_budget in the shared hooks/_command_classification.py; policy defaults small_file_max_bytes=5000, shell_max_inline_bytes=12000 (output_budget_guard.py:36-37), configurable via output_budget_policy in merged hook config.

Provenance

Investigation questions for the dedicated session

  1. Scoping: If the guard exists for Codex, why is it not gated to Codex/headless sessions (env-based scope like write_guard.py, or codex_status-driven applicability)? What was the design rationale in Output Budget Protocol Audit Remediation #4277 / the originating rectify plan for firing it everywhere, and does that rationale hold?
  2. False positives: Are the observed denials of bounded commands a classifier bug in _command_classification.py (R1–R3 disposition), and what is the actual false-positive surface (which command shapes, which pipe/redirect patterns)?
  3. Trust/UX: Should the deny reason self-identify (e.g., "AutoSkillit output_budget_guard:") so agents comply instead of classifying it as injection and routing around it? What do the three misidentification transcripts show about how agents respond?
  4. Composition: How should the guard's redirect-to-temp instruction compose with skill output contracts/manifests (rectify skill writes undeclared working files in interactive runs #4283) so the escape valve doesn't manufacture undeclared artifacts?
  5. Net effect: Given agents route around it, does the guard in Claude Code sessions currently produce net harm (retry churn, distrust, scratch files) versus its intended token savings — measured against the Codex-only problem it was built for?

Cross-references


Closed as superseded — 2026-07-27

This issue investigates output_budget_guard, which no longer exists. PR #4288 /
PR #4286 (feat: replace output_budget_guard with lossless shell capture, merged
2026-07-19)
removed it. Verified on develop @ c1c3e51ba: output_budget_guard.py
appears in the repo only as an entry in RETIRED_SCRIPT_BASENAMES
(src/autoskillit/hook_registry.py:406), and no live hook file exists.

There is no behaviour left to investigate. The programme priority map carried this at
rank 0 with "Done means: Closed" — carrying it further costs triage attention for nothing.
Closing per that ranking.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugExisting behavior is brokenrecipe:remediationRoute: investigate/decompose before implementation

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions