Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 17 additions & 2 deletions docs/decisions/0005-output-budget-protocol.md
Original file line number Diff line number Diff line change
Expand Up @@ -107,8 +107,13 @@ non-autoskillit repos, two lower-ceiling launch paths cap casual reads at ~40 KB
The ceiling clamps regular-path exec/tool output via
`min(model request, tool_output_token_limit)`, but code-mode models (gpt-5.6-sol)
honor model-declared `max_output_tokens` unclamped β€” for those sessions the ceiling
is not a hard cap and the intake discipline digest's numeric rule
(`max_output_tokens` <= 10000) is the operative bound.
is not a hard cap and the operative bound is enforced server-side by
`BackendCapabilities.effective_delivery_token_limit` (Codex: 10,000 tokens). The
response backstop reads that capability at spill time and spills any payload whose
estimated token count exceeds it, with the artifact path surfaced via the existing
`_autoskillit_response_spill` envelope (`reason="delivery_bound"`). The intake
discipline digest's numeric rule (`max_output_tokens` <= 10000) remains as the
prompt-level reinforcement.

## Corrections of Record

Expand Down Expand Up @@ -146,6 +151,16 @@ recorded explicitly in the issue body.
bounded slices enter worker memory). Still open for `run_skill` and `test_check`,
whose adjudication requires the full text.

## Resolved

- **Code-mode bypass (`gpt-5.6-sol` honoring `max_output_tokens` unclamped)**: prior to
this revision, the `tool_output_token_limit` ceiling did not constrain code-mode
responses and the intake discipline digest's numeric rule was the only operative
bound β€” prompt-level and unenforced. The server-side response backstop now reads
`BackendCapabilities.effective_delivery_token_limit` (Codex: 10,000 tokens;
Claude Code: 46,500 tokens) and spills payloads that exceed it. This makes the
delivery bound authoritative on every backend, not advisory.

## Operational Signals

Output-budget instrumentation uses low-cardinality structured counters or events for:
Expand Down
1 change: 1 addition & 0 deletions src/autoskillit/core/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ Sub-packages: types/ (see types/AGENTS.md) and runtime/ (see runtime/AGENTS.md).
| `logging.py` | Logging configuration |
| `paths.py` | `pkg_root()`, `is_git_worktree()`, `is_git_main_checkout()`, `is_in_git_repo()` |
| `_claude_env.py` | IDE-scrubbing canonical env builder for agent subprocesses |
| `_delivery_bounds.py` | `resolve_effective_delivery_bound()` β€” canonical accessor for per-backend effective delivery token limits |
| `_terminal_table.py` | IL-0 color-agnostic terminal table primitive |
| `_version_snapshot.py` | Process-scoped version snapshot for session telemetry (`lru_cache`'d) |
| `branch_guard.py` | Branch protection helpers |
Expand Down
1 change: 1 addition & 0 deletions src/autoskillit/core/__init__.pyi
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ from ._cmd_runner import CmdRunner as CmdRunner
from ._cmd_runner import default_cmd_runner as default_cmd_runner
from ._cmd_runner import run_gh as run_gh
from ._cmd_runner import run_git as run_git
from ._delivery_bounds import resolve_effective_delivery_bound as resolve_effective_delivery_bound
from ._execution_marker import execution_marker as execution_marker
from ._install_detect import DirectUrlInfo as DirectUrlInfo
from ._install_detect import _is_release_tag as _is_release_tag
Expand Down
19 changes: 19 additions & 0 deletions src/autoskillit/core/_delivery_bounds.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
"""Per-backend effective delivery-bound resolution.

The configured ``tool_output_token_limit`` written to Codex ``config.toml``
is a config-file ceiling. Code-mode models may bypass that ceiling when
they emit ``max_output_tokens`` without an upper bound, so the operative
bound for those paths is the harness default (~10K tokens). The
``BackendCapabilities.effective_delivery_token_limit`` field encodes this
worst-case operative bound; ``resolve_effective_delivery_bound`` is the
canonical accessor.
"""

from __future__ import annotations

from .types._type_backend import BackendCapabilities


def resolve_effective_delivery_bound(caps: BackendCapabilities) -> int:
"""Return the worst-case operative token bound for ``caps``'s backend transport."""
return caps.effective_delivery_token_limit
7 changes: 7 additions & 0 deletions src/autoskillit/core/types/_type_backend.py
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,12 @@ class BackendCapabilities:
# directory rather than written with gating frontmatter that the backend
# would ignore.
supports_model_invocation_gating: bool = True
# Worst-case delivery bound in tokens: the lowest operative bound that
# could apply to this backend's tool-output transport. Distinct from any
# config-file ceiling (e.g. `tool_output_token_limit`), which code-mode
# models may bypass. Used by `enforce_response_budget` to spill payloads
# the downstream transport cannot deliver at full size.
effective_delivery_token_limit: int = 0


ALL_PROJECT_LOCAL_SKILL_SEARCH_DIRS: tuple[str, ...] = (
Expand Down Expand Up @@ -291,6 +297,7 @@ def model_class(model: str) -> str:
skill_sigil="/",
session_dir_persistent=False,
supports_model_invocation_gating=True,
effective_delivery_token_limit=46_500,
)


Expand Down
1 change: 1 addition & 0 deletions src/autoskillit/execution/backends/codex.py
Original file line number Diff line number Diff line change
Expand Up @@ -639,6 +639,7 @@ def capabilities(self) -> BackendCapabilities:
skill_sigil="$",
session_dir_persistent=True,
supports_model_invocation_gating=False,
effective_delivery_token_limit=10_000,
)

@property
Expand Down
4 changes: 3 additions & 1 deletion src/autoskillit/hooks/formatters/_fmt_response_spill.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,9 @@
"reason",
}
)
_RESPONSE_SPILL_REASONS = frozenset({"oversized_values", "minimal_projection", "plain_text"})
_RESPONSE_SPILL_REASONS = frozenset(
{"oversized_values", "minimal_projection", "plain_text", "delivery_bound"}
)
_RESPONSE_SPILL_SCHEMA_DIGEST = hashlib.sha256(
json.dumps(
{
Expand Down
14 changes: 13 additions & 1 deletion src/autoskillit/server/_notify.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,12 @@
from anyio import ClosedResourceError as _ClosedResource

from autoskillit.config import OutputBudgetConfig
from autoskillit.core import RESERVED_LOG_RECORD_KEYS, get_logger
from autoskillit.core import (
RESERVED_LOG_RECORD_KEYS,
BackendCapabilities,
get_logger,
resolve_effective_delivery_bound,
)
from autoskillit.server._response_budget import (
bounded_response_budget_failure,
enforce_response_budget,
Expand Down Expand Up @@ -137,12 +142,19 @@ async def wrapper(*args: Any, **kwargs: Any) -> Any:
artifact_dir = (
temp_dir / "responses" / tool_name if isinstance(temp_dir, Path) else None
)
effective_delivery_token_limit: int | None = None
if ctx is not None:
backend = getattr(ctx, "backend", None)
caps = getattr(backend, "capabilities", None) if backend is not None else None
if isinstance(caps, BackendCapabilities):
effective_delivery_token_limit = resolve_effective_delivery_bound(caps)
try:
result = enforce_response_budget(
result,
tool_name=tool_name,
artifact_dir=artifact_dir,
config=budget,
effective_delivery_token_limit=effective_delivery_token_limit,
)
except Exception:
logger.error("track_response_size_enforcement_failed", tool_name=tool_name)
Expand Down
Loading
Loading