diff --git a/CHANGELOG.md b/CHANGELOG.md index 9bc0e40..596dc20 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,10 @@ adheres to [Semantic Versioning](https://semver.org/). ## Unreleased +### yoagent + +- **`ToolGate` judges large calls whole, and commands only whole.** Before, every string argument over 2,000 characters was cut to its head and tail, so a 100,000-character `write_file` was judged on 2,000 characters of content and a long `bash` script with `rm -rf` in its middle was judged without that line; a call still over 12,000 characters after that was denied. Now arguments are sent **whole when they fit** `ToolGate::with_max_argument_chars` (default 48,000 characters) and the decision backend's token limits; a call that does not fit has its longest strings cut only as far as needed, and the state lists what was cut under `tool_call.omitted`. **Full views** are never cut — judged whole or denied: `with_full_view(tool, json_pointer)`, by default the built-in `bash` tool's `/command` (`without_default_full_views` drops it). **Behaviour change:** a `bash` command too large to review in full (over about 48,000 characters) is now denied instead of judged from its head and tail; more calls are sent in full, so large calls cost more decision-model input tokens. + ### yoagent-frontend (experimental, new, not published) - **One frontend protocol** (`integrations/yoagent-frontend`): a `Session` runs one agent for any number of frontends — a pi-tui terminal UI (a rutis plugin) and browsers over WebSocket — with commands in (prompt, steer, follow-up, abort, reset, quit), and out every event (text deltas merged), a run lifecycle that is always delivered (`runStart` / `runEnd` with an `outcome`: completed, aborted, rejected or error — also when the agent's task or the session's driver fails), plugins' questions (`uiRequest`: confirm, select, input, notify; the first answer that fits wins, a safe default without a frontend or in time; `uiResolved` says why it closed), `notice`s, and `closed`, and **UI plugins**: a rutis plugin offers the browser an ES module (`renderTool` for a tool's results, `mountPanel` for a side panel). `PluginHost` replaces the rutis setup (runtimes, loader, explicit per-row runtime routing; `load` waits until plugins run — though a TypeScript plugin reads as running before its async start-up finishes, so wait for the handlers you need — and drops a batch that fails). The browser's WebSocket needs the token in the URL the server prints. Example `coding_agent` (terminal or `--web`), tried live in Chrome. yoagent's core and yoagent-rutis's API do not depend on it. diff --git a/CLAUDE.md b/CLAUDE.md index 19ab0ba..217bbb9 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -122,7 +122,7 @@ Behind the `gasp` Cargo feature (dep: `yoagent-state`, the GASP reference runtim Behind the `decision` Cargo feature (no deps; off by default; nothing is sent until a `DecisionModel` is constructed and used — an env key alone enables nothing, proven by `tests/decision_env_test.rs` against a wiremock `expect(0)`). Typed `Question`s (`Noul`/`Choice`/`Score`; "Noul" is SystemOne wire vocabulary) about a JSON/text `state` → `Evaluation` (private fields; getters `model()`, `usage()`, `cost_usd()`, `answers()` in request order; `new`/`with_answer`/`with_cost_usd` for backends); answers have private fields + getters (`NoulAnswer::p_true()`, `ChoiceAnswer` in option order, `ScoreAnswer`). `DecisionModel` (cheap clone) wraps a `DecisionBackend` trait (`capabilities()`, `evaluate(&Request)`): `SystemOneBackend` (HTTP `POST /v1/systemone`, or an exact URL via `with_endpoint_url`; its parser is public as `parse_systemone_response` for backends that move the request themselves; Cloudflare's `{"result": ...}` envelope unwrapped (a text `result` is parsed, a non-object one is `BadResponse`); a 2xx `"success": false` → `Invalid` for schema code 5006, else `Backend`; a 429 with Cloudflare code 3036 (daily allocation) → non-retryable `Http`, other 429/529 → `RateLimited` carrying the body; env keys trimmed, 401/403 name the variable; lenient parsing; 429/529 retried via `RetryConfig`, `retry-after` wins) and `MockBackend`. Presets `jev()` (`TYPESAFE_API_KEY`/`TYPESAFE_BASE_URL`, read at call time), `jev_opencode()`, `jev_opencode_free()` (`OPENCODE_API_KEY`), `clef(account_id)` / `clef_flash(account_id)` (Cloudflare Workers AI via `SystemOneBackend::workers_ai(account, "@cf/cloudflare/clef")`; token `CLOUDFLARE_API_TOKEN` then `CLOUDFLARE_AUTH_TOKEN`, read at call time), `local(url)` (sets `Capabilities::local` = self-hosted), `logprobs(url, model)` (`LogprobBackend`, see below), `gpt_6_luna()` / `from_openai_backend(backend, id)` (`OpenAiDecisionBackend` in `decision/openai.rs`, own `Slot::OpenAi` so `with_api_key`/`with_retry` apply: OpenAI's Decisions API `POST {base}/decisions`, base `https://api.openai.com/v1` (`/v1` included), `OPENAI_API_KEY` read at call time + trimmed, 401/403 name it; Noul→`predicate`, Choice→`choice` (string `value`s, criteria→`description`), Score→`score` (`levels[].label`, answer probabilities by level index `value`), id→`name`, JSON state/instructions → pretty JSON text, Noul criteria appended to instructions; answers matched by `name`, unnamed ones by position only when counts match; `refusal` → `DecisionError::Refused` naming the question (not retried, terminal in an `or` chain; gate/guard fail closed; its usage still recorded); no predicate confidence → computed; capabilities = `Capabilities::systemone()` (no published limits); `with_safety_identifier`; images unsupported; HTTP via `systemone::{post_json, with_retries}`; mock-only, never run live — `tests/decision_openai_test.rs`), `from_backend` / `from_arc`. Requests validated against `Capabilities` (2–255 options, 2–10 levels, 64k/32k token estimates); **answers validated centrally in `check_complete` for every backend** (kind, finite probs/confidence in [0,1], a probability for every Choice option / Score level, sums within `max(0.02, n × 0.005)` of 1 for n options/levels (TypeSafe rounds to 2 dp), choice ∈ options, score in range) → `BadResponse`; answers nobody asked for are dropped and the rest reordered to request order (so a non-batching merge cannot be overwritten). `DecisionError` is `Clone` without `PartialEq`; `Backend`/`Transport` are struct variants with an optional `Arc` source. Missing confidence = `(n*p_max-1)/(n-1)`. Pricing (all unpriced until the process opts in — `prices::enable_bundled()` etc.): `jev()` uses `typesafe/` from the process-wide table only when the base URL is TypeSafe's host; `clef*()` use `cloudflare/` (an `@cf/cloudflare/` prefix stripped) only while the host is `api.cloudflare.com`; `gpt_6_luna()` uses `openai-decisions/` (its own key, not chat `openai/gpt-6-luna`; $0.10/M input, output free) only while the host is `api.openai.com`, an unlisted id warns once — only while pricing is enabled — via the per-key `decision::warn_once` (also used by the no-usage warning and the two `deprecated:` keys: gate as middleware, guard as filter; once per process); gateways and `from_backend` unpriced; `local()` $0; an unpriced handle keeps a backend-reported cost; a response without usage is unpriced under a non-zero price, while a free handle (`local()`, all-zero rates) stays $0. -Agent integration (`decision::wire` returns extensions, appended in `Agent::build_config` / `SubAgentTool`): `with_decision_model` → advisory only (crate-private `Advisor`, installed as the `AdvisorExtension` extension's `before_model` note: one batched request per user request, memoized; skill hint via Choice+Noul at need ≥0.3 / confidence ≥0.5, tool hint at ≥40 tools; 2 s; failures warn and add nothing; does nothing without skills and with <40 tools). `with_tool_gate(ToolGate)` → installed as an extension (`impl Extension for ToolGate`, `rechecks_modified_calls` = true; its `ToolMiddleware` impl is deprecated (warns once), `ToolGate::decide` drives it directly) appended after the agent's own extensions, so it runs last; Nouls destructive/requested, deny when destructive ≥0.5 and requested <0.7; state `{user_request, tool_call}` with long strings head+tail-truncated and >12k chars denied; every comparison denies on NaN/missing; threshold setters and check-id collisions panic; **fails closed**. Not a security boundary. Spend: every `evaluate_request` inside a loop records into the `tokio::task_local!` `LoopScope` (`agent_loop::record_decision`), opened in `agent_loop_with_stats` **before the input filters** (a rejected run's `AgentEnd` carries the spend) and in `agent_loop_continue_with_stats` → `SessionStats::decision` (`DecisionStats`, with `unpriced` counting evaluations of unknown cost), child runs folded in (not into `sub_agents`); counted in `total_cost_usd`; a non-batching request that fails or times out partway records the usage already billed (accumulated outside the timed-out future). +Agent integration (`decision::wire` returns extensions, appended in `Agent::build_config` / `SubAgentTool`): `with_decision_model` → advisory only (crate-private `Advisor`, installed as the `AdvisorExtension` extension's `before_model` note: one batched request per user request, memoized; skill hint via Choice+Noul at need ≥0.3 / confidence ≥0.5, tool hint at ≥40 tools; 2 s; failures warn and add nothing; does nothing without skills and with <40 tools). `with_tool_gate(ToolGate)` → installed as an extension (`impl Extension for ToolGate`, `rechecks_modified_calls` = true; its `ToolMiddleware` impl is deprecated (warns once), `ToolGate::decide` drives it directly) appended after the agent's own extensions, so it runs last; Nouls destructive/requested, deny when destructive ≥0.5 and requested <0.7; state `{user_request, tool_call}`: arguments whole when they fit `with_max_argument_chars` (48k chars) and the backend's token limits (`question::check_token_limits`, shared with `Request::validate`; the strictest across an `or` chain via crate-private `DecisionModel::chain_capabilities`, so no fallback skips the call unsent), else `fit_arguments` binary-searches one per-string cap (≥400) cutting the longest strings first and lists cuts in `tool_call.omitted` (JSON pointers); full views (`with_full_view(tool, pointer)`, default `bash` `/command`, `without_default_full_views`) are never cut — a full view that does not fit denies, as does a call that does not fit cut to 400; never judged in pieces; every comparison denies on NaN/missing; threshold setters and check-id collisions panic; **fails closed**. Not a security boundary. Spend: every `evaluate_request` inside a loop records into the `tokio::task_local!` `LoopScope` (`agent_loop::record_decision`), opened in `agent_loop_with_stats` **before the input filters** (a rejected run's `AgentEnd` carries the spend) and in `agent_loop_continue_with_stats` → `SessionStats::decision` (`DecisionStats`, with `unpriced` counting evaluations of unknown cost), child runs folded in (not into `sub_agents`); counted in `total_cost_usd`; a non-batching request that fails or times out partway records the usage already billed (accumulated outside the timed-out future). Phase 3–4 (`decision/logprobs.rs`, `calibrate.rs`, `guard.rs`; fallback and the split path in `mod.rs`). **`LogprobBackend`** (`DecisionModel::logprobs(url, model)`, or `from_logprob_backend(backend, model)` to keep the logprobs slot — `with_api_key`/`with_retry`/loopback $0 — with custom settings; there is no `DecisionModel::with_temperature`): any OpenAI-compatible `/chat/completions` with logprobs; one request per question (`max_tokens:1`, `temperature:0`, `logprobs:true`, `top_logprobs:max(K=20, labels)`, `extra_body` deep-merged (nested objects key by key) and inserted under the core fields; `with_thinking_disabled` = `chat_template_kwargs.enable_thinking=false`, not default because OpenAI rejects unknown fields), single user message; labels A/B (Noul, A = yes), A.. (Choice, ≤26), 0–9 (Score); `KINDS` lists the three kinds explicitly and every match is exhaustive (a future kind → `Unsupported`). `label_distribution`: trim + upper-case, sum duplicates; no label or label mass < `min_label_mass` (0.5) → `BadResponse` (a `` first token or a word answer is noise, not a confident answer — the gate/guard then fail closed); an absent label gets `min(smallest reported p, 1 − Σreported)` clamped to `[1e-9, 1]`; temperature on ln(mass), softmax, then an absent label is re-floored at 1e-9 after scaling (a tiny temperature would underflow it), so absence never gives 0/1. Reports `batching:false` + `Capabilities::max_concurrent_requests = 8` (new field, default 1): `DecisionModel::evaluate_singles` splits the request into boxed futures built in a loop (a closure trips the `Send` HRTB check), runs them via `FuturesUnordered` up to the limit, stops starting new ones after the first failure but awaits those in flight, adds each completed one to `billed` (so errors and timeouts record the spend of completed questions), returns the earliest failed question's error. `local` only for a loopback host (`is_loopback_url`: localhost/`*.localhost`/127/8/::1; `0.0.0.0` is *not*, which the unix-only remote-pricing wiremock test relies on); loopback → `Pricing::Fixed(0)`, else unpriced. HTTP shared with SystemOne via `systemone::{post_json, with_retries, status_error}` (429/529/transport retried; other 5xx is `Http`; a reqwest `is_builder()` error — malformed URL — is `Invalid`). **Fallback** `DecisionModel::or`: `fallbacks: Vec` (flattened; each member sends its own model id), `evaluate_chain` — structural validation against `Capabilities::unlimited()` first (→ `Invalid`, no member tried), then per member: capability-invalid → skipped unsent (`FallbackAttempt { was_sent: false }`, not recorded); any member error, `Invalid` included, falls through — except `Refused`, which is returned at once; one `rt::Instant` deadline from the head's `timeout` (members' `timeout` ignored) judged by the clock after each failure (a backend's own `Timeout` does not end the chain), per-attempt limit `min(remaining, attempt_timeout)`; deadline passed → `Timeout(head timeout)`; exhaustion → `DecisionError::AllFailed { attempts: Vec }` (getters `model`/`error`/`was_sent`; `is_retryable` if any member's error was). Each sent attempt is recorded (`record_attempt`); nothing sent → one failure. `model()`/`capabilities()`/builders are the primary's. **`calibrate` / `calibrate_with`** (`CalibrationOptions` private fields + getters): one request per example, `buffer_unordered(concurrency)`; top-label ECE (10 bins), Brier binary for Noul / multi-class otherwise, Noul F1 and target-precision thresholds over observed `p_true`, temperature by NLL grid `1.05^k, k∈[-33,33]` (ties → closest to 1) where only p>0 entries are rescaled and only the true outcome's rescaled p is floored; `accuracy`/`brier`/`ece` are `Option` (None when nothing evaluated); `models` counts per `Evaluation::model` + `mixed_models()`; `errors: Vec`. **`InputGuard`** (`Agent`/`SubAgentTool::with_input_guard` = `assert_has_checks()` (panics on no checks) + `with_extension` (`impl Extension for InputGuard`, `on_input`; its `AsyncInputFilter` impl is deprecated (warns once), `InputGuard::screen` drives it directly)): built-ins `injection` / `harmful` at 0.8, stored as a `defaults` flag + `custom` list + `thresholds` overrides, resolved lazily in `checks()` (a custom check with a built-in id replaces it in place and survives `without_default_checks`, whatever the call order); state `{input}` with the **whole** text (no truncation); over `max_input_chars` (32k) → not sent, failure policy; 3 s, fail-closed (`with_fail_open` opt-out); empty text → pass; no checks used directly → Reject; steering/follow-ups are not filtered (loop unchanged). Tests: `decision_logprobs_test` (wiremock, incl. split-request spend via an input guard), `decision_fallback_test`, `decision_calibrate_test`, `decision_guard_test` (MockBackend). diff --git a/docs/concepts/decision-models.md b/docs/concepts/decision-models.md index 46f236e..a007b81 100644 --- a/docs/concepts/decision-models.md +++ b/docs/concepts/decision-models.md @@ -525,17 +525,50 @@ backend. 3. Otherwise there is no request to judge against, and the call is denied with a reason asking the user to restate it. -**Arguments.** Short values (paths, names, flags) are always sent whole. A -string over 2,000 characters keeps its head and its tail around an explicit -`[truncated N chars]` marker — so the `rm -rf` at the end of a long heredoc -is visible. If the arguments are still over 12,000 characters, the call is -denied without asking. +**Arguments.** A call's arguments are sent **whole when they fit**: within +`with_max_argument_chars` (default 48,000 characters, serialized) and the +decision backend's own token limits (with fallbacks, the strictest in the +chain, so no fallback skips the call). A full-file rewrite or a large patch is +judged on all of it. + +A call that does not fit is shortened only as far as needed: its longest +strings are cut first, each to its head and tail around an explicit +`[truncated N chars]` marker, while short values (paths, names, flags) stay +whole. The state then says what was cut, so the model knows it is judging a +partial view: + +```json +"tool_call": { + "tool": "write_file", + "arguments": { "path": "src/lib.rs", "content": "…[truncated 52310 chars]…" }, + "omitted": [{ "argument": "/content", "chars": 52310 }] +} +``` + +Some arguments must never be judged in part: a command or a script can hide +its dangerous line anywhere. Those are **full views**, judged whole or not +at all — a call whose full-view argument does not fit is denied without +asking ("more than the gate can review in full"). The built-in `bash` tool's +`/command` is one by default; add your own with +`with_full_view(tool, json_pointer)` (a pointer to an array or object covers +everything under it), and drop the default with `without_default_full_views()`. + +```rust,ignore +let gate = ToolGate::new(model) + .with_full_view("run_sql", "/query") // a query is judged whole or denied + .with_max_argument_chars(64_000); // more room before anything is cut +``` + +A call that does not fit even with every other long string cut to 400 +characters is denied too. Calls are never judged in pieces: what one piece +does can depend on another (a `cd` in one, an `rm` in the next). **What denies.** A call is denied — with a reason the model sees — when: - it looks destructive and is not clearly requested (the thresholds above); - a custom check (`with_check`) reaches its threshold; -- its arguments are over the size cap after shortening; +- a full-view argument (by default `bash`'s command) does not fit whole, or + the arguments do not fit even shortened; - there is no user request to judge against; - the decision model errors, times out (**5 s**), or answers with a missing, malformed, non-finite or out-of-range value (**fails closed**). diff --git a/src/decision/gate.rs b/src/decision/gate.rs index 7002762..6dd0179 100644 --- a/src/decision/gate.rs +++ b/src/decision/gate.rs @@ -3,7 +3,7 @@ //! it. Blocking, so strictly opt-in, and fail-closed. use super::advisory::{assert_threshold, truncate_middle}; -use super::question::Question; +use super::question::{check_token_limits, Question}; use super::DecisionModel; use crate::types::{ToolCallRequest, ToolDecision, ToolMiddleware}; use serde_json::{json, Value}; @@ -11,11 +11,14 @@ use std::time::Duration; /// Most characters of the user's request sent as state (head and tail kept). const MAX_REQUEST_CHARS: usize = 8_000; -/// A string argument longer than this is shortened to head and tail. -const MAX_STRING_CHARS: usize = 2_000; -/// Most characters of the arguments, serialized, after shortening. Larger -/// calls are denied: the gate will not approve what it could not read. -const MAX_ARGS_CHARS: usize = 12_000; +/// Default for [`ToolGate::with_max_argument_chars`]. +const DEFAULT_MAX_ARGUMENT_CHARS: usize = 48_000; +/// The least of a long string a shortened call keeps (head and tail). A call +/// that does not fit with every shortenable string cut to this is denied. +const MIN_KEPT_CHARS: usize = 400; +/// Arguments the gate never shortens by default: a command is judged whole +/// or not at all, since what it does can sit anywhere in it. +const DEFAULT_FULL_VIEWS: &[(&str, &str)] = &[("bash", "/command")]; const DESTRUCTIVE_ID: &str = "destructive"; const REQUESTED_ID: &str = "requested"; @@ -47,10 +50,25 @@ const DEFAULT_REQUESTED_QUESTION: &str = "Is `tool_call` something the user aske /// request, so a confirmation ("yes, go ahead") carries what it confirms — /// else the run's own prompts. When none of these exists (compaction left /// no user message and the run has no prompt), the call is denied and the -/// reason asks for the request to be restated. Small argument -/// values are sent in full; a string over 2,000 characters is shortened to -/// its head and tail around an explicit `[truncated N chars]` marker, and a -/// call whose arguments are still over 12,000 characters is denied. +/// reason asks for the request to be restated. +/// +/// **Arguments are sent whole when they fit**: within +/// [`with_max_argument_chars`](Self::with_max_argument_chars) (default +/// 48,000 characters, serialized) and the decision backend's own token +/// limits ([`Capabilities`](super::Capabilities)). A call that does not fit +/// has its longest strings shortened — only as far as needed, each to its +/// head and tail around an explicit `[truncated N chars]` marker — and the +/// state then lists what was cut under `tool_call.omitted` +/// (`[{"argument": "/content", "chars": N}]`, JSON pointers), so the model +/// knows it judges a partial view. Paths, names and flags are short and +/// stay whole. Arguments registered with +/// [`with_full_view`](Self::with_full_view) — by default the built-in +/// `bash` tool's `command` — are never shortened: a call whose full-view +/// argument does not fit is denied ("could not review it in full"), since a +/// dangerous line can sit anywhere in a command. A call that does not fit +/// even with every other long string cut to 400 characters is denied too. +/// Calls are never judged in pieces: what one piece does can depend on +/// another. /// /// **Fails closed.** A decision-model error or timeout (default 5 s), a /// missing or malformed answer, and a non-finite probability all deny. @@ -84,6 +102,9 @@ pub struct ToolGate { destructive_threshold: f64, requested_threshold: f64, checks: Vec<(String, String, f64)>, + max_argument_chars: usize, + default_full_views: bool, + full_views: Vec<(String, String)>, } impl ToolGate { @@ -97,6 +118,9 @@ impl ToolGate { destructive_threshold: 0.5, requested_threshold: 0.7, checks: Vec::new(), + max_argument_chars: DEFAULT_MAX_ARGUMENT_CHARS, + default_full_views: true, + full_views: Vec::new(), } } @@ -164,24 +188,67 @@ impl ToolGate { self } + /// Most characters of a call's arguments, serialized, sent to the + /// decision model (default 48,000). Arguments over it — or over the + /// backend's token limits, whichever is smaller — are shortened as + /// described on [`ToolGate`]. A larger budget means fewer partial views, + /// and more input tokens (cost, latency) on large calls. + /// + /// Panics on 0. + pub fn with_max_argument_chars(mut self, chars: usize) -> Self { + assert!(chars > 0, "a tool gate's argument budget must be above 0"); + self.max_argument_chars = chars; + self + } + + /// Never shorten `argument` of `tool`: it is judged whole, or the call is + /// denied. `argument` is a JSON pointer into the arguments (`"/command"`, + /// `"/edits"` for everything under it, `""` for all of them). Use it + /// for arguments that are executed — a script, a query, a program — and + /// whose every line can matter. The built-in `bash` tool's `/command` is + /// one by default ([`without_default_full_views`](Self::without_default_full_views) + /// removes it). + /// + /// Panics on a pointer that is neither empty nor starts with `/`. + pub fn with_full_view(mut self, tool: impl Into, argument: impl Into) -> Self { + let argument = argument.into(); + assert!( + argument.is_empty() || argument.starts_with('/'), + "a full-view argument is a JSON pointer (\"/command\", or \"\" for all), \ + got {argument:?}" + ); + self.full_views.push((tool.into(), argument)); + self + } + + /// Drop the default full view (`bash`'s `/command`); ones added with + /// [`with_full_view`](Self::with_full_view) stay, whatever the order. + pub fn without_default_full_views(mut self) -> Self { + self.default_full_views = false; + self + } + + /// The full-view pointers for `tool`. + fn full_views_for(&self, tool: &str) -> Vec<&str> { + let defaults = DEFAULT_FULL_VIEWS + .iter() + .filter(|_| self.default_full_views) + .map(|&(t, a)| (t, a)); + let custom = self + .full_views + .iter() + .map(|(t, a)| (t.as_str(), a.as_str())); + defaults + .chain(custom) + .filter(|&(t, _)| t == tool) + .map(|(_, a)| a) + .collect() + } + /// Judge one call: what the gate decides when installed. For testing a /// gate's settings outside the loop, with a call built by /// [`ToolCallRequest::new`]. pub async fn decide(&self, call: &ToolCallRequest<'_>) -> ToolDecision { - let arguments = shorten(call.args); - let size = serde_json::to_string(&arguments).map_or(usize::MAX, |s| s.chars().count()); - if size > MAX_ARGS_CHARS { - tracing::warn!( - tool = call.tool_name, - size, - "tool gate denied a call: arguments too large" - ); - return ToolDecision::Deny(format!( - "Tool gate: this call's arguments are too large to evaluate ({size} characters \ - after shortening long strings; the limit is {MAX_ARGS_CHARS}), so it was not run. \ - The gate fails closed; split the work into smaller calls or ask the user." - )); - } let Some(user_request) = call.user_request() else { tracing::warn!( tool = call.tool_name, @@ -194,10 +261,7 @@ impl ToolGate { .into(), ); }; - let state = json!({ - "user_request": truncate_middle(&user_request, MAX_REQUEST_CHARS), - "tool_call": { "tool": call.tool_name, "arguments": arguments }, - }); + let user_request = truncate_middle(&user_request, MAX_REQUEST_CHARS); let mut questions = vec![ ( @@ -219,6 +283,60 @@ impl ToolGate { questions.push((id.clone(), Question::noul(q.as_str()))); } + // The strictest limits in a fallback chain, so no member skips the call. + let caps = self.model.chain_capabilities(); + let full_views = self.full_views_for(call.tool_name); + let fits = |view: &View| { + let chars = + serde_json::to_string(&view.arguments).map_or(usize::MAX, |s| s.chars().count()); + chars <= self.max_argument_chars + && check_token_limits( + &state(&user_request, call.tool_name, view), + &questions, + &caps, + ) + .is_ok() + }; + let view = match fit_arguments(call.args, &full_views, fits) { + Ok(view) => view, + Err(Unfit::FullView { argument, chars }) => { + tracing::warn!( + tool = call.tool_name, + argument, + chars, + "tool gate denied a call: a full-view argument does not fit" + ); + return ToolDecision::Deny(format!( + "Tool gate: `{}`'s `{argument}` argument is {chars} characters, more than \ + the gate can review in full, and it does not judge part of it, so this \ + call was not run. The gate fails closed; split it into smaller calls or \ + ask the user.", + call.tool_name + )); + } + Err(Unfit::TooLarge { chars }) => { + tracing::warn!( + tool = call.tool_name, + chars, + "tool gate denied a call: arguments too large" + ); + return ToolDecision::Deny(format!( + "Tool gate: this call's arguments are too large to evaluate ({chars} \ + characters even with every long string cut to {MIN_KEPT_CHARS}), so it \ + was not run. The gate fails closed; split the work into smaller calls or \ + ask the user." + )); + } + }; + if !view.omitted.is_empty() { + tracing::debug!( + tool = call.tool_name, + omitted = view.omitted.len(), + "tool gate judges a shortened call" + ); + } + let state = state(&user_request, call.tool_name, &view); + let model = self.model.clone().with_timeout(self.timeout); let eval = match model.evaluate(state, questions).await { Ok(eval) => eval, @@ -281,21 +399,206 @@ fn fmt_p(p: Option) -> String { p.map_or_else(|| "unknown".into(), |p| format!("{p:.2}")) } -/// The arguments as the gate shows them: every value kept, except strings -/// longer than `MAX_STRING_CHARS`, which keep their head and tail around an -/// explicit `[truncated N chars]` marker — so a command's last line, and -/// every short field such as a path, stays visible. -fn shorten(v: &Value) -> Value { +/// The arguments as the gate shows them, and what was cut from them. +#[derive(Debug)] +struct View { + arguments: Value, + /// `(JSON pointer, characters cut)` per shortened string. + omitted: Vec<(String, usize)>, +} + +#[derive(Debug, PartialEq)] +enum Unfit { + /// A full-view argument is what does not fit. + FullView { argument: String, chars: usize }, + /// Nothing more can be cut. + TooLarge { chars: usize }, +} + +/// The state the decision model sees. +fn state(user_request: &str, tool: &str, view: &View) -> Value { + let mut call = json!({ "tool": tool, "arguments": view.arguments }); + if !view.omitted.is_empty() { + call["omitted"] = view + .omitted + .iter() + .map(|(argument, chars)| json!({ "argument": argument, "chars": chars })) + .collect(); + } + json!({ "user_request": user_request, "tool_call": call }) +} + +/// The largest view of `args` that `fits`: whole when it fits; else every +/// string outside `full_views` longer than some cap `c` cut to `c` (head and +/// tail), with `c` as large as fits (at least `MIN_KEPT_CHARS`). +fn fit_arguments( + args: &Value, + full_views: &[&str], + fits: impl Fn(&View) -> bool, +) -> Result { + let whole = View { + arguments: args.clone(), + omitted: Vec::new(), + }; + if fits(&whole) { + return Ok(whole); + } + let longest = longest_string(args, &mut String::new(), full_views, false); + if longest > MIN_KEPT_CHARS && fits(&cut(args, full_views, MIN_KEPT_CHARS)) { + // Binary search the largest cap that fits: `lo` fits, `hi` does not. + let (mut lo, mut hi) = (MIN_KEPT_CHARS, longest); + while hi - lo > 1 { + let mid = lo + (hi - lo) / 2; + if fits(&cut(args, full_views, mid)) { + lo = mid; + } else { + hi = mid; + } + } + return Ok(cut(args, full_views, lo)); + } + // It does not fit even cut to the minimum. Blame a full-view argument when + // cutting it too would have made it fit. + let mut views = Vec::new(); + collect_full_views(args, &mut String::new(), full_views, false, &mut views); + if let Some((argument, chars)) = views.into_iter().max_by_key(|&(_, chars)| chars) { + if chars > MIN_KEPT_CHARS && fits(&cut(args, &[], MIN_KEPT_CHARS)) { + return Err(Unfit::FullView { argument, chars }); + } + } + let chars = serde_json::to_string(&cut(args, &[], MIN_KEPT_CHARS).arguments) + .map_or(usize::MAX, |s| s.chars().count()); + Err(Unfit::TooLarge { chars }) +} + +fn is_full_view(pointer: &str, full_views: &[&str]) -> bool { + full_views.iter().any(|v| { + v.is_empty() + || pointer == *v + || (pointer.starts_with(*v) && pointer.as_bytes().get(v.len()) == Some(&b'/')) + }) +} + +/// `pointer` extended by one JSON-pointer segment (RFC 6901 escaping). +fn push_segment(pointer: &mut String, segment: &str) -> usize { + let len = pointer.len(); + pointer.push('/'); + pointer.push_str(&segment.replace('~', "~0").replace('/', "~1")); + len +} + +/// Visit every string in `v` with its pointer and whether it is full-view. +fn for_each_string( + v: &Value, + pointer: &mut String, + full_views: &[&str], + inherited: bool, + f: &mut impl FnMut(&str, &str, bool), +) { + let here = inherited || is_full_view(pointer, full_views); match v { - Value::String(s) if s.chars().count() > MAX_STRING_CHARS => { - Value::String(truncate_middle(s, MAX_STRING_CHARS)) + Value::String(s) => f(pointer, s, here), + Value::Array(items) => { + for (i, item) in items.iter().enumerate() { + let len = push_segment(pointer, &i.to_string()); + for_each_string(item, pointer, full_views, here, f); + pointer.truncate(len); + } } - Value::Array(items) => Value::Array(items.iter().map(shorten).collect()), Value::Object(map) => { - Value::Object(map.iter().map(|(k, v)| (k.clone(), shorten(v))).collect()) + for (k, item) in map { + let len = push_segment(pointer, k); + for_each_string(item, pointer, full_views, here, f); + pointer.truncate(len); + } + } + _ => {} + } +} + +/// The longest string outside the full views, in characters. +fn longest_string(v: &Value, pointer: &mut String, full_views: &[&str], inherited: bool) -> usize { + let mut longest = 0; + for_each_string(v, pointer, full_views, inherited, &mut |_, s, full| { + if !full { + longest = longest.max(s.chars().count()); + } + }); + longest +} + +/// Every string inside a full view, with its pointer and length. +fn collect_full_views( + v: &Value, + pointer: &mut String, + full_views: &[&str], + inherited: bool, + out: &mut Vec<(String, usize)>, +) { + for_each_string(v, pointer, full_views, inherited, &mut |p, s, full| { + if full { + out.push((p.to_string(), s.chars().count())); + } + }); +} + +/// `args` with every string outside `full_views` longer than `cap` cut to +/// its head and tail. +fn cut(args: &Value, full_views: &[&str], cap: usize) -> View { + fn go( + v: &Value, + pointer: &mut String, + full_views: &[&str], + inherited: bool, + cap: usize, + omitted: &mut Vec<(String, usize)>, + ) -> Value { + let here = inherited || is_full_view(pointer, full_views); + match v { + Value::String(s) if !here => { + let len = s.chars().count(); + if len > cap { + omitted.push((pointer.clone(), len - 2 * (cap / 2))); + Value::String(truncate_middle(s, cap)) + } else { + v.clone() + } + } + Value::Array(items) => Value::Array( + items + .iter() + .enumerate() + .map(|(i, item)| { + let len = push_segment(pointer, &i.to_string()); + let out = go(item, pointer, full_views, here, cap, omitted); + pointer.truncate(len); + out + }) + .collect(), + ), + Value::Object(map) => Value::Object( + map.iter() + .map(|(k, item)| { + let len = push_segment(pointer, k); + let out = go(item, pointer, full_views, here, cap, omitted); + pointer.truncate(len); + (k.clone(), out) + }) + .collect(), + ), + other => other.clone(), } - other => other.clone(), } + let mut omitted = Vec::new(); + let arguments = go( + args, + &mut String::new(), + full_views, + false, + cap, + &mut omitted, + ); + View { arguments, omitted } } /// **Deprecated since 0.25**, to be removed in a later release: install the @@ -353,3 +656,63 @@ impl crate::RunHooks for GateHooks { self.0.decide(call).await } } + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn full_views_match_whole_segments_only() { + assert!(is_full_view("/command", &["/command"])); + assert!(is_full_view("/edits/0/new", &["/edits"])); + assert!(!is_full_view("/commands", &["/command"])); + assert!(!is_full_view("/command", &["/commands"])); + assert!(is_full_view("/anything", &[""])); + assert!(!is_full_view("/command", &[])); + } + + #[test] + fn pointers_escape_tilde_and_slash() { + let args = json!({"a/b": "x".repeat(1_000), "c~d": "y".repeat(1_000)}); + let view = cut(&args, &[], MIN_KEPT_CHARS); + let mut cut: Vec<_> = view.omitted.iter().map(|(p, _)| p.as_str()).collect(); + cut.sort_unstable(); + assert_eq!(cut, ["/a~1b", "/c~0d"]); + // An escaped full view protects its key. + let view = super::cut(&args, &["/a~1b"], MIN_KEPT_CHARS); + assert_eq!(view.omitted.len(), 1); + assert_eq!(view.arguments["a/b"], args["a/b"]); + } + + #[test] + fn fit_cuts_the_longest_strings_first() { + // Budget 3,000: the short string stays whole; the long one takes the cut. + let args = json!({"short": "s".repeat(900), "long": "l".repeat(10_000)}); + let fits = |v: &View| serde_json::to_string(&v.arguments).unwrap().len() <= 3_000; + let view = fit_arguments(&args, &[], fits).unwrap(); + assert_eq!(view.arguments["short"], args["short"]); + assert_eq!(view.omitted.len(), 1); + assert_eq!(view.omitted[0].0, "/long"); + let len = serde_json::to_string(&view.arguments).unwrap().len(); + assert!((2_900..=3_000).contains(&len), "{len}"); + } + + #[test] + fn fit_blames_the_full_view_that_does_not_fit() { + let args = json!({"command": "c".repeat(10_000), "note": "n".repeat(10_000)}); + let fits = |v: &View| serde_json::to_string(&v.arguments).unwrap().len() <= 3_000; + assert_eq!( + fit_arguments(&args, &["/command"], fits).unwrap_err(), + Unfit::FullView { + argument: "/command".into(), + chars: 10_000 + } + ); + // With room for nothing at all, it is simply too large. + let none = |_: &View| false; + assert!(matches!( + fit_arguments(&args, &["/command"], none), + Err(Unfit::TooLarge { .. }) + )); + } +} diff --git a/src/decision/mod.rs b/src/decision/mod.rs index 368128b..12857ba 100644 --- a/src/decision/mod.rs +++ b/src/decision/mod.rs @@ -634,6 +634,26 @@ impl DecisionModel { self.backend.get().capabilities() } + /// The primary's capabilities with the strictest token limits of the + /// whole chain: a state sized to these is sent to every member, never + /// skipped by one with smaller limits. + pub(crate) fn chain_capabilities(&self) -> Capabilities { + let strictest = |a: Option, b: Option| match (a, b) { + (Some(a), Some(b)) => Some(a.min(b)), + (a, b) => a.or(b), + }; + let mut caps = self.capabilities(); + for member in &self.fallbacks { + let c = member.capabilities(); + caps.max_request_tokens = strictest(caps.max_request_tokens, c.max_request_tokens); + caps.max_state_and_question_tokens = strictest( + caps.max_state_and_question_tokens, + c.max_state_and_question_tokens, + ); + } + caps + } + /// The price of `usage` on the model that reported it. pub(crate) fn cost_usd(&self, model: &str, usage: &DecisionUsage) -> Option { let cost = match &self.pricing { diff --git a/src/decision/question.rs b/src/decision/question.rs index 67538b0..18f83e8 100644 --- a/src/decision/question.rs +++ b/src/decision/question.rs @@ -404,33 +404,43 @@ impl Request { } q.validate(id, caps)?; } - let state_tokens = estimate_json_tokens(&self.state); - let question_tokens: Vec = self - .questions - .iter() - .map(|(_, q)| estimate_json_tokens(q)) - .collect(); - if let Some(max) = caps.max_request_tokens { - let total = state_tokens + question_tokens.iter().sum::(); - if total > max { - return Err(DecisionError::Invalid(format!( - "request: about {total} tokens (estimated) exceeds the backend's \ - {max}-token request limit; trim the state or split the questions" - ))); - } + check_token_limits(&self.state, &self.questions, caps) + } +} + +/// The token-limit checks of [`Request::validate`] (on its estimate): the +/// whole request, and the state plus the longest question. Also used by the +/// tool gate to fit a call's arguments before asking. +pub(crate) fn check_token_limits( + state: &Value, + questions: &[(String, Question)], + caps: &Capabilities, +) -> Result<(), DecisionError> { + let state_tokens = estimate_json_tokens(state); + let question_tokens: Vec = questions + .iter() + .map(|(_, q)| estimate_json_tokens(q)) + .collect(); + if let Some(max) = caps.max_request_tokens { + let total = state_tokens + question_tokens.iter().sum::(); + if total > max { + return Err(DecisionError::Invalid(format!( + "request: about {total} tokens (estimated) exceeds the backend's \ + {max}-token request limit; trim the state or split the questions" + ))); } - if let Some(max) = caps.max_state_and_question_tokens { - let longest = question_tokens.iter().copied().max().unwrap_or(0); - if state_tokens + longest > max { - return Err(DecisionError::Invalid(format!( - "state: about {} tokens (estimated) for the state plus the longest \ - question exceeds the backend's {max}-token limit", - state_tokens + longest - ))); - } + } + if let Some(max) = caps.max_state_and_question_tokens { + let longest = question_tokens.iter().copied().max().unwrap_or(0); + if state_tokens + longest > max { + return Err(DecisionError::Invalid(format!( + "state: about {} tokens (estimated) for the state plus the longest \ + question exceeds the backend's {max}-token limit", + state_tokens + longest + ))); } - Ok(()) } + Ok(()) } fn estimate_json_tokens(v: &T) -> usize { diff --git a/tests/decision_agent_test.rs b/tests/decision_agent_test.rs index de7e54a..74db865 100644 --- a/tests/decision_agent_test.rs +++ b/tests/decision_agent_test.rs @@ -734,36 +734,45 @@ async fn gate_fails_closed_on_timeout() { // --- long arguments -------------------------------------------------------- +/// The arguments and `omitted` list of the first request the gate sent. +fn gate_saw(mock: &MockBackend) -> (serde_json::Value, serde_json::Value) { + let call = &mock.requests()[0].state["tool_call"]; + (call["arguments"].clone(), call["omitted"].clone()) +} + +/// `decide` on a call outside the loop, with a run prompt to judge against. +async fn decide(gate: &ToolGate, tool: &str, args: serde_json::Value) -> ToolDecision { + let prompts = [Message::user("rewrite the notes")]; + gate.decide(&ToolCallRequest::new("c1", tool, &args).with_run_prompts(&prompts)) + .await +} + +fn denied_with(decision: &ToolDecision, needle: &str) -> bool { + matches!(decision, ToolDecision::Deny(reason) if reason.contains(needle)) +} + #[tokio::test] -async fn gate_sees_the_tail_of_a_long_command() { - let heredoc = format!( - "cat > notes.txt <<'EOF'\n{}\nEOF\nrm -rf /srv/data", - "lorem ipsum ".repeat(700) - ); - assert!(heredoc.len() > 8_000); - let mock = gate_answers(0.9, 0.1); +async fn gate_sends_a_large_write_whole_when_it_fits() { + // A full-file rewrite of 30k characters: judged on all of it. + let content = format!("{}MIDDLE-MARK{}", "x".repeat(15_000), "y".repeat(15_000)); + let mock = gate_answers(0.1, 0.9); let (tools, ran) = make_tools(1); let agent = Agent::from_provider( - calls_rm_with(json!({"command": heredoc})), + calls_rm_with(json!({"path": "notes.md", "content": content})), ModelConfig::mock(), ) .with_tools(tools) .with_tool_gate(ToolGate::new(model(&mock))); - run(agent, "write some notes").await; - assert!(ran.lock().unwrap().is_empty()); - let shown = mock.requests()[0].state["tool_call"]["arguments"]["command"] - .as_str() - .unwrap() - .to_string(); - assert!(shown.ends_with("rm -rf /srv/data"), "the tail is visible"); - assert!(shown.starts_with("cat > notes.txt"), "the head is visible"); - assert!(shown.contains("[truncated "), "the cut is explicit"); - assert!(shown.len() < heredoc.len()); + run(agent, "rewrite the notes").await; + assert_eq!(ran.lock().unwrap().len(), 1, "allowed"); + let (args, omitted) = gate_saw(&mock); + assert_eq!(args["content"], content.as_str(), "sent whole"); + assert!(omitted.is_null(), "nothing omitted: {omitted}"); } #[tokio::test] -async fn gate_keeps_short_fields_whole_and_shortens_long_content() { - let content = format!("{}END-OF-FILE", "x".repeat(9 * 1024)); +async fn gate_shortens_only_as_needed_and_says_what_it_cut() { + let content = format!("{}END-OF-FILE", "x".repeat(100_000)); let mock = gate_answers(0.1, 0.9); let (tools, ran) = make_tools(1); let agent = Agent::from_provider( @@ -774,12 +783,19 @@ async fn gate_keeps_short_fields_whole_and_shortens_long_content() { .with_tool_gate(ToolGate::new(model(&mock))); run(agent, "update the config").await; assert_eq!(ran.lock().unwrap().len(), 1, "allowed"); - let args = &mock.requests()[0].state["tool_call"]["arguments"]; + let (args, omitted) = gate_saw(&mock); assert_eq!(args["path"], "/etc/app/config.toml"); assert_eq!(args["append"], false); let shown = args["content"].as_str().unwrap(); - assert!(shown.ends_with("END-OF-FILE")); - assert!(shown.contains("[truncated ")); + assert!(shown.ends_with("END-OF-FILE"), "the tail is visible"); + assert!(shown.contains("[truncated "), "the cut is explicit"); + // Cut only as far as the 48,000-character budget needs, not to a fixed size. + let sent = serde_json::to_string(&args).unwrap().chars().count(); + assert!((46_000..=48_000).contains(&sent), "{sent}"); + let cut = omitted[0]["chars"].as_u64().unwrap() as usize; + assert_eq!(omitted[0]["argument"], "/content"); + assert_eq!(omitted.as_array().unwrap().len(), 1); + assert!((50_000..56_000).contains(&cut), "cut {cut}"); // Positive control: the executed call still got the full content. assert_eq!( ran.lock().unwrap()[0]["content"].as_str().unwrap().len(), @@ -787,10 +803,97 @@ async fn gate_keeps_short_fields_whole_and_shortens_long_content() { ); } +#[tokio::test] +async fn gate_fits_the_backends_token_limits() { + // A backend that takes at most 4,000 tokens of state plus question + // (about 16,000 bytes): the call is cut to fit it, not refused by it. + let mock = gate_answers(0.1, 0.9).with_capabilities( + Capabilities::new(QuestionKind::all()).with_token_limits(None, Some(4_000)), + ); + let gate = ToolGate::new(model(&mock)); + let content = "z".repeat(60_000); + let decision = decide( + &gate, + "write_file", + json!({"path": "a.md", "content": content}), + ) + .await; + assert!(matches!(decision, ToolDecision::Allow), "{decision:?}"); + let state = &mock.requests()[0].state; + let bytes = serde_json::to_vec(state).unwrap().len(); + assert!((12_000..16_000).contains(&bytes), "{bytes}"); + assert_eq!(state["tool_call"]["omitted"][0]["argument"], "/content"); +} + +#[tokio::test] +async fn gate_reads_a_command_whole_or_not_at_all() { + // `bash`'s command is a full view by default: a dangerous line can sit + // anywhere in it, so it is never judged from its head and tail. + let command = |filler: usize| { + format!( + "echo start\n{}rm -rf ~\n{}echo done", + "true\n".repeat(filler), + "true\n".repeat(filler) + ) + }; + let mock = gate_answers(0.9, 0.1); + let gate = ToolGate::new(model(&mock)); + + // Fits: sent whole, the middle line included. + let decision = decide(&gate, "bash", json!({"command": command(2_000)})).await; + assert!(denied_with(&decision, "looks destructive"), "{decision:?}"); + let (args, omitted) = gate_saw(&mock); + assert!(args["command"].as_str().unwrap().contains("rm -rf ~")); + assert!(omitted.is_null()); + + // Does not fit: denied without asking. + let decision = decide(&gate, "bash", json!({"command": command(10_000)})).await; + assert!( + denied_with(&decision, "`bash`'s `/command` argument is"), + "{decision:?}" + ); + assert!(denied_with(&decision, "review in full"), "{decision:?}"); + assert_eq!( + mock.request_count(), + 1, + "the oversized command was never sent" + ); + + // Without the default full view the same call is judged shortened. + let gate = ToolGate::new(model(&mock)).without_default_full_views(); + decide(&gate, "bash", json!({"command": command(10_000)})).await; + assert_eq!(mock.request_count(), 2); + assert!(mock.requests()[1].state["tool_call"]["omitted"].is_array()); +} + +#[tokio::test] +async fn gate_full_views_cover_custom_tools_and_whole_subtrees() { + let mock = gate_answers(0.1, 0.9); + let gate = ToolGate::new(model(&mock)).with_full_view("sql", "/queries"); + let big = "SELECT 1; ".repeat(3_000); // 30k each: the two together do not fit + // Everything under `/queries` is a full view; `/note` is not. + let args = json!({"queries": [big.clone()], "note": big.clone()}); + let decision = decide(&gate, "sql", args).await; + assert!(matches!(decision, ToolDecision::Allow), "{decision:?}"); + let (args, omitted) = gate_saw(&mock); + assert_eq!(args["queries"][0], big.as_str(), "full view kept whole"); + assert_eq!(omitted[0]["argument"], "/note"); + // Another tool's arguments of the same name are not full views. + let decision = decide(&gate, "other", json!({"queries": [big.clone(), big]})).await; + assert!(matches!(decision, ToolDecision::Allow), "{decision:?}"); +} + +#[tokio::test] +#[should_panic(expected = "JSON pointer")] +async fn gate_full_view_must_be_a_pointer() { + let _ = ToolGate::new(model(&MockBackend::new())).with_full_view("bash", "command"); +} + #[tokio::test] async fn gate_denies_arguments_too_large_to_read() { - let fields: serde_json::Map = (0..10) - .map(|i| (format!("f{i}"), json!("y".repeat(5_000)))) + // 200 strings of 500 characters: even cut to 400 each, over the budget. + let fields: serde_json::Map = (0..200) + .map(|i| (format!("f{i}"), json!("y".repeat(500)))) .collect(); let mock = gate_answers(0.0, 1.0); let (tools, ran) = make_tools(1); @@ -803,6 +906,22 @@ async fn gate_denies_arguments_too_large_to_read() { assert_eq!(mock.request_count(), 0, "denied before asking"); } +#[tokio::test] +async fn gate_budget_is_configurable() { + let mock = gate_answers(0.1, 0.9); + let gate = ToolGate::new(model(&mock)).with_max_argument_chars(100_000); + let content = "q".repeat(90_000); + decide( + &gate, + "write_file", + json!({"path": "a", "content": content}), + ) + .await; + let (args, omitted) = gate_saw(&mock); + assert_eq!(args["content"], content.as_str()); + assert!(omitted.is_null()); +} + // --- what the user asked ----------------------------------------------------- /// Requested only when the state shows the confirmation of a deletion. @@ -1770,3 +1889,26 @@ async fn a_tree_gate_rechecks_a_rewritten_call() { ); assert_eq!(reqs[1].state["tool_call"]["arguments"]["path"], "/srv/data"); } + +#[tokio::test] +async fn gate_fits_the_strictest_limits_in_a_fallback_chain() { + // An unlimited primary that fails, then a fallback with SystemOne's + // 32k-token limit. 46,000 CJK characters are under the character budget + // but over the fallback's tokens (3 bytes each): sized to the primary + // alone, the fallback would skip the call unsent and the gate deny. + let primary = MockBackend::new().push_error(DecisionError::http(503, "unavailable")); + let fallback = gate_answers(0.1, 0.9).with_capabilities( + Capabilities::new(QuestionKind::all()).with_token_limits(Some(64_000), Some(32_000)), + ); + let chain = DecisionModel::from_backend(primary, "primary").or(model(&fallback)); + let gate = ToolGate::new(chain); + let decision = decide( + &gate, + "write_file", + json!({"path": "a.md", "content": "汉".repeat(46_000)}), + ) + .await; + assert!(matches!(decision, ToolDecision::Allow), "{decision:?}"); + assert_eq!(fallback.request_count(), 1, "the fallback was asked"); + assert!(fallback.requests()[0].state["tool_call"]["omitted"].is_array()); +}