Docs refresh + zero-config smart routing

bench/README.md

@@ -0,0 +1,76 @@
+# whittle benchmarks
+
+The receipts behind the README's numbers. Every figure is regenerable from this repo.
+
+Three tiers, in increasing order of realism - every number regenerable from
+this repo (`go run ./bench` for the deterministic rows; the prose row needs the
+model sidecar), reductions on an estimated-token basis (labeled).
+
+### 1. Synthetic corpus (ours - headline per content class)
+
+Authored fixtures in [`corpus/`](corpus/), designed to exercise each
+strategy and its guarantees. Full table: [`REPORT.md`](REPORT.md).
+
+| class | representative result |
+|---|---|
+| JSON (uniform/sparse/nested) | 57% - lossless, byte-exact reconstruction |
+| repetitive logs | 97% - omissions marked and exactly accounted |
+| terminal progress streams | 99% - final frame, rune-safe |
+| code / config (py, go, yaml) | **0% by design - skipped, never touched** |
+| prose | 30-40% extractive, fidelity-guarded (needs the model sidecar; not part of the deterministic `go run ./bench` output) |
+
+### 2. Side-by-side on headroom's data
+
+Inputs frozen from [headroom](https://github.com/headroomlabs-ai/headroom)'s own
+benchmark generators (Apache-2.0; pinned commit, seed 42 - they check in no
+corpora, so we froze what their numbers are computed on; `corpus_headroom/`,
+PROVENANCE.md). Both tools ran on identical bytes, defaults only, measured with
+the same tokenizer. Full table + methodology: [`SIDEBYSIDE.md`](SIDEBYSIDE.md).
+
+| | headroom-ai 0.30.0 | whittle 0.2.1 |
+|---|---|---|
+| aggregate token reduction (10 files, 116.5k tokens) | **41.8%** | 36.5% |
+| - conversation / agent-transcript JSON (3 files) | 2.1% | **5.4%** |
+| - bulk data arrays (7 files) | **48.3%** | 41.6% |
+| fidelity of that reduction | includes lossy row-dropping (recoverable via headroom's resident runtime) | **byte-exact lossless** on every file |
+| median latency, in-process (same files) | 2.93 ms | 2.36 ms |
+
+Read it straight: on the aggregate, headroom-ai's defaults compress ~5 points
+more - by dropping rows whittle refuses to drop. The category split shows where
+each position pays: on conversation-shaped content (the shape agent tool
+outputs actually take) whittle leads while staying lossless; on bulk data
+arrays headroom-ai's lossy sampling buys its margin. Latency is near parity.
+Which trade you want is the whole point of this project.
+
+### 3. Real-world: customer-service agents (two independent evaluations)
+
+Customer-service agents are whittle's strong-fit workload - their tool outputs
+are structured JSON on essentially every call, so the compressor engages on
+**100% of tool outputs**, all losslessly. Two evaluations, corroborating at
+different scales:
+
+**Breadth** - [Salesforce APIGen-MT-5k](https://huggingface.co/datasets/Salesforce/APIGen-MT-5k),
+5,000 verified multi-turn sessions: **22% tool-output reduction (o200k) at zero
+measured information loss**, verified two ways - mechanically lossless on
+**15,846 / 15,846** compressed items, and a blinded 4-judge panel finding
+**0 / 120** material loss on the lossy prose path (honeypot-validated, Gwet AC2
+0.96-1.00). [`datasets/salesforce_customer_support/`](datasets/salesforce_customer_support/)
+
+**Depth** - Sierra's [τ-bench](datasets/tau_bench/) (retail + airline),
+counterfactual replay of reference trajectories: **22.0% / 23.3% reduction, every
+record reconstructed field-for-field** (all 16 flight-search results recovered
+exactly). This eval isolates whittle's real structural value-add - on
+**multi-record results the columnar re-encoding goes beyond compact JSON: +24%
+on small flight searches, +45% on an 80-row result**, and the advantage grows
+with result size. [`datasets/tau_bench/`](datasets/tau_bench/)
+
+The product metric is **token/context reduction**: fewer tool-output tokens,
+removed from every later turn's context, compounding as the session grows.
+Dollar impact is a separate, caching-dependent question both reports publish in
+full: under prompt caching the same cut is ~3-5% of session cost (cheap
+cache-reads dominate the bill), so token savings and dollar savings are not the
+same thing.
+
+*(Both measured on the shared compression engine `content-aware-router` v0.1.0;
+whittle's `json_crusher` is lossless-only. See each report's provenance note.)*
+

cmd/whittle/main.go

@@ -95,7 +95,8 @@ route flags:
   -install         register the router as a background launchd agent (opt-in), then exit
   -uninstall       stop + unregister the router launchd agent, then exit
   env WHITTLE_ROUTER_UPSTREAM    upstream API (default api.anthropic.com)
-  env WHITTLE_ROUTER_MODEL_URL   classifier sidecar URL (unset → smart mode off)
+  env WHITTLE_ROUTER_MODEL_URL   classifier sidecar (default http://127.0.0.1:45872,
+                                 auto-detected; set to "off" to disable ML signals)
 
 env:
   WHITTLE_MODEL_URL        enable the ML prose path (model sidecar URL)

cmd/whittle/setup.go

@@ -255,22 +255,19 @@ func routerInstall() {
 		fmt.Println("whittle:", err)
 		return
 	}
-	// Point smart mode at the compress sidecar (it also serves /v1/route/*). If the
-	// sidecar is down, the router fails open to heuristics-only — never blocks.
+	// Smart mode is automatic (the router probes the standard sidecar address and
+	// degrades to heuristics if absent) — no env plumbing needed here.
 	plist := fmt.Sprintf(`<?xml version="1.0" encoding="UTF-8"?>
 <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
 <plist version="1.0"><dict>
   <key>Label</key><string>%s</string>
   <key>ProgramArguments</key><array><string>%s</string><string>route</string></array>
-  <key>EnvironmentVariables</key><dict>
-    <key>WHITTLE_ROUTER_MODEL_URL</key><string>http://%s</string>
-  </dict>
   <key>RunAtLoad</key><true/>
   <key>KeepAlive</key><true/>
   <key>StandardOutPath</key><string>%s/logs/router.log</string>
   <key>StandardErrorPath</key><string>%s/logs/router.log</string>
 </dict></plist>
-`, routerAgentLabel, self, modelAddr, dir, dir)
+`, routerAgentLabel, self, dir, dir)
 	if err := os.WriteFile(routerPlistPath(), []byte(plist), 0o644); err != nil {
 		fmt.Println("whittle: router agent registration failed:", err)
 		return

demo/hero-feed.sh

@@ -0,0 +1,90 @@
+#!/bin/sh
+# hero-feed.sh — whittle live-watch choreography for the README hero GIF.
+#
+# Emits a pre-sized, append-only ANSI stream (no scrolling, one clear) that
+# replays a real captured session: four agent turns routed to the right model,
+# tool outputs carved losslessly, then a `whittle stats`-style session summary.
+#
+# All color comes from this script's stdout (VHS `Type` can't emit ANSI).
+# Reproducible from a clone: `sh demo/hero-feed.sh`.
+#
+# Palette (flat 16-color): reset \033[0m · bold \033[1m · dim \033[2m ·
+# dim-italic \033[2;3m · green \033[32m · cyan \033[36m · blue \033[34m ·
+# red \033[31m. Percent signs are doubled (%%) for printf.
+
+# ── stream ──────────────────────────────────────────────────────────────────
+sleep 0.6
+printf '\033[2m🪓 whittle · live — carving outputs, routing requests · local · fail-open\033[0m\n'
+
+sleep 0.8
+printf '\n'
+
+# turn 1 — trivia drops to the cheapest model
+sleep 0.3
+printf '\033[2m»\033[0m "bigest country in europe?"\n'
+sleep 0.5
+printf '  \033[32m▸\033[0m casual-easy   \033[2mopus\033[0m\033[1;32m→haiku\033[0m\033[2m   cplx −0.27 · 52,848-tok ctx\033[0m\n'
+
+sleep 1.7
+printf '\n'
+
+# turn 2 — hard reasoning stays on the strongest model; its outputs get carved
+sleep 0.3
+printf '\033[2m»\033[0m "What is in embed.go file?"\n'
+sleep 0.6
+printf '  \033[34m▸\033[0m requested   \033[1;34mopus→opus\033[0m  \033[1;34mkept\033[0m\033[2m   dom computer-science 0.98\033[0m\n'
+
+sleep 1.8
+printf '\033[2m  🪓 \033[0m\033[1membed.go\033[0m\033[2m        source · \033[0m\033[32muntouched\033[0m\033[2m — code is never cut\033[0m\n'
+sleep 0.5
+printf '\033[2m  🪓 \033[0m\033[1mbuild.log\033[0m\033[2m       144 lines, 2 errors buried\033[0m\n'
+sleep 0.4
+printf '\033[31m       ERROR migrate failed: relation "users" does not exist\033[0m\n'
+sleep 0.4
+printf '\033[2;3m       … [118 lines omitted]\033[0m\n'
+sleep 0.4
+printf '\033[31m       ERROR shutdown: connection reset by peer\033[0m\n'
+sleep 0.4
+printf '       \033[2m1,904\033[0m \033[1;32m→ 47\033[0m\033[2m tok · \033[0m\033[1;32m−97%%\033[0m\n'
+sleep 0.8
+printf '\033[2m  🪓 \033[0m\033[1mtest-run.log\033[0m\033[2m     670 lines · 20,595\033[0m \033[1;32m→ 472\033[0m\033[2m tok · \033[0m\033[1;32m−98%%\033[0m\n'
+
+sleep 1.8
+printf '\n'
+
+# turn 3 — another trivial ask drops to haiku
+sleep 0.2
+printf '\033[2m»\033[0m "share command to rename a file"\n'
+sleep 0.5
+printf '  \033[32m▸\033[0m casual-easy   \033[2mopus\033[0m\033[1;32m→haiku\033[0m\033[2m   cplx −0.45\033[0m\n'
+
+sleep 1.2
+printf '\n'
+
+# turn 4 — mid-complexity drafting lands on sonnet
+sleep 0.2
+printf '\033[2m»\033[0m "draft a slack message for this content"\n'
+sleep 0.5
+printf '  \033[36m▸\033[0m casual-medium   \033[2mopus\033[0m\033[1;36m→sonnet\033[0m\033[2m   dom other 1.00\033[0m\n'
+
+# hold the full stream, then one clear into the session summary
+sleep 1.6
+sleep 1.2
+clear
+
+# ── closer (session summary; `whittle stats` house style) ────────────────────
+sleep 0.3
+printf '\033[1m🪓 whittle\033[0m \033[2m· session summary\033[0m\n'
+sleep 0.6
+printf '\n'
+printf '  \033[2mcontext\033[0m     \033[1;32m21,980\033[0m tokens carved      \033[2mlossless or marked · code untouched\033[0m\n'
+sleep 0.6
+printf '  \033[2mrouting\033[0m     4 requests   \033[32m●\033[0m haiku 2   \033[36m●\033[0m sonnet 1   \033[34m●\033[0m opus 1 held\n'
+sleep 0.6
+printf '  \033[2msavings\033[0m     ~$0.05 measured           \033[2mon billed tokens · this session\033[0m\n'
+sleep 0.6
+printf '\n'
+printf '  \033[2mnever cuts what doesn'\''t come back · github.com/firstops-dev/whittle\033[0m\n'
+
+# loop-pause hold, then the GIF loops
+sleep 4.0

demo/hero.tape

@@ -0,0 +1,20 @@
+Output demo/hero.gif
+Set Theme "Catppuccin Mocha"
+Set FontSize 18
+Set Width 1000
+Set Height 600
+Set Padding 24
+Set TypingSpeed 45ms
+Set CursorBlink false
+
+# Define whittle as this session's replay, hidden, so the visible command reads
+# as the real subcommand (`whittle watch`) driving whittle's own captured output.
+Hide
+Type "whittle() { sh demo/hero-feed.sh; }; clear"
+Enter
+Show
+
+Sleep 500ms
+Type "whittle watch"
+Enter
+Sleep 24s

docs/ROUTER.md

@@ -166,9 +166,14 @@ prints errors and cost-lint warnings.
 ## 5. Signals
 
 Two small models power three signals. Both run in the sidecar; the Go engine
-holds only thresholds. If the sidecar is down or smart mode is off, ML leaves
-evaluate false and the keyword/context heuristics still route (a route whose
-match *depended* on an errored signal never fires — so a `not` over an
+holds only thresholds. Smart mode is automatic: the router probes the standard
+sidecar address (`127.0.0.1:45872`, installed by `whittle setup`) — no
+configuration. An absent sidecar reads as smart-off (quiet; heuristics still
+route, and signals activate the moment it appears); a present-but-broken one is
+surfaced loudly (`err` traces + `ml-degraded`). Set
+`WHITTLE_ROUTER_MODEL_URL=off` to disable ML entirely, or point it elsewhere to
+override. When ML is unavailable, leaves evaluate false (a route whose match
+*depended* on an errored signal never fires — so a `not` over an
 unavailable signal can't invert fail-open; the log gains an `ml-degraded` tag).
 
 ### 5.1 `domain` — subject classification with mass thresholding

docs/compression.md

@@ -0,0 +1,97 @@
+# Compression: reference
+
+The full per-content-type contract, the ML prose path, architecture, performance, and limitations.
+
+## What it does per content type
+
+| detected | strategy | guarantee |
+|---|---|---|
+| JSON | minify + columnar reshape (union schema, typed CSV, nested flattening, constant factoring) | **lossless** - reconstructs byte-exact; rows are never dropped |
+| logs / build output | keep errors, warnings, stack traces, summaries | lossy but **marked** - `... [N lines omitted]`, exact accounting |
+| terminal | ANSI strip + CR-overwrite collapse (progress bars → final frame) | what the terminal actually displayed; rune-safe |
+| markdown file reads | structure-aware: prose compressed by the model, **code fences / tables / lists / headings passed through byte-exact** | code never reaches the model |
+| source code | untouched | routed away from every lossy path |
+| prose | extractive model (optional) with fidelity guards: entity protection, whole-token deletion, negation preservation | fails open on any guard trip |
+
+Every path is wrapped in fail-open guardrails: empty-output, expansion (both
+byte- and token-honest), panic recovery. The worst case is always "not
+compressed", never "corrupted".
+
+
+## The ML prose path (installed by `whittle setup`, optional by design)
+
+`whittle setup` installs and supervises the prose sidecar automatically: the
+Python source ships inside the binary, setup builds its venv, and the daemon
+keeps it running (GPU auto-selected: CUDA > Apple MPS > CPU). If `python3` is
+missing, setup says so and whittle runs deterministic-only - nothing breaks.
+
+"Optional" means the deterministic strategies never depend on it. Only if you
+use whittle as a bare library or `whittle serve` **without** running setup do
+you wire it manually (LLMLingua-2 + whittle's fidelity guards - see
+[model/](../model/)):
+
+```
+cd model && python -m venv .venv && .venv/bin/pip install -r requirements.txt
+.venv/bin/uvicorn app:app --port 45872
+export WHITTLE_MODEL_URL=http://127.0.0.1:45872
+```
+
+
+## Architecture
+
+```
+Claude Code ──PostToolUse (HTTP)──▶ whittle daemon  (:45871, launchd-kept)
+                                       │
+                                       ├─ dispatch ▶ json · log · terminal · markdown
+                                       │            (deterministic, in-process)
+                                       └─ prose ─▶ model sidecar (:45872, optional GPU)
+                                       │
+   whittle_get(id) ◀──MCP tool────────┘  reduced originals, retrievable on demand
+
+  ── opt-in, separate process, off by default ──
+Claude Code ──ANTHROPIC_BASE_URL──▶ whittle route (:45873) ──▶ Anthropic API
+                                      policy-based model-tier routing;
+                                      rewrites the model, not history; fail-open
+```
+
+The compression surfaces are one resident daemon, three ways in (hook · HTTP ·
+MCP) - compression happens where output is born, off the model-request path
+entirely. The model router is a **separate, opt-in** process on the request path;
+you run it only when you want tier routing.
+
+
+## Performance
+
+Deterministic strategies are pure CPU, single static binary, zero allocatable
+model state (Apple M-series, `go test -bench`):
+
+| input | size | latency |
+|---|---|---|
+| JSON array, 200 rows, pretty-printed | ~21 KB | ~1.0 ms |
+| terminal progress stream | ~12 KB | ~3.9 ms |
+| build log, 800 lines | ~56 KB | ~21 ms |
+
+The hook runs after the tool call completes, so this cost is **off the LLM
+request path entirely** - model-call latency is unchanged. (These are absolute
+in-path budgets on whittle's own corpus; for tool-vs-tool latency on identical
+inputs see the Benchmarks side-by-side above and [`bench/SIDEBYSIDE.md`](../bench/SIDEBYSIDE.md).) The optional ML
+prose path is capped by a fail-open budget (default 1.5 s) and never blocks
+beyond it.
+
+
+## Known limitations
+
+- **Replacements must match the tool's output shape.** Claude Code schema-validates
+  `updatedToolOutput` and silently keeps the original on mismatch; whittle rebuilds
+  the tool's own response shape around the compressed text (verified live on Claude
+  Code 2.1.203 — see [docs/hook-output-cap.md](hook-output-cap.md), which also
+  documents why the once-assumed 10k output cap does NOT apply to replacements).
+- **Prose needs the sidecar.** Without it, prose and markdown docs pass through
+  unchanged; deterministic strategies are unaffected.
+- **launchd is macOS-only.** Linux runs the daemon under systemd (unit in the README install notes).
+- **Prose latency ceiling** (default 100000 chars) trades a hard cap for predictable
+  in-path latency, sized for GPU/MPS inference (measured ~0.07s + 0.04s/KB on Apple
+  silicon: 100KB ≈ 4s, within the 8s prose timeout). CPU-only machines run ~8x
+  slower (~0.3s/KB) and should lower `WHITTLE_PROSE_MAX_CHARS` to ~12000, or large
+  prose burns the timeout and passes through unchanged.
+

docs/why-write-time.md

@@ -0,0 +1,50 @@
+# Why write-time compression, not read-time
+
+Whittle's core position, in full. (The [README](../README.md) carries the short version.)
+
+Context compressors typically integrate with coding agents as **request-path
+proxies**: your agent's base URL is redirected through a local server that
+rewrites conversation history at *read time*, on every LLM call. That position
+forces hard problems - prompt-cache stabilization (mutating history invalidates
+cached prefixes), per-call re-compression, terminating your API traffic (keys,
+system prompts and all), and a resident runtime that must stay up or your agent
+goes down with it. It also makes lossy compression the default, backed by a
+retrieval loop: the runtime is guaranteed present, so dropped content can be
+served back on demand.
+
+Whittle takes the other position: it is a **PostToolUse hook**. Each tool output
+is compressed **once, at the moment it is born**, before it ever enters
+conversation history. Everything else follows from that choice:
+
+- **Savings compound.** A tool output lives in context for every subsequent
+  turn. Tokens removed at write-time are removed from *every* later call -
+  no per-call rework, no cache surgery, because history is never mutated.
+- **No trust expansion.** A hook sees one tool output at a time, locally, with
+  zero credentials. Nothing terminates your API traffic.
+- **Failure is free.** The hook fails open; if whittle is down or declines, the
+  agent proceeds with the original output. A gateway outage is an agent outage.
+- **The loss budget is honest.** A read-time proxy can afford recoverable lossy
+  compression - its resident runtime serves dropped content back when the model
+  asks. Whittle keeps no runtime in your request path; reduced outputs carry a tiny
+  retrieval pointer served by the local daemon (`whittle_get`), and lossless
+  transforms carry nothing at all - lossless-or-marked stays the construction,
+  recovery is the safety net, never the license.
+
+The hook is whittle's default surface, not its only one: **library**
+(`whittle.New`) → **HTTP service** (`whittle serve`) → **hook adapters**
+(Claude Code PostToolUse today; Cursor, Codex, OpenCode adapters on the
+roadmap) - and the same library embeds in gateways or pipelines if that is
+where you need it. The position is the point: compression happens where output
+is born, whatever surface delivers it there.
+
+**This argument is about compression, not about routing.** Whittle's opt-in
+[model router](../README.md#model-routing-opt-in) *is* a request-path proxy - but it exists
+for a different job (send each request to the cheapest capable tier), and it is
+deliberately the minimal kind. It rewrites only the model field and reconciles
+capabilities; it never rewrites conversation history, so it inherits none of the
+prompt-cache surgery above. It forwards your credentials untouched rather than
+terminating them, and it fails open to your original request, so an outage is a
+passthrough, not an agent outage. The read-time-compression proxy is the pattern
+whittle rejects; a minimal, history-preserving, fail-open router is a different
+tool that happens to share the address bar.
+