Leave room for the reply, calibrate against the server, survive an unsupported parameter

[t-timms]

Why

Two failure modes seen while benchmarking local models (details in the Qwen3.8-27B run notes; the token-estimator half was started in #232).

1. Context headroom. On a hard 32K llama-server window, compaction fired only when the real prompt was already at the limit. The reply was then cut off in the middle of a tool call and llama-server answered HTTP 500 Failed to parse tool call arguments as JSON on every retry, ending the session (llm_error). Server-side evidence: stop processing: n_tokens = 32767, truncated = 1 (5 times in one KAT-REAP-50 run, 2 more at 65535 with a 64K window). Even after #232 the estimate is ~1.1x low on real Qwen3.8 requests, and nothing left room for the reply.

2. One unsupported optional parameter killed every call. reasoning_effort: medium with openai/gpt-oss-20b made LiteLLM raise UnsupportedParamsError: openai does not support parameters: ['reasoning_effort'] on every call, so all 8 tasks of a benchmark arm ended llm_error after 0 iterations.

Changes

• Conversation: completion_reserve_tokens (new setting, default 4096, capped at a quarter of the window) is subtracted from the budget compaction uses (usable_tokens); observe_prompt_tokens learns the estimate's overhead (template markup, tool schemas, tokenizer drift) from the provider's reported prompt size as a fixed offset, and the agent loop feeds it after each response (both input_tokens and prompt_tokens usage shapes). A reported size above twice the window is ignored; the overhead is capped at half the window and cleared when the estimate already over-counts.
• Agent loop: a tool-call parse failure with the conversation above 60% of the window is treated like a context overflow (the existing one-shot compaction + retry) instead of retrying the identical prompt.
• LLMClient._acompletion: on UnsupportedParamsError (matched by class name, so it works with mocked LiteLLM) drop the rejected optional parameter with one warning, retry once, and leave it out of later calls. Essential keys (model, messages, timeout, tools, tool_choice, stream) are never dropped; other errors propagate unchanged.
• Wiring: CLI, TUI, architect sub-conversation, benchmark runner. Docs/profile text updated; CHANGELOG.

Behavior change to review

completion_reserve_tokens defaults to 4096, so compaction happens up to 4096 tokens (at most 25% of the window) earlier than before for every model. Set it to 0 to restore the old behaviour.

Verification

• New tests: tests/test_context_headroom.py (calibration incl. large early overhead, both usage shapes, reserve arithmetic and cap, compaction firing earlier with a reserve, truncated-tool-call matching near/far from the limit, setting validation) and tests/test_llm_unsupported_params.py (parsing, essential keys never dropped, retry-once + later calls stripped + single warning, unrelated errors and repeated rejections propagate). 25 tests.
• Full suite on this branch: 6438 passed, 6 skipped, 3 deselected, 0 failed. ruff check . and ruff format --check . clean.

Not verified

• Not run against a live llama-server end to end (the GPU was not in use); the behaviour is covered by unit tests built from the observed error text and usage shapes. The 60%-of-window guard for the parse-failure path and the 4096 default are judgement calls from one hardware/model combination.
• Provider-specific token accounting (e.g. cached-token reporting) was not exercised beyond the plausibility guards.

[github-actions]

🤖 Godspeed Review

LLM review skipped (no API key configured).

Security

No secrets detected in changed files.