Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -785,7 +785,7 @@ A model call that stops making progress is cut, so an agent isn't held by a prov
- **A resident's streamed call**, and an MCPL server's streamed inference request, get membrane's progress bounds where the adapter's stream reports content (`AnthropicAdapter`, `BedrockAdapter`). Before content, a call ends 60 s after its send, or, once a thinking block has started, at the later of that and the block's allowance, its `max_tokens` at 87 tokens a second. Once content has begun, it ends after 150 s without a provider event, or after the block's allowance when the last event was a thinking block's start and the allowance is longer. Membrane's README gives the rules whole.
- **Every streamed call the framework makes** also gets a flat bound from its last provider event: the larger of a setup term plus that allowance and 1,862 s (`FLAT_MINIMUM_MS`). The setup term is the larger of the before-content and between-events bounds, 150 s by default. So on an adapter with progress bounds the flat bound never runs out first. On one without, such as `OpenAICompatibleAdapter`, it's the only progress bound, and an SSE keepalive comment doesn't restart it, though a data frame that parses as an event does, a provider's own ping object included.
- **A whole non-streamed call**, a compression's, the reply judge's, or an MCPL server's non-streamed inference request, ends the setup term plus its allowance after its send, or, when it names no `max_tokens`, at the larger of the setup term and the flat minimum, unless its caller passed a `progressTimeoutMs` of its own. Its caller gets the `timeout` error. The event gate's own 20 s judge timeout ends the reply judge's call first at the defaults. Context-manager takes a cut compression call as a transient failure: the lane pauses and tries again, so compression that never lands within its bound shows as a lane that keeps pausing, not as a failed turn, and is a reason to raise the bound. On an adapter whose `complete()` streams under the hood, such as membrane's OpenAI Responses adapter in subscription mode, membrane renews the bound at each parsed provider event, so there it runs from the last event, as a stream's flat bound does.
- **A turn's call a bound cuts** fails as a `timeout`, and `DefaultErrorPolicy` gives it one fresh call, 1 s later, whatever other failures its turn had, except in a turn sent again after its retries ran out, before one of its rounds has run or had its tool calls cut off, whose failures get no retries. A fresh call that's cut too, with no other failure between, ends the turn, and its `[inference-failed]` note says the call timed out, that the call was made again and timed out again, and the reason. For a bound's cut the reason names the bound that ran out and the host's setting for it. A deadline expiry membrane throws (the Anthropic adapter's own watchdogs, the SDK's request timeout, a runtime's fetch timeout on a silent connection) counts the same toward the fresh call and the turn's end, and its reason is its own message, which names no setting. A bound's cut isn't `retryable`, so a host's own `errorPolicy` that retries only retryable errors gives it no fresh call; an expiry membrane throws stays `retryable`, as membrane made it.
- **A turn's call a bound cuts** fails as a `timeout`, and `DefaultErrorPolicy` gives it one fresh call, 1 s later, whatever other failures its turn had, except in a turn sent again after its retries ran out, before one of its rounds has run or had its tool calls cut off, whose failures get no retries. A fresh call that's cut too, with no other failure between, ends the turn. If the turn got partway (see the `[inference-failed]` note below), its note says the call timed out, that the call was made again and timed out again, and the reason; a turn that did nothing gets no note, and its stderr line and `failures.log` record give the reason. For a bound's cut the reason names the bound that ran out and the host's setting for it. A deadline expiry membrane throws (the Anthropic adapter's own watchdogs, the SDK's request timeout, a runtime's fetch timeout on a silent connection) counts the same toward the fresh call and the turn's end, and its reason is its own message, which names no setting. A bound's cut isn't `retryable`, so a host's own `errorPolicy` that retries only retryable errors gives it no fresh call; an expiry membrane throws stays `retryable`, as membrane made it.
- **A stream that keeps sending content** isn't ended by any of these, since content is progress.

These bounds don't replace the deadlines below them: a silent connection still meets the runtime's own fetch timeout (360 s on Bun 1.4.4) and, on `AnthropicAdapter`, the adapter's 600 s first-event and idle watchdogs, so on a silent connection a bound raised past those is met by them first.
Expand Down Expand Up @@ -823,6 +823,8 @@ A caller's stop and an operator's quiesce abandon also reach a turn that has no

In XML tool mode, or with a membrane that gives a tool round no `roundContent`, such a round's speech reaches a channel only when its stream completes. When the stream ends any other way (a failure, a stop, a tool that ends the turn, an error policy's retry, a context-budget or tool-result-guard restart, a quiesce abandon or `stop()`), the rounds' words are already in the agent's context, and a `[speech-not-posted]` notice tells the agent that nothing it wrote before its tool calls in those rounds was posted, so it can send what should go out again. It's written once for those rounds: where the turn ends, or before a retry's or a restart's request is sent. A turn its own `skip_reply` ends, a silent turn, a round still waiting on its tools when its stream ends (which never reaches the context) and the rounds of a stream that completes get none. In XML tool mode with thinking on, a round whose text was only thinking gets the notice too, since the framework has only the round's raw text there. A failed turn's `[inference-failed]` note says that its earlier rounds ran when any round reached the context with its results, and that a round's tool calls were started and may have taken effect when any of its streams ended before that round's results came back; a caller's stop says the latter in its `[turn-interrupted]` note too. A turn that fails, or that a caller stops, writes no `[delivered]` receipt, so when its rounds' plain speech was delivered live (native tool mode, in locus or hybrid routing), its `[inference-failed]` note, or the stop's `[turn-interrupted]` note, says that speech was handed on for delivery as written and that no `[delivered]` line was written for it. When some of its speech was still being routed as the turn ended, the note also says it may yet be posted and to check the channel before sending it again. A delivery that finished undelivered gets neither: a channel delivery's `[discord-send-failed]` note, or a refused `>>>` envelope's bounce, already says what happened. A quiesce abandon and `stop()` write neither note.

A failed turn writes its `[inference-failed]` note only when it got partway: when one of its rounds reached the agent's context with its results, or had its tool calls started and cut off. A turn that failed before either writes no note. Everything else its failure does is as before: stderr's `[inference-failed] agent=…` line, `failures.log`'s `inference-exhausted` record, the consecutive-failure streak, the `inference:exhausted` trace, the hard-down alert, and, when the poison-history breaker sheds history, its `[refusal-rewind]` note, which carries the failure's reason. So a failure that comes back at every turn's first call, such as a provider that's down or a context that's too long, no longer adds a note per turn to the agent's context. A turn that got partway gets one note, which says how far it got and the reason, and, when its last call timed out, that a reply still owed can be given now. `SUPPRESS_INFERENCE_FAILED_MARKER=1` leaves that note out. Notes already in a store stay.

## API Server

WebSocket server for external clients (UIs, scripts, other agents).
Expand Down
59 changes: 59 additions & 0 deletions changelog.d/979-inference-failed-telemetry.changed.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
- **Residents:** a turn whose inference is exhausted before it got partway
writes no `[inference-failed]` note into the agent's context. Partway means
one of its rounds reached the context with its results, or had its tool
calls started and cut off. Everything else its failure does is as
before: stderr's `[inference-failed] agent=…` line, `failures.log`'s
`inference-exhausted` record, the consecutive-failure streak, the
`inference:exhausted` trace, the hard-down alert, and the poison-history
breaker, whose `[refusal-rewind]` note carries the failure's reason when
it sheds history. So a failure that comes back at every turn's first
call, such as a provider that's down or a context that's too long, no longer
adds a note per turn to the context it's sent with (#979).
- A turn that got partway still gets one note, which says how far it got
(its earlier rounds ran, or a round's tool calls were started and may
have taken effect) and the reason, without the advice
"If this recurs with the same cause, change approach rather than retrying
identically (e.g. drop an oversized attachment or an unsupported
setting)." A timed-out turn's note keeps "If a reply is still owed from
that turn, you can give it now."
- The reply judge reads the agent's context, so it no longer sees an
`[inference-failed]` note for a turn that did nothing.
- `SUPPRESS_INFERENCE_FAILED_MARKER=1` now leaves out only that note, since
no other is written.
- Notes already in a store stay as they are.
- What this changes in what's published:
- README, a turn's call a bound cuts: "A fresh call that's cut too, with
no other failure between, ends the turn, and its `[inference-failed]`
note says the call timed out, that the call was made again and timed
out again, and the reason" now reads that the turn's note says so if
the turn got partway, and that a turn that did nothing gets no note.
- README, after the failure and stop notes: a new paragraph says when the
note is written, as above.
- `FrameworkConfig.exhaustedTurnRearmMs`'s doc: "A re-armed turn's
transient failure writes no `[inference-failed]` marker unless a round
of it ran or had its tool calls cut off; any other failure of it writes
one." Now a failure of a re-armed turn writes a note only when a round
of it ran or had its tool calls cut off, as any turn's failure does.
- 0.22.1: "a non-retryable failure that states a wait keeps its terminal
disposition (its marker and failure count)". Now such a failure keeps
its failure count, and its marker only when its turn got partway.
- 0.29.0: "A re-armed turn's transient failure writes no
`[inference-failed]` marker unless it got partway, so the agent's
context keeps the one the first exhaustion wrote; a failure of another
kind writes its own." Now no failure of a turn that didn't get partway
writes one, the first exhaustion's included, whatever its kind.
- 0.31.0: "A turn that ends on a timeout says so in its
`[inference-failed]` note: the model call timed out, or its last call
did … Other failures keep their wording and advice." Now only a turn
that got partway has the note, which says its last call timed out, and
no failure's note carries the advice.
- The `[assistant-persist]` stderr line for a thinking-only reply the
store refuses ended "— the turn produced no output. See the
refusal/inference-failed record for why.", though a turn whose earlier
rounds ran produced output, and a refusal never wrote an
`[inference-failed]` note. It now ends "(… no text and no tool_use).
If it was refused, see its [inference-refusal] line and failures.log
record."
- Earlier entries' wording of the note for a turn that did nothing ("the
model call failed … and produced no response, so nothing was sent")
describes only notes already in stores.
12 changes: 5 additions & 7 deletions src/agent.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2152,8 +2152,7 @@ export class Agent {
// no content. The Anthropic API rejects empty text blocks with 400
// "messages: text content blocks must be non-empty"; when such a block
// reaches a live activation request the agent cannot complete ANY turn
// ([inference-failed] on every attempt) until the offending message ages
// out of the window. Empty text blocks occur naturally: tool-only turns,
// until the offending message ages out of the window. Empty text blocks occur naturally: tool-only turns,
// delivery-failure placeholders, silent/skip turns. Non-text blocks pass
// through unchanged, so tool pairing is unaffected. (Field-observed
// 2026-07-10 on a resident agent: one empty block muted the agent's live
Expand Down Expand Up @@ -2549,16 +2548,15 @@ export class Agent {
// in the latest assistant message cannot be modified", unrecoverable by
// retry (labclaude, 2026-07-27: hard down ~4h on exactly this pair).
//
// Refused turns are surfaced by the refusal path (stderr, failures.log,
// the [inference-failed] notice); the store does not also need a
// content-free artifact of them. Loud, never silent.
// Refused turns are surfaced by the refusal path; the store does not
// also need a content-free artifact of them. Loud, never silent.
if (content.length > 0 && content.every(
(b) => b.type === 'thinking' || b.type === 'redacted_thinking',
)) {
console.error(
`[assistant-persist] agent=${this.name}: refusing to store a thinking-only assistant ` +
`message (${content.length} block(s), no text and no tool_use) — the turn produced no ` +
`output. See the refusal/inference-failed record for why.`,
`message (${content.length} block(s), no text and no tool_use). If it was refused, see ` +
`its [inference-refusal] line and failures.log record.`,
);
return undefined;
}
Expand Down
Loading
Loading