Skip to content

fix: keep composer predictions focused on the next user message #441

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
iililiiilili merged 1 commit into from
Nov 4, 2026
Merged

fix: keep composer predictions focused on the next user message #441

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 22 additions & 9 deletions docs/platformclaw/web-ingress-runtime.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,13 +72,26 @@ configuration or unsupported API adapters yield no suggestion. No external LLM,
Agent Loop, tools, skills, memory retrieval, retries, or model fallback are used.
Normal conversation defaults and fallback settings remain unchanged.

Inference receives only the last six user/assistant text messages, each capped
at 300 Unicode characters. This bounded context supports two or three exchanges
without copying attachments, reasoning, tool output, or an unbounded transcript.
Output is capped at 64 tokens and 300 characters. The browser/BFF prediction deadline
includes Gateway history access. There is one active
prediction per Gateway process and no prediction queue. Ordinary model calls
preempt predictions. No vLLM load metrics are assumed; the single-slot policy is
Inference receives only the last eight user/assistant text messages, retaining
the first 2,000 Unicode code points from each. Attachments, reasoning, and tool
output are excluded. The bounded transcript is serialized as JSON in one fresh
user message with the upstream prediction system prompt, so the model predicts
the next user message.
The request uses a 160-token output limit. Tagged reasoning is removed before
display, and suggestions over 500 UTF-16 code units are rejected.

Predictions request thinking off through the selected model's existing
compatibility settings. Agent `thinkingDefault` controls ordinary agent runs;
`reasoningDefault` controls reasoning visibility. Neither configures this
prediction request. If the serving API requires an explicit thinking switch,
check the selected model row under
`models.providers.<provider>.models`: `reasoning` describes its capability, and
`compat.thinkingFormat` selects the request dialect. Set these only to match the
actual serving API; an opaque model alias does not identify its thinking controls.

The browser/BFF prediction deadline includes Gateway history access. There is
one active prediction per Gateway process and no prediction queue. Ordinary
model calls preempt predictions. No vLLM load metrics are assumed; the single-slot policy is
the conservative admission control. A multi-Gateway deployment needs a shared
admission owner before enabling this feature across instances.

Expand All @@ -88,7 +101,7 @@ pending prediction work immediately, before waiting for the send acknowledgment
or the next model call. Enter retains its existing behavior.
Cancellation closes the HTTP request to the model; actual GPU cancellation
depends on the company's serving stack and must be measured there. A server
that continues inference after disconnect can still consume up to the 64-token
that continues inference after disconnect can still consume up to the 160-token
budget. Real chat latency and GPU effects require a company-environment load test.

Knox, Cron, API adapters, and background runs do not trigger prediction inference.
Expand Down Expand Up @@ -122,7 +135,7 @@ Prediction responses retain `text` and add a fixed `reason` and integer
| `destination_rejected` | The model destination did not pass the private-address restriction. |
| `provider_http_error` | The provider returned a non-success response; `providerStatus` contains its numeric HTTP status. The response body is never copied. |
| `response_missing`, `response_invalid`, `response_empty`, `response_oversized` | The provider response could not be used. |
| `no_suggestion`, `format_rejected` | The model abstained or its output failed the existing format/length checks. |
| `no_suggestion`, `format_rejected` | The model abstained or its suggestion exceeded the output length limit. |
| `history_ineligible`, `history_changed` | The authorized text context was unsuitable or changed during inference. |
| `timeout`, `cancelled` | The operation reached its deadline or was cancelled. A browser cancellation can have no Response body. |
| `access_error`, `unauthenticated`, `forbidden`, `preferences_error`, `history_error`, `gateway_error` | A request, access, settings, transcript, or internal Gateway operation failed. |
Expand Down
14 changes: 13 additions & 1 deletion docs/plugins/sdk-subpaths.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,11 @@ are private-local.
| `plugin-sdk/provider-onboard` | Private-local after July 2026; Onboarding config patch helpers |
| `plugin-sdk/global-singleton` | Private-local after July 2026; Process-local singleton/map/cache helpers |
| `plugin-sdk/group-activation` | Private-local after July 2026; Narrow group activation mode and command parsing helpers |

The bundled `buildOpenAICompletionsParams` helper accepts read-only model
snapshots, including nested compatibility metadata. It does not mutate the
model; existing mutable model descriptors remain valid inputs.

</Accordion>

Provider usage snapshots normally report one or more quota `windows`, each with
Expand Down Expand Up @@ -327,7 +332,7 @@ Use `isLoopbackHost(host)` when a plugin must accept only the local machine. It
| `plugin-sdk/media-store` | Narrow media store helpers such as `saveMediaBuffer` and `saveMediaStream` |
| `plugin-sdk/media-generation-runtime` | Private-local after July 2026; Shared media-generation failover helpers, candidate selection, and missing-model messaging |
| `plugin-sdk/media-understanding` | Deprecated compatibility facade for media-understanding provider types and helpers; new providers register through the injected plugin API and keep request helpers plugin-owned |
| `plugin-sdk/text-chunking` | Outbound text and offset-preserving range chunking, markdown chunking/render helpers, quote-aware HTML tag tokenization, markdown table conversion, directive-tag stripping, and safe-text utilities |
| `plugin-sdk/text-chunking` | Outbound text and offset-preserving range chunking, markdown chunking/render helpers, quote-aware HTML tag tokenization, markdown table conversion, reasoning-tag stripping, directive-tag stripping, and safe-text utilities |
| `plugin-sdk/speech` | Private-local after July 2026; Speech provider types plus provider-facing directive, registry, validation, OpenAI-compatible TTS builder, and speech helper exports |
| `plugin-sdk/speech-core` | Private-local after July 2026; Shared speech provider types, registry, directive, normalization, and speech helper exports |
| `plugin-sdk/speech-settings` | Lightweight TTS config resolution and normalization primitives without provider registries or synthesis runtime |
Expand Down Expand Up @@ -359,6 +364,13 @@ Use `isLoopbackHost(host)` when a plugin must accept only the local machine. It
| `plugin-sdk/test-state` | Repo-local isolated OpenClaw state, config, workspace, environment, and auth-profile fixtures for plugin tests |
| `plugin-sdk/test-fixtures` | Repo-local generic CLI runtime capture, sandbox context, skill writer, agent-message, system-event, module reload, bundled plugin path, terminal-text, chunking, auth-token, and typed-case fixtures |
| `plugin-sdk/test-node-mocks` | Repo-local focused Node builtin mock helpers for use inside Vitest `vi.mock("node:*")` factories |

Use `stripReasoningTagsFromText(text, { mode: "hide" })` when unterminated
reasoning must remain hidden. It preserves literal Markdown code examples
and removes final-answer markers after classifying reasoning in the original
text. The default `strict` mode retains its recovery of malformed unclosed
opening tags; `preserve` also keeps its existing recovery behavior.

</Accordion>

<Accordion title="Memory subpaths">
Expand Down
Loading