Repository navigation
Memory LLM completion lane bypasses the host model catalog (cannot honor OpenRouter provider routing or other model config) #901
Description
Summary
memory-lancedb-pro's LLM completion lane builds a minimal OpenAI chat-completions request in its own client (src/llm-client.ts, createApiKeyClient) and posts it directly to the configured llm.baseURL. This bypasses the host model catalog and runtime entirely. When the base URL is OpenRouter, there is no way to apply OpenRouter's provider routing object, or any other catalog-level model configuration, so OpenRouter free-routes the model across all providers.
Where
src/llm-client.ts, createApiKeyClient(). The request carries only model, messages, temperature (plus a conditional chat_template_kwargs). The same client backs every completion lane (smart extraction, reflection, dedup/merge, admission-control, the memory upgrader).
Impact
On OpenRouter, an open-weight model such as openai/gpt-oss-120b is served by many providers with very different latency, throughput, quality, and rate limits. Because the plugin's calls carry no provider preference, they free-route (observed across SiliconFlow, DeepInfra, Together, Amazon Bedrock, NovitaAI, and Weights & Biases). A latency-sensitive memory and reflection lane cannot be pinned to fast providers (for example Cerebras or Groq), cannot avoid providers that return 429s, and cannot be attributed to the host app in OpenRouter's dashboard. Provider routing is only the first example: any catalog-level model setting (sampling params, request headers, request policy) is equally unavailable to this lane.
Proposed direction
Rather than adding a one-off passthrough per setting, let the completion lane route through the host managed LLM runtime using a catalog model reference, the way other OpenClaw plugins already do (for example lossless-claw's summaryModel and expansionModel, which set model: <ref> and let the host resolve and complete it). The plugin already holds a host-runtime handle (api.runtime.agent) for its recall sub-agent, so the completion lane could use the same managed path when running under OpenClaw with a catalog model reference, and fall back to the existing standalone client for CLI and non-host usage.
Benefit: the full catalog entry applies automatically (provider routing, sampling params, headers, auth profiles, app attribution, and anything added later), instead of re-patching the client for each new field.
OpenRouter provider routing reference: https://openrouter.ai/docs/features/provider-routing
Environment
- memory-lancedb-pro 1.1.0-beta.11 (master)
- OpenClaw 2026.6.1, host-native, Node 24
- Provider: OpenRouter (chat-completions)
Activity
AliceLJY commentedon Jun 19, 2026
Thanks for mapping the completion lane. This is a valid architecture gap.
The plugin currently has a direct OpenAI-compatible client so CLI / non-host usage can keep working, but under OpenClaw the memory LLM lane should be able to use the host-managed model runtime/catalog when the configured model is a catalog reference, then fall back to the direct client outside host runtime.
I would track this as an enhancement/design item rather than a one-off OpenRouter provider passthrough, because provider routing, sampling params, headers, auth profile, and attribution all belong to the catalog-level contract.
lylepratt commentedon Jun 29, 2026
also very interested in this. we deploy openclaw as a service behind the scenes and route all LLM requests through our centralized backend. The fact that this plugin can't use openclaw model definitions makes it very difficult for us to use it.
gorkem2020 commentedon Jul 7, 2026
Thanks @AliceLJY, that framing is exactly right and it matches the direction we were hoping for. Tracking it as a catalog-contract design item rather than a per-field passthrough is the durable fix: provider routing is just the first symptom, and sampling params, headers, auth profile, and app attribution all belong to the same catalog-level contract.
The seam looks clean because the plugin already holds a host-runtime handle (
api.runtime.agent) for the recall sub-agent, so the completion lane can take the managed path when the configured model is a catalog reference under OpenClaw, and fall back to the existing standalone OpenAI-compatible client for CLI and non-host usage. That keeps CLI and standalone users working while letting host deployments inherit the full model definition automatically.The centralized-backend case @lylepratt raises is the same need from the other end: when everything routes through a managed OpenClaw runtime, a memory lane that cannot see the host model catalog is the one component that escapes the routing policy. So this would unblock both the provider-pinning use case and the centralized-gateway one.
We run this lane in production on OpenRouter and feel the free-routing directly, so we would be glad to help move it forward, including prototyping the managed-runtime path if that is useful. Happy to align on the interface first (how the lane detects a catalog reference versus a raw base URL, and where the fallback boundary sits) before any code.
Update (2026-07-15): Ran into this concretely today: a lane-affinity admission call handed the core-style id
openrouter/anthropic/claude-opus-4-8straight to the direct client and got a 400 ("not a valid model ID") on every call, while the reflection distiller consumed the same string fine because it rides the host catalog.That confirmed the structural gap described above, so we have drafted a PR (#947) that adds a
llm.transport: "direct" | "host"knob."host"routes the completion lane through the host-managed runtime LLM surface (api.runtime.llm.complete) when available, feature-detecting it and falling back to today's direct client with a warning otherwise. Model strings pass through unmodified on the host path; the existingopenrouter/prefix-strip stays as a direct-path-only accommodation.feat(llm-client): add a host-managed completion transport alongside t…
feat(llm-client): add a host-managed completion transport alongside t…
feat(llm-client): add a host-managed completion transport alongside t…
feat(llm-client): add a host-managed completion transport alongside t…
gorkem2020 commentedon Sep 12, 2026
Update: PR #947 merged on 2026-08-29 and adds the opt-in
llm.transport: "host"path, which routes the extraction, admission, dedup, merge and upgrader completions through the host model runtime (runtime.llm.complete) with the reasoning effort forwarded explicitly. The direct client stays the default, so existing setups are unchanged. The core side prerequisite, reasoning passthrough on runtime completions, landed in openclaw#108759 and ships in OpenClaw 2026.9.2.From my side this issue is resolved by that PR, so it can be closed whenever convenient.