server-api.md

HTTP Server API

localm's inference server exposes an OpenAI-compatible REST API plus a small set of localm-specific management endpoints.

localm serve <model>          # default 127.0.0.1:8642, auto-bumps when busy

Authentication: open (no key required) until an API key exists - the LOCALM_API_KEY env var, an owner key generated by the launcher/GUI, or any scoped key minted with localm key create / the GUI's key manager / POST /v1/keys. Once any key is configured, every route below requires Authorization: Bearer <key> (or the GUI's session cookie) except GET /health, GET /localm-ca.crt, GET /whoami, and the three session routes (GET /api/session, POST /api/session, POST /api/session/logout), which stay public once a key exists. In open mode only GET /api/session is exempt from the management gate: POST /api/session without the GUI shell token is a 403 from that gate, and with it a 400 (there is no key to log in with). POST /v1/surfaces/gui and GET /v1/instances/status, POST /v1/instances/vouch and POST /v1/instances/cooperate-unload take no bearer key either, but carry a credential of their own (an attach token or an instance vouch; see SECURITY.md). Most authenticated routes also check the caller's key for a specific scope (noted per section below; the admin scope implies every other one). CORS is locked to localhost by default; widen it with the cors_origins config key. See tls.md before exposing the server beyond 127.0.0.1.

The OpenAI-compatible inference routes below (/v1/chat/completions, /v1/completions, /v1/embeddings, /v1/rerank, /v1/audio/transcriptions and /v1/images/generations) are the one exception: they accept a cross-origin request from any local app - LM Studio/Ollama-style clients and AI browsers included - without needing cors_origins widened first, so pointing one at localm works the same way it would against LM Studio or Ollama.

The Ollama HTTP API (/api/chat, /api/generate, /api/tags, ...) is served on the same port; see ollama-api.md.

Open mode does not open the management routes. With no key configured, every state-changing request and every GET under /api/ or /v1/ outside the inference routes and the routes above that carry their own credential still needs the GUI shell token or this instance's attach token as the bearer, from a same-origin or Origin-less caller (or one in the cors_origins allowlist). The config, keys, plugin, update and activity routes are among the gated ones. A missing or wrong token is a 403 Open-mode management requires the localm GUI shell on this machine, or an API key; a cross-origin state-changing request is refused earlier with a 403 Cross-origin request refused .... The exempt reads are every GET under /v1/models, GET /api/tags, GET /api/ps, GET /api/version and GET /api/session. See SECURITY.md.

Errors

A failed request answers {"detail": "<message>"}: 401 for a missing or invalid key, 403 for a key without the route's scope (Key lacks required scope: <scope>), a cookie-authenticated change without its CSRF token, or a cross-origin browser request to a route that does not accept one, and the statuses listed with each route below. A defect in the server is a 500 with {"detail": "Internal server error"}; its cause goes to the server log. A body that fails validation is a 422 with {"detail": "<one sentence per field>", "errors": [...]}, where errors is the structured list. Three groups answer in their own shape, and a body that fails validation is a 400 there: the Ollama routes ({"error": "..."}), POST /v1/messages (Anthropic's error object) and POST /v1/responses (OpenAI's error object).

OpenAI-compatible endpoints

POST /v1/chat/completions

Scope: any valid key (no specific scope required once auth is enabled).

Streaming and non-streaming chat. Standard OpenAI request body (see OpenAI request fields for what each field does here), plus localm request extras:

Field	Notes
top_k, repeat_penalty, min_p	extra sampling controls; min_p (0 to 1) drops tokens less likely than min_p times the most likely one
seed	the sampling seed: the same seed repeats the same random choices (see Behaviour notes for why a reply can still differ)
stop	A string or a list of up to 16 strings (each up to 1024 characters). The reply is cut before the first match, the generation ends there, and finish_reason is stop. A stop sequence inside a model's <think> block is not applied. Also accepted by POST /v1/completions.
tools, tool_choice, parallel_tool_calls	OpenAI function tools; see Tool calling.
grammar, grammar_lazy, grammar_triggers	GBNF grammar constraining the output (local models); a lazy grammar stays unconstrained until a trigger pattern appears, and requires grammar_triggers
cache_prompt	Default true. false computes the reply with nothing kept from earlier requests: a GGUF model empties its KV cache and processes the whole prompt, and the reply runs on its own, after the replies already running (it streams waiting until then). The same request then gets the same reply whatever came before it or runs beside it, unless speculative drafting is on or the model's context window had to grow (or move its cache to system RAM) in between. Only true or false is accepted (a string or a number is a 422; a 400 on /v1/messages and /v1/responses). Also accepted by POST /v1/completions, POST /v1/messages and POST /v1/responses. Other backends keep no prompt cache and accept it with no effect.
chat_template_kwargs	{"enable_thinking": false} asks a reasoning model that uses <think> blocks or the harmony format (gpt-oss) to answer without reasoning. Only enable_thinking is applied and must be a boolean (otherwise 422); other keys are accepted and ignored. A model with none of these conventions is unaffected.

While a streaming reply is being prepared, localm reports what it is doing in chunks whose delta carries status (English text) and status_code (a stable id): waiting (another request is running on this model), compacting (older messages are being summarised to fit the context window), processing, encoding_image, encoding_image_gpu, encoding_image_cpu, vision_cpu_retry, encoding_audio_gpu, encoding_audio_cpu, audio_cpu_retry, loading_model, recalling_memory, downloading_embedder (the embedding model memory recall needs, on its first download), chat_hooks (chat plugins' inlet hooks are running), checking_grammar and generating. A streaming request that is compacted carries X-Localm-Context-Compacted: 1; if the conversation still does not fit after compaction, the refusal arrives in the stream as the reply text with finish_reason: "error" instead of an HTTP 413.

A message may also carry untrusted_spans: [[start, end], ...] character ranges of its own content that came from an untrusted source, such as a fetched page or a tool result. localm tokenises those ranges with special-token parsing off, so a chat-template control token spelled inside them stays ordinary text instead of forging a role boundary. The field is optional and additive: omit it and the request behaves exactly as before. A range can only disable special-token parsing over your own prompt, never enable it, and out-of-range values are clamped to the content length.

A message may also carry origin to say the client, not the user, wrote it. Two values are accepted:

• "tool": a tool result or a client note sent as a user message only because some chat templates require user/assistant alternation.
• "client": text the client generated itself, such as the GUI's "Summarise the following conversation ..." prompt when it compacts a long chat.

localm then leaves that message out of what it treats as the user's own words: the long-term-memory recall query and the user line of the audit/session log (which memory consolidation learns from). A request whose only user messages are marked recalls no memories. The model still sees the message. The field is optional and additive: omit it (or send null) and the request behaves exactly as before. Any other value is rejected with a 422, and a response message never carries it.

The response's assistant message (and, for a stream, the delta) also carries reasoning_content: the model's <think> reasoning, split out of the visible content when the model emitted one. Present only when there was reasoning to report; OpenAI clients that don't know the field ignore it.

{
  "model": "mymodel",
  "messages": [{"role": "user", "content": "What is the capital of France?"}],
  "stream": true,
  "temperature": 0.8,
  "seed": 42
}

The usage block carries exact token counts from the model's own tokenizer plus performance numbers, but which extra fields appear depends on streaming and on the endpoint:

{
  "usage": {
    "prompt_tokens": 14, "completion_tokens": 3, "total_tokens": 17,
    "ttft_ms": 181.4, "tokens_per_sec": 38.2, "context_capacity": 8192
  }
}

	prompt/completion/total_tokens	tokens_per_sec	ttft_ms	context_capacity
/v1/chat/completions, streaming	yes	yes	yes	yes
/v1/chat/completions, non-streaming	yes	yes	yes	yes
/v1/completions, streaming	yes	yes	yes	no
/v1/completions, non-streaming	yes	no	no	no

ttft_ms (time to first token) is reported on streaming responses and on non-streaming chat completions. context_capacity (the model's total context window in tokens) is reported on both streaming and non-streaming chat completions.

Chat completions on a GGUF model with Multi-Token Prediction enabled (mtp_enabled) also carry usage.mtp:

{"mtp": {"state": "on", "drafted": 180, "accepted": 151, "paused_steps": 0, "reason": null}}

state is on (the reply speculated), paused (drafting was measured slower than generating one token at a time and was paused for at least as many steps as it ran), stopped (it stopped partway, or turned MTP off for the model; reason says why), off (this reply could not draft at all: reason is image for a turn with an image, audio for a turn with audio and no image), idle (it drafted nothing, for example on a very short reply) or unavailable (this model cannot speculate; reason says why). drafted and accepted count the draft tokens sent to verification and kept; paused_steps counts the steps generated without drafting because drafting was slower. The field is null when mtp_enabled is off.

With any draft source on (spec_source mtp, ngram or draft), chat completions also carry usage.speculation, the same object plus the source:

{"speculation": {"source": "ngram", "state": "on", "drafted": 96, "accepted": 71, "paused_steps": 0, "reason": null}}

For mtp the other fields equal usage.mtp. For ngram, idle means the reply drafted nothing, with reason not-paying when the measured step costs held drafting back (otherwise nothing in the reply matched earlier text), and unavailable carries the model status as reason (rewind-unsupported when the model's cache cannot drop a rejected draft, ngram-cannot-pay when drafts accepted 90% of the time were measured not to beat one-token decoding by 5%). draft reports the same states; its unavailable reasons also include draft-model-missing, draft-unsupported-role (the file is not a causal chat model), draft-load-failed, draft-vocab-mismatch, draft-context-refused and draft-cannot-pay (measured too slow to help at an 85% acceptance), and stopped carries draft-decode-failed:<code> when a draft decode failed partway through the reply. An on reply carries reason draft-on-cpu when the draft model runs on the CPU (the model runs on the CPU, or the draft model did not fit in VRAM beside it). The field is null when no draft source is on.

OpenAI request fields

Field	Behaviour
max_tokens, max_completion_tokens	The reply cap; either name works. Both with different values is a 400.
temperature, top_p, seed, stop	Applied. The finish chunk, or the non-streaming choice, carries stop_sequence (a localm extension): the stop sequence that ended the reply, or null.
presence_penalty, frequency_penalty	Applied (-2 to 2): a token's logit drops by presence_penalty once it has appeared, and by frequency_penalty times the number of times it appeared. On a GGUF model the window is the last 64 tokens, as in llama.cpp. A penalty of 0 applies nothing. A diffusion language model cannot apply a non-zero value of them or of min_p (400).
response_format	See Structured output.
tools, tool_choice, parallel_tool_calls	See Tool calling.
stream_options	{"include_usage": true} moves the usage from the finish chunk to a last chunk with empty choices, sent just before data: [DONE]. Without it the finish chunk carries the usage.
reasoning_effort	"none" asks a reasoning model to answer without thinking (like chat_template_kwargs.enable_thinking: false, which wins when both are sent). Other levels leave the model's default.
n	Only 1 (one choice per request); any other value is a 400.
logprobs, top_logprobs	Returned for a GGUF model: see Log probabilities. top_logprobs is 0 to 20 and needs logprobs: true when above 0; anything else is a 400.
logit_bias	Not served (a non-empty map is a 400): token ids differ between models.
functions, function_call	The deprecated function-calling fields are a 400 (function_call: "none" is accepted); send tools and tool_choice.
audio, modalities other than ["text"], web_search_options	A 400.
user, metadata, store, service_tier, prediction, verbosity, prompt_cache_key, safety_identifier	Accepted; no effect.

A message with role developer is read as system.

Log probabilities

logprobs: true returns the log probability of each token of the reply's content in choices[0].logprobs.content; top_logprobs adds that many of the most likely tokens at each position, most likely first:

{"token": " blue", "logprob": -0.12, "bytes": [32, 98, 108, 117, 101],
 "top_logprobs": [{"token": " blue", "logprob": -0.12, "bytes": [32, 98, 108, 117, 101]},
                  {"token": " red", "logprob": -2.31, "bytes": [32, 114, 101, 100]}]}

A streamed reply carries the entries of each delta's content in that chunk's choices[0].logprobs.content.

• The values are the model's own distribution at each step, a softmax over its raw logits: before temperature, top_k, top_p, min_p, the penalties or a grammar shape the choice. So the sampled token can rank below others, and a grammar can force a token with a very low value.
• Only the content's tokens are listed. Tokens of the reasoning (reasoning_content), of tool calls and of text cut off by a stop sequence are left out; a token that spells both kept and cut text (the last one before a stop sequence, for example) is listed.
• bytes holds each token's own UTF-8 bytes. A character spread over several tokens appears in each of them, and token shows an incomplete character as U+FFFD.
• A probability too small to represent is reported as -9999.0. So is a token whose step produced logits that are not finite numbers (a fault in the model or its runtime); it comes with no alternatives, and the server log gets a warning.
• GGUF models report them, except diffusion language models. A model that cannot (a Hugging Face model, a diffusion model) refuses the request with a 400, and so does a streamed request while a plugin rewrites the stream. When a chat plugin rewrites a non-streamed reply, logprobs is null and the log says why.
• Each token is scored over the whole vocabulary, which slows generation: on Qwen3-0.6B (a vocabulary of about 152,000 tokens) on a Radeon RX 6900 XT, 300-token replies ran at 321 tokens/s without logprobs, 222 with them and 218 with top_logprobs: 20, about 1.4 ms more per token. The added time per token follows the vocabulary size, not the model's, so it weighs most on small, fast models. On a model answering several requests at once, the replies decoding beside one that asked for logprobs wait for its scoring at each step too.

Structured output

response_format constrains the reply, token by token, with a grammar:

Value	Reply
{"type": "text"} (or no field)	Unconstrained.
{"type": "json_object"}	One JSON object.
{"type": "json_schema", "json_schema": {"name": "...", "schema": {...}, "strict": true}}	JSON matching the schema (the keywords listed for format in ollama-api.md). With strict: true a keyword that cannot be enforced (such as pattern) is a 400 naming it; otherwise that keyword is left out, the rest of the schema is still enforced, and the omission is written to the debug log. A keyword that gives the schema its shape (type, properties, required, items, anyOf, ...) is never left out: if it cannot be compiled where it is (for example type next to anyOf), the request is a 400 in either mode. Without schema the reply is any JSON object.

A reasoning model answers without thinking when a format is set, unless the request sends chat_template_kwargs: {"enable_thinking": true}; then the reply may start with a <think> block of up to 1900 characters (returned in reasoning_content) before the JSON. With tools and tool_choice: "auto" the reply is either tool calls or the formatted JSON; with required or a named function it is calls only and the format does not apply. response_format cannot be combined with grammar, and a model whose backend cannot apply grammars refuses it with a 400, as does a GGUF model whose grammar sampler faulted earlier in this server's life (until localm restarts). A malformed response_format is a 400, sent before any model is loaded.

Tool calling

tools takes OpenAI function tools (up to 64) and works with any chat model: the model is told about the functions in its system prompt, and its calls come back in message.tool_calls, with finish_reason "tool_calls". A streamed reply sends each call whole in one delta.tool_calls entry carrying its index, id, type and function; the call text never appears in content. A reply can hold text before its calls, and more than one call.

Request	Behaviour
tool_choice	auto (the default when tools is sent): the model may answer in text or call. none: the functions are not shown to the model and the reply is not read for calls. required: the reply is one or more calls. {"type": "function", "function": {"name": "..."}}: the reply is calls to that function.
parallel_tool_calls	false limits the reply to one call.
earlier turns	An assistant message with tool_calls (its content may be null) and the tool messages that follow it are given to the model as text, in order. A tool result is untrusted text: control tokens in it are defused and it cannot close its own fence.

With required or a named function the arguments are constrained, token by token, to the function's parameters JSON schema (the keywords listed for format in ollama-api.md). A keyword that cannot be enforced (pattern, bounds on a non-integer number, ...) is left out of that function's schema and everything else in it is still enforced; a function whose schema cannot be compiled, or a tool list too large for one grammar, takes any JSON object as its arguments. Each case is written to the debug log. With auto the arguments are constrained from the moment the model opens a call. A forced choice starts with the call, so the model has no room to reason before it. Whether a model opens a call under auto is the model's decision: a model that does not follow the call instruction answers in prose, and required or a named function is how to force a call. A call to a function that was not offered is left in the text.

stop sequences apply to the visible text only, never inside a call, and a call after a matched stop sequence is dropped. tools cannot be combined with grammar (400). On a model whose backend cannot apply grammars, auto still works, with calls read from the reply without a constraint, and required or a named function is a 400. A llama.cpp runtime too old to apply lazy grammars refuses auto at generation time: a non-streaming request gets a 400 that says so, and a streaming one an in-band [inference error: ...] with finish_reason "error". localm setup-llama installs a current runtime. A malformed tools or tool_choice is a 400.

How a generation failure is reported

Both generation endpoints answer failures the same way on both routes and in both streaming modes. Which of the two shapes you get depends on WHEN the failure happens, not on which endpoint you called:

Failure	Reported as
The request is refused before generation starts (unsupported input, an undecodable image, a grammar this model cannot apply, a malformed grammar or trigger pattern, malformed tools)	An HTTP error status with the reason in detail: 400; 413 when the images of a request exceed the size or count limits, when the prompt does not fit the context window, or when an embedding batch is too large; 501 when the server lacks an image decoder, or an audio decoder for the clip's format; 503 when the trigger-pattern validator was too busy to check your pattern
Generation started and then failed (not enough free VRAM for this prompt, a conversation that outgrew the context window, a native decode error)	200 whose text is \n[inference error: <reason>] (a leading newline, then the bracketed reason) and whose finish_reason is "error"

The second row is a 200 on purpose. A streaming response has already sent its headers by the time generation can fail, so a status is no longer available to it; the non-streaming responses match their own streaming twin rather than diverge from it. finish_reason is the machine-readable signal - check it rather than the status code to detect a failed generation. A successful generation never reports "error".

A streamed chat request whose preparation (loading the model, running the chat plugins' inlet hooks, checking the request) takes more than a fraction of a second opens its stream before preparing ends, so that it can send the status chunks above. A refusal after that point cannot be an HTTP status. It arrives as the reply text with finish_reason: "error", and the reply chunk carries localm_error: {"status": <the HTTP status the refusal would have had>, "detail": "<reason>"} (a refusal while preparing puts it on the final chunk too). A generation that fails before any text, with a status in the table above, carries localm_error on its error chunk the same way. The response headers of an early-opened stream (X-Localm-Memory, X-Localm-Model-Routing, X-Localm-Context-Compacted) cannot be set once it is open; they arrive in one chunk with an empty delta and a localm_headers object, sent before the reply.

A 503 is the only status here that a retry can clear on its own; the rest describe something about the request or the build that will fail identically next time.

Absolute paths are redacted out of an [inference error: ...] body before it leaves the server (a model that fails to load names its file, which would otherwise carry the account name and install layout to the caller). The reason, the file name and any line number are kept - the text is redacted, not muted.

An error that is neither of those - a genuine defect in the server - is a 500 with an opaque {"detail": "Internal server error"}, and its cause goes to the server log rather than to the client.

When the memory plugin is active and recalls facts for a turn, the response also carries an X-Localm-Memory header (see Memory endpoints).

When the conversation is too long for the answering model's context window, the server compacts it for that request (older turns summarised, recent ones kept) and the response carries X-Localm-Context-Compacted: 1. Only the server's copy is compacted; a client that keeps the history itself can compact its own copy from then on instead of sending the full history every turn.

Capability-based model routing

When a request leaves model unset, empty, or "localm", or names a model with "pin_model": false, the model that would answer (the loaded one, or the one named) is not automatically who answers: localm checks what the request actually needs (an attached image, a request for structured tool calls, a longer conversation than that model was trained to hold, or an explicit required_capabilities list) against what that model can do, and answers with an installed model that has it when it does not. A model you named is never swapped out from under you - naming one pins it, gap or no gap, unless you also send "pin_model": false.

Three localm extensions on the request body steer this:

Field	Meaning
pin_model	false: model is only the preferred model, which routing may replace. true: whatever answers is never replaced, named or not. Unset: a named model is pinned, an unnamed one is not.
required_capabilities	Capabilities the answering model must have, any of vision, audio_input, tool_use, reasoning. Vision, audio input and context length never need listing: an image part, an audio part and the prompt's size already say so.
min_context	Tokens the answering model's trained context window must hold. The larger of this and what the prompt's own size implies applies.

Only installed chat models whose file is present are candidates (never an embedding, diffusion, VAE, LoRA or projector entry), a candidate is only chosen when it is confirmed to have what is needed, and one that is already loaded is preferred. A routed model is loaded for the request without becoming the loaded model: the next request that needs nothing special is answered by the loaded model again. If a capable model fails to load, the next one is tried, then the model the request would otherwise use.

When no installed model has everything a request needs, what the request cannot be answered without decides: an image goes to a model that can read it, an audio clip to one that can take audio, and a conversation too long for the model goes to one that can hold it, even if that model lacks a listed capability such as tool_use. unmet in the routing header below names what it lacks. A request that only lists a capability no installed model has stays with the model it would otherwise use.

The response body's own "model" field names the model that actually answered whenever routing changed it; otherwise it echoes the name you sent (or, for an unnamed request, the model that answered). Whenever there is something to explain - a swap happened, or the model that answered was missing something the request needed - the response also carries an X-Localm-Model-Routing header, a compact JSON object:

{"resolved": "vision-model", "requested": "localm", "routed": true,
 "pinned": false, "gaps": {"vision": "absent"}, "unmet": []}

gaps maps each unmet or unknown capability to "absent" or "unknown" (a model that has not been inspected yet); unmet lists what the model that answered still lacks because no installed model could provide it, or none could be loaded, in which case load_errors says why. A pinned request reports its gaps with "pinned": true and an empty unmet. The header is omitted entirely when nothing needs explaining.

A request that resolves to a model with an accepted peer route (see Using a model another instance has loaded) is forwarded to that instance, named the way that instance names it.

Multimodal input uses the standard multipart content format with base64 data-URIs ({"type": "image_url", "image_url": {"url": "data:image/..."}}) and needs a vision-capable model (a GGUF paired with a multimodal projector via localm run --mmproj / localm pull --mmproj, or a HuggingFace-format vision model). Sending an image while a text-only model is loaded no longer just fails: if the request did not pin a model, localm looks for an installed model that can see and answers with that one instead (see routing above). Only when no such model is installed, or the request pinned the loaded text-only model by name, does it reject the attached image with a clear error.

Audio input uses the OpenAI input_audio part ({"type": "input_audio", "input_audio": {"data": "<base64>", "format": "wav"}}) and needs a GGUF model whose projector (mmproj) has an audio encoder, such as Qwen3-ASR, or a HuggingFace-format model with an audio processor. The model then transcribes or answers questions about the clip; it never replies with speech. On a GGUF model, WAV (8, 16, 24 or 32-bit PCM, or 32/64-bit float, any channel count) is always read and other formats (mp3, flac, ogg, ...) need the voice extra; the clip is converted to mono at the rate the projector expects, must be at least 0.1 seconds and at most 10 minutes long and 50 MB encoded, and anything else, or audio that cannot be decoded, is a 400 saying why. Audio sent to a model that cannot take it is routed or refused exactly like an image. GET /v1/models/{id} carries audio_input (true or false) next to vision when its files could be inspected.

Using a model another instance has loaded

When another localm instance on this machine already has a model loaded, chat and completion requests for it can be forwarded there instead of this instance loading its own copy into VRAM. It is always an explicit choice: the GUI offers it when you pick that model in the sidebar.

Route	Scope	Purpose
GET /v1/models/{model}/peer-offer	models:read	Whether a live instance has this model loaded. {"available": bool, "peer": {"instance_id", "host", "port", "scheme", "model", "requires_key"}}, with peer null when available is false. model is that instance's own name for it; requires_key is false for an instance with no API key. A lookup that takes too long is a 504.
POST /v1/models/{model}/peer-route	models:write	Accept an offer. Body {"instance_id": "...", "api_key": "..."}, with api_key empty for an instance that has no key. The key is checked against that instance first: a refused key is a 403 and nothing is stored. Returns {"status": "routed", "model", "peer": {"instance_id", "host", "port", "scheme", "model", "peer_model"}}. An empty instance_id or no model to route is a 400 (an absent instance_id field is a 422); an instance that is no longer live, no longer has the model loaded or is not reachable over loopback is a 409, as is a route that would have this instance record the chat in a persisting session mode; a check that takes too long is a 504.
DELETE /v1/models/{model}/peer-route	models:write	Stop forwarding. Returns {"status": "unrouted", "model", "was_routed": bool}; clearing a model with no route is a success.

A model is matched by its file (the same path, the same sha256, or the same file name and size), so a different model that merely shares a name is never offered; an instance that reports only its active model's name is matched by name. Any model that instance has loaded counts, not only its active one. Only an instance reached over loopback, with its identity verified, is ever offered or forwarded to. A route is dropped when that instance stops answering or refuses the key it was given (the request that found out gets a 502 saying so); the next request loads the model here. Routes live in this process's memory and do not survive a restart; GET /api/models lists them under peer_routes.

A route keeps this instance's session persistence. Unless the chat is recorded in full mode here, the other instance must confirm, when the route is accepted and again before every forwarded request, that it records the chat no more than this instance's mode allows (it reports this in the session_mode field of GET /api/session for a request carrying the X-Localm-Session-Mode header; it agrees to record less only for its owner key, or for any caller when it has no key). Otherwise accepting is a 409 and nothing is stored, and a forwarded request is a 409 saying why (naming that instance, and its mode when it reports one); nothing is sent and the route stays. An instance that does not answer this check, or refuses the key, makes accepting a 403 and drops an existing route with a 502 as above. A forwarded request carries X-Localm-Session-Mode. With a key that is not that instance's owner key, it records at its own mode, which is checked before each request; a change it makes to its mode while a request is running is not covered.

POST /v1/completions

Scope: any valid key (no specific scope required once auth is enabled).

Raw text completion (streaming and non-streaming), same request extras as chat. Its usage block is narrower than chat's - see the table above. prompt is one string, or a list holding exactly one string. echo: true starts the reply text with the prompt. presence_penalty, frequency_penalty, min_p and stream_options work as on chat. logprobs (0 to 20) returns the legacy logprobs object for a GGUF model: tokens, token_logprobs, top_logprobs (each a map of the most likely tokens, always including the sampled one) and text_offset (where each token starts in text); a streamed reply carries the entries of each chunk's text. The values are computed as on chat (Log probabilities), for the tokens of the returned text. echo together with logprobs is a 400 (the prompt's tokens are not scored). A batch of prompts, token-id prompts, best_of above 1, n other than 1, a non-empty logit_bias and suffix are each a 400 naming the field.

POST /v1/messages (Anthropic Messages API)

Scope: any valid key (no specific scope required once auth is enabled). Anthropic clients authenticate with x-api-key, which the API routes accept as well as Authorization: Bearer (the bearer token wins when both are sent; the GUI login takes only the bearer form); the anthropic-version and anthropic-beta headers are accepted and not needed.

An Anthropic SDK or an agent tool that speaks the Messages API can use localm by pointing its base URL at the server (ANTHROPIC_BASE_URL=http://127.0.0.1:8642) and naming a localm model. The request is answered by the same chat path as /v1/chat/completions, so capability routing, compaction, tool calling and the response headers work the same way.

Request field	Behaviour
model	A localm model name (localm or empty: the loaded model). Another name is a 404 not_found_error.
max_tokens	Required, the reply cap.
messages	user and assistant turns; content is a string or blocks: text, image (base64 or url source), tool_use (assistant), tool_result (user; text or image content, is_error marks a failed call). thinking blocks from earlier turns are dropped. Other block types (documents, server tool results) are a 400.
system	A string or a list of text blocks.
stop_sequences, temperature, top_p, top_k	Applied.
cache_prompt	localm's field, applied as on /v1/chat/completions.
tools, tool_choice	Custom tools (name, description, input_schema) work with any chat model, as in Tool calling. tool_choice auto, any, tool (by name) and none; disable_parallel_tool_use allows one call. Server tools (web search, code execution, ...) and mcp_servers are a 400.
thinking	{"type": "enabled"} (or adaptive, between_tools) lets a reasoning model think and returns its reasoning as a thinking block; without it, or with disabled, the model answers without thinking. budget_tokens is not applied.
output_config	format: {"type": "json_schema", "schema": {...}} constrains the reply to the schema, as a strict response_format does on chat (see Structured output). effort is accepted and not applied.
stream	Server-sent events in Anthropic's order: message_start, ping, per block content_block_start / content_block_delta (text_delta, thinking_delta then signature_delta, input_json_delta carrying a tool call's whole input) / content_block_stop, then message_delta with stop_reason and usage, then message_stop. message_start reports 0 input tokens; message_delta carries the real input_tokens and output_tokens. A generation that fails partway ends with an error event.
metadata, service_tier, cache_control, inference_geo	Accepted; no effect.

stop_reason is end_turn, max_tokens, stop_sequence (with stop_sequence naming the match; otherwise stop_sequence is null) or tool_use. Consecutive turns of the same role are joined, and a turn's text blocks stay separate blocks, as the Messages API treats them. A thinking block carries an empty signature. Errors use Anthropic's shape, {"type": "error", "error": {"type": "invalid_request_error", "message": "..."}}. The type follows the status: authentication_error (401, a missing or wrong key), permission_error (403), not_found_error (404), request_too_large (413), rate_limit_error (429), overloaded_error (503, 529), invalid_request_error for every other 4xx (a body that fails validation is a 400), and api_error for any other status.

POST /v1/messages/count_tokens

Same request fields as /v1/messages (without max_tokens); returns {"input_tokens": N}, counted with the model's own tokenizer and chat template, tool descriptions included, by the model that would answer the same /v1/messages request (capability routing applies, and a model another instance serves is counted there). The model is loaded if it is not already.

POST /v1/responses (OpenAI Responses API)

Scope: any valid key (no specific scope required once auth is enabled).

An OpenAI SDK or agent tool that speaks the Responses API can use localm by pointing its base URL at the server (OPENAI_BASE_URL=http://127.0.0.1:8642/v1) and naming a localm model. The request is answered by the same chat path as /v1/chat/completions, so capability routing, compaction, tool calling, structured output and the response headers work the same way.

Request field	Behaviour
model	A localm model name (localm or empty: the loaded model).
input	A string, or a list of items: messages (user, assistant, system, developer; content as a string or input_text, output_text, input_image with an image_url, and input_audio parts), function_call and function_call_output items (an output is a string, or input_text and input_image parts; the images reach the model in a user message after the tool results). reasoning items are dropped. input_file parts, item_reference and other item types are a 400.
instructions	A system message for this request only; it is not carried into a continuation.
tools, tool_choice, parallel_tool_calls	Function tools work with any chat model, as in Tool calling. tool_choice auto, none, required or a function by name. Built-in tools (web search, file search, code interpreter, ...) are a 400.
text.format	text, json_object, or json_schema (with strict), as response_format on chat (see Structured output).
max_output_tokens, temperature, top_p	Applied.
reasoning.effort, reasoning.summary	effort: none turns thinking off; another value leaves the model's default. A reasoning model's thinking comes back as a reasoning item whose content is the full reasoning text; its summary stays empty whatever reasoning.summary asks.
previous_response_id	Continues a stored response: its input and output are sent ahead of this request's input. Responses are kept in the server's memory only, never on disk, are lost when the server restarts, and are visible only to the key that created them. The store holds at most 256 responses and 64 MB of conversation JSON; past either bound, the oldest response of the key holding the most is dropped, so one key filling the store evicts its own responses first. A response expires an hour after it is stored and is removed at the next request to this route after that. A response too large to keep comes back with store: false. An unknown, expired, dropped or foreign id is a 404.
store	false keeps the response out of the store.
stream	Server-sent events: response.created, response.in_progress, per output item response.output_item.added, response.content_part.added, response.output_text.delta (or response.reasoning_text.delta), response.output_text.done, response.content_part.done, response.output_item.done, function calls as response.function_call_arguments.delta and .done carrying the whole arguments, then response.completed (or response.incomplete when the token cap was reached, response.failed when generation failed). Every event carries a sequence_number.
include	reasoning.encrypted_content and the built-in tool entries (file_search_call.results, web_search_call.results, web_search_call.action.sources, message.input_image.image_url, computer_call_output.output.image_url, code_interpreter_call.outputs) are accepted with no effect; a reasoning item carries no encrypted content. message.output_text.logprobs returns the log probabilities of the reply's text (see top_logprobs below). Any other value is a 400.
truncation	auto or disabled; either way an over-long conversation is compacted as on chat.
metadata, user, prompt_cache_key, safety_identifier, service_tier, max_tool_calls	Accepted; metadata is echoed, the rest have no effect.
localm chat fields	top_k, min_p, repeat_penalty, presence_penalty, frequency_penalty, seed, stop, grammar (with grammar_lazy, grammar_triggers), required_capabilities, pin_model, min_context, chat_template_kwargs and cache_prompt (for example through an SDK's extra_body) are applied as on /v1/chat/completions. Other unknown fields have no effect and are named in the debug log.
top_logprobs	0 to 20, anything else is a 400. With include: ["message.output_text.logprobs"], each token of the reply's text comes with that many of the most likely alternatives in the output_text part's logprobs (and, streamed, in each response.output_text.delta event, holding the entries the chat stream sent with that piece of text, so a token split across two deltas comes with the first, and in the response.output_text.done event, which carries all of them), in the shape of Log probabilities: token, logprob, bytes, top_logprobs. Without the include it has no effect and logprobs stays []. The rules and limits are the chat route's: a GGUF model reports them, and a model that cannot (or a streamed request while a plugin rewrites the stream) is a 400 naming the model; a reply a chat plugin rewrote comes back with logprobs: [].
background, conversation, prompt, text.verbosity other than medium	A 400 naming the field.

Only POST /v1/responses is served: a stored response cannot be fetched, listed, cancelled or deleted through the API. Errors use OpenAI's shape, {"error": {"message": "...", "type": "invalid_request_error", "param": ..., "code": null}}. The type follows the status: authentication_error (401, a missing or wrong key), permission_error (403), not_found_error (404, including an unknown previous_response_id, which sets param to that field), invalid_request_error for every other 4xx (a body that fails validation is a 400), and server_error for any other status.

POST /v1/embeddings

Scope: any valid key (no specific scope required once auth is enabled).

{"model": "mymodel", "input": ["text one", "text two"]}

Returns {"object": "list", "data": [{"object": "embedding", "index": 0, "embedding": [...]}], "model": "...", "usage": {"prompt_tokens": N, "total_tokens": N}}. encoding_format is float (default) or base64 (each vector as little-endian float32 bytes, base64-encoded); anything else is a 400. A model registered as an embedding model (or the configured embedding_model) is served by the dedicated embedder, whose usage counts are 0.

Errors: 400 when model is empty and there is no active or startup chat model, or (dedicated embedder) for an input the tokenizer refuses; 422 when the loaded model cannot embed, when the embedding model is unavailable (localm setup-embeddings), or when model names a reranker; 503 when the embedder worker fails. Against a HuggingFace-format model, 413 when the request exceeds the configured text-count or character-count cap (hf_embed_max_texts, hf_embed_max_chars).

POST /v1/rerank

Scope: any valid key (no specific scope required once auth is enabled).

{"model": "bge-reranker", "query": "what is a panda?",
 "documents": ["The giant panda is a bear.", {"text": "Pandas is a Python library."}],
 "top_n": 2, "return_documents": false}

Scores each document against the query with a registered reranker (a GGUF cross-encoder such as bge-reranker-v2-m3, or a decoder reranker such as Qwen3-Reranker) and returns them best first, in the Jina / Cohere / llama-server shape:

{"object": "list", "model": "bge-reranker",
 "usage": {"prompt_tokens": 61, "total_tokens": 61},
 "results": [{"index": 1, "relevance_score": 2.24}, {"index": 0, "relevance_score": 1.58}]}

index is the document's position in the request; a higher relevance_score is more relevant, on a scale that belongs to the model (bge returns an unbounded score, Qwen3-Reranker the probability of "yes"). top_n keeps the best N and return_documents adds "document": {"text": "..."} to each result. A classifier head with several outputs also reports them as label_scores, and the first is the relevance score. A document longer than the model's window is cut to fit and its result carries "truncated": true.

model names a registered reranker and may be left out when exactly one is registered; a path is never accepted. One reranker is resident at a time and a request for another replaces it. Errors: 400 for an empty query, a query too long for the model's window, or several rerankers with no model; 404 for an unregistered model or no reranker at all; 422 for an invalid body (including a top_n below 1 or an empty documents list), a model that is not a reranker (an embedding or chat model), a registered reranker whose file is missing, or a file without a classifier head; 503 when the model cannot be loaded or its worker fails.

POST /v1/audio/transcriptions

Scope: voice. Served by the voice plugin, so the route exists while that plugin is enabled. It answers with Whisper (pip install "localm[voice]"), or with an installed GGUF model that hears audio (for example localm pull ggml-org/Qwen3-ASR-0.6B-GGUF:Qwen3-ASR-0.6B-Q8_0.gguf), described below. The official openai SDK works against it:

from openai import OpenAI
client = OpenAI(base_url="http://127.0.0.1:8642/v1", api_key="<key, or any text in open mode>")
with open("speech.wav", "rb") as f:
    result = client.audio.transcriptions.create(model="whisper-1", file=f)
print(result.text)

A multipart/form-data upload. The audio is decoded and transcribed in memory by the speech worker and is never written to disk, in any session mode, and the file name is never used as a path.

Field	Meaning
file	Required. One audio file, any format PyAV decodes (wav, mp3, m4a, ogg, flac, webm). At most 25 MB; larger is a 413 before it is fully read.
model	Optional. whisper-1 or the configured voice_stt_model for Whisper, or the name of an installed model that hears audio. A registered model that cannot take audio is a 400; any other name is a 404 naming what this server transcribes with.
language	Optional ISO 639-1 code such as en. Detected when omitted. An unsupported code is a 400.
prompt	Optional text that biases the transcript (names, jargon).
response_format	json (default, {"text": ...}), text, srt, vtt, or verbose_json (task, language name, duration, text, segments with timing and confidence).
temperature	Optional, 0 to 1.
timestamp_granularities[]	segment and/or word, with verbose_json only. word adds a words list; asking for word alone omits segments.

Which engine answers. Whisper answers whenever faster-whisper is installed and model is left out or is a Whisper name. An installed model that hears audio answers when model names it (even with faster-whisper installed), and when faster-whisper is not installed and model is left out or is a Whisper name; with several such models, one that is already loaded is used, else the first by name. The response carries X-Localm-Transcription-Model with the model that answered. That path gives json and text only (the model returns plain text without timestamps; srt, vtt and verbose_json are a 400), takes WAV without any extra and other formats with the voice extra (501 otherwise), accepts clips up to 10 minutes and 25 MB, decodes at 0.0 temperature unless temperature is sent, passes language and prompt to the model as part of its instruction (a dedicated speech model such as Qwen3-ASR ignores them), and answers 413 when the clip does not fit the model's context and 502 when the transcript hit the reply limit instead of returning it cut short. The audio is not recorded in the audit log or the transcript.

stream, include[], chunking_strategy and known_speaker_* change the output and are not implemented, so sending them is a 400; other unknown fields are ignored. A recording with no speech is a 200 with empty text. Failures: 400 an unreadable file or bad field, 403 a cross-origin browser request from an origin that is not this server, a local page or a cors_origins entry, 413 too large, 415 not multipart, 501 the speech package is not installed and no installed model hears audio, 409 the first-use model download is blocked by the network policy, 503 the speech engine could not start or load its model, 504 the speech engine hung and was restarted, 500 the audio decoder failed, 502 any other engine failure.

POST /v1/images/generations

Scope: image. Served by the image plugin (ComfyUI today), which ships disabled: enable it on the Plugins page first. The route exists while the plugin is enabled.

result = client.images.generate(model="any", prompt="a lighthouse at dusk",
                                size="1024x1024", response_format="b64_json")

A JSON body. The request waits for the generation (it runs as the same background job as POST /api/imagine, so it shows in the activity list and frees and restores VRAM the same way) and answers {"created": <unix time>, "data": [{"b64_json": ...} | {"url": ...}]}.

Field	Meaning
prompt	Required, up to 32000 characters.
n	1 to 4 images, generated one after another.
size	WIDTHxHEIGHT (each side 64 to 2048 and a multiple of 8, at most 2048 x 2048 pixels) or auto. Omitted or auto keeps the workflow's own size. A workflow with no EmptyLatentImage or EmptySD3LatentImage node cannot take a size and the request fails saying so.
response_format	b64_json or url.
output_format	Only png (the default).
model, quality, style, user	Accepted and ignored: the image model is the one the active workflow loads.

Outside privacy mode the image is kept in the gallery with its sidecar. With an API key configured, response_format defaults to url: the gallery file route (/api/imagine/file/<name>), fetched with the same key. With no key configured that route cannot be fetched by an API client, so b64_json is the default and asking for url is a 400. In privacy mode nothing is kept: the image is generated into a private directory, returned as b64_json (the default there; asking for url is a 400), and the directory is deleted before the response is sent.

Failures: 400 an empty or over-long prompt, a bad size, an output_format other than png, a bad response_format, or url where none can be served (above); 422 an n outside 1 to 4 or a body that does not validate; 409 the job was cancelled (for example through the job's cancel route); 502 the generation failed (the reason is in detail); 503 the server has no background job registry or could not determine its own address; 500 in privacy mode, the generated image could not be deleted, so it was not returned.

POST /v1/audio/speech

Scope: tts. Served by the tts plugin. Speaks text with a registered text-to-speech model: a Qwen3-TTS GGUF and its mmproj (localm pull ggml-org/Qwen3-TTS-12Hz-1.7B-Base-GGUF:Qwen3-TTS-12Hz-1.7B-Base-Q4_K_M.gguf fetches both). The browser voices of the tts plugin are separate and unaffected.

audio = client.audio.speech.create(model="tts-1", voice="default",
                                   input="Hello there.", response_format="wav")
audio.write_to_file("hello.wav")

A JSON body, or multipart/form-data with the same fields plus an optional voice_file. The request waits for the whole file: it runs in the isolated speech worker as a background job of kind speak, so it shows in the activity list with its progress, and a client that disconnects cancels it. The audio is returned from memory and never written to disk, in any session mode. The seed used is in the X-Localm-Seed response header.

Field	Meaning
input	Required, 1 to 4096 characters. Text containing the model's control-token strings (such as <|im_end|>) is a 400.
model	A registered model of type tts. Omitted, localm, tts-1, tts-1-hd and gpt-4o-mini-tts name the only one; with several registered, name one.
voice	default (the model's own voice), or the name of a WAV recording saved as <name>.wav in the voices folder of the data directory. Any other name is a 400 listing the voices.
voice_file	Multipart only: a WAV recording (PCM or float, at most 30 s and 16 MB) whose voice to imitate. Not together with a named voice.
response_format	wav (default; 16-bit mono at 24 kHz) or pcm (the same samples, raw little-endian, no header; a model that speaks at another rate than 24 kHz gets a 400 for it).
language	Optional code such as en or name such as english; the model's default when omitted.
seed	Optional, 0 to 4294967294, for a reproducible result. Random when omitted.
speed, instructions, stream_format	Only 1.0, empty and audio: other values change the output and are not implemented, so they are a 400.

mp3, opus, aac and flac are a 400; other unknown fields are ignored. Failures: 400 a bad field, text or recording, 403 a cross-origin browser request from an origin that is not this server, a local page or a cors_origins entry, 404 no such model, 422 a model that is not a text-to-speech model or lacks its mmproj, 413 too large, 501 the llama.cpp runtime or the model cannot synthesize speech (run localm setup-llama for the pinned runtime), 503 the model failed to load, 504 the speech worker hung and was stopped, 502 the generation failed or did not end within its frame budget, 409 cancelled.

GET /v1/models

Scope: models:read.

Lists every registered model (and the startup model when it is not registered) as {"object": "list", "data": [{"id", "object": "model", "created", "owned_by", "loaded", "active"}]}. GET /v1/models/{id} (same scope) returns registry detail for any registered model: id, object, owned_by, path (basename only), source, sha256, size_bytes, aliases, active, loaded, model_type, plus model_type_recorded: false when the registry entry records no type, vision and audio_input when the model's files could be inspected, adapters when LoRA adapters are applied to the loaded model, and peer_route when an accepted route forwards it to another instance. An unregistered {id} is a 404.

GET /health

Public - no authentication, even when a key is configured. 200 with {"status": "ok", "model": "...", "loaded": true|false}; 503 when no engine is initialised.

Model lifecycle

Scope: models:write.

POST /v1/models/unload?model=<name> releases model(s) from VRAM (e.g. before image generation hands the GPU to ComfyUI; omit model to unload everything currently loaded); POST /v1/models/load?model=<name> reloads one (omit model for the active or default model). Both wait for any in-flight generation to finish first, and for the freed VRAM to actually be reclaimed before returning.

POST /v1/models/load returns:

{"status": "loaded", "model": "mymodel",
 "gpu_layers_offloaded": 24, "gpu_layers_total": 32, "degraded": true}

status is "already_loaded" when the model was resident already, or "peer_routed" (with model and peer, and no load) when an accepted route forwards the model to another instance. The gpu_layers_offloaded/gpu_layers_total/degraded fields appear only when the backend can report per-layer placement (GGUF models, once the load has completed), with moe_cpu_layers when a Mixture-of-Experts load kept the routed experts of that many layers in system RAM (which also makes degraded true). A model too large to fully fit VRAM still loads deliberately, offloading as many layers as fit and running the rest on CPU rather than refusing outright; degraded is true whenever fewer than the full layer count landed on the GPU, so a caller can tell that apart from a full GPU load. parallel_slots appears when the loaded model answers more than one request at once, and says how many. A load that reports its memory-mapping also carries use_mmap, mmap, mmap_from_disk and mmap_cpu_from_disk, with mmap_note and mmap_ram_short when there is something to say, and adapters lists the LoRA adapters applied. Without model and with nothing active, loaded or configured to fall back to, the load is a 503; an unregistered name is a 404.

POST /v1/models/unload with no model returns status ("unloaded", "in_use" when every loaded model was mid-request and none could be freed, or "already_unloaded"), unloaded_models, embedder_unloaded, and skipped_in_use when something was pinned by an in-flight request. It adds confirm_required (what is still using a model) when a model stayed loaded because it was still in use after a short grace period; this route has no force parameter to override that. With ?model=<name> it returns {"status": "unloaded", "model": ..., "was_active": ...}, or {"status": "already_unloaded", "model": ...}, or {"status": "confirm_required", "model": ..., "detail": ...} when the model was still in use after the grace period and was left loaded. Treat anything other than "unloaded" or "already_unloaded" as "the model is still resident. An unregistered model is a 404. The unload routes (not the load route) add vram_freed, vram_before_bytes, and vram_after_bytes when a GPU free-memory reading is available, plus vram_reading_uncertain and a vram_note when that reading cannot be trusted (a stale probe, a driver that reports only this process's own allocations, or an unmeasurable post-unload reading). Unloading is implicit-recovery: the next chat request against an unloaded model reloads it automatically.

POST /v1/models/rename?model=<name>&new_name=<name> renames a registered model in place: re-keys the registry entry and, if the model is currently loaded, its in-memory engine, so it keeps serving under the new name with no reload. Unlike an alias, the old name stops resolving. It returns {"status": "renamed", "model": "<old name>", "new_name": "<new name as sanitised>", "notes": [...]}, where notes reports the references to the old name it migrated or could not (a per-project coder config, for example); an unregistered model is a 404, a taken name a 409, and a failed rename a 400. This is what localm rename calls; it exists on the always-present /v1 surface (not only under localm gui) because a headless localm serve has no GUI routes to do it from instead.

Server control

Scope: config:write.

POST /v1/server/restart unloads every resident model, waits for the freed VRAM to actually be reclaimed, then restarts the process in place on the same port (a fresh model load right after re-exec would otherwise race the still-reclaiming GPU driver). An API-only instance that a second localm gui attached to comes back with the GUI mounted again. POST /v1/server/shutdown unloads and exits without relaunching - a clean alternative to force-closing the window or a Ctrl+C that does not always land. Both return immediately ({"restarting": true} / {"stopping": true}) while the actual restart/shutdown runs a moment later, so the response reaches the client first.

Management endpoints

GET /v1/config / PATCH /v1/config

Scope: config:read for GET (also covers GET /v1/config/schema, GET /v1/media/config, GET /v1/tts/config, GET /v1/plugins/settings and GET /v1/comfy/status); config:write for PATCH (also covers POST /v1/media/config/{name}, POST /v1/tts/config, POST /v1/plugins/{name}/settings, POST /v1/comfy/stop, and POST /v1/comfy/restart).

Read and update <data dir>/config.json. PATCH accepts only known keys and persists immediately; engine values (context sizes, GPU layers) apply on the next model load. These settings widen a trust boundary and are hidden from, and refused for, a non-owner config:write key - only an admin-scoped key or the open-mode owner may read or change them: allow_network_drives, binary_dir, bind_host, browser_allow, browser_custom_domain_rules, browser_deny, browser_enabled, browser_engine, bugreport_upload_token, bugreport_upload_url, chat_mode, civitai_api_key, coder_mode, coder_reviewer, coder_reviewer_model, coder_untrusted_provenance, comfy_api_url, comfy_launch_cmd, comfy_workdir, cors_origins, embedding_model, gui_preview_enabled, gui_preview_owner_only, gui_proxy_remote_images, hf_token, hf_trust_remote_code, keep_diagnostics, llama_runtime_history, llama_runtime_pin, metrics_enabled, mode, net_allow_model_downloads, net_allow_private, net_deny, net_search_url, rag_allowed_roots, rag_denied_roots, rag_indexing_mode, require_auth, spec_draft_model, tls_cert, tls_enabled, tls_key, update_allow_prerelease, update_ignore_net_policy, update_token, update_url.

Two further keys, plugins and plugins_enabled, are plugin STATE rather than settings: this endpoint cannot validate what is inside them, and their own write surfaces enforce stronger permissions (POST /v1/tts/config and POST /v1/media/config/{name} require an owner for the fields that become a script URL, a shell command, or a render target; POST /api/plugins/{name}/enable requires plugins:admin). PATCH therefore refuses both to a non-owner config:write key, with a 403 naming the endpoint to use instead, so this general route cannot be used to bypass a specific one. Their values stay readable.

GET /v1/config returns the whole config plus seven read-only extras: effective_mode, effective_chat_mode, effective_coder_mode, effective_ctx_max, effective_ctx_model, instance_id and instance_port. PATCH ignores those seven, so a client can send back what it read, and returns the full merged config; a non-owner caller gets neither the owner-only keys nor their values. hf_token and civitai_api_key are kept in a separate owner-only file, not in config.json, and are written last, so an error leaves them unchanged. A change to embedding_model that would leave an existing collection's vectors unusable is not applied by the first request: it answers 200 {"needs_confirm": true, "model", "collections", "note"}, and the same body with "confirm": true applies it. PATCH errors: 400 for an unknown key, an invalid value, or require_auth: true while no API key exists; 403 for an owner-only key or plugin state from a non-owner key; 409 when config.json or the credentials file exists but cannot be read; 504 when the save times out; 500 when the credentials file cannot be written.

GET /v1/config/schema returns {"fields": [...]}. GET /v1/media/config returns {"plugins": [{"plugin", "label", "fields"}]} for image, music and video; POST /v1/media/config/{name} merges one plugin's block (a blank field clears its override) and returns {"plugin", "fields"}, with 404 for an unknown plugin, 403 for an owner-only field from a non-owner key, 400 for an invalid value, 409 for an unreadable config and 504 on a timeout. GET and POST /v1/tts/config return {"plugin", "active", "fields"} and fail the same way (no 404). GET /v1/comfy/status returns {"alive", "launched_by_localm"}; POST /v1/comfy/stop and POST /v1/comfy/restart return {"ok", "message"}; all three answer 504 when they time out.

GET /v1/plugins/settings / POST /v1/plugins/{name}/settings

Scope: config:read for GET, config:write for POST.

The generic counterpart to /v1/tts/config and /v1/media/config/{name} for a plugin that contributes settings dynamically via host.add_settings() rather than through a bespoke schema (the Open WebUI Valves interop seam - see plugin-interop.md). GET returns every ACTIVE plugin's declared fields with their resolved values; an inactive or unknown plugin simply has no entry. POST merges a plugin's block key by key (a blank field clears that override back to its declared default) and 404s for a plugin with no active add_settings() fields. GET returns {"plugins": [{"plugin", "label", "fields"}]} and POST returns {"plugin", "fields"}; POST also answers 400 for an invalid value, 409 for an unreadable config and 504 on a timeout. A field marked owner-only in the plugin's own declaration is hidden from, and refused for, a non-owner config:write key, the same rule /v1/config applies to its own admin_only keys.

Diagnostics, issues, and updates

These routes are part of the base server, registered unconditionally. They are present on a headless localm serve with no plugins installed, not only under localm gui. /v1/server/restart and /v1/server/shutdown are documented under Server control above.

Route	Scope	Purpose
GET /api/issues	config:read	List the project's issues through the bug-report proxy. Takes ?state= (open, closed or all; default and fallback all). Returns {"available": bool, "issues": [...]}, with available false when no tracker is configured, and an error string when the proxy fails.
GET /api/update/check	config:read	Ask whether a newer build is available. Does not apply anything. Returns {"available": true, "current", "latest", "newer", "comparable", "notes", "published_at", "asset", "signature"}; {"available": false} when no updater is configured; {"available": true, "error": "..."} when the check fails.
GET /api/changelog	config:read	Return the shipped CHANGELOG.md. Local file read, no network call, works offline. Returns {"available": true, "version", "markdown"} with the released history and without the [Unreleased] section, or {"available": false} when the file is absent from this build.
POST /api/update/apply	config:write	Apply an available update. See SECURITY.md for the signing and rollback model. Returns {"applied": false, "reason": "already up to date", "current"} when there is nothing newer, {"applied": false, "error": "..."} when the apply failed and was rolled back, and otherwise {"applied": true, "version", "klass", "backup", "restart_needed", "available": true} plus "restarting": true for every class except a setup-class update. 400 when the updater is not configured or the release has no downloadable build; 502 when the check for the latest release fails. The backup path and any error text carry the full path for the owner and open mode, and a scrubbed one (data dir as <data>, file name kept) for a scoped key.
GET /api/update/rollback	config:read	Whether a rollback is possible, and which build it would restore. Read-only; never rolls anything back. Returns {"available", "backup", "version", "current"}, with version null when the backup does not record one. backup is the full path for the owner and open mode, and a scrubbed one for a scoped key.
POST /api/update/rollback	admin	Restore the previous build from the last update's local backup and restart into it. No signature check and no network request - it restores files, not a download. Requires admin specifically (not merely config:write), and restores files only, not a deps-class update's package installs. Returns {"rolled_back": true, "backup", "version", "restarting": true}; 403 for a key without admin, 409 when there is no backup or an update or rollback is already running, 500 when the restore failed partway (the backup is kept). Only open mode or an owner key reaches it, so its backup path and error texts are never scrubbed.
POST /api/bug-report	config:write	Generate and save a bug report for review.
POST /api/bug-report/send	config:write	File a saved, reviewed bug report.

A proxy failure is reported in the body, not the status. GET /api/issues and GET /api/update/check answer 200 with an error string when the proxy cannot be reached, rather than a 5xx. Check for error rather than branching on the status code alone. They are not blanket "always 200" routes though: the scope dependency still rejects an unauthorised caller before the handler runs.

net_mode: off stops GET /api/issues and GET /api/update/check. Each answers 200 with an error naming the setting (the update check still runs when the owner-only update_ignore_net_policy is on), and POST /api/update/apply fails with its 502 for the same reason. GET /api/changelog makes no network call at all.

GET /metrics

Scope: admin. Off by default.

Prometheus text-format metrics. The route does not exist until you turn on the metrics_enabled setting (Settings > Server, or localm config metrics_enabled true; LOCALM_METRICS=1 does the same for one run) and restart. With it off nothing is collected and /metrics answers 404.

Metric	Type	Labels
localm_http_requests_total	counter	method, route, status
localm_http_request_duration_seconds	histogram	method, route
localm_http_requests_in_flight	gauge	none
localm_prompt_tokens_total, localm_generated_tokens_total	counter	none
localm_time_to_first_token_seconds	histogram	none
localm_tokens_per_second	histogram	none
localm_inference_queue_depth	gauge	none
localm_models_loaded	gauge	none
localm_vram_used_bytes, localm_vram_total_bytes	gauge	none

route is the route template (/v1/models/{model_id}), never the requested path; anything that is not a plain route is other. No metric carries a prompt, a reply, a model name or a model path. The VRAM gauges come from the cached GPU reading and are left out until a trustworthy one exists; localm_inference_queue_depth counts requests waiting for a loaded model's inference slot (not requests still waiting on a model that was just evicted). A request that ends before any response is sent (the client left first) is counted with status 499. A reply the client abandons partway (a closed connection or a stop button) still counts its prompt, the tokens generated before it stopped, and its time to first token; a streamed request that leaves while still queued generated nothing and counts nothing. Counters live in memory and start from zero at every server start.

Access: a server with an API key needs an admin key (the owner key from localm key generate, or LOCALM_API_KEY). A server with no key answers only on a loopback bind, and then only with the per-process shell token the local GUI holds or this instance's local attach token, so a scraper on a keyless server needs a key first. A cross-origin browser request is refused.

scrape_configs:
  - job_name: localm
    authorization:
      credentials: <admin key>
    static_configs:
      - targets: ["127.0.0.1:8642"]

POST /api/bug-report

Scope: config:write.

Takes a JSON object. Recognised fields are description (alias message), what_i_expected, what_happened, include_log and client. It saves the report and never files it anywhere.

• At least one of description/message or what_happened must be non-empty, otherwise 400. what_i_expected on its own does NOT satisfy that check.
• A body that sets upload is refused with 400 and nothing is saved: filing goes through POST /api/bug-report/send with a report the user has seen.
• An absent or non-object body is a FastAPI validation 422, not the 400.
• A report that cannot be written to disk returns 500. A failed save is never reported as success. A save that takes too long returns 504.

On success the response carries saved (true), filename, path (scrubbed of machine paths for a caller other than the local operator), maintainer, and, when the saved file can be read back, report_markdown (its full text, for review and download) and report_sha256 (the hex SHA-256 of that text).

POST /api/bug-report/send

Scope: config:write.

Takes {"filename": ..., "sha256": ...}: a report saved by POST /api/bug-report, and the report_sha256 that route returned for it. The saved file is filed as an issue through the configured proxy only when its current text still has that digest, so what is filed is the text that was shown, apart from its closing note about editing the report, which is left out. The issue title is the report's own heading.

• A missing digest, or a filename that is not a bare saved-report name (bug-<stamp>.md), returns 400. A file that cannot be read returns 404. A file whose text no longer matches the digest returns 409, and nothing is sent.
• Past those checks the response is a 200 carrying filename and maintainer. A failed upload (including one that times out) has uploaded: false plus upload_stage, upload_message and upload_error; the file stays saved. A rate-limited upload additionally sets rate_limited: true and retry_after in seconds. A successful upload sets uploaded: true and, when the proxy returned one, issue_url.

The optional client object is reduced server-side before it is written into the report, so a client cannot inflate a report with arbitrary content.

The bugreport_upload_* and update_* config keys that back these routes widen a trust boundary and are owner-only; see the note under GET /v1/config above.

Plugin management endpoints

These endpoints are present under localm serve too, not just localm gui. They are scope-gated: GET /api/plugins requires PLUGINS_READ, and the mutations require PLUGINS_ADMIN - except the install-deps event stream below, which is a GET that also requires PLUGINS_ADMIN. A {name} that is not a legal plugin id is a 404 on every {name} route.

GET /api/plugins

Returns {plugins: [...], errors: {...}, suggest_plugins, auto_install_plugin_deps}. Each plugin entry carries the state flags installed, enabled, active, available, and loaded, plus name, version, description, scope, builtin, protected, tab, label, icon, group, client_entry, assets_base, requires, requires_extras, missing_deps, missing_requires, tool_exports, commands, extra, and error.

POST /api/plugins/{name}/install / POST /api/plugins/{name}/uninstall

Move a plugin between the available catalog and the installed set. Install copies the plugin from the store into <data dir>/plugins/ and enables it; uninstall removes the installed directory and accepts ?delete_data= to also drop the plugin's stored data. They return {"status": "installed" | "uninstalled", "name"}. Uninstalling a protected plugin (chat) returns 409; an unknown plugin returns 404; a failed install or uninstall returns 400; an uninstall that disabled and unloaded the plugin but could not remove everything it owns from disk returns 500.

POST /api/plugins/refresh / POST /api/plugins/{name}/refresh

Refresh plugin state from disk without a server restart. The bare route refreshes every installed plugin and returns {"status": "ok" | "partial", "refreshed": [...], "failed": {name: error}}, with partial and the failed plugins named in failed when some could not be refreshed; the {name} route refreshes just that plugin and returns {"status": "refreshed"|"up-to-date", "name": name}. An unknown, uninstalled or non-builtin plugin returns 404; any other refresh failure of the {name} route returns 500.

POST /api/plugins/{name}/enable / POST /api/plugins/{name}/disable

Toggle an installed plugin active or inactive in config. Disabling a protected plugin (chat) returns 409; enabling a plugin that is not installed returns 409; an unknown plugin returns 404 from enable; any other failure to enable returns 400. They return {"status": "enabled" | "disabled", "name"}. disable does not check whether a well-formed name is a real plugin - an unknown name returns 200 the same as a real one.

POST /api/plugins/install-external

PLUGINS_ADMIN scope, same as the routes above. Installs a third-party plugin from a local directory ({"source": "/path/to/plugin"}, which must contain a plugin.toml; force replaces an installed copy) - the HTTP sibling of localm plugin install <dir>. The plugin loads on the next server start. Returns {"status": "installed", "name", "version"}; a missing source, a directory without a plugin.toml, or a rejected plugin is a 400.

POST /api/plugins/{name}/install-deps / GET /api/plugins/{name}/install-deps/events

PLUGINS_ADMIN scope. Installs a plugin's pip extras on this host (the HTTP sibling of localm plugin install-deps NAME). The POST starts the install in the background and returns {"status": "running"|"done"|"error", "name", "missing", "lines"}; an unknown plugin is a 404. Both routes are a 403 on a server bound beyond loopback, because a remote client may never trigger a server-side pip. The events route replays the whole log and then follows it as server-sent events: data: {"type": "log", "line": "..."} per line, then data: {"type": "end", "ok", "installed", "failed", "error"}, with : keepalive comments between; it is a 404 when no install was ever started for that plugin.

API key endpoints

Scope: keys:admin.

Route	Purpose
GET /v1/keys	List every scoped key (metadata only, never the plaintext key). Returns {"keys": [{"id", "name", "scopes", "created", "expires", "last_used", "fs_access", "rag_roots"}], "is_owner", "presets", "owner_key_warning"}: whether the caller is the owner, the configured key presets, and owner_key_warning, a notice (or null) that the owner key file exists but holds no key, so the server runs keyless; it names the key file: the full path for the owner and open mode, a scrubbed one for a scoped key.
POST /v1/keys	Mint a new scoped key. Body: name, scopes (list of scope strings), fs_access (none or host), rag_roots (list of folder paths to confine RAG indexing to; omit for the key's normal, unconfined RAG reach), and either expires_in (seconds from now) or expires (epoch seconds); returns {"id", "name", "scopes", "expires", "fs_access", "rag_roots", "key"} (plus warnings when a granted scope unlocks a plugin that is not installed or not ready), with the plaintext key exactly once. A scopes that is not a list, an expires_in or expires that is not a number, a rag_roots that is not a list, or a scope that is neither a shipped scope nor a plain plugin-name identifier is a 400. Only an owner/admin caller may grant a privileged scope (admin, plugins:admin, keys:admin, config:write, coder:full), fs_access=host, or a non-empty rag_roots list - each is host-filesystem reach in some form, and a non-owner request for any of them gets 403. Minting the very first key from an open-mode loopback GUI also seeds an owner key and hands that browser a session cookie, so the local user is never locked out. 409 when the key store file exists but cannot be read as a JSON list; nothing is written.
DELETE /v1/keys/{key_id}	Revoke a key. Returns {"status": "revoked", "id"}; 404 if no such key exists, 409 when the key store file cannot be read as a JSON list.

POST /api/auth/key/clear (scope config:write; CSRF-protected for cookie-authenticated callers) deletes the server-side owner key and signs out every session - the HTTP sibling of localm key clear and the GUI's clear button (403 when a cookie-authenticated caller sends no valid CSRF token). It does two things that can fail independently, and reports on both: removing the stored credential, and revoking every browser session. Either one surviving still grants access, so {"cleared": true, "warnings": []} is returned ONLY when both completed. If either could not be finished, the response is {"cleared": false, "warnings": [...]}, naming what survived: a credential the server could not remove (a virus scanner or search indexer holding the file open, for example), or "browser sessions (some devices may still be signed in)" when the session store could not be written. The labels are deliberately path-free.

POST /api/auth/key/rotate (scope admin - not merely config:write, since setting a caller-chosen key is privilege-equivalent to minting yourself owner) rolls or sets the owner key - the HTTP sibling of localm key generate / localm key set <key>. An empty or omitted body mints a fresh random key; {"key": "<value>"} persists that exact one (an empty/whitespace key still GENERATES, so this route can never drop the server to open mode - use /api/auth/key/clear for that). Returns {"rotated", "active", "key", "warnings"}: key is the new key, shown once; active is whether the server's next request will actually accept it (false when LOCALM_API_KEY in the environment overrides the stored key, or on an unexplained write/read-back mismatch, either surfaced in warnings). Existing browser sessions are NOT revoked by a roll (sessions are decoupled from the key's value); the compromise-recovery path that does revoke them is localm key recover, run locally. A body that is not a JSON object, a key that is not a string, or a key the store refuses (too short, or with characters an Authorization header cannot carry) is a 400, and the previous key stays.

POST /api/session/logout reports the same way: it always clears the calling browser's own cookie, and answers {"authed": false, "warnings": [...]} with a warnings entry when the server-side session record could not be dropped (403 for a cookie-authenticated call without a valid CSRF token).

The two other session routes stay public. POST /api/session exchanges a key, sent as {"key": "..."} or as a bearer token, for the HttpOnly localm_session cookie and answers {"authed": true, "scopes": [...], "csrf": "..."}, where csrf is the token to send as X-CSRF-Token on later changes; it is a 401 for a key that does not verify and, once past the open-mode gate above, a 400 in open mode, where there is nothing to log into. GET /api/session answers {"authed", "scopes", "required", "csrf", "session_mode"}: required is true when a key is configured (or require_auth is on), csrf is empty unless the caller holds a cookie session, and session_mode is the mode a chat from this caller is recorded under.

Knowledge (RAG) endpoints (rag plugin)

These routes exist only when the rag plugin is installed and enabled; otherwise they 404. Scope: rag. See rag.md for the retrieval design, indexing confinement, and CLI equivalents.

Across the collection routes: a name that is not a legal collection name is a 400, and an unknown collection a 404 (create excepted). The routes that read or change an existing collection's content (detail, delete, query, reembed, repair, remove-doc) answer 403 when the key's rag_roots confine it to folders and the collection holds documents indexed from outside them; the listing leaves such collections out. Create, delete and remove-doc answer 409 while another process is writing the collection. A collection's stats are name, created, n_docs, n_missing, n_roots, n_chunks, has_vectors, corrupt, chunks_bad_lines, vector_degrade_reason and vector_dim.

Route	Purpose
GET /api/rag/collections	List every collection with its stats (chunk/doc counts, has_vectors, dim_mismatch against whatever embedder is currently loaded). Returns {"collections": [...]}; dim_mismatch is null when it cannot be told.
POST /api/rag/collections	Create an empty collection. {"name": ...}; returns its stats; 409 if it already exists.
GET /api/rag/collections/{name}	One collection's stats plus docs (each {"path", ...}) and dim_mismatch. 404 for an unknown name.
DELETE /api/rag/collections/{name}	Delete a collection and its on-disk index. Returns {"status": "deleted", "name"}; 404 if it does not exist.
POST /api/rag/collections/{name}/add	Index paths into a collection. Always starts a background job and returns {"job_id": ...} immediately, on localm gui and a headless localm serve alike. Follow progress with GET /api/jobs/{id}/events (see jobs.md). Body: paths (1 to 50), embed (default true; degrades to lexical-only when no embedder is available), reindex (default false; re-index unchanged files). 400 for no paths, more than 50, a path the indexing policy denies outright, or one that does not exist; for a path outside the allowed folders the owner gets a 409 {"needs_consent": true, "reason": "outside_allowed", "addable": [...], "detail"} and any other key a 403; 503 when the server has no job registry.
POST /api/rag/collections/{name}/upload	Same, for uploaded files; same {"job_id": ...} response. Body: files (1 to 50 of {"filename", "content_b64"}), embed, reindex. 400 for no files, more than 50, or invalid base64; 413 for a file over 30 MB or a request over 100 MB.
POST /api/rag/collections/{name}/query	Semantic/lexical search against one collection; body query (empty is a 400), k (default 4, clamped to 1 to 20), relevant_only, rerank (null follows the rag_rerank setting), context; returns up to k hits; with "relevant_only": true it drops hits below the collection's relevance floor, so an unrelated query returns none; "context" (the user's previous message, cut to 600 characters) is searched together with a relevant_only query that refers back to the conversation, and the response's context_used says whether it was. The response is {"collection", "query", "hits", "relevant_only", "reranked", "rerank_model", "rerank_note", "context_used"}. Chunk text is defanged of control/frame tokens before it comes back, since it is untrusted content headed for a chat prompt.
POST /api/rag/collections/{name}/reembed	Recompute the collection's vectors with the currently-loaded embedding model, from the stored chunk text (no re-read of the source files). Job-backed ({"job_id": ...}); 400 when no embedding model is available.
POST /api/rag/collections/{name}/repair	Re-index every rebuildable document (mirrors localm rag repair). Upload-only documents cannot be rebuilt (the uploaded bytes are never retained) and are reported, not silently dropped. Returns {"needs_confirm": true, ...} instead of starting a job when repairing without embeddings would remove existing vectors; re-POST with confirm: true to proceed. Body: embed (default true), confirm. Returns {"job_id": ...} otherwise; 400 when the collection has no documents or every document was uploaded.
POST /api/rag/collections/{name}/remove-doc	Remove one indexed document ({"path": ...}) from a collection. Returns {"status": "removed", "path"}; 404 if it is not indexed there.
POST /api/rag/extract	Extract plain text from an uploaded chat attachment, entirely in memory (30 MB cap). Body: filename, content_b64, max_chars (omit for the whole text; otherwise clamped to 500 to 200000). Returns {"filename", "text", "chars", "truncated"}; 400 for invalid base64, 413 over 30 MB, 422 when no text can be extracted. Not used by collection indexing. The GUI's Chat and Coder attachments use POST /api/chat/extract instead: the same request and answer, served by the built-in chat plugin (chat scope), so it works without this plugin.
GET /api/rag/rerank	Whether Knowledge results are reranked. Returns {"enabled", "installed", "model", "candidates", "note"}: the rag_rerank setting, the registered reranker names, the reranker a query would use (null when none), how many hits it would see, and why a wanted rerank cannot run. Never loads a model.
GET /api/rag/embedding	Current embedding-model config and availability: {"model", "default", "internal", "installed", "dim", "error", "gpu_fallback_reason", "status", "can_download"}, with status one of ready, not_installed and unknown. Never loads a model; 504 if the embedder state cannot be read. A non-owner caller gets a withheld/generic answer for a model that is not one of localm's known internal ones or a registered model.
POST /api/rag/embedding/download	One-time download of the currently-configured embedding model, when the network policy would otherwise not fetch it automatically (net_mode: ask). Needs config:write - the same scope that governs net_mode itself. Writes nothing to config; net_mode: off refuses it with a 409 unless net_allow_model_downloads is on. Returns {"job_id", "model"}, or {"status": "already_installed", "model"} when the model is on disk; 403 without config:write, 409 when the configured model is not one of the internal downloadable ones. See network.md.
POST /api/rag/embedding	Select the embedding model (and install it, if it is one of localm's known internal models and not yet on disk). Two-step by confirm: without it, this is a dry run - no job starts, and the response is {"needs_confirm": true, "model", "collections", "note"} describing what switching would affect. Re-POST with confirm: true to actually switch; that starts a job and returns {"job_id": ...}. Changing the model requires an owner (admin-scoped) key (403 otherwise, for the dry run too); an empty model is a 400, and a second setup while one is running is a 409.

Memory endpoints (memory plugin)

These routes exist only when the memory plugin is installed and enabled; otherwise they 404. Scope: memory. Memory is owner-scoped and off in privacy mode: every write returns 403 in privacy mode (no new traces). See memory.md. A write also answers 409 while another process holds the namespace.

Route	Purpose
GET /api/memory	The current facts (text + per-item metadata), pending correction proposals (see below), and whether writes are allowed. Returns {"text", "writable", "items": [{"id", "text", "importance", "source", "kind", "uses", "updated"}], "corrections": [{"id", "action", "proposed_text", "target_id", "target_text", "confidence", "created", "target_updated"}], "path", "can_download_embedder", "embedder_model"}; corrections is empty when writes are not allowed.
PUT /api/memory	Bulk edit. A line matching an existing fact keeps that record; new lines are added; omitted lines are deleted. Body {"text": ...}; returns {"status": "saved", "count"}. 413 past the 256-record or 64k-char cap.
POST /api/memory/append	Add one fact. Body {"text": ...}; returns {"status": "appended", "id"}; 400 for empty text, 413 at the record cap.
PATCH /api/memory/{id}	Edit one record's text or importance. Returns {"status": "saved", "item"}; 404 for an unknown id.
DELETE /api/memory/{id}	Delete one record. Returns {"status", "id"} with status deleted or absent; an unknown id is a 200 with absent.
POST /api/memory/corrections/{id}/accept	Apply a proposed correction (the auto-synthesis job spotted a new fact that contradicts a saved one): replaces or deletes the target record, archiving its old value. Returns {"status", "id", "target_id", "action"} with status updated, deleted or target_gone; 404 for an unknown correction, 500 when the old value could not be archived (nothing is applied).
POST /api/memory/corrections/{id}/reject	Discard a proposed correction, leaving the target record untouched. Returns the same object with status rejected or target_gone, and fails like accept.
POST /api/memory/consolidate	Distil durable facts from recent sessions now; 503 until a model is loaded. Returns {"status", "added", "updated", "deleted", "proposed", "pending", "episodic", "pruned", "facts"}, or {"status": "skipped", "reason": "no_sessions", "added": 0} when there is no session to read.
GET /api/memory/forgotten	Archived (forgotten) records available for recovery - read-only, available even in privacy mode. Returns {"items": [{"id", "text", "importance", "source", "kind", "forgotten_at"}]}.
POST /api/memory/forgotten/{id}/restore	Recover one archived record back into the live store; 404 if no matching archive entry exists. Returns {"status": "restored", "item"}.

When the memory plugin recalls facts for a chat turn, POST /v1/chat/completions returns an X-Localm-Memory response header: a compact JSON blob with the count of injected facts, the items, and a degrade reason when semantic recall could not be used (for example no_embedder before localm setup-embeddings).

Client example

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8642/v1", api_key="localm")

stream = client.chat.completions.create(
    model="mymodel",
    messages=[{"role": "user", "content": "Tell me a story."}],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="", flush=True)

Behaviour notes

• Concurrency: a GGUF model answers up to parallel_slots requests at the same time (default auto: 4, or 1 while speculative drafting is on), decoding them together in one batch per step; further requests queue in order and stream a waiting status meanwhile. All of them share the model's one context window: a request that does not fit beside the running ones waits for them. A reply decoded beside others is not bit-identical to the same request decoded alone, even at temperature 0. A turn with an image runs on its own, after the replies already running. Other backends answer one request at a time per model.
• Prompt cache: a GGUF model keeps the tokens of earlier requests in its KV cache and computes only the part of a new prompt that they do not already cover. llama.cpp's results can depend slightly on how many tokens it computes in one step (it picks kernels by batch size, and with model layers on the CPU and a GPU present it can run a large step on the GPU and a small one on the CPU), so the same request can get a different reply depending on the requests before it, even at temperature 0 or with a fixed seed. Greedy output changes where the two most likely tokens are close. Send cache_prompt: false for a reply that depends only on the request.
• Speculative drafting: with spec_source set to ngram, the same request at temperature 0 can get a different reply from one server run to the next, and two identical requests in a row can differ. With spec_source off that did not happen. The mtp and draft sources have not been tested for this.
• Context: the window starts at n_ctx and grows on demand up to n_ctx_max (see the dynamic context window section of architecture.md). Conversations that outgrow the ceiling get a clear error instead of an OOM.
• GUI-only routes: only a handful of /api/models/* model-switching routes are actually gated behind localm gui. Everything else plugin-contributed - coder sessions (/api/coder/*), the Knowledge routes above, image/music/video generation, and the /api/plugins* management routes above - mounts the moment its plugin is enabled, on a headless localm serve too. See gui.md.