Skip to content

memory: allow a self-hosted embeddings endpoint - #420

Open
nikita-vanyasin wants to merge 1 commit into
ClickHouse:mainfrom
nikita-vanyasin:feat/configurable-embeddings-endpoint
Open

memory: allow a self-hosted embeddings endpoint#420
nikita-vanyasin wants to merge 1 commit into
ClickHouse:mainfrom
nikita-vanyasin:feat/configurable-embeddings-endpoint

Conversation

@nikita-vanyasin

@nikita-vanyasin nikita-vanyasin commented Sep 2, 2026

Copy link
Copy Markdown
Member

Problem

The memU bridge hardcoded the embeddings host, so embeddings only ever worked against OpenAI:

if self.config.openai_api_key:
    llm_profiles["embedding"] = {
        "base_url": "https://api.openai.com/v1",
        ...

On a Bedrock install that means no vector recall at all. _BedrockLLMClient.embed() raises NotImplementedError and the client is AsyncAnthropicBedrock, which only speaks Anthropic models, so Titan is not reachable through it either. Those installs fall back to LLM-based recall, which spends Sonnet or Haiku tokens on every query. Any other install has to send memory text to OpenAI and hold a static third-party key.

memU needs no change for this. OpenAISDKClient.__init__ already takes a base_url and builds a normal AsyncOpenAI, then embeds with client.embeddings.create(model=self.embed_model, input=inputs). Nerve was the only thing pinning the host.

Change

New memory.embed_base_url accepts any endpoint implementing OpenAI /v1/embeddings, self-hosted or proxied.

The gate widens from openai_api_key to a key or a base URL, so an endpoint on its own enables embeddings. It does its own auth, which makes openai_api_key optional; AsyncOpenAI still rejects an unset key, hence the "placeholder".

run_non_interactive reads NERVE_EMBEDDINGS_API_ENDPOINT and NERVE_EMBEDDINGS_MODEL. The model variable is needed because a self-hosted gateway will not serve text-embedding-3-small under that name, and without it the hardcoded default is baked in.

The profile logic moved into a pure build_embedding_profile(). It was previously inline in _initialize_impl, which stands up a real MemoryService and so could not be tested.

Two follow-on fixes:

  • _check_openai_key probes the configured host rather than always OpenAI. It would otherwise report a failure at the end of nerve init for any custom endpoint.
  • nerve doctor reports the endpoint, and returns [ERR] when a base URL is set with no embed_model.

Compatibility

Defaults are unchanged: no base URL means api.openai.com gated on the key, as before. embed_base_url is a new dataclass field, so unknown-key validation picks it up automatically and older configs parse identically.

An endpoint set without embed_model disables embeddings with a warning rather than calling the API with model="", which fails opaquely at query time. from_dict coerces None to "", so a bare embed_base_url: key does not become the string "None".

embed_base_url is written to the portable settings.yaml layer, since it describes the deployment. The Bedrock model rewrite only touches recall_model, memorize_model and fast_model, so it leaves this key alone.

Tests

13 new tests. TestBuildEmbeddingProfile covers the gating matrix: endpoint alone, key alone, endpoint overriding a set key, and both no-model paths. TestNonInteractiveEmbeddingsEndpoint covers the env plumbing, including that the Bedrock rewrite leaves the embed keys intact and that an omitted variable leaves the config key absent. Config parsing covers defaults, whitespace and None.

Full suite: 3378 passed.

Three TestCredentialWaterfall tests fail on my machine, but they also fail on a clean main worktree and still fail with ANTHROPIC_API_KEY unset, so a locally stored credential is leaking into patch.dict(..., clear=False). Unrelated to this change and expected to pass in CI.

Untested assumption

This assumes the target endpoint returns the OpenAI response shape, data[].embedding. A gateway that differs needs its own client. Bedrock embeddings through a separate bedrock-runtime call against amazon.titan-embed-text-v2:0 would be the fallback for that case, and is out of scope here.

🤖 Generated with Claude Code

The memU bridge hardcoded base_url to api.openai.com, so embeddings were
only available from OpenAI. A Bedrock-only install could not use vector
recall at all, because the Bedrock client's embed() raises
NotImplementedError.

memU's OpenAISDKClient already accepts an arbitrary base_url, so make it
configurable as memory.embed_base_url, seeded from
NERVE_EMBEDDINGS_API_ENDPOINT and NERVE_EMBEDDINGS_MODEL for unattended
setup. A base URL alone now enables embeddings; the endpoint does its own
auth, so openai_api_key becomes optional.

Defaults are unchanged. An endpoint set without embed_model disables
embeddings with a warning instead of calling the API with model="".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@nikita-vanyasin
nikita-vanyasin force-pushed the feat/configurable-embeddings-endpoint branch from 952b8e3 to 1c50752 Compare September 2, 2026 10:09
# work goes through Anthropic — use Haiku for extraction
# too to avoid saturating the rate-limit budget.
"memory_extract_llm_profile": (
fast_profile if not self.config.openai_api_key

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This still uses the old gate based on the openai_api_key presence.

},
},
retrieve_config={
"method": "llm" if not self.config.openai_api_key else "rag",

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above.

# memorize pipeline's "categorize_items" step with one that
# stores items and resources with embedding=None. This
# avoids KeyError on the missing "embedding" LLM profile.
if not self.config.openai_api_key:

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above.

@property
def _has_embeddings(self) -> bool:
"""Whether an embedding provider (e.g. OpenAI) is configured."""
return bool(self.config.openai_api_key)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants