memory: allow a self-hosted embeddings endpoint - #420
Open
nikita-vanyasin wants to merge 1 commit into
Open
Conversation
The memU bridge hardcoded base_url to api.openai.com, so embeddings were only available from OpenAI. A Bedrock-only install could not use vector recall at all, because the Bedrock client's embed() raises NotImplementedError. memU's OpenAISDKClient already accepts an arbitrary base_url, so make it configurable as memory.embed_base_url, seeded from NERVE_EMBEDDINGS_API_ENDPOINT and NERVE_EMBEDDINGS_MODEL for unattended setup. A base URL alone now enables embeddings; the endpoint does its own auth, so openai_api_key becomes optional. Defaults are unchanged. An endpoint set without embed_model disables embeddings with a warning instead of calling the API with model="". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
nikita-vanyasin
force-pushed
the
feat/configurable-embeddings-endpoint
branch
from
September 2, 2026 10:09
952b8e3 to
1c50752
Compare
| # work goes through Anthropic — use Haiku for extraction | ||
| # too to avoid saturating the rate-limit budget. | ||
| "memory_extract_llm_profile": ( | ||
| fast_profile if not self.config.openai_api_key |
Collaborator
There was a problem hiding this comment.
This still uses the old gate based on the openai_api_key presence.
| }, | ||
| }, | ||
| retrieve_config={ | ||
| "method": "llm" if not self.config.openai_api_key else "rag", |
| # memorize pipeline's "categorize_items" step with one that | ||
| # stores items and resources with embedding=None. This | ||
| # avoids KeyError on the missing "embedding" LLM profile. | ||
| if not self.config.openai_api_key: |
| @property | ||
| def _has_embeddings(self) -> bool: | ||
| """Whether an embedding provider (e.g. OpenAI) is configured.""" | ||
| return bool(self.config.openai_api_key) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The memU bridge hardcoded the embeddings host, so embeddings only ever worked against OpenAI:
On a Bedrock install that means no vector recall at all.
_BedrockLLMClient.embed()raisesNotImplementedErrorand the client isAsyncAnthropicBedrock, which only speaks Anthropic models, so Titan is not reachable through it either. Those installs fall back to LLM-based recall, which spends Sonnet or Haiku tokens on every query. Any other install has to send memory text to OpenAI and hold a static third-party key.memU needs no change for this.
OpenAISDKClient.__init__already takes abase_urland builds a normalAsyncOpenAI, then embeds withclient.embeddings.create(model=self.embed_model, input=inputs). Nerve was the only thing pinning the host.Change
New
memory.embed_base_urlaccepts any endpoint implementing OpenAI/v1/embeddings, self-hosted or proxied.The gate widens from
openai_api_keyto a key or a base URL, so an endpoint on its own enables embeddings. It does its own auth, which makesopenai_api_keyoptional;AsyncOpenAIstill rejects an unset key, hence the"placeholder".run_non_interactivereadsNERVE_EMBEDDINGS_API_ENDPOINTandNERVE_EMBEDDINGS_MODEL. The model variable is needed because a self-hosted gateway will not servetext-embedding-3-smallunder that name, and without it the hardcoded default is baked in.The profile logic moved into a pure
build_embedding_profile(). It was previously inline in_initialize_impl, which stands up a realMemoryServiceand so could not be tested.Two follow-on fixes:
_check_openai_keyprobes the configured host rather than always OpenAI. It would otherwise report a failure at the end ofnerve initfor any custom endpoint.nerve doctorreports the endpoint, and returns[ERR]when a base URL is set with noembed_model.Compatibility
Defaults are unchanged: no base URL means
api.openai.comgated on the key, as before.embed_base_urlis a new dataclass field, so unknown-key validation picks it up automatically and older configs parse identically.An endpoint set without
embed_modeldisables embeddings with a warning rather than calling the API withmodel="", which fails opaquely at query time.from_dictcoercesNoneto"", so a bareembed_base_url:key does not become the string"None".embed_base_urlis written to the portablesettings.yamllayer, since it describes the deployment. The Bedrock model rewrite only touchesrecall_model,memorize_modelandfast_model, so it leaves this key alone.Tests
13 new tests.
TestBuildEmbeddingProfilecovers the gating matrix: endpoint alone, key alone, endpoint overriding a set key, and both no-model paths.TestNonInteractiveEmbeddingsEndpointcovers the env plumbing, including that the Bedrock rewrite leaves the embed keys intact and that an omitted variable leaves the config key absent. Config parsing covers defaults, whitespace andNone.Full suite: 3378 passed.
Three
TestCredentialWaterfalltests fail on my machine, but they also fail on a cleanmainworktree and still fail withANTHROPIC_API_KEYunset, so a locally stored credential is leaking intopatch.dict(..., clear=False). Unrelated to this change and expected to pass in CI.Untested assumption
This assumes the target endpoint returns the OpenAI response shape,
data[].embedding. A gateway that differs needs its own client. Bedrock embeddings through a separatebedrock-runtimecall againstamazon.titan-embed-text-v2:0would be the fallback for that case, and is out of scope here.🤖 Generated with Claude Code