OpenAI compatible LLM endpoint? #480
tftranslate
started this conversation in
Ideas
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
I just discovered AnythingMCP as a potential MCP for my bank. This sounds great! Before venturing further, I have a question:
I see that for the (optional) AI-baed features you support OpenRouter, OpenAI and Anthropic.
I would much appreciate if you could add "OpenAI compatible" here, i.e. another environment variable setting where I can specify a different endpoint URL than the OpenAI default one (otherwise, the code logic is the same).
Reason: With the advent of platforms such as Strix Halo and DGX Spark, self-hosted models now are capable enough to run tasks similiar to the ones anythingmcp can use without the need for cloud services. In order to connect to a self-hosted llama.cpp, we need to be able to connect to an "OpenAI compatible" endpoint.
All reactions