feat: add livekit-plugins-thegrid - #7138
Conversation
|
|
cd232dc to
730f263
Compare
The Grid is an OpenAI-compatible inference marketplace that addresses capability tiers rather than a specific lab's model name. Adds an LLM plugin wrapping the OpenAI plugin, following livekit-plugins-perplexity.
730f263 to
8180fae
Compare
|
Good catch, and your suggestion was right. Applied, with one extra step it needed. Guarding the raise on It now resolves the key into a local and passes Verified all three paths:
Separately, I have added the missing |
What
Adds
livekit-plugins-thegrid, an LLM plugin for The Grid — an OpenAI-compatible inference marketplace.Modeled on
livekit-plugins-perplexity. The plugin subclasses the OpenAI plugin'sLLMand sets the base URL andTHEGRID_API_KEYhandling; there's no bespoke client.Registration points, matching how perplexity/cerebras are wired:
livekit-plugins/livekit-plugins-thegrid/(new package)livekit-plugins-thegrid = { workspace = true }in the rootpyproject.tomlthegrid = ["livekit-plugins-thegrid>=1.8.0"]extra inlivekit-agents/pyproject.tomltests/test_plugin_thegrid.pyLeft
CHANGELOG.mdand the version manifests alone per CONTRIBUTING.One deliberate difference from the perplexity template
Perplexity passes
_strict_tool_schema=False. I tested The Grid withstrict: truetool schemas (andadditionalProperties: false) and it returns well-formed tool calls, so I did not carry that flag over — the default applies.Verification
Run against the live API, not mocked:
That covers chat, a tool call round-trip on
agent-standard, and streaming. Also checkedruff format --checkandruff checkclean on the new files, that the missing-key path raises, and thatGET /v1/modelsreturns 200 directly with no redirect (relevant because_prewarm_impl()calls it with a 5s timeout).Note on models
The Grid addresses capability tiers —
text-standard,code-prime,agent-maxeach route to a current model for that tier, so an agent keeps working when the underlying model is swapped.models.pylists the tiers plus the lab-pinned*-latestids;modelstill accepts anystr, andGET /v1/modelsis authoritative.Disclosure: I work on The Grid. Happy to adjust anything to house style.