Skip to content

feat: add livekit-plugins-thegrid - #7138

Open
CastilloLuis wants to merge 1 commit into
livekit:mainfrom
the-gridai:thegridai/thegrid-llm-plugin
Open

feat: add livekit-plugins-thegrid#7138
CastilloLuis wants to merge 1 commit into
livekit:mainfrom
the-gridai:thegridai/thegrid-llm-plugin

Conversation

@CastilloLuis

Copy link
Copy Markdown

What

Adds livekit-plugins-thegrid, an LLM plugin for The Grid — an OpenAI-compatible inference marketplace.

Modeled on livekit-plugins-perplexity. The plugin subclasses the OpenAI plugin's LLM and sets the base URL and THEGRID_API_KEY handling; there's no bespoke client.

Registration points, matching how perplexity/cerebras are wired:

  • livekit-plugins/livekit-plugins-thegrid/ (new package)
  • livekit-plugins-thegrid = { workspace = true } in the root pyproject.toml
  • thegrid = ["livekit-plugins-thegrid>=1.8.0"] extra in livekit-agents/pyproject.toml
  • tests/test_plugin_thegrid.py

Left CHANGELOG.md and the version manifests alone per CONTRIBUTING.

One deliberate difference from the perplexity template

Perplexity passes _strict_tool_schema=False. I tested The Grid with strict: true tool schemas (and additionalProperties: false) and it returns well-formed tool calls, so I did not carry that flag over — the default applies.

Verification

Run against the live API, not mocked:

$ pytest tests/test_plugin_thegrid.py --plugin thegrid -q
...                                                                      [100%]
3 passed in 6.34s

That covers chat, a tool call round-trip on agent-standard, and streaming. Also checked ruff format --check and ruff check clean on the new files, that the missing-key path raises, and that GET /v1/models returns 200 directly with no redirect (relevant because _prewarm_impl() calls it with a 5s timeout).

Note on models

The Grid addresses capability tiers — text-standard, code-prime, agent-max each route to a current model for that tier, so an agent keeps working when the underlying model is swapped. models.py lists the tiers plus the lab-pinned *-latest ids; model still accepts any str, and GET /v1/models is authoritative.

Disclosure: I work on The Grid. Happy to adjust anything to house style.

@CastilloLuis
CastilloLuis requested a review from a team as a code owner September 5, 2026 17:18
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

devin-ai-integration[bot]

This comment was marked as resolved.

@CastilloLuis
CastilloLuis force-pushed the thegridai/thegrid-llm-plugin branch 2 times, most recently from cd232dc to 730f263 Compare September 6, 2026 15:03
The Grid is an OpenAI-compatible inference marketplace that addresses
capability tiers rather than a specific lab's model name. Adds an LLM
plugin wrapping the OpenAI plugin, following livekit-plugins-perplexity.
@CastilloLuis
CastilloLuis force-pushed the thegridai/thegrid-llm-plugin branch from 730f263 to 8180fae Compare September 6, 2026 17:36
@CastilloLuis

Copy link
Copy Markdown
Author

Good catch, and your suggestion was right. Applied, with one extra step it needed.

Guarding the raise on client is None alone was not sufficient: this class still forwarded api_key="" to the OpenAI plugin, and llm.py:152 rejects an explicitly-given empty string with "OpenAI API key is required". So an injected client failed one line further down instead.

It now resolves the key into a local and passes resolved_key or NOT_GIVEN, which leaves the OpenAI plugin free to use the supplied client's own authentication.

Verified all three paths:

supplied client without key: OK, model = text-standard
no client, no key: still raises ValueError OK
env key path still works: text-standard

pytest tests/test_plugin_thegrid.py --plugin thegrid still passes 3/3 against the live API, and ruff check / ruff format --check are clean on 0.15.22.

Separately, I have added the missing uv.lock entry for the new workspace member in this branch, which is what was failing the ruff job (it died at uv sync before lint ran).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants