feat: add native Groq provider - #73
Merged
Merged
Conversation
JNK234
force-pushed
the
feat/groq-provider
branch
from
August 13, 2026 23:46
881202f to
7f3aca4
Compare
Groq serves open-weight models on LPU hardware through an OpenAI-compatible API. Its free tier is fast and generous, making it the best free option for workshops and classrooms. Previously reachable only via an untested OpenAI-compatible base_url workaround. - Add GroqProvider extending OpenAICompatibleProvider, with reasoning_format "parsed" on reasoning requests and thinking extraction from both message.reasoning and <think> tags - Register the groq descriptor with gsk_ key prefix and exposesThinking=true - Add groq section to models.yaml listing only currently live models - Add groq to the ModelRegistry fallback map and reasoning model detection - Add deterministic tests for descriptor defaults, help text, model switching, and chat history; bump provider counts from 6 to 7 - Document Groq setup in PROVIDER-GUIDE.md and SETUP.md Default model is openai/gpt-oss-20b rather than a Llama model: Groq's llama-3.1-8b-instant and llama-3.3-70b-versatile are scheduled for shutdown on 2026-08-16, and gpt-oss-20b is Groq's own recommended replacement. Closes #51
llm:set-reasoning-effort validated against one hardcoded global set, so "xhigh" was accepted by the extension and then rejected by Groq with an HTTP 400 mid-run. Verified live before and after this change. Effort levels are now declared per provider on ProviderDescriptor, with the OpenAI-style set as the default for providers that do not override. Groq's accepted values were probed one at a time against the live API rather than read from its docs, which turned out to matter: the API's own rejection message lists none|default|low|medium|high, but that is the union across model families. The default model (openai/gpt-oss-20b) accepts only low|medium|high — "none" and "default" both 400 against it. The descriptor declares the set that is safe on the default model. Verified end to end against live Groq: "xhigh" and "none" are now refused up front with a message naming the provider and the valid values, and "high" completes a real chat. 74 tests pass, including a new block covering both directions.
JNK234
force-pushed
the
feat/groq-provider
branch
from
August 13, 2026 23:55
7f3aca4 to
91c38b9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #51 (roadmap B1).
Why
Groq was reachable only through the OpenAI-compatible workaround — untested and undocumented as a supported path. Its free tier is fast and generous, which makes it the most useful zero-cost option for workshops and classrooms, where throughput matters more than model quality.
What
GroqProvider.scalaextendingOpenAICompatibleProvider(90 lines, mirrorsTogetherProvider)groqdescriptor inProviderRegistrations.scala,apiKeyPrefix = Some("gsk_")groq:section inmodels.yamllisting only currently live modelsgroqadded toModelRegistry.FALLBACK_CONFIG— every other provider had an entry; omitting it would have left Groq validation broken if the YAML failed to loadgroqcase inReasoningModelDetectorPROVIDER-GUIDE.mdandSETUP.mdThe default model choice matters here
The obvious picks were traps. Verified against Groq's deprecation page:
llama-3.1-8b-instantllama-3.3-70b-versatileqwen3-32bdeepseek-r1-distill-llama-70bqwen-qwq-32bBoth Llama models are still listed under "Production Models" with no banner. Either would have shipped a provider that broke within days. The default is
openai/gpt-oss-20b— Groq's own recommended replacement forllama-3.1-8b-instant, 131K context.Groq retires models aggressively.
models.yamlwill need periodic revisiting; noted in both docs. This is an argument for #64.Verified vs. assumed
Verified from Groq documentation:
max_tokensis deprecated in favour ofmax_completion_tokens; both still accepted, and the base class already sends the right one per path — no change neededreasoning_formatacceptsraw|parsed|hidden, defaultraw(thinking inline in<think>tags). HenceextractThinkingkeeps both paths rather than only readingmessage.reasoning.reasoning_formatentirely and always populatemessage.reasoning— the parameter is a harmless no-op for the default model, correct for Qwenlogprobs,logit_bias,top_logprobs,presence_penalty,n>1) return 400. The extension sends none of them, so no filtering was required.Assumed, flagged honestly:
reasoning_effortdefault for gpt-oss is undocumented — the provider passes through only what the user sets rather than injecting a defaultmessage.reasoningappears in the reasoning docs but not the formal response schema; extraction is defensive and degrades toNoneTests
73 passing, 0 failed on the rebased branch (71 from #71, plus 2 new Groq blocks). Deterministic — the harness swaps in
DeterministicTestProvider, so no key is required. The drift guard added in #71 now also validates Groq's default model.Left out
include_reasoningsupport — needs new config surface, beyond this issuerequiresReasoningFormatentry — reserved for models that 400 without it (OpenAI o-series); Groq works without