Skip to content

feat: add native Groq provider - #73

Merged
JNK234 merged 2 commits into
mainfrom
feat/groq-provider
Aug 13, 2026
Merged

feat: add native Groq provider#73
JNK234 merged 2 commits into
mainfrom
feat/groq-provider

Conversation

@JNK234

@JNK234 JNK234 commented Aug 13, 2026

Copy link
Copy Markdown
Collaborator

Closes #51 (roadmap B1).

Stacked PR. Targets fix/provider-defaults (#71), not main, because both touch ProviderRegistrations.scala, models.yaml, and the provider counts in tests.txt. Merge #71 first and this retargets to main cleanly.

Why

Groq was reachable only through the OpenAI-compatible workaround — untested and undocumented as a supported path. Its free tier is fast and generous, which makes it the most useful zero-cost option for workshops and classrooms, where throughput matters more than model quality.

What

  • GroqProvider.scala extending OpenAICompatibleProvider (90 lines, mirrors TogetherProvider)
  • groq descriptor in ProviderRegistrations.scala, apiKeyPrefix = Some("gsk_")
  • groq: section in models.yaml listing only currently live models
  • groq added to ModelRegistry.FALLBACK_CONFIG — every other provider had an entry; omitting it would have left Groq validation broken if the YAML failed to load
  • groq case in ReasoningModelDetector
  • Docs in PROVIDER-GUIDE.md and SETUP.md

The default model choice matters here

The obvious picks were traps. Verified against Groq's deprecation page:

Model Status
llama-3.1-8b-instant shuts down 16 Aug 2026
llama-3.3-70b-versatile shuts down 16 Aug 2026
qwen3-32b decommissioned 17 Jul 2026
deepseek-r1-distill-llama-70b decommissioned Oct 2025
qwen-qwq-32b decommissioned Jul 2025

Both Llama models are still listed under "Production Models" with no banner. Either would have shipped a provider that broke within days. The default is openai/gpt-oss-20b — Groq's own recommended replacement for llama-3.1-8b-instant, 131K context.

Groq retires models aggressively. models.yaml will need periodic revisiting; noted in both docs. This is an argument for #64.

Verified vs. assumed

Verified from Groq documentation:

  • max_tokens is deprecated in favour of max_completion_tokens; both still accepted, and the base class already sends the right one per path — no change needed
  • reasoning_format accepts raw|parsed|hidden, default raw (thinking inline in <think> tags). Hence extractThinking keeps both paths rather than only reading message.reasoning.
  • gpt-oss models ignore reasoning_format entirely and always populate message.reasoning — the parameter is a harmless no-op for the default model, correct for Qwen
  • Unsupported OpenAI fields (logprobs, logit_bias, top_logprobs, presence_penalty, n>1) return 400. The extension sends none of them, so no filtering was required.

Assumed, flagged honestly:

  • reasoning_effort default for gpt-oss is undocumented — the provider passes through only what the user sets rather than injecting a default
  • message.reasoning appears in the reasoning docs but not the formal response schema; extraction is defensive and degrades to None
  • No live API call was made — there is no Groq key in the build environment, so wire-format correctness is documentation-derived, not empirically confirmed. Worth one real call before release.

Tests

73 passing, 0 failed on the rebased branch (71 from #71, plus 2 new Groq blocks). Deterministic — the harness swaps in DeterministicTestProvider, so no key is required. The drift guard added in #71 now also validates Groq's default model.

Left out

  • include_reasoning support — needs new config surface, beyond this issue
  • requiresReasoningFormat entry — reserved for models that 400 without it (OpenAI o-series); Groq works without
  • Groq in the interactive demo assets — those need a real key and are not on the CI path

@JNK234
JNK234 force-pushed the feat/groq-provider branch from 881202f to 7f3aca4 Compare August 13, 2026 23:46
@JNK234
JNK234 changed the base branch from fix/provider-defaults to main August 13, 2026 23:50
JNK234 added 2 commits August 13, 2026 18:50
Groq serves open-weight models on LPU hardware through an OpenAI-compatible
API. Its free tier is fast and generous, making it the best free option for
workshops and classrooms. Previously reachable only via an untested
OpenAI-compatible base_url workaround.

- Add GroqProvider extending OpenAICompatibleProvider, with reasoning_format
  "parsed" on reasoning requests and thinking extraction from both
  message.reasoning and <think> tags
- Register the groq descriptor with gsk_ key prefix and exposesThinking=true
- Add groq section to models.yaml listing only currently live models
- Add groq to the ModelRegistry fallback map and reasoning model detection
- Add deterministic tests for descriptor defaults, help text, model switching,
  and chat history; bump provider counts from 6 to 7
- Document Groq setup in PROVIDER-GUIDE.md and SETUP.md

Default model is openai/gpt-oss-20b rather than a Llama model: Groq's
llama-3.1-8b-instant and llama-3.3-70b-versatile are scheduled for shutdown
on 2026-08-16, and gpt-oss-20b is Groq's own recommended replacement.

Closes #51
llm:set-reasoning-effort validated against one hardcoded global set, so
"xhigh" was accepted by the extension and then rejected by Groq with an
HTTP 400 mid-run. Verified live before and after this change.

Effort levels are now declared per provider on ProviderDescriptor, with
the OpenAI-style set as the default for providers that do not override.

Groq's accepted values were probed one at a time against the live API
rather than read from its docs, which turned out to matter: the API's own
rejection message lists none|default|low|medium|high, but that is the
union across model families. The default model (openai/gpt-oss-20b)
accepts only low|medium|high — "none" and "default" both 400 against it.
The descriptor declares the set that is safe on the default model.

Verified end to end against live Groq: "xhigh" and "none" are now
refused up front with a message naming the provider and the valid values,
and "high" completes a real chat.

74 tests pass, including a new block covering both directions.
@JNK234
JNK234 force-pushed the feat/groq-provider branch from 7f3aca4 to 91c38b9 Compare August 13, 2026 23:55
@JNK234
JNK234 merged commit ab878c8 into main Aug 13, 2026
2 checks passed
@JNK234
JNK234 deleted the feat/groq-provider branch August 14, 2026 00:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat: native Groq provider

1 participant