diff --git a/docs/PROVIDER-GUIDE.md b/docs/PROVIDER-GUIDE.md index a7ca7ca..8292f2d 100644 --- a/docs/PROVIDER-GUIDE.md +++ b/docs/PROVIDER-GUIDE.md @@ -360,6 +360,68 @@ let result llm:chat-with-thinking "What is 15 * 17?" ; result is [answer thinking-text] ``` +## Groq Configuration + +### API Setup + +1. **Get API Key**: Visit [console.groq.com/keys](https://console.groq.com/keys) +2. **Check Usage**: Monitor at [console.groq.com/settings/billing](https://console.groq.com/settings/billing) +3. **Browse Models**: Explore at [console.groq.com/docs/models](https://console.groq.com/docs/models) + +### Configuration Parameters + +```ini +# Required Parameters +provider=groq +groq_api_key=gsk_your-groq-key-here +model=openai/gpt-oss-20b + +# Optional Parameters +groq_base_url=https://api.groq.com/openai/v1 +temperature=0.7 +max_tokens=1000 +timeout_seconds=30 +``` + +### Available Models + +| Model | Description | Context | +|-------|-------------|---------| +| `openai/gpt-oss-20b` | Fast open-weight reasoning model | 131K | +| `openai/gpt-oss-120b` | Larger open-weight reasoning model | 131K | +| `openai/gpt-oss-safeguard-20b` | Safety-tuned gpt-oss variant | 131K | +| `groq/compound` | Agentic system with built-in tool use | 131K | +| `groq/compound-mini` | Smaller agentic system | 131K | +| `qwen/qwen3.6-27b` | Qwen 3.6 reasoning model (preview) | 131K | +| `minimaxai/minimax-m2.7` | MiniMax M2.7 (preview) | 131K | + +**Recommended for NetLogo**: `openai/gpt-oss-20b` (fast, cheap, generous free tier) + +### Why Groq? + +- **Very fast inference** — custom LPU hardware delivers high tokens/second +- **Generous free tier** — the best free option for workshops and classrooms +- **Open-weight models** — gpt-oss, Qwen, MiniMax +- **Reasoning support** — thinking text exposed via `llm:chat-with-thinking` + +### Thinking/Reasoning Models + +The gpt-oss models are reasoning models and return their thinking separately: + +```netlogo +llm:set-provider "groq" +llm:set-model "openai/gpt-oss-20b" +llm:set-thinking true +llm:set-reasoning-effort "low" +let result llm:chat-with-thinking "What is 15 * 17?" +; result is [answer thinking-text] +``` + +`llm:set-reasoning-effort` accepts `low`, `medium`, and `high` for gpt-oss models. +Groq's model lineup changes frequently — check the +[deprecations page](https://console.groq.com/docs/deprecations) before relying on +a specific model in a long-lived model file. + ## Ollama (Local) Configuration ### Setup Requirements diff --git a/docs/SETUP.md b/docs/SETUP.md index 523fae5..d0e65ac 100644 --- a/docs/SETUP.md +++ b/docs/SETUP.md @@ -151,13 +151,39 @@ Browse the full catalog: [api.together.ai/models](https://api.together.ai/models **Reasoning model note:** DeepSeek-R1 emits its thinking inside `...` tags (sometimes filling the entire response). Use `llm:chat-with-thinking` to get the answer and reasoning split out, and bump `max_tokens` to 2000+ so the model has room to finish its answer after thinking. +### Groq (fast free-tier inference) + +Groq runs open-weight models on custom LPU hardware via an OpenAI-compatible API. Its free tier is fast and generous, which makes it a good choice for workshops and classrooms. + +1. **Get API Key**: Visit [console.groq.com/keys](https://console.groq.com/keys) +2. **Create config.txt**: +``` +provider=groq +model=openai/gpt-oss-20b +groq_api_key=gsk_your-key-here +temperature=0.7 +max_tokens=1000 +``` + +**Available Models**: +- `openai/gpt-oss-20b` - Fast open-weight reasoning model (recommended) +- `openai/gpt-oss-120b` - Larger open-weight reasoning model +- `groq/compound-mini` - Agentic system with built-in tool use +- `qwen/qwen3.6-27b` - Qwen 3.6 reasoning model (preview) + +Browse the full catalog: [console.groq.com/docs/models](https://console.groq.com/docs/models) + +**Reasoning model note:** the gpt-oss models always produce reasoning output. Use `llm:chat-with-thinking` to get the answer and reasoning split out, and `llm:set-reasoning-effort "low"` to keep latency and token use down in classroom settings. + +**Model lifecycle note:** Groq retires models often. If a model stops working, check [console.groq.com/docs/deprecations](https://console.groq.com/docs/deprecations). + ## Configuration Parameters ### Core Settings | Parameter | Description | Required | Default | |-----------|-------------|----------|---------| -| `provider` | LLM provider (`openai`, `anthropic`, `gemini`, `ollama`, `openrouter`, `together`) | Yes | - | +| `provider` | LLM provider (`openai`, `anthropic`, `gemini`, `ollama`, `openrouter`, `together`, `groq`) | Yes | - | | `model` | Model identifier | Yes | Provider-specific | | `api_key` | API authentication key | Yes* | - | | `temperature` | Response randomness (0.0-1.0) | No | 0.7 | @@ -222,6 +248,11 @@ call the LLM every N ticks) rather than only raising `retry_max_elapsed_seconds` - `max_tokens`: 1000 - API key config field: `together_api_key` +**Groq**: +- `base_url`: `https://api.groq.com/openai/v1` +- `max_tokens`: 1000 +- API key config field: `groq_api_key` + ## Testing Your Setup 1. **Create Test Model**: diff --git a/src/main/LLMExtension.scala b/src/main/LLMExtension.scala index 6bb2302..450131c 100644 --- a/src/main/LLMExtension.scala +++ b/src/main/LLMExtension.scala @@ -3,7 +3,7 @@ package org.nlogo.extensions.llm import org.nlogo.api._ import org.nlogo.core.{LogoList, Syntax} import org.nlogo.extensions.llm.config.{ConfigLoader, ConfigStore} -import org.nlogo.extensions.llm.providers.{LLMProvider, ProviderFactory, ProviderRegistry, ProviderRegistrations, ModelRegistry, OllamaProvider, ReadinessCheck, RetryPolicy} +import org.nlogo.extensions.llm.providers.{LLMProvider, ProviderDescriptor, ProviderFactory, ProviderRegistry, ProviderRegistrations, ModelRegistry, OllamaProvider, ReadinessCheck, RetryPolicy} import org.nlogo.extensions.llm.models.{ChatMessage, ChatResponse} import scala.collection.mutable.{ArrayBuffer, WeakHashMap} import scala.concurrent.{Await, ExecutionContext, Future} @@ -705,9 +705,21 @@ class LLMExtension extends DefaultClassManager { override def perform(args: Array[Argument], context: Context): Unit = { val effort = args(0).getString.toLowerCase.trim - if (!Set("none", "low", "medium", "high", "xhigh").contains(effort)) { + + // Validate against the ACTIVE provider, not a global set. Providers + // disagree — Groq rejects "xhigh" with a 400 and accepts "default", the + // inverse of OpenAI — so one shared list either lets a request through to + // a hard failure or blocks a value the provider would have taken. + val providerName = configStore.get(ConfigStore.PROVIDER).getOrElse("") + val allowed = ProviderRegistry.get(providerName) + .map(_.reasoningEffortValues) + .getOrElse(ProviderDescriptor.DefaultReasoningEffortValues) + + if (!allowed.contains(effort)) { + val forProvider = if (providerName.nonEmpty) s" for provider '$providerName'" else "" throw new ExtensionException( - s"Invalid reasoning effort: '$effort'. Must be one of: none, low, medium, high, xhigh" + s"Invalid reasoning effort: '$effort'$forProvider. " + + s"Must be one of: ${allowed.toSeq.sorted.mkString(", ")}" ) } configStore.set(ConfigStore.REASONING_EFFORT, effort) diff --git a/src/main/providers/GroqProvider.scala b/src/main/providers/GroqProvider.scala new file mode 100644 index 0000000..3570658 --- /dev/null +++ b/src/main/providers/GroqProvider.scala @@ -0,0 +1,90 @@ +// ABOUTME: Groq provider — very fast inference on open-weight models via LPU hardware +// ABOUTME: Extends OpenAICompatibleProvider with Groq-specific reasoning fields and thinking extraction +package org.nlogo.extensions.llm.providers + +import org.nlogo.extensions.llm.models.ChatRequest +import scala.concurrent.ExecutionContext + +/** + * Groq provider implementation. + * + * Groq serves open-weight models (gpt-oss, Qwen, Llama) on custom LPU hardware, + * giving very high tokens/second through an OpenAI-compatible API. The generous + * free tier makes it a good fit for classroom and workshop use. + * + * Key differences from direct OpenAI: + * - max_tokens is deprecated in favor of max_completion_tokens (both accepted) + * - Reasoning models expose thinking in message.reasoning, or in ... + * tags within content when reasoning_format is "raw" (the API default) + * - gpt-oss models ignore reasoning_format and always use the reasoning field + * - Unsupported OpenAI fields (logprobs, logit_bias, top_logprobs, presence_penalty) + * are never sent by this extension, so no filtering is required + */ +class GroqProvider(implicit ec: ExecutionContext) extends OpenAICompatibleProvider { + + override def providerName: String = "groq" + + override def defaultModel: String = "openai/gpt-oss-20b" + + override protected def defaultBaseUrl: String = "https://api.groq.com/openai/v1" + + override protected def baseUrlConfigKey: String = "groq_base_url" + + override protected def apiKeyConfigKey: String = "groq_api_key" + + override protected def defaultMaxTokens: String = "1000" + + override protected def requiresApiKey: Boolean = true + + // No extra headers needed (unlike OpenRouter) + + /** + * Groq accepts OpenAI's top-level reasoning_effort. Valid values vary by model + * (gpt-oss accepts low/medium/high; qwen3 accepts none/default), so we pass the + * user's choice straight through rather than guessing a default. + * + * We also request reasoning_format "parsed" so thinking arrives in the dedicated + * message.reasoning field instead of inline tags. gpt-oss models ignore + * this parameter and use message.reasoning anyway, so it is safe either way. + */ + override protected def applyReasoningFields(baseObj: ujson.Obj, request: ChatRequest): Unit = { + baseObj("reasoning_format") = "parsed" + request.thinkingConfig.flatMap(_.reasoningEffort).foreach { effort => + baseObj("reasoning_effort") = effort + } + } + + /** + * Extract thinking text from Groq responses. + * + * Two extraction paths: + * 1. message.reasoning field (reasoning_format "parsed", and always for gpt-oss) + * 2. ... tags in message.content (reasoning_format "raw") + */ + override protected def extractThinking(message: ujson.Value): Option[String] = { + // Path 1: check message.reasoning field + val fromReasoning = try { + message.obj.get("reasoning").flatMap { v => + val text = v.str.trim + if (text.nonEmpty) Some(text) else None + } + } catch { + case _: Exception => None + } + + if (fromReasoning.isDefined) return fromReasoning + + // Path 2: parse ... tags from content (reasoning_format "raw") + // (?s) makes dot match newlines. Content is returned unmodified — we only + // mirror the reasoning into a separate field for llm:chat-with-thinking. + try { + val content = message("content").str + val pattern = """(?s)(.*?)""".r + pattern.findFirstMatchIn(content) + .map(_.group(1).trim) + .filter(_.nonEmpty) + } catch { + case _: Exception => None + } + } +} diff --git a/src/main/providers/ModelRegistry.scala b/src/main/providers/ModelRegistry.scala index 387f1b8..1696be1 100644 --- a/src/main/providers/ModelRegistry.scala +++ b/src/main/providers/ModelRegistry.scala @@ -42,7 +42,8 @@ object ModelRegistry { "gemini" -> ProviderModels(Set("gemini-2.5-pro", "gemini-2.5-flash", "gemini-3-pro-preview"), isCustom = false), "ollama" -> ProviderModels(Set("llama3.2:3b", "llama3.2:1b", "mistral", "phi4"), isCustom = false), "openrouter" -> ProviderModels(Set("openai/gpt-4o", "openai/gpt-4o-mini", "anthropic/claude-3.5-sonnet", "deepseek/deepseek-r1"), isCustom = false), - "together" -> ProviderModels(Set("meta-llama/Llama-3.3-70B-Instruct-Turbo", "deepseek-ai/DeepSeek-R1", "Qwen/Qwen2.5-72B-Instruct-Turbo"), isCustom = false) + "together" -> ProviderModels(Set("meta-llama/Llama-3.3-70B-Instruct-Turbo", "deepseek-ai/DeepSeek-R1", "Qwen/Qwen2.5-72B-Instruct-Turbo"), isCustom = false), + "groq" -> ProviderModels(Set("openai/gpt-oss-20b", "openai/gpt-oss-120b", "groq/compound-mini"), isCustom = false) ) /** diff --git a/src/main/providers/ProviderDescriptor.scala b/src/main/providers/ProviderDescriptor.scala index a9dee6c..cb77d46 100644 --- a/src/main/providers/ProviderDescriptor.scala +++ b/src/main/providers/ProviderDescriptor.scala @@ -31,6 +31,12 @@ enum ReadinessCheck: * @param apiKeyPrefix Optional prefix for API key validation hint (e.g. Some("sk-")) * @param readinessCheck How to determine if the provider is ready to use * @param exposesThinking Whether thinking/reasoning text appears in API responses + * @param reasoningEffortValues Effort levels this provider's API accepts. Providers + * disagree: Groq rejects anything outside + * none|default|low|medium|high with a 400, while OpenAI and + * Anthropic take xhigh. Validating against one global set let + * `llm:set-reasoning-effort "xhigh"` through to a hard failure + * on Groq, and rejected "default", which Groq accepts. * @param helpText Multi-line setup instructions shown by llm:provider-help * @param factory Function that creates a new LLMProvider instance given an ExecutionContext */ @@ -47,5 +53,13 @@ case class ProviderDescriptor( readinessCheck: ReadinessCheck, exposesThinking: Boolean, helpText: String, - factory: ExecutionContext => LLMProvider + factory: ExecutionContext => LLMProvider, + reasoningEffortValues: Set[String] = ProviderDescriptor.DefaultReasoningEffortValues ) + +object ProviderDescriptor { + /** Effort levels accepted by OpenAI-style APIs, and the fallback for providers + * that have not declared their own. */ + val DefaultReasoningEffortValues: Set[String] = + Set("none", "low", "medium", "high", "xhigh") +} diff --git a/src/main/providers/ProviderRegistrations.scala b/src/main/providers/ProviderRegistrations.scala index b6d8b9b..680326c 100644 --- a/src/main/providers/ProviderRegistrations.scala +++ b/src/main/providers/ProviderRegistrations.scala @@ -205,5 +205,52 @@ object ProviderRegistrations { |Browse models: https://api.together.ai/models""".stripMargin, factory = ec => new TogetherProvider()(using ec) )) + + ProviderRegistry.register(ProviderDescriptor( + name = "groq", + displayName = "Groq", + apiKeyConfigKey = "groq_api_key", + baseUrlConfigKey = "groq_base_url", + defaultBaseUrl = "https://api.groq.com/openai/v1", + defaultModel = "openai/gpt-oss-20b", + defaultMaxTokens = "1000", + requiresApiKey = true, + apiKeyPrefix = Some("gsk_"), + readinessCheck = ReadinessCheck.ApiKey, + exposesThinking = true, + // Probed against the live API 13 Aug 2026, one value at a time. + // + // Groq's rejection message lists none|default|low|medium|high, but that is + // the union across its model families, not what any one model takes. On the + // default model (openai/gpt-oss-20b) only low|medium|high are accepted — + // "none" and "default" both 400 with "must be one of low, medium, or high". + // Those two belong to the qwen3 family instead. + // + // We declare the intersection that is safe on the default model. "xhigh", + // which OpenAI accepts, is rejected by every Groq model. + reasoningEffortValues = Set("low", "medium", "high"), + helpText = + """Groq Setup Instructions: + | + |1. Get an API key: + | - Visit https://console.groq.com/keys + | - Create a new API key + | + |2. Set the key: + | - In config file: groq_api_key=gsk_your-key-here + | - Or at runtime: llm:set-api-key "gsk_your-key-here" + | + |3. Set a model: + | - llm:set-model "openai/gpt-oss-20b" + | - llm:set-model "openai/gpt-oss-120b" + | - llm:set-model "groq/compound-mini" + | + |4. Verify: + | - Check llm:provider-status for "has-key: true" + | + |Groq's free tier is fast and generous — a good fit for classrooms. + |Browse models: https://console.groq.com/docs/models""".stripMargin, + factory = ec => new GroqProvider()(using ec) + )) } } diff --git a/src/main/providers/ReasoningModelDetector.scala b/src/main/providers/ReasoningModelDetector.scala index 77c0b0b..143a397 100644 --- a/src/main/providers/ReasoningModelDetector.scala +++ b/src/main/providers/ReasoningModelDetector.scala @@ -108,6 +108,9 @@ object ReasoningModelDetector { case "together" => val m = model.toLowerCase m.contains("deepseek-r1") || m.contains("qwq") || m.contains("qwen3") + case "groq" => + val m = model.toLowerCase + m.contains("gpt-oss") || m.contains("qwen3") || m.contains("deepseek-r1") case _ => false } } diff --git a/src/main/resources/config/models.yaml b/src/main/resources/config/models.yaml index 478fa11..1603d32 100644 --- a/src/main/resources/config/models.yaml +++ b/src/main/resources/config/models.yaml @@ -131,6 +131,20 @@ together: # Mistral - mistralai/Mistral-Small-24B-Instruct-2501 +groq: + # OpenAI open-weight models (reasoning, exposed via message.reasoning) + - openai/gpt-oss-120b + - openai/gpt-oss-20b + - openai/gpt-oss-safeguard-20b + + # Groq agentic systems (built-in tool use) + - groq/compound + - groq/compound-mini + + # Preview models + - qwen/qwen3.6-27b + - minimaxai/minimax-m2.7 + ollama: # Top reasoning models - deepseek-r1:70b diff --git a/tests.txt b/tests.txt index 0ba20c0..b3cedb8 100644 --- a/tests.txt +++ b/tests.txt @@ -10,19 +10,20 @@ LLMConfigPrimitives LLMProvidersAll extensions [llm] - length llm:providers-all => 6 + length llm:providers-all => 7 member? "openai" llm:providers-all => true member? "anthropic" llm:providers-all => true member? "gemini" llm:providers-all => true member? "ollama" llm:providers-all => true member? "openrouter" llm:providers-all => true member? "together" llm:providers-all => true + member? "groq" llm:providers-all => true LLMProviderStatus extensions [llm] O> llm:set-api-key "test-key" O> llm:set-provider "openai" - length llm:provider-status => 6 + length llm:provider-status => 7 LLMListModels extensions [llm] @@ -81,6 +82,18 @@ LLMReasoningEffortValidation O> llm:set-reasoning-effort "high" O> llm:set-reasoning-effort "xhigh" +LLMReasoningEffortIsProviderAware + extensions [llm] + O> llm:set-api-key "test-key" + O> llm:set-provider "groq" + O> llm:set-reasoning-effort "low" + O> llm:set-reasoning-effort "high" + O> llm:set-reasoning-effort "xhigh" => ERROR Extension exception: Invalid reasoning effort: 'xhigh' for provider 'groq'. Must be one of: high, low, medium + O> llm:set-reasoning-effort "none" => ERROR Extension exception: Invalid reasoning effort: 'none' for provider 'groq'. Must be one of: high, low, medium + O> llm:set-provider "openai" + O> llm:set-reasoning-effort "xhigh" + O> llm:set-reasoning-effort "none" + LLMThinkingBudgetValidation extensions [llm] O> llm:set-thinking-budget 1024 @@ -270,6 +283,27 @@ LLMTogetherProvider O> llm:set-model "deepseek-ai/DeepSeek-R1" item 1 llm:active => "deepseek-ai/DeepSeek-R1" +LLMGroqProvider + extensions [llm] + O> llm:set-api-key "test-key" + O> llm:set-provider "groq" + item 0 llm:active => "groq" + item 1 llm:active => "openai/gpt-oss-20b" + empty? (llm:provider-help "groq") => false + O> llm:set-model "openai/gpt-oss-120b" + item 1 llm:active => "openai/gpt-oss-120b" + +LLMGroqChat + extensions [llm] + globals [response] + O> llm:set-api-key "test-key" + O> llm:set-provider "groq" + O> llm:clear-history + O> set response llm:chat "Say hello" + is-string? response => true + empty? response => false + length llm:history => 2 + LLMConfigLoading extensions [llm] O> llm:load-config "demos/config"