Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 62 additions & 0 deletions docs/PROVIDER-GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -360,6 +360,68 @@ let result llm:chat-with-thinking "What is 15 * 17?"
; result is [answer thinking-text]
```

## Groq Configuration

### API Setup

1. **Get API Key**: Visit [console.groq.com/keys](https://console.groq.com/keys)
2. **Check Usage**: Monitor at [console.groq.com/settings/billing](https://console.groq.com/settings/billing)
3. **Browse Models**: Explore at [console.groq.com/docs/models](https://console.groq.com/docs/models)

### Configuration Parameters

```ini
# Required Parameters
provider=groq
groq_api_key=gsk_your-groq-key-here
model=openai/gpt-oss-20b

# Optional Parameters
groq_base_url=https://api.groq.com/openai/v1
temperature=0.7
max_tokens=1000
timeout_seconds=30
```

### Available Models

| Model | Description | Context |
|-------|-------------|---------|
| `openai/gpt-oss-20b` | Fast open-weight reasoning model | 131K |
| `openai/gpt-oss-120b` | Larger open-weight reasoning model | 131K |
| `openai/gpt-oss-safeguard-20b` | Safety-tuned gpt-oss variant | 131K |
| `groq/compound` | Agentic system with built-in tool use | 131K |
| `groq/compound-mini` | Smaller agentic system | 131K |
| `qwen/qwen3.6-27b` | Qwen 3.6 reasoning model (preview) | 131K |
| `minimaxai/minimax-m2.7` | MiniMax M2.7 (preview) | 131K |

**Recommended for NetLogo**: `openai/gpt-oss-20b` (fast, cheap, generous free tier)

### Why Groq?

- **Very fast inference** — custom LPU hardware delivers high tokens/second
- **Generous free tier** — the best free option for workshops and classrooms
- **Open-weight models** — gpt-oss, Qwen, MiniMax
- **Reasoning support** — thinking text exposed via `llm:chat-with-thinking`

### Thinking/Reasoning Models

The gpt-oss models are reasoning models and return their thinking separately:

```netlogo
llm:set-provider "groq"
llm:set-model "openai/gpt-oss-20b"
llm:set-thinking true
llm:set-reasoning-effort "low"
let result llm:chat-with-thinking "What is 15 * 17?"
; result is [answer thinking-text]
```

`llm:set-reasoning-effort` accepts `low`, `medium`, and `high` for gpt-oss models.
Groq's model lineup changes frequently — check the
[deprecations page](https://console.groq.com/docs/deprecations) before relying on
a specific model in a long-lived model file.

## Ollama (Local) Configuration

### Setup Requirements
Expand Down
33 changes: 32 additions & 1 deletion docs/SETUP.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,13 +151,39 @@ Browse the full catalog: [api.together.ai/models](https://api.together.ai/models

**Reasoning model note:** DeepSeek-R1 emits its thinking inside `<think>...</think>` tags (sometimes filling the entire response). Use `llm:chat-with-thinking` to get the answer and reasoning split out, and bump `max_tokens` to 2000+ so the model has room to finish its answer after thinking.

### Groq (fast free-tier inference)

Groq runs open-weight models on custom LPU hardware via an OpenAI-compatible API. Its free tier is fast and generous, which makes it a good choice for workshops and classrooms.

1. **Get API Key**: Visit [console.groq.com/keys](https://console.groq.com/keys)
2. **Create config.txt**:
```
provider=groq
model=openai/gpt-oss-20b
groq_api_key=gsk_your-key-here
temperature=0.7
max_tokens=1000
```

**Available Models**:
- `openai/gpt-oss-20b` - Fast open-weight reasoning model (recommended)
- `openai/gpt-oss-120b` - Larger open-weight reasoning model
- `groq/compound-mini` - Agentic system with built-in tool use
- `qwen/qwen3.6-27b` - Qwen 3.6 reasoning model (preview)

Browse the full catalog: [console.groq.com/docs/models](https://console.groq.com/docs/models)

**Reasoning model note:** the gpt-oss models always produce reasoning output. Use `llm:chat-with-thinking` to get the answer and reasoning split out, and `llm:set-reasoning-effort "low"` to keep latency and token use down in classroom settings.

**Model lifecycle note:** Groq retires models often. If a model stops working, check [console.groq.com/docs/deprecations](https://console.groq.com/docs/deprecations).

## Configuration Parameters

### Core Settings

| Parameter | Description | Required | Default |
|-----------|-------------|----------|---------|
| `provider` | LLM provider (`openai`, `anthropic`, `gemini`, `ollama`, `openrouter`, `together`) | Yes | - |
| `provider` | LLM provider (`openai`, `anthropic`, `gemini`, `ollama`, `openrouter`, `together`, `groq`) | Yes | - |
| `model` | Model identifier | Yes | Provider-specific |
| `api_key` | API authentication key | Yes* | - |
| `temperature` | Response randomness (0.0-1.0) | No | 0.7 |
Expand Down Expand Up @@ -222,6 +248,11 @@ call the LLM every N ticks) rather than only raising `retry_max_elapsed_seconds`
- `max_tokens`: 1000
- API key config field: `together_api_key`

**Groq**:
- `base_url`: `https://api.groq.com/openai/v1`
- `max_tokens`: 1000
- API key config field: `groq_api_key`

## Testing Your Setup

1. **Create Test Model**:
Expand Down
18 changes: 15 additions & 3 deletions src/main/LLMExtension.scala
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ package org.nlogo.extensions.llm
import org.nlogo.api._
import org.nlogo.core.{LogoList, Syntax}
import org.nlogo.extensions.llm.config.{ConfigLoader, ConfigStore}
import org.nlogo.extensions.llm.providers.{LLMProvider, ProviderFactory, ProviderRegistry, ProviderRegistrations, ModelRegistry, OllamaProvider, ReadinessCheck, RetryPolicy}
import org.nlogo.extensions.llm.providers.{LLMProvider, ProviderDescriptor, ProviderFactory, ProviderRegistry, ProviderRegistrations, ModelRegistry, OllamaProvider, ReadinessCheck, RetryPolicy}
import org.nlogo.extensions.llm.models.{ChatMessage, ChatResponse}
import scala.collection.mutable.{ArrayBuffer, WeakHashMap}
import scala.concurrent.{Await, ExecutionContext, Future}
Expand Down Expand Up @@ -705,9 +705,21 @@ class LLMExtension extends DefaultClassManager {

override def perform(args: Array[Argument], context: Context): Unit = {
val effort = args(0).getString.toLowerCase.trim
if (!Set("none", "low", "medium", "high", "xhigh").contains(effort)) {

// Validate against the ACTIVE provider, not a global set. Providers
// disagree — Groq rejects "xhigh" with a 400 and accepts "default", the
// inverse of OpenAI — so one shared list either lets a request through to
// a hard failure or blocks a value the provider would have taken.
val providerName = configStore.get(ConfigStore.PROVIDER).getOrElse("")
val allowed = ProviderRegistry.get(providerName)
.map(_.reasoningEffortValues)
.getOrElse(ProviderDescriptor.DefaultReasoningEffortValues)

if (!allowed.contains(effort)) {
val forProvider = if (providerName.nonEmpty) s" for provider '$providerName'" else ""
throw new ExtensionException(
s"Invalid reasoning effort: '$effort'. Must be one of: none, low, medium, high, xhigh"
s"Invalid reasoning effort: '$effort'$forProvider. " +
s"Must be one of: ${allowed.toSeq.sorted.mkString(", ")}"
)
}
configStore.set(ConfigStore.REASONING_EFFORT, effort)
Expand Down
90 changes: 90 additions & 0 deletions src/main/providers/GroqProvider.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
// ABOUTME: Groq provider — very fast inference on open-weight models via LPU hardware
// ABOUTME: Extends OpenAICompatibleProvider with Groq-specific reasoning fields and thinking extraction
package org.nlogo.extensions.llm.providers

import org.nlogo.extensions.llm.models.ChatRequest
import scala.concurrent.ExecutionContext

/**
* Groq provider implementation.
*
* Groq serves open-weight models (gpt-oss, Qwen, Llama) on custom LPU hardware,
* giving very high tokens/second through an OpenAI-compatible API. The generous
* free tier makes it a good fit for classroom and workshop use.
*
* Key differences from direct OpenAI:
* - max_tokens is deprecated in favor of max_completion_tokens (both accepted)
* - Reasoning models expose thinking in message.reasoning, or in <think>...</think>
* tags within content when reasoning_format is "raw" (the API default)
* - gpt-oss models ignore reasoning_format and always use the reasoning field
* - Unsupported OpenAI fields (logprobs, logit_bias, top_logprobs, presence_penalty)
* are never sent by this extension, so no filtering is required
*/
class GroqProvider(implicit ec: ExecutionContext) extends OpenAICompatibleProvider {

override def providerName: String = "groq"

override def defaultModel: String = "openai/gpt-oss-20b"

override protected def defaultBaseUrl: String = "https://api.groq.com/openai/v1"

override protected def baseUrlConfigKey: String = "groq_base_url"

override protected def apiKeyConfigKey: String = "groq_api_key"

override protected def defaultMaxTokens: String = "1000"

override protected def requiresApiKey: Boolean = true

// No extra headers needed (unlike OpenRouter)

/**
* Groq accepts OpenAI's top-level reasoning_effort. Valid values vary by model
* (gpt-oss accepts low/medium/high; qwen3 accepts none/default), so we pass the
* user's choice straight through rather than guessing a default.
*
* We also request reasoning_format "parsed" so thinking arrives in the dedicated
* message.reasoning field instead of inline <think> tags. gpt-oss models ignore
* this parameter and use message.reasoning anyway, so it is safe either way.
*/
override protected def applyReasoningFields(baseObj: ujson.Obj, request: ChatRequest): Unit = {
baseObj("reasoning_format") = "parsed"
request.thinkingConfig.flatMap(_.reasoningEffort).foreach { effort =>
baseObj("reasoning_effort") = effort
}
}

/**
* Extract thinking text from Groq responses.
*
* Two extraction paths:
* 1. message.reasoning field (reasoning_format "parsed", and always for gpt-oss)
* 2. <think>...</think> tags in message.content (reasoning_format "raw")
*/
override protected def extractThinking(message: ujson.Value): Option[String] = {
// Path 1: check message.reasoning field
val fromReasoning = try {
message.obj.get("reasoning").flatMap { v =>
val text = v.str.trim
if (text.nonEmpty) Some(text) else None
}
} catch {
case _: Exception => None
}

if (fromReasoning.isDefined) return fromReasoning

// Path 2: parse <think>...</think> tags from content (reasoning_format "raw")
// (?s) makes dot match newlines. Content is returned unmodified — we only
// mirror the reasoning into a separate field for llm:chat-with-thinking.
try {
val content = message("content").str
val pattern = """(?s)<think>(.*?)</think>""".r
pattern.findFirstMatchIn(content)
.map(_.group(1).trim)
.filter(_.nonEmpty)
} catch {
case _: Exception => None
}
}
}
3 changes: 2 additions & 1 deletion src/main/providers/ModelRegistry.scala
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,8 @@ object ModelRegistry {
"gemini" -> ProviderModels(Set("gemini-2.5-pro", "gemini-2.5-flash", "gemini-3-pro-preview"), isCustom = false),
"ollama" -> ProviderModels(Set("llama3.2:3b", "llama3.2:1b", "mistral", "phi4"), isCustom = false),
"openrouter" -> ProviderModels(Set("openai/gpt-4o", "openai/gpt-4o-mini", "anthropic/claude-3.5-sonnet", "deepseek/deepseek-r1"), isCustom = false),
"together" -> ProviderModels(Set("meta-llama/Llama-3.3-70B-Instruct-Turbo", "deepseek-ai/DeepSeek-R1", "Qwen/Qwen2.5-72B-Instruct-Turbo"), isCustom = false)
"together" -> ProviderModels(Set("meta-llama/Llama-3.3-70B-Instruct-Turbo", "deepseek-ai/DeepSeek-R1", "Qwen/Qwen2.5-72B-Instruct-Turbo"), isCustom = false),
"groq" -> ProviderModels(Set("openai/gpt-oss-20b", "openai/gpt-oss-120b", "groq/compound-mini"), isCustom = false)
)

/**
Expand Down
16 changes: 15 additions & 1 deletion src/main/providers/ProviderDescriptor.scala
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,12 @@ enum ReadinessCheck:
* @param apiKeyPrefix Optional prefix for API key validation hint (e.g. Some("sk-"))
* @param readinessCheck How to determine if the provider is ready to use
* @param exposesThinking Whether thinking/reasoning text appears in API responses
* @param reasoningEffortValues Effort levels this provider's API accepts. Providers
* disagree: Groq rejects anything outside
* none|default|low|medium|high with a 400, while OpenAI and
* Anthropic take xhigh. Validating against one global set let
* `llm:set-reasoning-effort "xhigh"` through to a hard failure
* on Groq, and rejected "default", which Groq accepts.
* @param helpText Multi-line setup instructions shown by llm:provider-help
* @param factory Function that creates a new LLMProvider instance given an ExecutionContext
*/
Expand All @@ -47,5 +53,13 @@ case class ProviderDescriptor(
readinessCheck: ReadinessCheck,
exposesThinking: Boolean,
helpText: String,
factory: ExecutionContext => LLMProvider
factory: ExecutionContext => LLMProvider,
reasoningEffortValues: Set[String] = ProviderDescriptor.DefaultReasoningEffortValues
)

object ProviderDescriptor {
/** Effort levels accepted by OpenAI-style APIs, and the fallback for providers
* that have not declared their own. */
val DefaultReasoningEffortValues: Set[String] =
Set("none", "low", "medium", "high", "xhigh")
}
47 changes: 47 additions & 0 deletions src/main/providers/ProviderRegistrations.scala
Original file line number Diff line number Diff line change
Expand Up @@ -205,5 +205,52 @@ object ProviderRegistrations {
|Browse models: https://api.together.ai/models""".stripMargin,
factory = ec => new TogetherProvider()(using ec)
))

ProviderRegistry.register(ProviderDescriptor(
name = "groq",
displayName = "Groq",
apiKeyConfigKey = "groq_api_key",
baseUrlConfigKey = "groq_base_url",
defaultBaseUrl = "https://api.groq.com/openai/v1",
defaultModel = "openai/gpt-oss-20b",
defaultMaxTokens = "1000",
requiresApiKey = true,
apiKeyPrefix = Some("gsk_"),
readinessCheck = ReadinessCheck.ApiKey,
exposesThinking = true,
// Probed against the live API 13 Aug 2026, one value at a time.
//
// Groq's rejection message lists none|default|low|medium|high, but that is
// the union across its model families, not what any one model takes. On the
// default model (openai/gpt-oss-20b) only low|medium|high are accepted —
// "none" and "default" both 400 with "must be one of low, medium, or high".
// Those two belong to the qwen3 family instead.
//
// We declare the intersection that is safe on the default model. "xhigh",
// which OpenAI accepts, is rejected by every Groq model.
reasoningEffortValues = Set("low", "medium", "high"),
helpText =
"""Groq Setup Instructions:
|
|1. Get an API key:
| - Visit https://console.groq.com/keys
| - Create a new API key
|
|2. Set the key:
| - In config file: groq_api_key=gsk_your-key-here
| - Or at runtime: llm:set-api-key "gsk_your-key-here"
|
|3. Set a model:
| - llm:set-model "openai/gpt-oss-20b"
| - llm:set-model "openai/gpt-oss-120b"
| - llm:set-model "groq/compound-mini"
|
|4. Verify:
| - Check llm:provider-status for "has-key: true"
|
|Groq's free tier is fast and generous — a good fit for classrooms.
|Browse models: https://console.groq.com/docs/models""".stripMargin,
factory = ec => new GroqProvider()(using ec)
))
}
}
3 changes: 3 additions & 0 deletions src/main/providers/ReasoningModelDetector.scala
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,9 @@ object ReasoningModelDetector {
case "together" =>
val m = model.toLowerCase
m.contains("deepseek-r1") || m.contains("qwq") || m.contains("qwen3")
case "groq" =>
val m = model.toLowerCase
m.contains("gpt-oss") || m.contains("qwen3") || m.contains("deepseek-r1")
case _ => false
}
}
Expand Down
14 changes: 14 additions & 0 deletions src/main/resources/config/models.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,20 @@ together:
# Mistral
- mistralai/Mistral-Small-24B-Instruct-2501

groq:
# OpenAI open-weight models (reasoning, exposed via message.reasoning)
- openai/gpt-oss-120b
- openai/gpt-oss-20b
- openai/gpt-oss-safeguard-20b

# Groq agentic systems (built-in tool use)
- groq/compound
- groq/compound-mini

# Preview models
- qwen/qwen3.6-27b
- minimaxai/minimax-m2.7

ollama:
# Top reasoning models
- deepseek-r1:70b
Expand Down
Loading
Loading