Repository navigation
Conversation
… AI, and Together AI One Worker script covers all three providers: a provider table holds each endpoint's base URL, API key environment variable, and a default open-weight model with tool calling (gpt-oss-120b everywhere), and an OpenAICompatibleModelProvider turns an entry into a Chat Completions model with client retries off, so Temporal owns the retries. The Workflow names no model, so it runs unchanged on whichever provider the Worker was started with (--provider baseten|fireworks|together). Each provider entry carries its own snipsync marker so the Temporal docs page for that provider can quote it. Offline tests cover the provider table, the client configuration, and the Workflow with a scripted model.
DABH
changed the title
| # No model is named here, so the Worker's model provider supplies its | ||
| # default. Set model= to any model the provider serves to override it. |
There was a problem hiding this comment.
Nit: I think this demo might be a little bit more compelling if you didn't need to restart the entire worker to change providers/models. I think it'd be great to demonstrate the ModelProvider having all providers to choose from and it picks which provider based on the model name for that particular workflow instance.
Basically could just make the workflow take some init arg for the chosen model, and then you could have several workflows running concurrently on the same worker each with different providers. A more realistic deployment than deploying a worker per provider URL.
There was a problem hiding this comment.
Done in 53b9456. Model names now carry a provider prefix (together/openai/gpt-oss-120b, fireworks/accounts/fireworks/models/gpt-oss-120b, baseten/openai/gpt-oss-120b), one Worker serves every provider whose API key is set, the model travels in the Workflow input, and the starter runs one Workflow per --model concurrently on that Worker. A name for a provider without a key fails its Activity without retrying, and the starter prints that root cause. Verified live with a Fireworks Workflow and a keyless Together Workflow side by side. Side benefit: naming the model lets the Agents SDK drop its OpenAI-default settings, so the explicit ModelSettings() workaround is gone.
Review feedback: restarting the Worker to switch providers is not how a real deployment works. The model provider now holds every provider whose API key is set and routes `<provider>/<model>` names, such as `together/openai/gpt-oss-120b`, to the matching endpoint. The model name travels in the Workflow input, so Workflows on different providers run side by side on the same Worker, and the starter runs one Workflow per `--model` concurrently. Naming the model on the agent also lets the Agents SDK drop the default settings it keeps for OpenAI's default model, so the explicit `ModelSettings()` workaround is gone. A model name for a provider without a key, or with an unknown prefix, fails its Activity without retrying; the starter prints that root cause. Verified live on Fireworks AI alongside a keyless Together AI Workflow on the same Worker: the first completed through two model Activities, the second failed on its first attempt with the "set TOGETHER_API_KEY" error.
What changed
One sample in
openai_agents/model_providers/runs the same agent on Baseten, Fireworks AI, or Together AI, all of which serve open-weight models behind the OpenAI Chat Completions API:openai_compatible_providers.py: a provider table (base URL, API key env var, an example model with tool calling: gpt-oss-120b on all three) andOpenAICompatibleModelProvider, which builds a client for every provider whose API key is set and routes<provider>/<model>names such astogether/openai/gpt-oss-120bto the matching endpoint as anOpenAIChatCompletionsModelwithmax_retries=0, so the Activity's Retry Policy is the only retry policy. A name for an unknown prefix or a provider without a key fails the Activity without retrying.run_openai_compatible_worker.py: one Worker serves every configured provider and prints which ones at startup.workflows/openai_compatible_workflow.py: the model name travels in the Workflow input and goes on theAgent, so Workflows on different providers run side by side on the same Worker.run_openai_compatible_workflow.py --model ... [--model ...]: starts one Workflow per model concurrently and prints each result or root-cause failure.tests/openai_agents/model_providers_test.py: offline tests for the table, routing, the non-retryable failures (unit and through the Workflow), two concurrent Workflows on different providers sharing one Worker, and that no OpenAI-default model settings reach the provider.Each provider entry carries its own
@@@SNIPSTARTmarker (python-openai-agents-{baseten,fireworks,together}-provider), plus markers for the provider class, Worker wiring, and Workflow, so the matching Temporal docs pages can quote them.Testing
ruff,mypy, and the new tests pass locally. Run end to end against Fireworks AI with a Fireworks Workflow and a keyless Together Workflow on the same Worker: the first called the tool and answered through two completed model Activities, the second failed on its first attempt with the "Set TOGETHER_API_KEY" error. Baseten and Together AI were verified against their current docs (base URL, env var, model ID, tool calling) but not called.Related
main).