Repository navigation
docs: document UberLLM OpenAI-compatible gateway setup - #3376
Merged
Yeachan-Heo merged 1 commit intoNov 4, 2026
Merged
Conversation
Yeachan-Heo
approved these changes
Nov 4, 2026
Yeachan-Heo
left a comment
Contributor
There was a problem hiding this comment.
✓ Verified Claims
local/ prefix routing:
- Function
strip_routing_prefixinrust/crates/api/src/providers/openai_compat.rs:1002recognizes "local" and strips it. - Remainder is returned verbatim:
&model[pos + 1..]
Native route avoidance:
- Bare
qwen/models route to DashScope (confirmed inrust/crates/api/src/providers/mod.rs:288check:if canonical.starts_with("qwen/")) - Test at L1280 confirms: qwen-prefixed models resolve to DashScope auth, not UberLLM
Default max_tokens:
max_tokens_for_modelinrust/crates/api/src/providers/mod.rs:628defaults to64_000- No OpenAI-compatible-specific override; default applies uniformly
Vendor prefixes verified against live UberLLM catalog:
deepseek/deepseek-v4-flashexists, 128k max_output_tokensmoonshotai/kimi-k2.6exists, 65.5k max_output_tokensaion-labs/aion-3-0-miniexists, 32,768 max_output_tokens (example in doc)- curl command confirmed functional and accurate (returns JSON with
max_output_tokensfield)
✓ Consistency & Quality
- Structure, length, and scope match precedent LLMTR section (merged 6ac59ab)
- Properly distinguishes UberLLM behavior (bounds output) vs LLMTR (rejects request)
- No duplication of generic OpenAI-compatible guidance from OmniRoute section
- Placeholder key format consistent:
ull_...follows pattern - No unverified claims or marketing language
✓ Validation
python .github/scripts/check_doc_source_of_truth.py→ passedgit diff --check→ no trailing whitespace
—
[repo owner's gaebal-gajae (clawdbot) 🦞]
Contributor
|
Merged, thanks. The routing and token-limit details check out against the provider code, and the disclosure is clear. — |
|
Thanks for merging this — the UberLLM section with the |
5 of 7 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
docs/local-openai-compatible-providers.md, after the LLMTR section. Docs only.local/<catalog id>for every id (a bareqwen/id goes to DashScope), and how the 64,000-token request relates to each model's output limit.Anti-slop triage
claw-linux-x64, sha256 checked):claw --model local/deepseek/deepseek-v4-flash prompt "Reply exactly HELLO_WORLD_123"printedHELLO_WORLD_123, and so didlocal/moonshotai/kimi-k2.6andlocal/qwen/qwen3.8-flash. A bareqwen/qwen3.8-flashfailed withmissing DashScope credentials, as the section says. A direct request withmax_tokens: 64000toaion-labs/aion-3-0-mini(output limit 32,768) answered normally.Verification
python .github/scripts/check_doc_source_of_truth.pyandcheck_release_readiness.pypass. No Rust code changed, socargo testwasn't run; I tested with the published release binary rather than a local build.git diff --checkpasses.curl ... | jqmodel-list command in the section was run as written.🤖 Generated with Claude Code