Skip to content

docs: document UberLLM OpenAI-compatible gateway setup - #3376

Merged
Yeachan-Heo merged 1 commit into
ultraworkers:mainfrom
PSA-Team-source:docs-uberllm-gateway
Nov 4, 2026
Merged

Yeachan-Heo merged 1 commit into
ultraworkers:mainfrom
PSA-Team-source:docs-uberllm-gateway

Conversation

@PSA-Team-source

Copy link
Copy Markdown
Contributor

Summary

  • Adds an UberLLM section to docs/local-openai-compatible-providers.md, after the LLMTR section. Docs only.
  • Covers the base URL and key, using local/<catalog id> for every id (a bare qwen/ id goes to DashScope), and how the 64,000-token request relates to each model's output limit.
  • Disclosure: I work on UberLLM. It is a paid, prepaid service with no free tier, and the section says so.
  • Closes docs: document UberLLM OpenAI-compatible gateway setup #3375

Anti-slop triage

  • Classification: docs-only
  • Evidence: docs: document UberLLM OpenAI-compatible gateway setup #3375. Release binary v0.1.11 (claw-linux-x64, sha256 checked): claw --model local/deepseek/deepseek-v4-flash prompt "Reply exactly HELLO_WORLD_123" printed HELLO_WORLD_123, and so did local/moonshotai/kimi-k2.6 and local/qwen/qwen3.8-flash. A bare qwen/qwen3.8-flash failed with missing DashScope credentials, as the section says. A direct request with max_tokens: 64000 to aion-labs/aion-3-0-mini (output limit 32,768) answered normally.
  • Non-destructive review result: merge candidate

Verification

  • python .github/scripts/check_doc_source_of_truth.py and check_release_readiness.py pass. No Rust code changed, so cargo test wasn't run; I tested with the published release binary rather than a local build.
  • git diff --check passes.
  • The curl ... | jq model-list command in the section was run as written.

🤖 Generated with Claude Code

@Yeachan-Heo Yeachan-Heo left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✓ Verified Claims

local/ prefix routing:

  • Function strip_routing_prefix in rust/crates/api/src/providers/openai_compat.rs:1002 recognizes "local" and strips it.
  • Remainder is returned verbatim: &model[pos + 1..]

Native route avoidance:

  • Bare qwen/ models route to DashScope (confirmed in rust/crates/api/src/providers/mod.rs:288 check: if canonical.starts_with("qwen/"))
  • Test at L1280 confirms: qwen-prefixed models resolve to DashScope auth, not UberLLM

Default max_tokens:

  • max_tokens_for_model in rust/crates/api/src/providers/mod.rs:628 defaults to 64_000
  • No OpenAI-compatible-specific override; default applies uniformly

Vendor prefixes verified against live UberLLM catalog:

  • deepseek/deepseek-v4-flash exists, 128k max_output_tokens
  • moonshotai/kimi-k2.6 exists, 65.5k max_output_tokens
  • aion-labs/aion-3-0-mini exists, 32,768 max_output_tokens (example in doc)
  • curl command confirmed functional and accurate (returns JSON with max_output_tokens field)

✓ Consistency & Quality

  • Structure, length, and scope match precedent LLMTR section (merged 6ac59ab)
  • Properly distinguishes UberLLM behavior (bounds output) vs LLMTR (rejects request)
  • No duplication of generic OpenAI-compatible guidance from OmniRoute section
  • Placeholder key format consistent: ull_... follows pattern
  • No unverified claims or marketing language

✓ Validation

  • python .github/scripts/check_doc_source_of_truth.py → passed
  • git diff --check → no trailing whitespace

—
[repo owner's gaebal-gajae (clawdbot) 🦞]

@Yeachan-Heo
Yeachan-Heo merged commit ab8561d into ultraworkers:main Nov 4, 2026
6 checks passed
@Yeachan-Heo

Copy link
Copy Markdown
Contributor

Merged, thanks. The routing and token-limit details check out against the provider code, and the disclosure is clear.

—
[repo owner's gaebal-gajae (clawdbot) 🦞]

@XinwangTao

Copy link
Copy Markdown

Thanks for merging this — the UberLLM section with the local/<catalog id> prefix guidance and the verified examples (deepseek/deepseek-v4-flash, moonshotai/kimi-k2.6, and the qwen3.8-flash DashScope fallback case) makes the setup easy to follow for new users.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

docs: document UberLLM OpenAI-compatible gateway setup

3 participants