Skip to content

fix(knowledge): support Doubao multimodal online embeddings (#1299) - #1405

Open
lymerin wants to merge 1 commit into
TencentCloud:developfrom
lymerin:codex/1299
Open

lymerin wants to merge 1 commit into
TencentCloud:developfrom
lymerin:codex/1299

Conversation

@lymerin

@lymerin lymerin commented Sep 30, 2026

Copy link
Copy Markdown

Summary

Fixes #1299.

Support Doubao vision embedding models in knowledge-base online embeddings. These models use /embeddings/multimodal, whose request and response formats differ from the OpenAI text embedding API.

  • Use the multimodal protocol for doubao-embedding-vision-* models or an explicitly configured /embeddings/multimodal URL.
  • Send typed text inputs and read the dense vector from data.embedding.
  • Apply the same protocol selection to provider probing, document indexing, and retrieval.
  • Request one vector per chunk to preserve chunk/vector alignment. Existing text embedding batching remains unchanged.

Endpoint IDs (ep-*) are supported through an explicitly configured multimodal URL.

Target branch

  • Base is develop (feature / fix — default)
  • Base is main (release/* or hotfix/* only)

Type of change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation
  • Refactor / chore
  • Release / hotfix

Test plan

  • make all passes locally — 3920 passed, 120 skipped
  • Added/updated tests

Regression tests cover protocol selection, explicit endpoint URLs, endpoint IDs, request payloads, authentication/custom headers, ordered vectors across 21 chunks, and indexing/retrieval through the real SQLite knowledge index. Existing OpenAI and Doubao text embedding behavior is also covered.

The pre-commit checks and dashboard build passed. Provider responses were mocked; no live Ark API validation was performed.

Checklist

  • Updated CHANGELOG.md (if user-facing)
  • README / docs updated (if needed) — not required for this fix

@lymerin lymerin changed the title Codex/1299 fix(knowledge): support Doubao multimodal online embeddings (#1299) Sep 30, 2026
@lymerin
lymerin changed the base branch from main to develop September 30, 2026 19:24

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug] 使用知识库功能,添加在线 Embedding模型,使用火山引擎的doubao-embedding-vision-251215模型,会报错

1 participant