[Bug] OpenAI embedder skips the max_input_tokens truncation guard for plain list[str] inputs — oversized directory-meta overviews reach the provider (embedding 1210) #5760

Summary

OpenAICompatibleEmbedder._prepare_embedding_input() bypasses the max_input_tokens truncation guard entirely when the input is a plain list[str]. The directory-overview vectorization path (vectorize_directory_meta) submits its text as a plain string list, so long .overview.md bodies reach the provider untruncated and get rejected with 400 {'code': '1210'} (bigmodel embedding-3).

Affected version

v0.4.23 (HEAD 9d9bc85 verified)

Reproduction

1. A memory-type directory whose .overview.md is large (ours: memories/trajectories/.overview.md at 9.4 KB, listing every trajectory file).
2. Any write under that directory triggers MemoryUpdater.generate_overview → vectorize_directory_meta(uri, abstract, overview, ...) (utils/embedding_utils.py).
3. The overview text is embedded as a plain list[str]; _prepare_embedding_input (models/embedder/openai_embedders.py:322-325) returns it untruncated:

if isinstance(text, list) and all(isinstance(item, str) for item in text):
    return text                      # ← guard skipped
return self.prepare_embedding_input(text)

4. Provider rejects: 400 {'error': {'code': '1210', 'message': 'API 调用参数有误'}}.

Observed in production: every session commit touching memories/trajectories/ failed its embedding step until the overview was bounded.

Why the per-file path does not hit this

The single-string file path truncates correctly via truncate_embedding_input(content, embedding_cfg.max_input_tokens) (embedding_utils.py:669/702). Only the plain-list branch lacks the guard.

Suggested fix

Apply the same truncation to plain string lists, mirroring prepare_embedding_input:

if isinstance(text, list) and all(isinstance(item, str) for item in text):
    if self.max_input_tokens is None:
        return text
    return [truncate_embedding_input(item, self.max_input_tokens) for item in text]
return self.prepare_embedding_input(text)

Additional measurement (estimator calibration, FYI)

While diagnosing, we measured estimate_embedding_input_tokens against bigmodel embedding-3 usage.total_tokens:

The formula is well-calibrated for mixed CJK+Latin content (the dominant memory case); English-only text is under-counted ~15% (other/4 → measured ≈ other/3.5). Pure-CJK is over-counted (harmless direction). Optionally worth revisiting the Latin coefficient.

Impact

Any deployment using a directory-overview-producing memory type with an embedding provider that enforces an input token limit (bigmodel embedding-3 confirmed) fails embedding for that directory on every write until the overview is manually bounded.