fix(embedding): truncate plain list inputs to max_input_tokens (#5760)

[akushonkamen]

Description

OpenAI-compatible embed() and embed_async() preserve plain list[str] inputs, but previously skipped max_input_tokens for these batches. Apply the existing truncation helper to each item when a limit is configured.

For example, with max_input_tokens=1, ["first text", "x"] previously reached the provider unchanged; it now becomes ["firs\n...(truncated for embedding)", "x"]. Item boundaries and short inputs are preserved. Without a configured limit, the batch passes through unchanged.

Human Involvement

• A human participated in the implementation or review loop
• This PR was generated entirely by AI agents without human participation in the loop

Related Issue

Related to #5760. This fixes the direct OpenAI batch-input guard; it does not establish an end-to-end fix for the reported directory-overview error.

Type of Change

• Bug fix (non-breaking change that fixes an issue)
• New feature (non-breaking change that adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Refactoring (no functional changes)
• Performance improvement
• Test update

Changes Made

• Bound plain string batches in the existing OpenAI-specific input guard using truncate_embedding_input.
• Keep the shared EmbedderBase contract unchanged: plain text or multimodal content parts. Queue acceptance of a list does not establish text-batch support for every provider.
• Update the existing batch-input test to cover truncation, short-item preservation, and item boundaries. No new test functions or files. Total diff: 10 additions and 6 deletions across two files.

Testing

• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes
• I have tested this on the following platforms:
  • Linux
  • macOS
  • Windows

Updated an existing test instead of adding tests. python -m pytest tests/unit/test_openai_embedder.py --no-cov -q: 10 passed. The no-new-tests check and git diff --check pass. Lint and format findings are unchanged from the base.

Checklist

• My code follows the project's coding style
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• Any dependent changes have been merged and published

Screenshots (if applicable)

Not applicable.

Additional Notes

The current directory-overview converter emits a string, so this change is limited to the existing OpenAI direct-call batch behavior. Single-string and multimodal-part handling retain their existing behavior.