feat(embeddings): batch ollama embeds via /api/embed

[dkhokhlov]

Summary

Cuts per-chunk HTTP round-trips for ollama embeddings by switching the fast path to POST /api/embed with input: string[] (one request for N chunks instead of N requests). Amortizes N HTTP
round-trips into 1, which dominates indexing throughput for a remote or shared ollama host.

Closes #301.

Problem

OllamaEmbeddingProvider.embedBatch issued one POST /api/embeddings (single prompt) per chunk, and getDynamicBatchOptions pinned ollama maxBatchItems: 1, so the indexer's batching
machinery never packed texts — every chunk was its own HTTP round-trip. The cost is the embedding HTTP wait, not local disk or parse.

Ollama's newer POST /api/embed endpoint accepts input: string[] and returns embeddings: number[][] (one vector per input, in order) in a single request. Ollama encodes each input
independently (each element of input[] produces its own vector, not a concatenation), so the model context length applies per input, not over the batch — the upstream splitter already bounds
each input.

Changes

• src/embeddings/providers/ollama.ts: new embedMany (batched /api/embed); embedBatch now tries the batch path and falls back to the legacy per-text /api/embeddings loop on 404 (old
  ollama without /api/embed), context-length overflow (per-text truncation net), or a malformed batch response (per-chunk isolation — one bad vector fails only its own chunk).
  embedSingle/embedSingleWithFallback unchanged.
• src/indexer/index.ts: getDynamicBatchOptions ollama defaults to maxBatchItems: 16, maxBatchTokens: 65536. New forceSingleItemBatches option forces maxBatchItems: 1 on the two
  failed-batch recovery paths (index retry loop + retryFailedBatchesUnlocked) so a permanently-failing chunk is isolated from healthy chunks on the retry run; scoped to ollama.
• src/config/schema.ts: new embedding.batch.{maxBatchItems,maxBatchTokens} knobs (parsed with Number.isFinite, clamped to ≥1).
• docs/configuration.md: "Batching Ollama embeddings" section documenting the endpoint, fallbacks, knobs, and the resource-pressure note (up to 5 concurrent ollama requests ×
  maxBatchItems chunks in flight; ollama concurrency is fixed at 5, not configurable).
• Tests: new tests/ollama-embed.test.ts (batched success, empty, single-text-uses-legacy, 404 fallback, context-length fallback, malformed-batch fallback, non-JSON/null-body fallback,
  non-404 error propagation); tests/indexer-failed-batches.test.ts pins per-text recovery tests to maxBatchItems: 1 and adds a batched isolation regression test; tests/custom-provider.test.ts
  updated to assert batching into a single /api/embed request; tests/config.test.ts covers the new knobs.

Verified

• Affected suites green (npx vitest run); npm run typecheck clean; npm run build:ts exit 0.
• Batched code confirmed in dist/cli.js + dist/index.js.

[Helweg]

I found one correctness issue before this can be approved:

getDynamicBatchOptions() applies embedding.batch.maxBatchItems and maxBatchTokens to every provider, despite the new public config/docs explicitly stating that these options are currently Ollama-only. For example, a custom, OpenAI, or Google setup with embedding.batch.maxBatchItems: 1 will now be split by the indexer into one-item calls, changing existing request behavior and potentially defeating a provider's own batching. Please return {} immediately for non-Ollama providers, and add a regression test showing a non-Ollama provider ignores this Ollama-specific configuration.

The targeted implementation tests, config tests, typecheck, and lint otherwise pass locally on e37b013.

[Helweg]

Thanks for the follow-up. We are planning the next release and would like to include this Ollama batching feature.

The PR is currently a draft and has conflicts with main, and its head changed after the last review. Please rebase onto current main, resolve the conflicts, and mark it ready for review. Once pushed, we will re-review the final head and run the hosted validation.

[dkhokhlov]

@Helweg
Let me profile the indexing throughput.

[Helweg]

Re-reviewed the current head . The prior non-Ollama batching concern is addressed: now returns empty indexer options for non-Ollama providers. Focused native-backed validation passed: 192 tests across Ollama batching, batch pooling, failed-batch recovery, custom-provider compatibility, and config parsing. The implementation is approved pending the required GitHub-hosted workflows.

[Helweg]

Clarification to my approval: I reviewed head 7dc5d21. The prior non-Ollama batching concern is addressed because embedding.batch now returns empty indexer options for non-Ollama providers. Native-backed focused validation passed: 192 tests covering Ollama batching, batch pooling, failed-batch recovery, custom-provider compatibility, and config parsing. Approval remains conditional on the required GitHub-hosted workflows.