Impact: High (performance)
The main indexing loop in `server/indexer/pipeline.py:192-248` processes one file at a time: blob fetch → await dense embed → await sparse embed → delete → upsert, then the next file. There is no `asyncio.gather`, no semaphore, and no accumulation of symbols across files into full embedding batches.
Providers support batches of 128 (`server/embeddings/voyage.py:15`, `openai.py:16`, `jina_api.py:16`), but a single file yields only a handful of symbols, so the pipeline sends dozens-to-hundreds of tiny requests instead of a few full ones. This is the largest performance problem in the codebase.
Recommended changes (in order of value)
- Fetch blobs concurrently with a bounded semaphore — the pattern already exists in `github_source.py` (`_walk_tree_recursive`).
- Accumulate `CodeSymbol`s across files and embed in batches of the provider batch size.
- At minimum,
asyncio.gather the dense and sparse embed_batch calls (pipeline.py:221-224) — dense is network-bound and sparse runs in a thread executor (bm25.py:19-21), so they overlap for free.
Impact: High (performance)
The main indexing loop in `server/indexer/pipeline.py:192-248` processes one file at a time: blob fetch → await dense embed → await sparse embed → delete → upsert, then the next file. There is no `asyncio.gather`, no semaphore, and no accumulation of symbols across files into full embedding batches.
Providers support batches of 128 (`server/embeddings/voyage.py:15`, `openai.py:16`, `jina_api.py:16`), but a single file yields only a handful of symbols, so the pipeline sends dozens-to-hundreds of tiny requests instead of a few full ones. This is the largest performance problem in the codebase.
Recommended changes (in order of value)
asyncio.gatherthe dense and sparseembed_batchcalls (pipeline.py:221-224) — dense is network-bound and sparse runs in a thread executor (bm25.py:19-21), so they overlap for free.