Repository navigation
perf: embed() round-trips one value per worker call; 3.4x the model's batch encode #1204
Description
tests/e2e/speed_of_light_test.py in the plugin suite (#1200) is red at 3.4x the model's own batch encode on 32k short abstracts, cold cache, conftest's on-disk model2vec model:
| pipeline | wall |
|---|---|
one Python process: StaticModel.from_pretrained + model.encode(all_texts) |
1.31 s |
dirsql query "SELECT path, embed(content, '<model>') AS emb FROM './abstracts/*.md' ORDER BY path" |
4.46 s (startup 0.04 s) |
Vectors are identical. The gap is the protocol: one JSON line per value through the worker, one encode([text]) per line, one cache lookup per value. On the real model the per-value round trip measured 0.7 ms warm and 4.1 ms cold per abstract.
The bar is batch encode plus 10%, so the fix is batching on the core-to-worker path: the core collects the values a statement needs (or a chunk of them), the worker encodes them in one encode(list) call. Both ends of the protocol live in this repo ([[dirsql.function]] in the core, Worker.embed in the plugin).
Done when
plugins/dirsql-plugin-embeddings/tests/e2e/speed_of_light_test.py passes with its pytest.mark.xfail(strict=True) marker removed in the same PR: identical rows, and dirsql wall time minus launcher startup within 1.1x the native pipeline. Part of epic #1205.
Activity