Repository navigation
Perf ollama embedder - #6
Merged
Merged
Conversation
added 5 commits
October 28, 2026 06:50
- OllamaEmbedder gains an optional keep_alive parameter (default "30m"), included in every /api/embed and /api/embeddings request body, including _probe_dimensions and embed_batch. - _post_json now distinguishes HTTP 404 (OllamaNotFoundError) from other failures. embed/embed_batch fall back to the legacy /api/embeddings endpoint only on 404; timeouts and connection errors raise immediately without a second attempt. - tests/test_ollama_keep_alive.py: 8 tests with a mocked _post_json covering keep_alive on probe/embed/embed_batch, custom keep_alive, the 404 fallback, and one-request behaviour for timeouts/connection errors.
benchmarks/latency_probe.py measures warm single-embed p50/p95 against a running Ollama and CrossEncoderVerifier scoring latency on 20 pairs. Library behaviour is unchanged.
- _probe_dimensions no longer returns a "last resort: 768" guess and no longer swallows all exceptions. Unreachable Ollama raises ConnectionError; a missing model (HTTP 404) raises RuntimeError with 'model not found; run: ollama pull <model_name>'. Only the explicit dimensions= constructor argument skips the probe. - The legacy /api/embeddings fallback in the probe is kept only for a 404 that may not be a missing-model error; if the legacy endpoint also 404s, the model-not-found message is raised. - manual_api_ollama.py lifespan already wrapped OllamaEmbedder construction in a RuntimeError, so both failure modes propagate. - tests/test_audit_fixes.py: the old test that relied on the 768 fallback is replaced with a fail-closed assertion; test_explicit_dimensions_skips_probe confirms dimensions= never makes a request. - tests/test_ollama_keep_alive.py: 5 new probe tests (unreachable, 404, 404-then-legacy-404, explicit dimensions, empty response). - tests/test_manual_api_ollama.py: TestLifespanFailsWhenOllamaDown asserts the app fails to start when the embedder probe fails.
- base_url default is now None, resolved by _resolve_ollama_base_url(): explicit argument wins; otherwise OLLAMA_HOST (with "http://" prepended when it has no scheme); otherwise http://127.0.0.1:11434. - manual_api_ollama.py calls OllamaEmbedder(model_name=...) with no base_url, so it picks up the new resolution automatically. - tests/test_ollama_keep_alive.py: 4 base_url tests (explicit URL wins, OLLAMA_HOST used with scheme added, default 127.0.0.1, OLLAMA_HOST with scheme kept).
- semantic_cache/embedders.py: split the 122-char timeout-detection line in OllamaEmbedder._post_json across an is_timeout helper. - audit_ollama_scores.py: wrap the 101-char header print line. - tests/test_numeric_facts_kind_order.py: wrap the 106-char check_match call in test_range_swap_still_rejects.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.