Skip to content

Image index search cache holds ~600 MB after stop(), and misses entirely with 3+ active scopes #96

Description

@lstein

Severity: low. Memory retention and a cache-sizing limit; correctness is unaffected.

What

ImageIndexService._search_cache (invokeai/app/services/image_index/image_index_default.py) keeps up to 2 entries, each a full accessible embedding matrix. By the code's own arithmetic that is roughly 300 MB per entry at 100k images × 768 dims, so ~600 MB at capacity.

Two problems:

1. stop() does not release it. _search_cache is not cleared on shutdown, and neither are _text_encoder or _cpu_model. Disabling the image index at runtime therefore does not give the memory back — the process keeps holding both matrices and the loaded encoders until it exits.

2. Two entries is below the natural working set. The cache is keyed per scope, and an admin has their own all-images scope in addition to each user's. With three or more active scopes — three users, or two users plus an admin — the LRU thrashes to a zero hit rate, and every search re-reads every embedding BLOB from SQLite. That is precisely the cost the cache exists to avoid, and it degrades silently: the feature still works, just slowly, and only on multi-user installs.

Suggested directions

  • Clear _search_cache (and drop _text_encoder / _cpu_model) in stop().
  • Either size the cache per active scope rather than at a flat 2, or go the other way and keep a single entry — a flat 2 is the one size that is both large enough to retain a lot of memory and small enough to miss constantly.

Context

Found by an adversarial review of #43 (feat/image-map-10-semantic-search). The medium and critical findings from that review are fixed on that branch.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions