Skip to content

fix(mlx): validate vision prefix cache fail closed - #2277

Open
RezaGolriz wants to merge 1 commit into
exo-explore:mainfrom
RezaGolriz:codex/fail-closed-vision-prefix-cache
Open

fix(mlx): validate vision prefix cache fail closed#2277
RezaGolriz wants to merge 1 commit into
exo-explore:mainfrom
RezaGolriz:codex/fail-closed-vision-prefix-cache

Conversation

@RezaGolriz

Copy link
Copy Markdown

Summary

  • require positive media-region validation before reusing a vision KV prefix
  • truncate the reusable prefix before the first unvalidated image region
  • reject missing, shifted, duplicated, empty, or hash-mismatched media metadata instead of silently trusting it
  • preserve normal text-only prefix-cache behavior

Fixes #2272.

Why this approach

Image placeholder tokens can be identical even when the image bytes differ. A token-prefix match is therefore insufficient once it reaches a media region. The cache now fails closed unless the cached and incoming media spans have a one-to-one positional and hash match.

Validation

  • uv run pytest -q src/exo/worker/tests/unittests/test_mlx/test_kv_prefix_cache.py -m 'not slow' — 28 passed
  • uv run pytest -q src/exo/worker/tests/unittests -m 'not slow' — 161 passed, 184 deselected
  • uv run basedpyright src/exo/worker/engines/mlx/cache.py src/exo/worker/tests/unittests/test_mlx/test_kv_prefix_cache.py
  • uv run ruff check src/exo/worker/engines/mlx/cache.py src/exo/worker/tests/unittests/test_mlx/test_kv_prefix_cache.py

The regression tests cover different image hashes, missing metadata, positional shifts, duplicate spans, empty hashes, exact matches, and text-only prefixes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] KV prefix cache replays a previous image's features for token-identical vision requests (different images → identical answers)

1 participant