server: unify text and vision KV prefix reuse

[HariboApfel]

Goal

Make KV-cache reuse modality-agnostic: restore the longest prefix that produced identical model inputs—text or images—and compute only the new or changed suffix.

Motivation

For ds4 on Apple M-series hardware, practical agent serving lives or dies by cache reuse. OMP system and tool prompts can exceed 17,000 tokens and take tens of seconds to prefill, whereas restoring a compatible KV checkpoint takes only tens of milliseconds. Initial prefill is a much larger bottleneck here than on the NVIDIA serving hardware typically used for these models, so cache reuse needs to be a first-class execution path rather than a best-effort optimization.

The previous split between text and vision lookup paths caused that optimization to disappear exactly when a conversation gained an image. The underlying KV tensors are not inherently text or vision specific; reuse is correct whenever the retained token prefix and all conditioning inputs before the cache frontier are identical.

That gives predictable behavior:

• cached text + first image → reuse the text prefix
• cached image A + appended image B → reuse through image A
• image changed, moved, or removed → rewind safely before the affected image

Images are represented in cache metadata as conditioning spans, identified by token location, row count, and a SHA-256 fingerprint of the actual encoder output. This preserves correctness without forcing a full prefill at modality boundaries.

In the OMP validation run, this reused 18,073 of 18,435 tokens for the first image and 18,807 of 19,079 tokens when adding a second image.

Summary

• search text-only and image-conditioned checkpoints through one longest-compatible-prefix path
• reuse text KV when the first image is appended, and image A KV when image B is appended
• validate every retained conditioning span by token location, row count, and embedding SHA-256; rewind before changed, moved, or removed images
• preserve compatibility with existing text cache files and make recovered live-slot misses explicit in logs

Dependency

Stacked on #961 at commit d28b29e, which adds exact multimodal checkpoint persistence. Until #961 lands, this PR intentionally includes that prerequisite commit.

Test plan

• make -j4 ds4-server ds4_test
• DS4_LOCK_FILE=/tmp/ds4-codex-vision-kv-test.lock ./ds4_test --server
• Metal + OMP: append first image to a text conversation; restored 18,073 / 18,435 prompt tokens
• Metal + OMP: append a second image; restored the one-image checkpoint at 18,807 / 19,079 prompt tokens
• changed/removed/relocated conditioning-span unit coverage

Generated with Codex.