Skip to content

ds4-agent: context compaction rebuild fails with "metal prefill failed" in a Vision-Exp session #945

Description

@ChengHuPacer

What happened

In a long-running ds4-agent session on Metal using DeepSeek V4 Flash Vision-Exp, automatic context compaction generated its durable summary successfully but failed while rebuilding the compacted context:

COMPACTING rebuilding context: old=85462 summary+tail=13924 tail=10000
ds4-agent: metal prefill failed
COMPACTING soft limit before user turn: summarizing durable task state

The previous transcript is restored after the failure, but the agent remains above the compaction threshold, so another user turn can trigger the same failure again.

What I expected

Compaction should preserve or omit multimodal observations atomically, rebuild a valid compacted context, and continue processing the pending user turn.

Steps to reproduce

  1. Build ds4-agent from current main and run it on an Apple Silicon Mac with the Metal backend, a DeepSeek V4 Flash Vision-Exp model, and the DeepSeek V4 vision encoder.
  2. Run a long coding session that includes one or more view_image observations.
  3. Continue until the transcript reaches the automatic compaction threshold (about 85k tokens with the default 100k context).
  4. Submit another user turn so automatic compaction runs.
  5. Observe that summary generation completes, but the compacted-context Metal prefill fails.

Additional context

  • Commit: 819cef0 (local build; the vision encoder is configured to load by default, with inference code otherwise matching this commit).
  • Hardware: Apple Silicon Mac with 128 GB unified memory.
  • Backend: Metal.
  • Context size: 100,000 tokens.
  • Model: DeepSeek-V4-Flash-Vision-Exp-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8.gguf.
  • Vision encoder: DeepSeek-V4-Flash-Vision-Encoder.gguf.
  • The rebuilt prompt contains only 13,924 tokens, so this is not a context-length overflow.
  • The retained tail was exactly the 10,000-token budget. Code inspection suggests that when no later user-turn boundary exists, the raw cutoff can land inside a DeepSeek image token block, while only fully retained image spans/embeddings are carried into the rebuilt transcript. That would leave a partial image block and cause Vision-Exp prefill to reject the prompt.
  • If available, the lower-level stderr line immediately before ds4-agent: metal prefill failed would help distinguish a malformed image block from a Metal command-buffer failure.
  • This appears distinct from GLM-5.3 branch: compaction fails when think-mode generation overshoots the soft limit — analysis + 4 local patches #906 (insufficient room to create a summary) and DeepSeek V4 Flash Vision: image inference fails on Metal (M4 Max) #928 (vision encoding itself fails before inference).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions