ds4-agent: fix every tool call failing on DeepSeek without --vision (+ regression test) - #963
Open
rinaldofesta wants to merge 1 commit into
Open
Conversation
Since 4771329 the agent builds every tool observation with ds4_chat_append_multimodal_message, whose guard rejects any model that is not GLM unless a DeepSeek vision encoder is loaded (fc8bf3c relaxed only that case). On a DeepSeek text model the observation never builds; agent_tool_observation_fits reports the failure as "tool result would exceed context", compaction runs, and the turn ends with "context full after compaction" at 1.9k of 262k tokens. A message with zero images is an ordinary chat message: delegate it to ds4_chat_append_message, which keeps the per-family rendering the agent used before 4771329 (DeepSeek: user role plus <tool_result>; GLM: same tokens as the multimodal branch). The vision guard still applies to messages that carry images. Test: --chat-multimodal-text-only asserts token-identical output between the two entry points for the tool and user roles. It fails on the previous engine (ok == 1) and passes with this change. Validation on a MacBook Pro M5 Max 128 GB, Metal, DeepSeek V4 Flash 0731 IQ2_XXS/Q2_K: ./ds4_test --chat-multimodal-text-only (before: 4 assertion failures, after: OK), ./ds4_test --server OK, ./ds4_test --tool-call-quality OK, ./ds4_agent_test OK, a scripted ds4-agent --non-interactive list tool call OK, make ds4_cpu.o clean. Not run: CUDA, ROCm, GLM, --logprob-vectors. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
rinaldofesta
force-pushed
the
fix/agent-textonly-tool-observation
branch
from
September 3, 2026 16:10
96f7166 to
4113327
Compare
Contributor
Author
|
Unrelated follow-up in #965 (Metal DSpark scheduler bypass). The two do not touch the same lines and merge in either order. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Since 4771329 ds4-agent cannot complete a single tool call on a DeepSeek text model. The first tool result dies with
at 1.9k of 262k context, on a 29-entry
list .. Reproduced on an M5 Max 128 GB with DeepSeek V4 Flash 0731 IQ2_XXS, no--vision.Root cause:
agent_tool_observation_buildsends every tool observation throughds4_chat_append_multimodal_message, whose guard rejects any model that is not GLM unless a DeepSeek vision encoder is loaded (fc8bf3c relaxed only that case). The observation never builds.agent_tool_observation_fitsreturns false on a build error as well, so the agent reads it as a context overflow, compacts, and gives up.Fix, 8 lines in
ds4.c: a user or tool message with zero images delegates tods4_chat_append_message, the rendering the agent used before 4771329 (DeepSeek: user role plus<tool_result>; GLM: the same tokens the multimodal branch emits). The vision guard is untouched and still applies to messages carrying images.Test,
tests/ds4_test.c,--chat-multimodal-text-only: asserts token-identical output between the two entry points for the tool and user roles. On the engine before this patch it fails twice per role (assertion failed: ok == 1); with the patch it passes.Validation on a MacBook Pro M5 Max 128 GB, Metal, Flash 0731 IQ2_XXS/Q2_K:
./ds4_test --chat-multimodal-text-only: before 4 assertion failures, after OK./ds4_test --serverOK,./ds4_test --tool-call-qualityOK,./ds4_agent_testOKds4-agent --non-interactivewith onelisttool call in a scratch project: no compaction, correct answer. An interactive session then ran 11 tool calls on a Next.js repo without a hitch.make ds4_cpu.oclean. The patch applies on current main b0a147a.Not run: CUDA, ROCm, GLM models,
--logprob-vectors.Overlap with #924: same idea for the
ds4.chunk. #924 no longer applies to main after fc8bf3c, adds a second copy of the dispatch inds4_agent.c, and has no test. If you would rather land #924 rebased, I can move the test there instead.Two things I left out on purpose, both separate changes:
agent_tool_observation_fitsstill reports any build error as a context overflow and drops the error text, and the image-bearing tool branch renders DeepSeek observations with GLM tags and no role token.Implemented with the help of a coding agent (Claude Code) and reviewed locally by the author.