Skip to content

ds4-agent: fix every tool call failing on DeepSeek without --vision (+ regression test) - #963

Open
rinaldofesta wants to merge 1 commit into
antirez:mainfrom
rinaldofesta:fix/agent-textonly-tool-observation
Open

ds4-agent: fix every tool call failing on DeepSeek without --vision (+ regression test)#963
rinaldofesta wants to merge 1 commit into
antirez:mainfrom
rinaldofesta:fix/agent-textonly-tool-observation

Conversation

@rinaldofesta

Copy link
Copy Markdown
Contributor

Since 4771329 ds4-agent cannot complete a single tool call on a DeepSeek text model. The first tool result dies with

COMPACTING tool result would exceed context: summarizing durable task state
ds4-agent: context full after compaction

at 1.9k of 262k context, on a 29-entry list .. Reproduced on an M5 Max 128 GB with DeepSeek V4 Flash 0731 IQ2_XXS, no --vision.

Root cause: agent_tool_observation_build sends every tool observation through ds4_chat_append_multimodal_message, whose guard rejects any model that is not GLM unless a DeepSeek vision encoder is loaded (fc8bf3c relaxed only that case). The observation never builds. agent_tool_observation_fits returns false on a build error as well, so the agent reads it as a context overflow, compacts, and gives up.

Fix, 8 lines in ds4.c: a user or tool message with zero images delegates to ds4_chat_append_message, the rendering the agent used before 4771329 (DeepSeek: user role plus <tool_result>; GLM: the same tokens the multimodal branch emits). The vision guard is untouched and still applies to messages carrying images.

Test, tests/ds4_test.c, --chat-multimodal-text-only: asserts token-identical output between the two entry points for the tool and user roles. On the engine before this patch it fails twice per role (assertion failed: ok == 1); with the patch it passes.

Validation on a MacBook Pro M5 Max 128 GB, Metal, Flash 0731 IQ2_XXS/Q2_K:

  • ./ds4_test --chat-multimodal-text-only: before 4 assertion failures, after OK
  • ./ds4_test --server OK, ./ds4_test --tool-call-quality OK, ./ds4_agent_test OK
  • ds4-agent --non-interactive with one list tool call in a scratch project: no compaction, correct answer. An interactive session then ran 11 tool calls on a Next.js repo without a hitch.
  • make ds4_cpu.o clean. The patch applies on current main b0a147a.

Not run: CUDA, ROCm, GLM models, --logprob-vectors.

Overlap with #924: same idea for the ds4.c hunk. #924 no longer applies to main after fc8bf3c, adds a second copy of the dispatch in ds4_agent.c, and has no test. If you would rather land #924 rebased, I can move the test there instead.

Two things I left out on purpose, both separate changes: agent_tool_observation_fits still reports any build error as a context overflow and drops the error text, and the image-bearing tool branch renders DeepSeek observations with GLM tags and no role token.

Implemented with the help of a coding agent (Claude Code) and reviewed locally by the author.

Since 4771329 the agent builds every tool observation with
ds4_chat_append_multimodal_message, whose guard rejects any model that is
not GLM unless a DeepSeek vision encoder is loaded (fc8bf3c relaxed only
that case). On a DeepSeek text model the observation never builds;
agent_tool_observation_fits reports the failure as "tool result would
exceed context", compaction runs, and the turn ends with "context full
after compaction" at 1.9k of 262k tokens.

A message with zero images is an ordinary chat message: delegate it to
ds4_chat_append_message, which keeps the per-family rendering the agent
used before 4771329 (DeepSeek: user role plus <tool_result>; GLM: same
tokens as the multimodal branch). The vision guard still applies to
messages that carry images.

Test: --chat-multimodal-text-only asserts token-identical output between
the two entry points for the tool and user roles. It fails on the
previous engine (ok == 1) and passes with this change.

Validation on a MacBook Pro M5 Max 128 GB, Metal, DeepSeek V4 Flash 0731
IQ2_XXS/Q2_K: ./ds4_test --chat-multimodal-text-only (before: 4
assertion failures, after: OK), ./ds4_test --server OK,
./ds4_test --tool-call-quality OK, ./ds4_agent_test OK, a scripted
ds4-agent --non-interactive list tool call OK, make ds4_cpu.o clean.
Not run: CUDA, ROCm, GLM, --logprob-vectors.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@rinaldofesta
rinaldofesta force-pushed the fix/agent-textonly-tool-observation branch from 96f7166 to 4113327 Compare September 3, 2026 16:10
@rinaldofesta

Copy link
Copy Markdown
Contributor Author

Unrelated follow-up in #965 (Metal DSpark scheduler bypass). The two do not touch the same lines and merge in either order.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant