fix(response_codec): suppress deepseek-v4-pro tool markup leaked into content

[jatmn]

Problem

DeepSeek-v4-pro emits native tool_calls AND leaks XML-ish tool markup (e.g. <parameter, <tool, <invoke, <function, <function_call, <think) as delta.content. Those fragments were forwarded verbatim as assistant output text, causing the reported <parameter> spam. This can happen even before the first native tool_calls chunk arrives.

Fix

• Detect any content fragment that begins with a tool markup tag and suppress it before it is accumulated or emitted as output_text.delta.
• Preserve native tool_calls and legitimate prose/content.
• Add regression test deepseek_v4_pro_tool_markup_in_content_is_suppressed covering markup arriving before tool_calls, legitimate prose, and non-tool XML-like content.

Validation

• cargo mutants --in-diff git.diff: 14/14 mutants caught (previously 6 missed).
• response_codec module: 183 passed / 0 failed.
• Full crate: 623 passed / 25 failed with patch vs 622/25 pristine. The 25 failures are pre-existing network-dependent webui/upstream tests unrelated to this change.

Final reviewed SHA

4105a4c7b9d420a2b0157ccd2889da06c7792c32

Summary by Sourcery

Prevent DeepSeek-v4-pro tool markup from appearing as assistant output while retaining valid content and native tool calls.

Bug Fixes:

• Suppress leaked DeepSeek-v4-pro tool-invocation markup from assistant content while preserving native tool calls and legitimate text.

Enhancements:

• Make duplicate tool-markup suppression configurable and enable it for the DeepSeek-v4-pro model family.

Tests:

• Add regression coverage for leaked markup before and alongside native tool calls, legitimate prose, and non-tool XML-like content.

[sourcery-ai]

Hey - I've found 1 issue, and left some high level feedback:

• The suppression check using chars().take_while(...).last().is_some_and(...) is a bit convoluted; consider simplifying this logic (e.g., by directly trimming leading whitespace and checking the first non-whitespace character) to improve readability.
• In looks_like_tool_markup, converting the entire string to lowercase allocates a new String; if this function is called frequently, consider a cheaper case-insensitive prefix check (e.g., manual char comparison or limiting the lowercase conversion to the maximum tag prefix length).

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- The suppression check using `chars().take_while(...).last().is_some_and(...)` is a bit convoluted; consider simplifying this logic (e.g., by directly trimming leading whitespace and checking the first non-whitespace character) to improve readability.
- In `looks_like_tool_markup`, converting the entire string to lowercase allocates a new `String`; if this function is called frequently, consider a cheaper case-insensitive prefix check (e.g., manual char comparison or limiting the lowercase conversion to the maximum tag prefix length).

## Individual Comments

### Comment 1
<location path="src/response_codec_tests.rs" line_range="4991-5000" />
<code_context>
+            }]}
+        }]
+    }));
+    accum.apply_chat_chunk(&json!({
+        "choices": [{
+            "delta": {"content": "<tool>"}
+        }]
+    }));
+    accum.apply_chat_chunk(&json!({
+        "choices": [{
+            "delta": {"content": "Working on it."}
+        }]
+    }));
+    accum.apply_chat_chunk(&json!({
+        "choices": [{
+            "delta": {"content": "the `<parameter>` tag is not markup"}
+        }]
+    }));
+    accum.apply_chat_chunk(&json!({
+        "choices": [{
+            "delta": {"content": "<hello>this is not a tool tag</hello>"}
+        }]
+    }));
+    accum.apply_chat_chunk(&json!({
+        "choices": [{
+            "delta": {"content": "<function>fn x() {}</function>"}
+        }]
+    }));
+    accum.apply_chat_chunk(&json!({
+        "choices": [{
+            "delta": {"content": "<think>some reasoning</think>"}
+        }]
+    }));
</code_context>
<issue_to_address>
**suggestion (testing):** Extend the regression to cover all tool tags handled by `looks_like_tool_markup`, especially `<invoke` and `<function_call`.

The production code suppresses `<parameter`, `<tool`, `<invoke`, `<function`, `<function_call`, and `<think>`, but this test only covers a subset (`<parameter`, `<tool`, `<function`, `<think>`). Please add chunks containing `<invoke ...>` and `<function_call ...>` and assert that their content is also excluded from `output_text` to fully exercise `looks_like_tool_markup` behavior.

Suggested implementation:

```rust
    accum.apply_chat_chunk(&json!({
        "choices": [{
            "delta": {"content": "<tool>"}
        }]
    }));
    accum.apply_chat_chunk(&json!({
        "choices": [{
            "delta": {"content": "Working on it."}
        }]
    }));
    accum.apply_chat_chunk(&json!({
        "choices": [{
            "delta": {"content": "the `<parameter>` tag is not markup"}
        }]
    }));
    accum.apply_chat_chunk(&json!({
        "choices": [{
            "delta": {"content": "<hello>this is not a tool tag</hello>"}
        }]
    }));
    accum.apply_chat_chunk(&json!({
        "choices": [{
            "delta": {"content": "<function>fn x() {}</function>"}
        }]
    }));
    accum.apply_chat_chunk(&json!({
        "choices": [{
            "delta": {"content": "<think>some reasoning</think>"}
        }]
    }));
    accum.apply_chat_chunk(&json!({
        "choices": [{
            "delta": {"content": "<invoke name=\"exec_command\">echo hi</invoke>"}
        }]
    }));
    accum.apply_chat_chunk(&json!({
        "choices": [{
            "delta": {"content": "<function_call name=\"exec_command\">{\"cmd\":\"ls\"}</function_call>"}
        }]
    }));

```

You will also need to update the final assertion(s) in `deepseek_v4_pro_tool_markup_in_content_is_suppressed` to ensure the `<invoke ...>` and `<function_call ...>` chunks are excluded from `output_text`. For example, if the test currently asserts only the non-tool-markup content is present (like `"Working on it.the `<parameter>` tag is not markup<hello>this is not a tool tag</hello>"`), extend the expected string so that it still **does not** include any of `<parameter ...>`, `<tool>`, `<function>...`, `<think>...`, `<invoke ...>`, or `<function_call ...>`, while preserving any legitimate assistant text that follows these chunks.
</issue_to_address>

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[sourcery-ai]

suggestion (testing): Extend the regression to cover all tool tags handled by looks_like_tool_markup, especially <invoke and <function_call.

The production code suppresses <parameter, <tool, <invoke, <function, <function_call, and <think>, but this test only covers a subset (<parameter, <tool, <function, <think>). Please add chunks containing <invoke ...> and <function_call ...> and assert that their content is also excluded from output_text to fully exercise looks_like_tool_markup behavior.

Suggested implementation:

    accum.apply_chat_chunk(&json!({
        "choices": [{
            "delta": {"content": "<tool>"}
        }]
    }));
    accum.apply_chat_chunk(&json!({
        "choices": [{
            "delta": {"content": "Working on it."}
        }]
    }));
    accum.apply_chat_chunk(&json!({
        "choices": [{
            "delta": {"content": "the `<parameter>` tag is not markup"}
        }]
    }));
    accum.apply_chat_chunk(&json!({
        "choices": [{
            "delta": {"content": "<hello>this is not a tool tag</hello>"}
        }]
    }));
    accum.apply_chat_chunk(&json!({
        "choices": [{
            "delta": {"content": "<function>fn x() {}</function>"}
        }]
    }));
    accum.apply_chat_chunk(&json!({
        "choices": [{
            "delta": {"content": "<think>some reasoning</think>"}
        }]
    }));
    accum.apply_chat_chunk(&json!({
        "choices": [{
            "delta": {"content": "<invoke name=\"exec_command\">echo hi</invoke>"}
        }]
    }));
    accum.apply_chat_chunk(&json!({
        "choices": [{
            "delta": {"content": "<function_call name=\"exec_command\">{\"cmd\":\"ls\"}</function_call>"}
        }]
    }));

You will also need to update the final assertion(s) in deepseek_v4_pro_tool_markup_in_content_is_suppressed to ensure the <invoke ...> and <function_call ...> chunks are excluded from output_text. For example, if the test currently asserts only the non-tool-markup content is present (like "Working on it.the tag is not markup<hello>this is not a tool tag</hello>"), extend the expected string so that it still does not include any of <parameter ...>, <tool>, <function>..., <think>..., <invoke ...>, or <function_call ...>, while preserving any legitimate assistant text that follows these chunks.

[jatmn]

@sourcery-ai review

[sourcery-ai]

Sorry @jatmn, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery