Accept sanitised Kimi tool-call ids and return the canonical form - #2282
Open
Bdwg222 wants to merge 1 commit into
Open
Accept sanitised Kimi tool-call ids and return the canonical form#2282Bdwg222 wants to merge 1 commit into
Bdwg222 wants to merge 1 commit into
Conversation
Fixes exo-explore#2281 (part 1). Kimi K2 emits functions_<name>_<n> when a client has echoed a sanitised id back in the history; the parser only accepted functions.<name>:<n> and failed hard. Accept both, always return canonical.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Fixes #2281 (part 1 of 2).
_parse_kimi_tool_callsrequires the id immediately before<|tool_call_argument_begin|>to matchfunctions.<name>:<n>. Kimi K2 models also emitfunctions_<name>_<n>— they do so whenever a client has echoed a sanitised id back in the conversation history, which is common (OpenAI-style clients restrict ids to[A-Za-z0-9_-]). That produces a hard parse failure, surfaced to the client as anErrorChunkwhose message is the raw tool-call markup (streaming) or a bare 500 (non-streaming). In practice the second tool call in any agent session fails deterministically.Changes
src/exo/worker/engines/mlx/utils_mlx.py—_parse_kimi_tool_calls: the id regex now accepts.or_afterfunctions, and:or_before the index. The returned id is always the canonicalfunctions.<name>:<n>regardless of which form the model emitted.src/exo/worker/tests/unittests/test_runner/test_kimi_tool_parser.py(new) — unit tests for the canonical form, the sanitised form, underscores inside function names, a missingfunctions.prefix, multiple calls with mixed forms, and the still-rejected no-index case.Why It Works
The trailing
[:_]\d+is unambiguous even when the function name contains underscores (functions_read_file_3→read_file,3): the lazy name group can only stop where separator + digits + the argument marker follow, so it cannot split the name early.Returning the canonical id also breaks the feedback loop that causes the bug — the client now receives
functions.<name>:<n>, echoes that back in the history, and the model keeps emitting the form it was trained on.Test Plan
Manual Testing
Hardware: two Mac Studio M3 Ultra 512GB, two-node MLX ring, serving
moonshotai/Kimi-K2.7-Code.functions_Write_3→functions.Write:3.gh pr create; previously failed on the second call, now completes.Automated Testing
test_kimi_tool_parser.pycovers the cases listed under Changes.test_parse_tool_calls.pyunaffected; behaviour on parse failure is unchanged in this PR.Not in this PR
Part 2 of #2281 — having
parse_tool_callsdegrade to plain text on a parse failure instead of emittingfinish_reason="error"— is a deliberate behaviour change (the current behaviour is asserted intest_failed_parse_yields_text), so it is kept out of this PR for discussion on the issue first.