Guard Qwen3.5+ chat-template failure modes; stop deriving the low reasoning tier - #229
Merged
Merged
Conversation
…easoning tier
Ternary-Bonsai-2-27B's KNOWN_ISSUES page documents chat-template failures that
also apply to any Qwen3.8 build (the template is shared). Checking Godspeed
against them found three exposures.
- Resumed sessions appended a mid-conversation `system` message
(Conversation.add_system_message, called from the TUI resume bootstrap).
Qwen3.5+ templates raise "System message must be at the beginning" -> HTTP 500.
It now merges into the single leading system prompt; once messages exist it
adds a labelled `user` note instead.
- Empty / None tool-call arguments were stored as-is in history, and the
templates reject them (HTTP 500/400). add_assistant_message now stores "{}".
- _parse_tool_call called json.loads("") on an empty argument string and dropped
a valid zero-argument call as malformed. Empty/None now parse as no arguments.
Malformed JSON is still rejected.
- The derived reasoning-effort tiers picked "low" for thinking_budget <= 2048 and
for the "minimal" alias, but Prism ML measured "low" using about as many
reasoning tokens as the default xhigh on Qwen3.8. Derived tiers now pick
"medium" (budget <= 8192) or "xhigh"; an explicit `reasoning_effort: low` is
still honoured. Follows up #228.
Tests: tests/test_qwen38_template_safety.py (new), and the two resume tests that
asserted the old two-system-message behavior now pin the fixed behavior.
🤖 Godspeed Review
SecurityNo secrets detected in changed files. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The Ternary-Bonsai-2-27B
KNOWN_ISSUES.md(updated 2026-09-24) documents Qwen3.8 chat-template failures. The template is shared by every Qwen3.8 build, so I checked Godspeed against each item. Three real exposures, plus a follow-up to #228.Changes
systemmessage after the first message; Qwen3.5+ templates raiseSystem message must be at the beginning-> HTTP 500Conversation.add_system_message(only caller: TUI resume bootstrap when only a summary is available)usernote.Nonetool-callargumentsstored in history; templates reject them (HTTP 500/400)Conversation.add_assistant_message"{}". Non-empty (even malformed) strings untouched; caller dicts not mutated.""is dropped as malformed_parse_tool_call(json.loads(""))Noneparse as no arguments. Malformed JSON still rejected.low(budget <= 2048,minimalalias), butlowdoes not shorten Qwen3.8 reasoning (Prism ML measured aboutxhightoken use)LLMClient._qwen_template_kwargsmedium, otherwisexhigh;minimal->medium. An explicitreasoning_effort: lowis still honoured. Follow-up to #228.Docs (
GODSPEED_ARCHITECTURE.md) andCHANGELOG.mdupdated.Verification
ruff check .,ruff format --check .,mypy src/(CI flags),bandit(CI flags): clean.test_qwen38_template_safety,test_session_resume,test_thinking,test_agent_loop,test_llm_client,test_context/,test_tui_commands). Full suite: see CI.tests/test_qwen38_template_safety.py. Two existing resume tests asserted the old behavior (roles == ["system", "system"]) and now pin the fixed behavior.Not verified
chat_template.jinja(which raises on a non-first system message) and from Prism ML's KNOWN_ISSUES; I have not yet reproduced it against llama-server. That is part of the next measured run.add_assistant_message/_parse_tool_call, which only touch empty arguments.