fix(memory): add dedicated SGLang and vLLM thinking transports - #312
fix(memory): add dedicated SGLang and vLLM thinking transports#312LeslieWylie wants to merge 1 commit into
Conversation
|
Thanks for working on this! I suggest adding chat_template_kwargs later as part of dedicated SGLang/vLLM providers. Adding it to the generic OpenAI-compatible path now could affect existing cloud-provider calls and introduce unnecessary risk. |
Route Qwen enable_thinking through chat_template_kwargs only for explicit sglang and vllm providers. Keep the generic OpenAI-compatible request body unchanged so cloud-provider calls retain their existing contract.
415f802 to
5410110
Compare
|
Thanks — I reworked the PR around that boundary. |
|
Superseded by clean-history replacement #348. An amend from a shallow checkout accidentally removed this branch's base ancestry, so GitHub closed the PR and would not reopen it after the ancestry was repaired. The replacement keeps the maintainer-requested provider isolation and the passing 21-test validation. |
Summary
Add dedicated
sglangandvllmmemory LLM providers for the Qwenchat_template_kwargs.enable_thinkingtransport.The generic
openai_compatibleprovider keeps its existing top-level request shape, so cloud-provider calls are unaffected.Changes
sglangandvllmas explicit LLM provider valuesenable_thinkingintochat_template_kwargsonly for those dedicated providerschat_template_kwargs, including an explicitenable_thinkingoverrideenable_thinkingbehavior unchanged for DashScope and generic OpenAI-compatible endpointsValidation
vitest run Memory/tests/llm-thinking.test.ts— 21 tests passedenable_thinking=falsewas ignored whilechat_template_kwargs.enable_thinking=falseproduced zero reasoning tokens and valid JSON