Skip to content

fix(llm): double max_tokens on finish_reason=length before retrying JSON - #2321

Open
smoryan wants to merge 2 commits into
MemTensor:mainfrom
smoryan:fix/llm-length-retry-budget
Open

fix(llm): double max_tokens on finish_reason=length before retrying JSON#2321
smoryan wants to merge 2 commits into
MemTensor:mainfrom
smoryan:fix/llm-length-retry-budget

fix(llm): address review — guarantee the truncation retry one attempt…

bd9bd17
Select commit
Loading
Failed to load commit list.

Workflow runs completed with no jobs