Skip to content

fix: time to first token for generations that stream no text - #43

Merged
hassiebp merged 2 commits into
mainfrom
fix/ttft-on-first-output-delta
Sep 25, 2026
Merged

hassiebp merged 2 commits into
mainfrom
fix/ttft-on-first-output-delta

Conversation

@milanagm

@milanagm milanagm commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Fixes LFE-16664

completionStartTime was only set when a message_update carried a non-empty text part, so a generation that never streams prose reported no TTFT at all — a model that calls a tool straight away, or one that reasons before answering. The gate now accepts the first delta of any kind: text, reasoning or tool call, since each is a streamed completion token.

Measured in a real pi project (90 days, 49 generations): 13 of 49 (27 %) carried no TTFT, and every one was a tool-call-only completion. The gate has been in place since the plugin's first commit.

Verification

  • Two end-to-end tests, one per trigger, driven by the new mock scenarios [tool-only] (tool call, no text, no reasoning) and [think-only] (reasoning then tool call, no text). Both fail against the previous gate. Each also asserts its own premise — no text part, no reasoning part — so it cannot keep passing while no longer covering the bug.
  • 130/130 tests, tsc --noEmit clean. main is 128; the two new tests are the whole delta.

Relation to #39

Supersedes the TTFT half of #39, which extended the same gate to reasoning only and explicitly left tool-call-only generations broken ("There is no token to time there") — i.e. 100 % of the cases actually observed. If this lands, #39 narrows to the inline-<think> split.

@milanagm
milanagm requested a review from hassiebp September 23, 2026 14:45
@hassiebp
hassiebp merged commit 5f3c083 into main Sep 25, 2026
10 of 12 checks passed
@hassiebp
hassiebp deleted the fix/ttft-on-first-output-delta branch September 25, 2026 08:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants