Drop the duplicated trailing user turn in Foundation Models sessions - #217
Merged
Merged
Conversation
LanguageModelSession appends the prompt entry to its transcript before invoking the model. The Foundation Models adapter then passes that transcript to FoundationModels.LanguageModelSession and also sends the prompt through respond/streamResponse, which appends it again, so every FM-backed turn reaches the model with the final user message duplicated. The MLX adapter shows the intended contract: it renders the chat from session.transcript alone and treats the prompt parameter as a fallback. FM's API requires passing the prompt separately, so the fix on this adapter is to drop a trailing transcript prompt that matches the outgoing prompt before converting. Verified by decoding the rendered prompt of a custom LanguageModel backend on OS 27, where the doubled turn is directly observable; the same transcript construction applies to SystemLanguageModel on OS 26. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
🟡 Changes recommended
The new helper can drop a trailing transcript prompt based on text-only comparison and may inadvertently remove structured prompt context that the outgoing Prompt parameter cannot represent.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR fixes prompt duplication when using the Foundation Models–backed SystemLanguageModel by ensuring the outgoing prompt is not effectively included twice (once via LanguageModelSession.transcript, and again via the FM respond/streamResponse APIs).
Changes:
- Drops a trailing transcript prompt entry when it matches the outgoing
promptbefore converting the transcript toFoundationModels.Transcript. - Applies the fix consistently to both
respondandstreamResponsepaths inSystemLanguageModel.
File summaries
| File | Description |
|---|---|
| Sources/AnyLanguageModel/Models/SystemLanguageModel.swift | Avoids duplicating the final user prompt in FM session construction by trimming a matching trailing transcript prompt. |
Review details
- Files reviewed: 1/1 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+370
to
+383
| @available(macOS 26.0, iOS 26.0, watchOS 26.0, tvOS 26.0, visionOS 26.0, *) | ||
| func fmTranscriptDroppingDuplicatePrompt(_ transcript: Transcript, prompt: Prompt) -> Transcript { | ||
| guard let lastEntry = transcript.last, case .prompt(let lastPrompt) = lastEntry else { | ||
| return transcript | ||
| } | ||
| let lastText = lastPrompt.segments.compactMap { segment -> String? in | ||
| if case .text(let textSegment) = segment { return textSegment.content } | ||
| return nil | ||
| }.joined() | ||
| guard lastText == prompt.description else { | ||
| return transcript | ||
| } | ||
| return Transcript(entries: transcript.dropLast()) | ||
| } |
Collaborator
|
Nice catch. I pushed a small follow-up that fixes the formatting for the linter and limits the drop to prompt entries whose segments are all text, so a structured segment can never go missing. Merging now. Thanks, @james-333i! |
james-333i
added a commit
to james-333i/AnyLanguageModel
that referenced
this pull request
Sep 4, 2026
Brings in the merged huggingface#195, huggingface#205, huggingface#212 and huggingface#217 along with the follow-ups applied on merge (all-text prompt drop, boolean and number probe items). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
LanguageModelSessionappends the prompt entry to its transcript before invoking the model. The Foundation Models adapter then passes that transcript toFoundationModels.LanguageModelSessionand also sends the prompt throughrespondandstreamResponse, which appends it again. Every FM-backed turn therefore reaches the model with the final user message duplicated.The MLX adapter shows the intended contract. It renders the chat from
session.transcriptalone and treats the prompt parameter as afallbackPrompt. FM's API requires passing the prompt separately, so the fix on this adapter is to drop a trailing transcript prompt that matches the outgoing prompt before conversion.This is directly observable on OS 27 by decoding the rendered prompt of a custom
LanguageModelbackend, where the user turn appears twice. The same transcript construction applies toSystemLanguageModelon OS 26, where it wastes prompt tokens on every turn.