Skip to content

fix: parse list-of-parts content in ChatMessage.from_openai_dict_format - #12446

Open
Vedant-Agarwal wants to merge 1 commit into
deepset-ai:mainfrom
Vedant-Agarwal:fix/chat-message-openai-list-content
Open

fix: parse list-of-parts content in ChatMessage.from_openai_dict_format#12446
Vedant-Agarwal wants to merge 1 commit into
deepset-ai:mainfrom
Vedant-Agarwal:fix/chat-message-openai-list-content

Conversation

@Vedant-Agarwal

Copy link
Copy Markdown
Contributor

Related Issues

Proposed Changes:

ChatMessage.from_openai_dict_format passed list-of-parts content verbatim into a single TextContent for user, system, and developer messages (only the tool branch handled lists). This made msg.text return a list (violating its str | None contract) and made to_dict() / to_openai_dict_format() emit invalid content:

from haystack.dataclasses import ChatMessage

b64 = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+ip1sAAAAASUVORK5CYII="
msg = ChatMessage.from_openai_dict_format({
    "role": "user",
    "content": [
        {"type": "text", "text": "What is in this image?"},
        {"type": "image_url", "image_url": {"url": f"data:image/png;base64,{b64}"}},
    ],
})
type(msg.text)  # was <class 'list'>, now str
msg.images      # was [], now [ImageContent(...)]

This PR makes list parsing the exact inverse of to_openai_dict_format:

  • user messages: text parts become TextContent; image_url parts with base64 data URLs become ImageContent (MIME type and base64 payload split from the data URL, detail preserved); file parts with inline file_data become FileContent (filename preserved). The message is built via ChatMessage.from_user(content_parts=...).
  • system/developer messages: lists of text parts are accepted (joined with a newline, since from_system stores a single text); non-text parts raise ValueError.
  • unsupported parts — non-data image URLs, files referenced by file_id, unknown part types — raise ValueError instead of silently corrupting the message.

String content behaves exactly as before.

How did you test it?

Added unit tests in test/dataclasses/test_chat_message.py covering: list of text parts, text + image data URL part (with detail), text + file part, unsupported parts (http image URL, file_id file, unknown part type), system message with text parts / with non-text parts, and a full multimodal round-trip from_openai_dict_format(msg.to_openai_dict_format()) == msg.

pytest test/dataclasses/test_chat_message.py  ->  103 passed
ruff check / ruff format --check              ->  clean
mypy haystack/dataclasses/chat_message.py     ->  no issues

Notes for the reviewer

  • The data-URL parser and part converter are small private helpers (_parse_openai_data_url, _from_openai_content_parts) next to _validate_openai_message.
  • Joining multiple system text parts with "\n" is a judgment call — happy to change to rejecting multiple parts if you prefer.

Checklist

  • I have read the contributors guidelines and the code of conduct.
  • I have updated the related issue with new insights and changes.
  • I have added unit tests and updated the docstrings.
  • I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
  • I have documented my code.
  • I have added a release note file, following the contributors guidelines.
  • I have run pre-commit hooks and fixed any issue.

(AI-assisted: implemented with Claude Code.)

Previously, user/system/developer messages whose content was a list of
OpenAI content parts had the list wrapped verbatim in a single
TextContent. As a result, msg.text returned a list (violating its
str | None contract) and to_dict() / to_openai_dict_format() emitted
invalid content.

Now user messages convert text parts to TextContent, image_url parts
with base64 data URLs to ImageContent (splitting MIME type and base64
payload, preserving detail), and file parts with inline file_data to
FileContent (preserving filename). System and developer messages accept
lists of text parts. Unsupported parts (non-data image URLs, file_id
references, unknown part types) raise ValueError.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Vedant-Agarwal
Vedant-Agarwal requested a review from a team as a code owner August 24, 2026 08:08
@Vedant-Agarwal
Vedant-Agarwal requested review from julian-risch and removed request for a team August 24, 2026 08:08
@vercel

vercel Bot commented Aug 24, 2026

Copy link
Copy Markdown

@Vedant-Agarwal is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@github-actions github-actions Bot added topic:tests type:documentation Improvements on the docs labels Aug 24, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  haystack/dataclasses
  chat_message.py
Project Total  

This report was generated by python-coverage-comment-action

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

topic:tests type:documentation Improvements on the docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ChatMessage.from_openai_dict_format wraps list-of-parts content in TextContent, breaking msg.text contract

1 participant