Python: [Bug]: Content.from_uri regression - Service failed to complete the prompt: Error code: 400

[savvasp-123]

Description

Using the new FoundryChatClient with agent framework version 1.0.0, sending image data via base64 no longer works.

Code sample:

# Note: "agent" is made using FoundryClient.as_agent and some default options for structured outputs
 
with open("my_image.jpg", "rb") as f:
    image_bytes = f.read()    
    image_b64 = base64.b64encode(image_bytes).decode("utf-8")
    c = [Content.from_uri(uri=f"data:image/jpeg;base64,{image_b64}")]
    m = Message(role="user", contents=c)
    result = await agent.run(m)

Error:
{'error': {'code': 'invalid_payload', 'message': 'The provided data does not match the expected schema', 'param': '/', 'type': 'invalid_request_error', 'details': [], 'additionalInfo': {'request_id': '5e748b2875102c59bd87bc285f8d6b4a'}}}

Package Versions

agent-framework-core:1.0.0

Python Version

3.13

Additional Context

No response

[ManniArora]

+1, running into the same issue (using AzureAIProjectAgentProvider)

[Serjbory]

Hello @ManniArora , @savvasp-123

I investigated this bug and what I've found:

Root Cause

In _chat_client.py (the OpenAI Responses API client), the _prepare_content_for_openai
method unconditionally includes "file_id": None in the serialized input_image payload:

return {
    "type": "input_image",
    "image_url": content.uri,
    "detail": ...,
    "file_id": content.additional_properties.get("file_id", None)   # ← always present
              if content.additional_properties
              else None,
}

This produces the following JSON on the wire:

{
  "type": "input_image",
  "image_url": "data:image/jpeg;base64,/9j/4AAQ...",
  "detail": "auto",
  "file_id": null          ← rejected by the API
}

Why the API rejects it

The agent framework builds raw Python dicts, not Pydantic ResponseInputImage models.
The OpenAI Responses API input_image schema defines file_id as Optional[str] — meaning
the field can be absent, but when present it must be a string. Sending null
violates that constraint, producing the 400 error.

This is standard JSON Schema behavior: "type": "string" does not accept null unless
the schema explicitly allows it (e.g., "type": ["string", "null"]).

Fix

Only include file_id in the dict when it actually has a value:

# AFTER (fixed)
image_item: dict[str, Any] = {
    "type": "input_image",
    "image_url": content.uri,
    "detail": ...,
}
file_id = (
    content.additional_properties.get("file_id")
    if content.additional_properties
    else None
)
if file_id is not None:
    image_item["file_id"] = file_id
return image_item

If this solution makes sense - I would be glad to make a PR to fix it

[ManniArora]

Yes, the solution makes sense!

GitHub Copilot also arrived at this root cause for me :P

OK so I've confirmed that file_id: null is being included. But I need to verify this is actually the cause of the 400 error. Let me check if the newer version of the SDK fixes this.

Actually, let me take a step back. The openai python SDK passes input as a parameter to responses.create(). The openai SDK would serialize this into JSON, and according to the openai SDK, file_id: Optional[str] = None should be serializable. The openai SDK typically strips None values from the request body when serializing.

Let me check how the openai SDK handles None values in the responses.create parameters.

The openai v2 SDK uses Pydantic models with exclude_none=True during serialization, but when raw dicts are passed in the input parameter, they bypass this filtering and get serialized as-is, converting None to null in JSON. This might be what's causing the Azure API to reject the request.

The Azure Responses API with the preview version likely has stricter schema validation that doesn't accept null values for fields like file_id, whereas the OpenAI API itself might be more lenient. This is probably a known issue with the beta SDK version being used, and upgrading to a newer release or manually filtering out None values from the input dicts would resolve it.

[chetantoshniwal]

fixed as part of #4944