Skip to content

Python: [Bug]: Agent behaves differently with input when sent as string vs array - Difference is consistent #5112

@ashishmundra

Description

@ashishmundra

Description

In Code Sample, I have shown how I create chat_client and then Agent.

I host it as a Foundry Hosted Agent in Azure Foundry. So I am using Hosting Adapter Library:
from azure.ai.agentserver.agentframework import from_agent_framework
from_agent_framework(agent).run()

Then I send it same requests as below to the /responses endpoint:

INPUT A - as a plain string
"input": "Which Google phones are allowed in University? What is the color and Capacity? Answer using tools that you have available."

INPUT B - as a array, but same text
"input": [
{
"role": "user",
"content": "Which Google phones are allowed in University? What is the color and Capacity? Answer using tools that you have available."
}
],

When sent as INPUT A, consistently I get Google Phone, in logs I can see mcp tool called which returns apple, samsung, google, but response is only the Google Phone. This is good.

When sent as INPUT B (which is desirable way I want to use), it always return Google, Apple and Samsung, so ignores the user query.

This is consistent, so if I call 10 times, Input A always work, Input B always give more phone models and not meets user query. Every time in both scenario the mcp_tool is invoked which returns same results whichever is the input.

I test it on local machine, or even post deployment to Azure, so deployment location is not a problem.

Is there some problem here? As I have a Super Agent which in future will have around 6 MCP Tools, so I do not want to hardcode any special phone related system, developer prompt. In responses api, in array, I sent developer, system, some examples in prompt, some continuing conversation, but Input B just always give me more phones and Input A always is precise.

Code Sample

To create Chat Client


        self.chat_client = AzureOpenAIChatClient(
            credential=...,
            endpoint=self.config.azure_openai_endpoint,
            deployment_name=self.config.azure_openai_chat_deployment_name
        )



To create Agent


    BASE_INSTRUCTIONS = "When using tools, treat tool output as raw data.  \nYou must analyze it and return only the parts that satisfy the user’s request.  \nNever return items that do not match the user’s criteria.  \nIf the tool returns mixed or irrelevant results, filter them out before responding.\n"

    AGENT_NAME = "SuperAgent"

        async with mcp_tool:
            agent = self.chat_client.as_agent(
                name=self.AGENT_NAME,
                instructions=instructions,
                tools=[mcp_tool]
            )

Error Messages / Stack Traces

Package Versions

azure-ai-agentserver-agentframework==1.0.0b17 agent-framework==1.0.0rc3

Python Version

3.12

Additional Context

No response

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

Projects

Status

No status

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions