-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Python: [Bug]: Agent behaves differently with input when sent as string vs array - Difference is consistent #5112
Description
Description
In Code Sample, I have shown how I create chat_client and then Agent.
I host it as a Foundry Hosted Agent in Azure Foundry. So I am using Hosting Adapter Library:
from azure.ai.agentserver.agentframework import from_agent_framework
from_agent_framework(agent).run()
Then I send it same requests as below to the /responses endpoint:
INPUT A - as a plain string
"input": "Which Google phones are allowed in University? What is the color and Capacity? Answer using tools that you have available."
INPUT B - as a array, but same text
"input": [
{
"role": "user",
"content": "Which Google phones are allowed in University? What is the color and Capacity? Answer using tools that you have available."
}
],
When sent as INPUT A, consistently I get Google Phone, in logs I can see mcp tool called which returns apple, samsung, google, but response is only the Google Phone. This is good.
When sent as INPUT B (which is desirable way I want to use), it always return Google, Apple and Samsung, so ignores the user query.
This is consistent, so if I call 10 times, Input A always work, Input B always give more phone models and not meets user query. Every time in both scenario the mcp_tool is invoked which returns same results whichever is the input.
I test it on local machine, or even post deployment to Azure, so deployment location is not a problem.
Is there some problem here? As I have a Super Agent which in future will have around 6 MCP Tools, so I do not want to hardcode any special phone related system, developer prompt. In responses api, in array, I sent developer, system, some examples in prompt, some continuing conversation, but Input B just always give me more phones and Input A always is precise.
Code Sample
To create Chat Client
self.chat_client = AzureOpenAIChatClient(
credential=...,
endpoint=self.config.azure_openai_endpoint,
deployment_name=self.config.azure_openai_chat_deployment_name
)
To create Agent
BASE_INSTRUCTIONS = "When using tools, treat tool output as raw data. \nYou must analyze it and return only the parts that satisfy the user’s request. \nNever return items that do not match the user’s criteria. \nIf the tool returns mixed or irrelevant results, filter them out before responding.\n"
AGENT_NAME = "SuperAgent"
async with mcp_tool:
agent = self.chat_client.as_agent(
name=self.AGENT_NAME,
instructions=instructions,
tools=[mcp_tool]
)Error Messages / Stack Traces
Package Versions
azure-ai-agentserver-agentframework==1.0.0b17 agent-framework==1.0.0rc3
Python Version
3.12
Additional Context
No response
Metadata
Metadata
Assignees
Labels
Type
Projects
Status