diff --git a/docs/specs/004-python-function-calling-loop.md b/docs/specs/004-python-function-calling-loop.md index 81019bb5bbe..7639183cc91 100644 --- a/docs/specs/004-python-function-calling-loop.md +++ b/docs/specs/004-python-function-calling-loop.md @@ -369,6 +369,10 @@ that manually replay messages own the equivalent rule: do not resend an approval already executing in a worker thread cannot be interrupted and may complete its side effects — its result is discarded either way and never reaches the transcript, the model, or history. Middleware must not catch `MiddlewareFailure` — swallowing it converts a fail-closed abort back into a running, possibly unguarded loop. +- `Content.exception` is host-internal diagnostic state. Default `Content.to_dict()` and nested response serialization replace it with a fixed non-sensitive failure marker, + while the original field remains directly available to trusted local code. Remote protocol serializers use the + marker only for status and use the channel-visible `result` or `items` for output text. `include_detailed_errors=False` keeps the channel-visible + result generic; enabling it explicitly may place diagnostic text in the result for that configured channel. - Parallel calls retain model order in the returned transcript. - `call_id` remains the provider/service correlation id; a locally actionable `function_call` also carries a stable Agent Framework occurrence identity in `Content.id`. @@ -582,6 +586,7 @@ that manually replay messages own the equivalent rule: do not resend an approval |---|---|---| | Rejected execution | Rejection is a normal terminal result, not an exception to the caller. | `test_unapproved_tool_execution_raises_exception` | | Approved tool exception | Generic and detailed error modes preserve one result and one execution. | `test_approved_function_call_with_error_without_detailed_errors`, `test_approved_function_call_with_error_with_detailed_errors` | +| Tool exception diagnostics | Internal diagnostics remain available to trusted local code, serialization preserves only a fixed failure marker, and explicit detailed-error configuration affects only the channel-visible result. | `packages/core/tests/core/test_types.py::test_function_result_exception_is_internal_by_default`, `packages/core/tests/core/test_function_invocation_logic.py::test_function_invocation_config_include_detailed_errors_false`, `test_function_invocation_config_include_detailed_errors_true`, `test_streaming_function_invocation_config_include_detailed_errors_false`, `test_streaming_function_invocation_config_include_detailed_errors_true` | | Approved validation error | Validation failure returns one result without invoking the function body. | `test_approved_function_call_with_validation_error` | | Approved success | Successful approved execution returns one result. | `test_approved_function_call_successful_execution` | | Consecutive error cap | Error threshold stops repeated failures, submits collected results, and makes only the required final no-tool model call. | `test_function_invocation_config_max_consecutive_errors`, `test_streaming_function_invocation_config_max_consecutive_errors`, `test_approval_resume_error_limit_forces_final_no_tool_response` | @@ -611,6 +616,7 @@ that manually replay messages own the equivalent rule: do not resend an approval | OpenAI end-to-end hosted approval | Hosted request parses, response sends, and continuation completes. | `test_end_to_end_mcp_approval_flow` | | Stored function call/result | Service-side storage drops server-issued calls but keeps new outputs. | `test_prepare_options_with_conversation_id_strips_server_issued_items`, `test_prepare_messages_for_openai_full_conversation_with_reasoning` | | Stateless reasoning replay | Replay reconstructs reasoning, call, and result together; missing required reasoning fails before the request. | `test_tool_loop_store_false_replays_encrypted_reasoning_group`, `test_stateless_request_rejects_non_replayable_reasoning_bound_mcp_output`, `test_prepare_messages_for_openai_full_conversation_with_reasoning` | +| Remote tool error serialization | Foundry Responses, Responses hosting, and AG-UI preserve channel-visible results without host-internal diagnostics. A2A and MCP hosting omit unsupported intermediate function results entirely, including their diagnostics. | `packages/foundry_hosting/tests/test_responses.py::TestNonStreaming::test_function_result_omits_internal_exception`, `TestStreaming::test_function_result_omits_internal_exception`, `packages/hosting-responses/tests/hosting_responses/test_parsing.py::TestResponsesRunHelpers::test_responses_from_run_omits_internal_function_exception`, `test_responses_from_streaming_run_omits_internal_function_exception`, `packages/ag-ui/tests/ag_ui/test_run_common.py::TestEmitToolResult::test_tool_result_does_not_emit_internal_exception`, `packages/hosting-a2a/tests/hosting_a2a/test_conversion.py::test_a2a_from_run_omits_unsupported_content`, `packages/hosting-mcp/tests/hosting_mcp/test_conversion.py::test_mcp_from_run_omits_content_not_supported_in_tool_results` | | Foundry encrypted reasoning opt-in | Foundry clients omit `reasoning.encrypted_content` by default and preserve an explicit caller opt-in. | `packages/foundry/tests/foundry/test_foundry_chat_client.py::test_get_response_does_not_request_encrypted_reasoning_by_default`, `test_get_response_preserves_explicit_encrypted_reasoning_opt_in`, `packages/foundry/tests/foundry/test_foundry_agent.py::test_foundry_agent_basic_call_does_not_request_unsupported_encrypted_reasoning`, `test_foundry_agent_preserves_caller_requested_encrypted_reasoning`, `packages/foundry_hosting/tests/test_responses_int.py::TestReasoningHostedMcpReplay::test_second_turn_replays_mcp_call_with_encrypted_reasoning` | | Opaque reasoning signature replay | Provider-specific opaque reasoning metadata is captured and restored on reconstructed calls. | `packages/gemini/tests/test_gemini_client.py::test_function_call_part_captures_thought_signature_as_reasoning_content`, `test_reconstructed_function_call_replays_thought_signature_from_reasoning_content` | | Chat Completions approval wrappers | Framework approval wrappers are not sent as chat messages. | `packages/openai/tests/openai/test_openai_chat_completion_client.py` approval serialization tests | diff --git a/python/packages/ag-ui/tests/ag_ui/test_run_common.py b/python/packages/ag-ui/tests/ag_ui/test_run_common.py index 72ab8f8f98a..ba921638299 100644 --- a/python/packages/ag-ui/tests/ag_ui/test_run_common.py +++ b/python/packages/ag-ui/tests/ag_ui/test_run_common.py @@ -253,6 +253,27 @@ def test_tool_result_closes_open_text_message(self): assert flow.message_id is None assert flow.accumulated_text == "" + def test_tool_result_does_not_emit_internal_exception(self): + """AG-UI events and snapshots contain only the channel-visible result.""" + diagnostic = "test-token-value at /srv/private/tool.py" + content = Content.from_function_result( + call_id="call_1", + result="Error: Function failed.", + exception=diagnostic, + ) + flow = FlowState() + + events = _emit_tool_result(content, flow) + payload = json.dumps( + { + "events": [event.model_dump(mode="json", by_alias=True) for event in events], + "snapshot": flow.tool_results, + } + ) + + assert "Error: Function failed." in payload + assert diagnostic not in payload + class TestStateUpdateHelper: """Tests for the public ``state_update`` helper.""" diff --git a/python/packages/anthropic/tests/test_anthropic_client.py b/python/packages/anthropic/tests/test_anthropic_client.py index c671076505f..36a65228abd 100644 --- a/python/packages/anthropic/tests/test_anthropic_client.py +++ b/python/packages/anthropic/tests/test_anthropic_client.py @@ -374,6 +374,31 @@ def test_prepare_message_for_anthropic_function_result( assert result["content"][0]["is_error"] is False +def test_prepare_message_for_anthropic_preserves_redacted_error_status( + mock_anthropic_client: MagicMock, +) -> None: + """Persisted tool failures retain error status without exposing diagnostics.""" + client = create_test_anthropic_client(mock_anthropic_client) + diagnostic = "test-token-value at /srv/private/tool.py" + failed_result = Content.from_function_result( + call_id="call_123", + result="Error: Function failed.", + exception=diagnostic, + ) + restored_result = Content.from_dict(failed_result.to_dict()) + message = Message(role="tool", contents=[restored_result]) + + result = client._prepare_message_for_anthropic(message) + + tool_result = result["content"][0] + assert tool_result["type"] == "tool_result" + assert tool_result["is_error"] is True + assert diagnostic not in str(tool_result) + tool_content = tool_result["content"] + assert isinstance(tool_content, list) + assert tool_content[0]["text"] == "Error: Function failed." + + def test_prepare_message_for_anthropic_function_result_with_data_image( mock_anthropic_client: MagicMock, ) -> None: diff --git a/python/packages/bedrock/agent_framework_bedrock/_chat_client.py b/python/packages/bedrock/agent_framework_bedrock/_chat_client.py index d719027db0a..6d2d222d1dd 100644 --- a/python/packages/bedrock/agent_framework_bedrock/_chat_client.py +++ b/python/packages/bedrock/agent_framework_bedrock/_chat_client.py @@ -572,24 +572,13 @@ def _convert_content_to_bedrock_block(self, content: Content) -> dict[str, Any] tool_result_blocks = self._convert_tool_result_to_blocks(tool_result_text) else: tool_result_blocks = self._convert_tool_result_to_blocks(content.result) - tool_result_block = { + return { "toolResult": { "toolUseId": content.call_id, "content": tool_result_blocks, - "status": "error" if content.exception else "success", + "status": "error" if content.exception is not None else "success", } } - if content.exception: - tool_result = tool_result_block["toolResult"] - existing_content = tool_result.get("content") - content_list: list[dict[str, Any]] - if isinstance(existing_content, list): - content_list = existing_content - else: - content_list = [] - tool_result["content"] = content_list - content_list.append({"text": str(content.exception)}) - return tool_result_block case _: # Bedrock does not support other content types at this time pass diff --git a/python/packages/bedrock/tests/test_bedrock_client.py b/python/packages/bedrock/tests/test_bedrock_client.py index 41e277d75f3..ee17483e84d 100644 --- a/python/packages/bedrock/tests/test_bedrock_client.py +++ b/python/packages/bedrock/tests/test_bedrock_client.py @@ -448,24 +448,34 @@ def test_align_tool_results_handles_pending_edge_cases() -> None: def test_convert_content_to_bedrock_block_handles_errors_and_missing_items() -> None: """Function result conversion should serialize items, rich content warnings, and fallback results.""" client = _make_client() + diagnostic = "test-token-value at /srv/private/tool.py" rich_result = Content.from_function_result( call_id="call-1", result=[Content.from_text(text="summary"), Content.from_data(data=b"x", media_type="image/png")], - exception="tool failed", + exception=diagnostic, ) + restored_rich_result = Content.from_dict(rich_result.to_dict()) fallback_result = Content.from_function_result(call_id="call-2", result={"answer": 42}) fallback_result.items = None - rich_block = client._convert_content_to_bedrock_block(rich_result) + rich_block = client._convert_content_to_bedrock_block(restored_rich_result) fallback_block = client._convert_content_to_bedrock_block(fallback_result) assert rich_block == { "toolResult": { "toolUseId": "call-1", - "content": [{"text": "summary"}, {"text": "tool failed"}], + "content": [{"text": "summary"}], "status": "error", } } + assert diagnostic not in str(rich_block) + + empty_diagnostic_block = client._convert_content_to_bedrock_block( + Content.from_function_result(call_id="call-empty", result="failed", exception="") + ) + assert empty_diagnostic_block is not None + assert empty_diagnostic_block["toolResult"]["status"] == "error" + assert fallback_block == { "toolResult": { "toolUseId": "call-2", diff --git a/python/packages/core/agent_framework/_compaction.py b/python/packages/core/agent_framework/_compaction.py index b9080d0a814..b1d627b6359 100644 --- a/python/packages/core/agent_framework/_compaction.py +++ b/python/packages/core/agent_framework/_compaction.py @@ -1273,8 +1273,8 @@ def _format_summary_content(content: Content) -> str: result_text = _format_summary_result_items(content.items) if content.items else "" if not result_text: result_text = _tool_result_text(content.result) if content.result is not None else "no result" - if content.exception: - result_text = f"error({content.exception}): {result_text}" + if content.exception is not None: + result_text = f"error: {result_text}" call_id_suffix = f" [call_id={content.call_id}]" if content.call_id else "" return f"function_result: {result_text}{call_id_suffix}" if content.type == "mcp_server_tool_call": @@ -1285,8 +1285,8 @@ def _format_summary_content(content: Content) -> str: return call if content.type == "mcp_server_tool_result": result_text = _tool_result_text(content.output) - if content.exception: - result_text = f"error({content.exception}): {result_text}" + if content.exception is not None: + result_text = f"error: {result_text}" call_id_suffix = f" [call_id={content.call_id}]" if content.call_id else "" return f"mcp_tool_result: {result_text}{call_id_suffix}" if content.type in ("function_approval_request", "function_approval_response"): diff --git a/python/packages/core/agent_framework/_tools.py b/python/packages/core/agent_framework/_tools.py index 7c471bd5d93..a42ca1ef4a2 100644 --- a/python/packages/core/agent_framework/_tools.py +++ b/python/packages/core/agent_framework/_tools.py @@ -1462,7 +1462,8 @@ class FunctionInvocationConfiguration(TypedDict, total=False): - ``additional_tools``: Extra tools available during execution but not advertised to the model in the tool list. - ``include_detailed_errors``: Whether to include exception details in the - function result returned to the model. + function result returned to the model. Exception text may contain sensitive + information regardless of its source, so enable this only for a trusted channel. Note: ``max_iterations``, ``max_function_calls``, and ``max_duration_seconds`` diff --git a/python/packages/core/agent_framework/_types.py b/python/packages/core/agent_framework/_types.py index 7b0efa634e8..66e9a95e530 100644 --- a/python/packages/core/agent_framework/_types.py +++ b/python/packages/core/agent_framework/_types.py @@ -40,6 +40,8 @@ logger = logging.getLogger("agent_framework") +_SERIALIZED_EXCEPTION_MARKER: Final[str] = "FunctionInvocationError" + if TYPE_CHECKING: from pydantic import BaseModel @@ -272,16 +274,24 @@ def _validate_uri(uri: str, media_type: str | None) -> dict[str, Any]: raise ContentError("URI must contain a scheme (e.g., http://, data:, file://)") -def _serialize_value(value: Any, exclude_none: bool) -> Any: +def _serialize_value(value: Any, exclude_none: bool, *, redact_exception: bool = True) -> Any: """Recursively serialize a value for to_dict.""" if value is None: return None if isinstance(value, Content): - return value.to_dict(exclude_none=exclude_none) + return value._to_dict( # pyright: ignore[reportPrivateUsage] + exclude_none=exclude_none, redact_exception=redact_exception + ) if isinstance(value, Sequence) and not isinstance(value, (str, bytes, bytearray)): - return [_serialize_value(item, exclude_none) for item in cast(Iterable[Any], value)] + return [ + _serialize_value(item, exclude_none, redact_exception=redact_exception) + for item in cast(Iterable[Any], value) + ] if isinstance(value, Mapping): - return {k: _serialize_value(v, exclude_none) for k, v in value.items()} # type: ignore[reportUnknownVariableType] + return { + k: _serialize_value(v, exclude_none, redact_exception=redact_exception) + for k, v in cast(Mapping[Any, Any], value).items() + } if hasattr(value, "to_dict"): return value.to_dict() # type: ignore[call-arg] return value @@ -480,6 +490,10 @@ class Content: This class provides a single unified type that handles all content variants. Use the class methods like `Content.from_text()`, `Content.from_data()`, `Content.from_uri()`, etc. to create instances. + + The ``exception`` field is host-internal diagnostic state. Its value may originate from a tool, middleware, + provider, or caller and must always be treated as potentially sensitive. Dictionary serialization replaces it + with a fixed marker that preserves failure state; channel-visible error information belongs in the public result. """ _SHALLOW_COPY_FIELDS: ClassVar[set[str]] = {"raw_representation"} @@ -857,8 +871,8 @@ def from_function_call( Keyword Args: arguments: The arguments for the requested function call. May be a JSON string, a mapping that can be serialized as arguments, or None when no arguments were provided. - exception: Error information associated with the function call, if the provider returned the call in an - error state. + exception: Host-internal diagnostic information when the provider returned the call in an error state. + Treat it as potentially sensitive regardless of its source; serialization replaces it with a marker. informational_only: Whether the function call is present only for transcript fidelity and should not be executed by Agent Framework function invocation. id: Stable Agent Framework identity for this occurrence. When omitted, the function invocation layer @@ -907,7 +921,9 @@ def from_function_result( result: The tool output. Accepts a ``list[Content]`` (the canonical form produced by :meth:`~FunctionTool.parse_result`), a plain ``str``, or any other value (which is stringified). - exception: The exception message if the function call failed. + exception: Host-internal diagnostic information when the function call failed. Treat it as potentially + sensitive regardless of whether it came from a tool, middleware, provider, or caller. Serialization + replaces it with a fixed failure marker; use ``result`` for channel-visible error text. annotations: Optional annotations for the content. additional_properties: Optional additional properties. raw_representation: Optional raw representation from the provider. @@ -1408,7 +1424,21 @@ def to_function_approval_response( ) def to_dict(self, *, exclude_none: bool = True, exclude: set[str] | None = None) -> dict[str, Any]: - """Serialize the content to a dictionary.""" + """Serialize content without host-internal exception diagnostics. + + Exception diagnostics are replaced with a fixed marker regardless of their source because they may contain + sensitive information. The marker preserves failure status across persistence round-trips. + """ + return self._to_dict(exclude_none=exclude_none, exclude=exclude, redact_exception=True) + + def _to_dict( + self, + *, + exclude_none: bool, + exclude: set[str] | None = None, + redact_exception: bool, + ) -> dict[str, Any]: + """Serialize content with explicit control over internal exception redaction.""" fields_to_capture = ( "text", "protected_data", @@ -1456,11 +1486,13 @@ def to_dict(self, *, exclude_none: bool = True, exclude: set[str] | None = None) value = getattr(self, field, None) if field in exclude: continue + if field == "exception" and value is not None and redact_exception: + value = _SERIALIZED_EXCEPTION_MARKER if field == "informational_only" and (self.type != "function_call" or not value): continue if exclude_none and value is None: continue - result[field] = _serialize_value(value, exclude_none) + result[field] = _serialize_value(value, exclude_none, redact_exception=redact_exception) if "annotations" not in exclude and self.annotations is not None: result["annotations"] = [dict(annotation) for annotation in self.annotations] @@ -1471,7 +1503,10 @@ def __eq__(self, other: object) -> bool: """Check if two Content instances are equal by comparing their dict representations.""" if not isinstance(other, Content): return False - return self.to_dict(exclude_none=False) == other.to_dict(exclude_none=False) + return self._to_dict(exclude_none=False, redact_exception=False) == other._to_dict( + exclude_none=False, + redact_exception=False, + ) def __str__(self) -> str: """Return a string representation of the Content.""" diff --git a/python/packages/core/tests/core/test_compaction.py b/python/packages/core/tests/core/test_compaction.py index 2cfadff942b..bc62b0b6d90 100644 --- a/python/packages/core/tests/core/test_compaction.py +++ b/python/packages/core/tests/core/test_compaction.py @@ -1016,19 +1016,28 @@ def test_format_summary_message_includes_function_call_details() -> None: assert "[call_id=call_1]" in rendered -def test_format_summary_message_includes_function_result_and_exception() -> None: +def test_format_summary_message_redacts_function_result_exception() -> None: + diagnostic = "test-token-value at /srv/private/tool.py" message = Message( role="tool", - contents=[Content.from_function_result(call_id="call_1", result="42", exception="ValueError")], + contents=[Content.from_function_result(call_id="call_1", result="42", exception=diagnostic)], ) rendered = _format_summary_message(2, message) assert "function_result" in rendered assert "42" in rendered - assert "error(ValueError)" in rendered + assert "error" in rendered + assert diagnostic not in rendered assert "[call_id=call_1]" in rendered + empty_diagnostic_message = Message( + role="tool", + contents=[Content.from_function_result(call_id="call_2", result="failed", exception="")], + ) + empty_diagnostic_rendered = _format_summary_message(3, empty_diagnostic_message) + assert "error: failed" in empty_diagnostic_rendered + def test_format_summary_message_renders_function_result_without_call_id() -> None: message = Message( diff --git a/python/packages/core/tests/core/test_function_invocation_logic.py b/python/packages/core/tests/core/test_function_invocation_logic.py index 7e688a2e13c..a9689453e9b 100644 --- a/python/packages/core/tests/core/test_function_invocation_logic.py +++ b/python/packages/core/tests/core/test_function_invocation_logic.py @@ -37,6 +37,7 @@ _EXPECTED_FUNCTION_INVOCATION_LIMIT_FALLBACK_TEXT = ( "Function invocation limit reached before a final answer could be produced." ) +_PRIVATE_ERROR_DETAIL = "test-token-value at /srv/private/tool.py" def _group_id(message: Message) -> str | None: @@ -2920,7 +2921,7 @@ async def test_function_invocation_config_include_detailed_errors_false(chat_cli @tool(name="error_function", approval_mode="never_require") def error_func(arg1: str) -> str: - raise ValueError("Specific error message that should not appear") + raise ValueError(f"Specific error message that should not appear: {_PRIVATE_ERROR_DETAIL}") chat_client_base.run_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute] ChatResponse( @@ -2949,6 +2950,8 @@ def error_func(arg1: str) -> str: assert error_result.exception is not None assert "Specific error message" not in error_result.result assert "Error:" in error_result.result # Generic error prefix + assert _PRIVATE_ERROR_DETAIL in error_result.exception + assert _PRIVATE_ERROR_DETAIL not in json.dumps(response.to_dict()) async def test_function_invocation_config_include_detailed_errors_true(chat_client_base: SupportsChatGetResponse): @@ -2956,7 +2959,7 @@ async def test_function_invocation_config_include_detailed_errors_true(chat_clie @tool(name="error_function", approval_mode="never_require") def error_func(arg1: str) -> str: - raise ValueError("Specific error message that should appear") + raise ValueError(f"Specific error message that should appear: {_PRIVATE_ERROR_DETAIL}") chat_client_base.run_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute] ChatResponse( @@ -2986,6 +2989,7 @@ def error_func(arg1: str) -> str: assert "Specific error message that should appear" in error_result.result # The error format includes "Function failed. Exception:" prefix assert "Exception:" in error_result.result + assert _PRIVATE_ERROR_DETAIL in json.dumps(response.to_dict()) async def test_function_invocation_config_validation_max_iterations(): @@ -5249,7 +5253,7 @@ async def test_streaming_function_invocation_config_include_detailed_errors_true @tool(name="error_function", approval_mode="never_require") def error_func(arg1: str) -> str: - raise ValueError("Specific error message that should appear") + raise ValueError(f"Specific error message that should appear: {_PRIVATE_ERROR_DETAIL}") chat_client_base.streaming_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute] [ @@ -5282,6 +5286,7 @@ def error_func(arg1: str) -> str: assert error_result.exception is not None assert "Specific error message that should appear" in error_result.result assert "Exception:" in error_result.result + assert _PRIVATE_ERROR_DETAIL in json.dumps([update.to_dict() for update in updates]) async def test_streaming_function_invocation_config_include_detailed_errors_false( @@ -5291,7 +5296,7 @@ async def test_streaming_function_invocation_config_include_detailed_errors_fals @tool(name="error_function", approval_mode="never_require") def error_func(arg1: str) -> str: - raise ValueError("Specific error message that should not appear") + raise ValueError(f"Specific error message that should not appear: {_PRIVATE_ERROR_DETAIL}") chat_client_base.streaming_responses = [ # type: ignore[attr-defined] # ty: ignore[unresolved-attribute] [ @@ -5324,6 +5329,8 @@ def error_func(arg1: str) -> str: assert error_result.exception is not None assert "Specific error message" not in error_result.result assert "Error:" in error_result.result # Generic error prefix + assert _PRIVATE_ERROR_DETAIL in error_result.exception + assert _PRIVATE_ERROR_DETAIL not in json.dumps([update.to_dict() for update in updates]) async def test_streaming_argument_validation_error_with_detailed_errors(chat_client_base: SupportsChatGetResponse): diff --git a/python/packages/core/tests/core/test_types.py b/python/packages/core/tests/core/test_types.py index f7669221bd3..45085ab77d3 100644 --- a/python/packages/core/tests/core/test_types.py +++ b/python/packages/core/tests/core/test_types.py @@ -2477,6 +2477,49 @@ def test_content_to_dict_exclude_fields() -> None: assert parsed["type"] == "text" +def test_function_result_exception_is_internal_by_default() -> None: + diagnostic = "test-token-value at /srv/private/tool.py" + content = Content.from_function_result( + call_id="call-1", + result="Error: Function failed.", + exception=diagnostic, + ) + + assert content.exception == diagnostic + assert content.to_dict()["exception"] == "FunctionInvocationError" + assert content.to_dict(exclude_none=False)["exception"] == "FunctionInvocationError" + response = AgentResponse(messages=[Message(role="tool", contents=[content])]) + serialized = json.dumps(response.to_dict()) + assert diagnostic not in serialized + assert "FunctionInvocationError" in serialized + + restored = Content.from_dict(content.to_dict()) + assert restored.exception == "FunctionInvocationError" + assert restored != Content.from_function_result( + call_id="call-1", + result="Error: Function failed.", + exception="different diagnostic", + ) + + empty_diagnostic = Content.from_function_result(call_id="call-2", exception="") + assert empty_diagnostic.to_dict()["exception"] == "FunctionInvocationError" + + +def test_content_equality_compares_nested_raw_exception_diagnostics() -> None: + first = Content( + "function_result", + call_id="outer", + items=[Content.from_function_result(call_id="inner", exception="diagnostic-a")], + ) + second = Content( + "function_result", + call_id="outer", + items=[Content.from_function_result(call_id="inner", exception="diagnostic-b")], + ) + + assert first != second + + def test_chat_response_roundtrip_preserves_compaction_annotation_dict() -> None: response = ChatResponse( messages=[ diff --git a/python/packages/foundry_hosting/tests/test_responses.py b/python/packages/foundry_hosting/tests/test_responses.py index 9265346a9f5..b4522d29107 100644 --- a/python/packages/foundry_hosting/tests/test_responses.py +++ b/python/packages/foundry_hosting/tests/test_responses.py @@ -85,6 +85,7 @@ ) _OPENAI_HTTPX = cast(Any, import_module(DefaultAsyncHttpxClient.__mro__[1].__module__.partition(".")[0])) +_PRIVATE_ERROR_DETAIL = "test-token-value at /srv/private/tool.py" def _function_approval_store(request: Content) -> MagicMock: @@ -1634,6 +1635,34 @@ async def test_function_call_and_result(self) -> None: assert "function_call_output" in types assert "message" in types + async def test_function_result_omits_internal_exception(self) -> None: + agent = _make_agent( + response=AgentResponse( + messages=[ + Message( + role="assistant", + contents=[Content.from_function_call("call_1", "get_weather", arguments="{}")], + ), + Message( + role="tool", + contents=[ + Content.from_function_result( + "call_1", + result="Error: Function failed.", + exception=_PRIVATE_ERROR_DETAIL, + ) + ], + ), + ] + ) + ) + + resp = await _post(_make_server(agent), stream=False) + + assert resp.status_code == 200 + assert "Error: Function failed." in resp.text + assert _PRIVATE_ERROR_DETAIL not in resp.text + @pytest.mark.parametrize( ("result", "expected_output"), [ @@ -2041,6 +2070,32 @@ async def test_function_call_streaming(self) -> None: assert len(args_done) == 1 assert args_done[0]["data"]["arguments"] == '{"q": "hello"}' + async def test_function_result_omits_internal_exception(self) -> None: + agent = _make_agent( + stream_updates=[ + AgentResponseUpdate( + role="assistant", + contents=[Content.from_function_call("call_1", "get_weather", arguments="{}")], + ), + AgentResponseUpdate( + role="tool", + contents=[ + Content.from_function_result( + "call_1", + result="Error: Function failed.", + exception=_PRIVATE_ERROR_DETAIL, + ) + ], + ), + ] + ) + + resp = await _post(_make_server(agent), stream=True) + + assert resp.status_code == 200 + assert "Error: Function failed." in resp.text + assert _PRIVATE_ERROR_DETAIL not in resp.text + @pytest.mark.parametrize(("arguments", "expected_count"), [(None, 1), ("", 2)]) async def test_declaration_only_metadata_replay_requires_none_arguments( self, arguments: str | None, expected_count: int diff --git a/python/packages/hosting-a2a/tests/hosting_a2a/test_conversion.py b/python/packages/hosting-a2a/tests/hosting_a2a/test_conversion.py index b8491f2ea79..d30dcc8e67a 100644 --- a/python/packages/hosting-a2a/tests/hosting_a2a/test_conversion.py +++ b/python/packages/hosting-a2a/tests/hosting_a2a/test_conversion.py @@ -165,6 +165,11 @@ def test_a2a_from_run_omits_unsupported_content() -> None: "assistant", [ Content(type="function_call", call_id="call-1", name="get_weather", arguments="{}"), + Content.from_function_result( + call_id="call-1", + result="Error: Function failed.", + exception="test-token-value at /srv/private/tool.py", + ), Content.from_text("hello"), ], ) @@ -172,6 +177,7 @@ def test_a2a_from_run_omits_unsupported_content() -> None: assert len(parts) == 1 assert parts[0].text == "hello" + assert "test-token-value" not in str([MessageToDict(part) for part in parts]) def test_a2a_from_run_omits_user_messages() -> None: diff --git a/python/packages/hosting-mcp/tests/hosting_mcp/test_conversion.py b/python/packages/hosting-mcp/tests/hosting_mcp/test_conversion.py index 0cfcf1f88a6..21de73f7afc 100644 --- a/python/packages/hosting-mcp/tests/hosting_mcp/test_conversion.py +++ b/python/packages/hosting-mcp/tests/hosting_mcp/test_conversion.py @@ -1,5 +1,7 @@ # Copyright (c) Microsoft. All rights reserved. +import json + from agent_framework import AgentResponse, Content, Message from mcp import types from pytest import raises @@ -113,6 +115,11 @@ def test_mcp_from_run_omits_content_not_supported_in_tool_results() -> None: "assistant", [ Content.from_function_call(call_id="call-1", name="get_weather", arguments="{}"), + Content.from_function_result( + call_id="call-1", + result="Error: Function failed.", + exception="test-token-value at /srv/private/tool.py", + ), Content.from_text("hello"), ], ) @@ -120,6 +127,9 @@ def test_mcp_from_run_omits_content_not_supported_in_tool_results() -> None: assert len(blocks) == 1 assert isinstance(blocks[0], types.TextContent) + payload = json.dumps([block.model_dump(mode="json", by_alias=True) for block in blocks]) + + assert "test-token-value" not in payload def test_mcp_from_run_omits_user_messages() -> None: diff --git a/python/packages/hosting-responses/agent_framework_hosting_responses/_parsing.py b/python/packages/hosting-responses/agent_framework_hosting_responses/_parsing.py index 6cfd77f63e4..62e4bd2b095 100644 --- a/python/packages/hosting-responses/agent_framework_hosting_responses/_parsing.py +++ b/python/packages/hosting-responses/agent_framework_hosting_responses/_parsing.py @@ -773,9 +773,15 @@ def _function_call_output_item(content: Content, *, status: str) -> ResponseOutp def _function_result_output_item(content: Content, *, status: str) -> ResponseOutputItem: - if content.exception: - output: str | list[Any] = content.exception - elif output_parts := _content_parts_to_input_items(content.items): + """Project channel-visible output without exposing potentially sensitive exception diagnostics.""" + output_parts = _content_parts_to_input_items(content.items) + has_visible_output = any( + getattr(part, "type", None) != "input_text" or bool(getattr(part, "text", None)) for part in output_parts + ) + result_is_empty = content.result is None or (isinstance(content.result, str) and not content.result) + if content.exception is not None and result_is_empty and not has_visible_output: + output: str | list[Any] = "Error: Function failed." + elif output_parts: output = output_parts elif isinstance(content.result, str): output = content.result diff --git a/python/packages/hosting-responses/tests/hosting_responses/test_parsing.py b/python/packages/hosting-responses/tests/hosting_responses/test_parsing.py index 23a0753f768..e3c3079de15 100644 --- a/python/packages/hosting-responses/tests/hosting_responses/test_parsing.py +++ b/python/packages/hosting-responses/tests/hosting_responses/test_parsing.py @@ -340,6 +340,78 @@ def test_responses_from_run_preserves_tool_role_function_result(self) -> None: assert payload["output"][0]["output"] == [{"type": "input_text", "text": "sunny"}] assert payload["output"][0]["status"] == "completed" + def test_responses_from_run_omits_internal_function_exception(self) -> None: + diagnostic = "test-token-value at /srv/private/tool.py" + result = AgentResponse( + messages=Message( + role="tool", + contents=[ + Content.from_function_result( + "call_1", + result="Error: Function failed.", + exception=diagnostic, + ) + ], + ) + ) + + payload = responses_from_run(result, response_id="resp_new") + serialized = json.dumps(payload) + + assert "Error: Function failed." in serialized + assert diagnostic not in serialized + + @pytest.mark.parametrize("diagnostic", ["test-token-value at /srv/private/tool.py", ""]) + def test_responses_from_run_uses_generic_output_for_exception_only_result(self, diagnostic: str) -> None: + result = AgentResponse( + messages=Message( + role="tool", + contents=[Content.from_function_result("call_1", exception=diagnostic)], + ) + ) + + payload = responses_from_run(result, response_id="resp_new") + serialized = json.dumps(payload) + + assert "Error: Function failed." in serialized + if diagnostic: + assert diagnostic not in serialized + + @pytest.mark.parametrize( + ("result", "expected_output"), + [ + (0, "0"), + (False, "false"), + ([], "[]"), + ({}, "{}"), + ], + ) + def test_responses_from_run_preserves_falsey_error_result(self, result: object, expected_output: str) -> None: + diagnostic = "test-token-value at /srv/private/tool.py" + content = Content("function_result", call_id="call_1", result=result, exception=diagnostic) + response = AgentResponse(messages=Message(role="tool", contents=[content])) + + payload = responses_from_run(response, response_id="resp_new") + serialized = json.dumps(payload) + + assert payload["output"][0]["output"] == expected_output + assert diagnostic not in serialized + + def test_responses_from_run_uses_generic_error_when_items_project_to_nothing(self) -> None: + diagnostic = "test-token-value at /srv/private/tool.py" + content = Content.from_function_result( + "call_1", + result=[Content("uri", uri=None)], + exception=diagnostic, + ) + response = AgentResponse(messages=Message(role="tool", contents=[content])) + + payload = responses_from_run(response, response_id="resp_new") + serialized = json.dumps(payload) + + assert payload["output"][0]["output"] == "Error: Function failed." + assert diagnostic not in serialized + def test_responses_from_run_rejects_standalone_media(self) -> None: result = AgentResponse( messages=Message( @@ -809,6 +881,34 @@ def finalizer(items: Sequence[AgentResponseUpdate]) -> AgentResponse: completed_output = cast("list[dict[str, object]]", completed_response["output"]) assert done_item["id"] == completed_output[0]["id"] + async def test_responses_from_streaming_run_omits_internal_function_exception(self) -> None: + diagnostic = "test-token-value at /srv/private/tool.py" + + async def updates() -> AsyncIterator[AgentResponseUpdate]: + yield AgentResponseUpdate( + role="tool", + contents=[ + Content.from_function_result( + "call_1", + result="Error: Function failed.", + exception=diagnostic, + ) + ], + ) + + stream = ResponseStream(updates(), finalizer=AgentResponse.from_updates) + events = [ + event + async for event in responses_from_streaming_run( + stream, + response_id="resp_new", + ) + ] + serialized = "".join(events) + + assert "Error: Function failed." in serialized + assert diagnostic not in serialized + async def test_responses_from_streaming_run_preserves_marked_refusal_deltas(self) -> None: marker = {"model_output_kind": "refusal"} diff --git a/python/packages/openai/agent_framework_openai/_chat_client.py b/python/packages/openai/agent_framework_openai/_chat_client.py index 0cc2bc3e34c..60c6c2aafd9 100644 --- a/python/packages/openai/agent_framework_openai/_chat_client.py +++ b/python/packages/openai/agent_framework_openai/_chat_client.py @@ -2133,10 +2133,8 @@ def _to_local_shell_output_payload(content: Content) -> str: payload = { "stdout": "" if content.result is None else str(content.result), } - if content.exception is not None and "stderr" not in payload: - payload["stderr"] = str(content.exception) if "exit_code" not in payload: - payload["exit_code"] = 1 if content.exception else 0 + payload["exit_code"] = 1 if content.exception is not None else 0 return json.dumps(payload, ensure_ascii=False) @staticmethod @@ -2149,8 +2147,6 @@ def _to_shell_call_output_payload(content: Content) -> list[dict[str, Any]]: payload = { "stdout": "" if content.result is None else str(content.result), } - if content.exception is not None and "stderr" not in payload: - payload["stderr"] = str(content.exception) # Pass through native payload shape when tool already returns shell output entries. direct_output = payload.get("output") @@ -2165,9 +2161,11 @@ def _to_shell_call_output_payload(content: Content) -> list[dict[str, Any]]: else: exit_code_raw = payload.get("exit_code") try: - exit_code = int(exit_code_raw) if exit_code_raw is not None else (1 if content.exception else 0) + exit_code = ( + int(exit_code_raw) if exit_code_raw is not None else (1 if content.exception is not None else 0) + ) except (TypeError, ValueError): - exit_code = 1 if content.exception else 0 + exit_code = 1 if content.exception is not None else 0 outcome = {"type": "exit", "exit_code": exit_code} return [ { diff --git a/python/packages/openai/tests/openai/test_openai_chat_client.py b/python/packages/openai/tests/openai/test_openai_chat_client.py index 7258c50cae9..58d625fb0ec 100644 --- a/python/packages/openai/tests/openai/test_openai_chat_client.py +++ b/python/packages/openai/tests/openai/test_openai_chat_client.py @@ -2159,6 +2159,32 @@ def local_exec(command: str) -> str: assert response_tools[0]["environment"]["type"] == "local" +def test_shell_output_payloads_do_not_expose_exception_diagnostics() -> None: + diagnostic = "test-token-value at /srv/private/tool.py" + content = Content.from_function_result( + call_id="call-1", + result="Error: Function failed.", + exception=diagnostic, + ) + + local_payload = json.loads(OpenAIChatClient._to_local_shell_output_payload(content)) + shell_payload = OpenAIChatClient._to_shell_call_output_payload(content) + serialized = json.dumps({"local": local_payload, "shell": shell_payload}) + + assert local_payload["stdout"] == "Error: Function failed." + assert local_payload["exit_code"] == 1 + assert shell_payload == [ + {"stdout": "Error: Function failed.", "stderr": "", "outcome": {"type": "exit", "exit_code": 1}} + ] + assert diagnostic not in serialized + + empty_diagnostic = Content.from_function_result(call_id="call-2", result="failed", exception="") + empty_local_payload = json.loads(OpenAIChatClient._to_local_shell_output_payload(empty_diagnostic)) + empty_shell_payload = OpenAIChatClient._to_shell_call_output_payload(empty_diagnostic) + assert empty_local_payload["exit_code"] == 1 + assert empty_shell_payload[0]["outcome"] == {"type": "exit", "exit_code": 1} + + def test_prepared_local_shell_tool_survives_make_tools() -> None: """Regression: the prepared shell tool must be a subscriptable dict.