fix(crewai): trace tools run by native function calling by pylaterreur · Pull Request #292 · alibaba/loongsuite-python

pylaterreur · 2026-10-28T11:20:57Z

Description

When an agent's LLM supports function calling, as crewAI's native providers do, crewAI runs the tool calls returned by the model through BaseTool.run (AgentExecutor._execute_single_native_tool_call), not through ToolUsage._use. The instrumentation only wrapped ToolUsage._use, so these tools ran without a TOOL span.

This PR also wraps BaseTool.run, and Tool.run (the class behind @tool, which overrides it), with the same execute_tool span: same name, attributes and parent (the AGENT span) as on the ReAct path. ToolUsage._use reaches the tool through its CrewStructuredTool and _run, not run. Even so, a run of the same tool inside a traced _use is not traced again, so a tool call never gets two spans. Parallel tool calls run in copied contexts and keep the AGENT span as parent.

Fixes #291

Type of change

Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

New tests/test_native_tool_calls.py, with the OpenAI-compatible API mocked by respx, so it needs no cassette and runs with block_network. It covers one tool call, a failing tool (ERROR status), two parallel calls (both under the AGENT span), no second span for run inside a traced _use, and uninstrumentation. Without the fix, 4 of the 5 fail; the duplicate-span test fails when the guard is removed.
The spans of the 41 existing span-checking tests (name, kind, operation, status, parent) are unchanged on crewai 1.15.30 and 1.15.32. The native TOOL span has the same attribute keys and values as the ReAct one for the same tool call.
tox -c tox-loongsuite.ini -e py3{10,11,12,13}-test-loongsuite-instrumentation-crewai (tox-uv 1.35.2, GitHub CPython builds), together with test(crewai): install a litellm that crewAI 1.15.31+ can load #290, which the existing tests need on crewai 1.15.31 and later: 53 passed on each version with crewai 1.15.32, online and offline (bwrap --unshare-net), and offline with crewai 1.15.30. Without test(crewai): install a litellm that crewAI 1.15.31+ can load #290, on crewai 1.15.30 (litellm 1.95.0), Python 3.12: 53 passed.
ruff check and ruff format --check pass, and so does tox -e precommit (Python 3.11).

Does This PR Require a Core Repo Change?

No.

Checklist:

Followed the style guidelines of this project
Changelogs have been updated
Unit tests have been added
Documentation has been updated

🤖 Generated with Claude Code

https://claude.ai/code/session_01GwUvhmzgfkiZ6vo4uGyesN

@tool

When the LLM supports function calling, as crewAI's native providers do, crewAI sends the tools to the model and runs the returned tool calls through BaseTool.run (AgentExecutor's _execute_single_native_tool_call, for one), not through ToolUsage._use. The instrumentation only wrapped ToolUsage._use, so those tools ran without a TOOL span. Also wrap BaseTool.run, and Tool.run, which overrides it for @tool functions, with the same execute_tool span: same name, attributes and parent as the ReAct path. ToolUsage._use calls the tool's _run through its CrewStructuredTool, not run, but a run of the same tool inside a traced _use is skipped anyway, so a call never gets two spans. crewAI runs parallel tool calls in copied contexts, so their spans keep the agent span as parent. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GwUvhmzgfkiZ6vo4uGyesN

ralf0131 · 2026-10-29T12:13:23Z

+            "tool",
+            create_tool_invocation,
+            instance,
+            None,


Preserve the native tool call ID. _ToolRunWrapper passes None as tool_calling, causing tool_call_id to fall back to the tool name. Two calls to get_weather therefore receive the same ID instead of their original call_1 / call_2, preventing reliable correlation with the model’s tool calls. Could we propagate the original ID from the native execution context and assert distinct IDs in the parallel-call test?

Fixed in 0a4aded. The instrumentation now wraps CrewAI's native dispatch (AgentExecutor._execute_single_native_tool_call, and execute_single_native_tool_call as step_executor calls it). It reads the model's id with CrewAI's own extract_tool_call_info, and the BaseTool.run span uses it. The parallel test asserts call_1/call_2. A direct run(), or a CrewAI version without these functions, keeps the tool name as before.

🤖 Generated with Claude Code

ralf0131 · 2026-10-29T12:13:23Z

+            _ACTIVE_TOOL.reset(token)
+
+        _finish_tool_success(
+            self._handler, invocation, started, result, None, instance


Handle structured tool failures. CrewAI supports returning a ToolFailure without raising an exception. The new wrapper treats this as a successful return and marks the span OK, even though the tool explicitly reported failure. Could we recognize this result, mark the span ERROR, and add a test covering a returned ToolFailure?

Fixed in 0a4aded. A returned ToolFailure now ends the span with ERROR: the description is as_agent_message() and error.type is RuntimeError. This applies on the native path, and on the ReAct path through ToolUsage.last_failure, which reported OK before. Testing this also showed that a raw result JSON can't encode (a ToolFailure, a pydantic model) made the handler raise before ending the span. The span now records the text the agent gets, as _use does. New tests cover both paths.

🤖 Generated with Claude Code

Copilot

🟡 Changes recommended

The wrapper discards the native model call ID, causing parallel invocations of the same tool to share a fallback ID and preventing reliable correlation with provider tool calls.

1 open issue

Propagate native tool call IDs into invocation spans · New

What changed in this PR

Adds CrewAI instrumentation for tools executed through native function calling, ensuring they emit execute_tool spans consistently with the existing ReAct path.

Changes:

Wraps BaseTool.run and conditionally wraps Tool.run.
Prevents duplicate spans for nested tool executions and supports uninstrumentation.
Adds mocked tests for successful, failing, parallel, duplicate, and cleanup scenarios.
Documents the fix in the CrewAI changelog.

File	Summary and review comments
`instrumentation-loongsuite/loongsuite-instrumentation-crewai/tests/test_native_tool_calls.py`	Adds native tool-call span coverage. Moderate (1 vote): Add a precondition verifying that `Tool.run` is wrapped before uninstrumentation, ideally alongside a direct `Tool` execution span assertion.
`instrumentation-loongsuite/loongsuite-instrumentation-crewai/src/opentelemetry/instrumentation/crewai/__init__.py`	Adds native tool-run wrappers, active-tool tracking, and uninstrumentation support. Moderate (1 vote): Add native-call coverage for a `Tool`/`@tool` instance. Moderate (2 votes): Propagate the native model call ID instead of passing `None`, so parallel calls can be correlated correctly. Moderate (1 vote): Preserve both positional and keyword arguments when recording mixed invocations.
`instrumentation-loongsuite/loongsuite-instrumentation-crewai/CHANGELOG.md`	Documents the native function-calling tracing fix.

_{Review effort: Lite}

pylaterreur · 2026-10-29T13:10:37Z

+            create_tool_invocation,
+            instance,
+            None,
+            tool_call_arguments=kwargs or (list(args) if args else None),


Fixed in 0a4aded, as in the reply above. The overview's other points are also covered: a @tool (Tool.run) native-call test, a check that Tool.run is wrapped before uninstrumentation, and mixed positional and keyword run() arguments are both recorded.

🤖 Generated with Claude Code

Address the review of the native tool spans: - Give each native tool span the model's tool call id. CrewAI runs one native call in AgentExecutor._execute_single_native_tool_call or, for planned steps, agent_utils.execute_single_native_tool_call (as imported by step_executor). Wrap both to keep the id that CrewAI's extract_tool_call_info reads from the call, and use it in the BaseTool.run span. Two parallel calls to one tool no longer share the tool name as their id. Where neither runs (other CrewAI versions, or a direct run), the span keeps the tool name. - Mark a span ERROR when the tool returns a ToolFailure instead of raising: BaseTool.run returns it, ToolUsage._use keeps it in last_failure. Both paths now report it the same way: status description from ToolFailure.as_agent_message(), error.type RuntimeError, no exception event. - Record the result as the text the agent gets (the tool's format_output_for_agent), as ToolUsage._use returns it. A raw result that JSON can't encode, such as a pydantic model or a ToolFailure, made the handler raise before ending the span, so the span was lost and its context stayed attached. - Record both positional and keyword run() arguments. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01GwUvhmzgfkiZ6vo4uGyesN

pylaterreur · 2026-10-29T22:13:00Z

The red docs job is not from this PR: it fails on every PR since #294 and is fixed by #295.

🤖 Generated with Claude Code

https://claude.ai/code/session_01GwUvhmzgfkiZ6vo4uGyesN

github-actions Bot assigned 123liuziming, Cirilla-zmh and ralf0131 Oct 28, 2026

github-actions Bot requested review from 123liuziming, Cirilla-zmh and ralf0131 October 28, 2026 11:21

pylaterreur mentioned this pull request Oct 28, 2026

test(crewai): install a litellm that crewAI 1.15.31+ can load #290

Merged

4 of 6 tasks

Merge branch 'main' into crewai-native-tool-spans

f178a2f

ralf0131 requested a lite review from Copilot October 29, 2026 12:05

ralf0131 reviewed Oct 29, 2026

View reviewed changes

Copilot started reviewing on behalf of ralf0131 October 29, 2026 12:19

View session

Copilot AI reviewed Oct 29, 2026

View reviewed changes

pylaterreur and others added 2 commits October 29, 2026 12:48

Merge remote-tracking branch 'origin/main' into crewai-native-tool-spans

4d4d2b0

Navigation Menu

Repository navigation

fix(crewai): trace tools run by native function calling - #292

fix(crewai): trace tools run by native function calling#292
pylaterreur wants to merge 4 commits into
alibaba:mainfrom
pylaterreur:crewai-native-tool-spans

pylaterreur commented Oct 28, 2026

Uh oh!

ralf0131 Oct 29, 2026

Uh oh!

pylaterreur Oct 29, 2026

Uh oh!

ralf0131 Oct 29, 2026

Uh oh!

pylaterreur Oct 29, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

pylaterreur Oct 29, 2026

Uh oh!

pylaterreur commented Oct 29, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

pylaterreur commented Oct 28, 2026

Description

Type of change

How Has This Been Tested?

Does This PR Require a Core Repo Change?

Checklist:

Uh oh!

ralf0131 Oct 29, 2026

Choose a reason for hiding this comment

Uh oh!

pylaterreur Oct 29, 2026

Choose a reason for hiding this comment

Uh oh!

ralf0131 Oct 29, 2026

Choose a reason for hiding this comment

Uh oh!

pylaterreur Oct 29, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

pylaterreur Oct 29, 2026

Choose a reason for hiding this comment

Uh oh!

pylaterreur commented Oct 29, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants