Skip to content

Commit 5c93bd7

Browse files
starfolkai[bot]starfolkbotclaude
authored
fix(adk): fix llm span metadata (provider, tools, errors) (#580)
## Summary Brings the Google ADK integration in line with `.agents/skills/sdk-integrations/SKILL.md` and cuts the test suite over to real VCR-backed coverage. ### Integration behavior (`tracing.py`) - **`metadata.provider="google"` and `metadata.tools` on every `llm` span.** Tool definitions are lifted out of `input.config` into `metadata.tools`, preserving Google-native shape. - **`_capture_config` is now a strict allowlist** — no more `dict(config)` pass-through. Fields outside the allowlist (including `tools`/`tool_config`) are dropped from `input.config` automatically. `_extract_tool_metadata` is the single source of truth for what goes into `metadata.tools`. - **`error=e` instead of `error=str(e)`** in the tool and MCP-tool wrappers — the `Exception` instance flows through to the log pipeline. - **Trim raw ADK objects from span metadata.** `invocation_context`, `model_response_event`, `parent_context`, and the `**kwargs` spread across every wrapper are dropped; only small named fields (`app_name`, `user_id`, `session_id`, `state_delta`, `agent_name`, `flow_class`, `tool_class`, `model`, `provider`, `tools`) remain. - **Drop `_get_field` and `bt_safe_deep_copy(tools)`.** Production ADK always passes Pydantic objects, so plain `getattr(x, field, default)` is enough. The log pipeline serializes Pydantic tool objects at send time (SKILL: "do not over-serialize"). ### Tests - **`test_adk_braintrust_integration`** now filters spans by `context.span_origin.instrumentation.name == "adk-auto"`, asserts `{task, llm, tool}` types are covered, and validates `metadata.provider`, `metadata.tools`, and no tool leakage into `input.config`. - **`test_adk_captures_metrics`** asserts `metadata.provider="google"` and the `direct_response` call-type span name. - **`test_adk_max_tokens_captures_content`** now also asserts the sampling config (`max_output_tokens`, `temperature`) is captured in the LLM span input. - **`test_adk_sync_runner_run_does_not_duplicate_invocation_spans`** now also asserts thread-bridge context propagation — every `adk-auto` span shares the outer parent's `root_span_id`. - **Delete `test_adk_mcp_tool.py` entirely** (the two remaining patcher-wiring tests were duplicative of the auto-instrument subprocess test) and drop its line from `noxfile.py`. - **Delete fake-LLM, mock, and non-ADK unit tests** — 15 tests removed. Every branch they covered (`_determine_llm_call_type`, `_capture_config` allowlist, `_serialize_content/_serialize_part`, `_serialize_pydantic_schema`, `bt_safe_deep_copy`, sampling config, thread bridge, PDF attachment, MCP-in-flow parenting, async context) is exercised by an existing VCR test. Final test count: **13 tests**, 12 VCR-backed + one plain-Python regression for the `_create_thread_wrapper` double-invoke bug. ## Test plan - [x] `nox -s "test_google_adk(latest)"` — 13 passed - [x] `nox -s "test_google_adk(1.14.1)"` — 13 passed - [x] No cassettes re-recorded (one orphaned cassette, `test_adk_agent_metadata_with_attachment.yaml`, removed on both matrix versions) 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- sfk:created-approved-by --> Created by abhijeet <!-- sfk:slack-thread --> [Slack thread](https://starfolkai.slack.com/archives/C0AQDETAVT3/p1784214360713409?thread_ts=1784214360.713409&cid=C0AQDETAVT3) --------- Co-authored-by: Starfolk <noreply@starfolk.ai> Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
1 parent c056762 commit 5c93bd7

6 files changed

Lines changed: 128 additions & 1645 deletions

File tree

‎py/noxfile.py‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -442,6 +442,11 @@ def test_pydantic_ai_wrap_openai(session, version):
442442
"""Test pydantic_ai with wrap_openai() approach - supports older versions."""
443443
_install_test_deps(session)
444444
_install_matrix_dep(session, "pydantic-ai-wrap-openai", version)
445+
# pydantic-ai 0.1.9 unconditionally imports ``from opentelemetry._events import Event``.
446+
# The ``_events`` module was removed from ``opentelemetry-api`` in 1.40+, so a fresh
447+
# resolution on newer opentelemetry releases picks a version that breaks import.
448+
if version == "0.1.9":
449+
session.install("opentelemetry-api<1.40", silent=SILENT_INSTALLS)
445450
_run_tests(session, f"{INTEGRATION_DIR}/pydantic_ai/test_pydantic_ai_wrap_openai.py", version=version)
446451

447452

@@ -513,7 +518,6 @@ def test_google_adk(session, version):
513518
_install_test_deps(session)
514519
_install_matrix_dep(session, "google-adk", version)
515520
_run_tests(session, f"{INTEGRATION_DIR}/adk/test_adk.py", version=version)
516-
_run_tests(session, f"{INTEGRATION_DIR}/adk/test_adk_mcp_tool.py", version=version)
517521

518522

519523
LANGCHAIN_VERSIONS = _get_matrix_versions("langchain-core")

‎py/src/braintrust/integrations/adk/cassettes/1.14.1/test_adk_agent_metadata_with_attachment.yaml‎

Lines changed: 0 additions & 254 deletions
This file was deleted.

0 commit comments

Comments
 (0)