diff --git a/docs/observability/logging.md b/docs/observability/logging.md index d93869f177..3ea06d598a 100644 --- a/docs/observability/logging.md +++ b/docs/observability/logging.md @@ -35,8 +35,10 @@ for GenAI](https://github.com/open-telemetry/semantic-conventions/blob/main/docs/gen-ai/gen-ai-events.md). By default prompt content is elided in logs for security. You can enable prompt -logging using environment variables or programmatic configuration (see Setup -section below). +logging using environment variables or programmatic configuration. See +[Capture prompt content](#capture-prompt-content-in-adk-web) for `adk web`, and +[Capture prompt content programmatically](#capture-prompt-content) +for setup in code. ### Log levels (Python) @@ -55,15 +57,13 @@ using the standard logger: Only enable `DEBUG` when actively troubleshooting an issue, as `DEBUG` logs can be very verbose and may contain sensitive information. -## Logging setup - -### Logging in ADK Web +## Logging in ADK Web When running agents using the ADK's `adk web`, `adk api_server`, `adk deploy cloud_run` and `adk deploy gke` commands, you can control the log verbosity or destination. -#### Logging level +### Logging level in ADK Web To start the web server with `DEBUG` level logging, run: @@ -74,7 +74,7 @@ adk web --log_level DEBUG path/to/your/agents_dir The available log levels for the `--log_level` option are: `DEBUG`, `INFO` (default), `WARNING`, `ERROR`, `CRITICAL`. -#### Capture prompt content +### Capture prompt content in ADK Web By default a prompt content is elided in logs for security. You can enable prompt logging using the environment variable: @@ -96,7 +96,7 @@ and `SPAN_AND_EVENT` also require debugging but may capture sensitive data or PII. In production, set this to false or ensure you have appropriate data handling policies in place. -#### OTLP export +### OTLP export in ADK Web To export logs to an OTLP-compatible backend, set the standard OTel environment variables: @@ -112,7 +112,7 @@ adk web path/to/your/agents_dir in addition to logs. -#### GCP export setup +### GCP export setup in ADK Web You can enable GCP export using the `--otel_to_cloud` flag: @@ -120,188 +120,306 @@ You can enable GCP export using the `--otel_to_cloud` flag: adk web --otel_to_cloud path/to/your/agents_dir ``` -### Python programmatic setup +## Programmatic setup -In Python, ADK uses the standard `logging` module and OpenTelemetry for -structured GenAI logs. +Programmatic setup configures the underlying logging framework and +OpenTelemetry exporters from your own code, for system-level diagnostics and +production observability. ADK uses the following logging facilities: -#### Logging level +- **Python:** ADK uses the standard `logging` module and OpenTelemetry for + structured GenAI logs. +- **Go:** ADK uses the `google.golang.org/adk/v2/telemetry` package for + OpenTelemetry configuration, and the standard `log` package for general + events, which it writes to `stderr` by default. +- **Kotlin:** ADK uses standard JVM logging facilities, defaulting to Flogger, + and OpenTelemetry for structured GenAI logs. -To enable detailed logging, including `DEBUG` level messages, add the following -to the top of your script: +### Logging level -```python -import logging +You can set the logging level for your ADK agent using standard logging controls, as follows: -logging.basicConfig( - level=logging.DEBUG, - format='%(asctime)s - %(levelname)s - %(name)s - %(message)s' -) -``` +=== "Python" -#### Capture prompt content + To enable detailed logging, including `DEBUG` level messages, add the following + to the top of your script: -You can enable full prompt logging programmatically by setting an environment -variable: + ```python + import logging -```python -import os + logging.basicConfig( + level=logging.DEBUG, + format='%(asctime)s - %(levelname)s - %(name)s - %(message)s' + ) + ``` -os.environ["OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT"] = "true" -``` +=== "Go" -To scope content capture to a single run instead of the whole process, set -`RunConfig.telemetry` rather than the environment variable: + General events (such as server startup or HTTP requests) are logged using the standard Go `log` package and written to `stderr` by default. -```python -from google.adk.agents.run_config import RunConfig -from google.adk.telemetry import ContentCapturingMode, TelemetryConfig +=== "Kotlin" -run_config = RunConfig( - telemetry=TelemetryConfig( - capture_message_content=ContentCapturingMode.SPAN_AND_EVENT, - ), -) -``` + ADK uses standard JVM logging facilities (defaulting to Flogger). Configure your JVM logger backend, such as `java.util.logging` or SLF4J, to adjust log verbosity. -#### OTLP export +### Capture prompt content -To export logs to an OpenTelemetry Collector (or an OTLP-compatible backend) -programmatically: +=== "Python" -```python -from google.adk.telemetry.setup import maybe_set_otel_providers -import os + You can enable full prompt logging programmatically by setting an environment + variable: -os.environ["OTEL_EXPORTER_OTLP_LOGS_ENDPOINT"] = "http://your-collector:4318/v1/logs" -os.environ["OTEL_SERVICE_NAME"] = "your-adk-agent" -os.environ["OTEL_RESOURCE_ATTRIBUTES"] = "key1=value1,key2=value2" -maybe_set_otel_providers() -``` + ```python + import os -#### GCP export setup + os.environ["OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT"] = "true" + ``` -To export logs to Google Cloud Logging programmatically, use the OpenTelemetry -Google Cloud exporter. Here is an example in Python: + To scope content capture to a single run instead of the whole process, set + `RunConfig.telemetry` rather than the environment variable: -```python -from google.adk.telemetry.google_cloud import get_gcp_exporters -from google.adk.telemetry.setup import maybe_set_otel_providers -import os + ```python + from google.adk.agents.run_config import RunConfig + from google.adk.telemetry import ContentCapturingMode, TelemetryConfig -gcp_exporters = get_gcp_exporters( - enable_cloud_logging = True, -) -os.environ["OTEL_SERVICE_NAME"] = "your-adk-agent" -os.environ["OTEL_RESOURCE_ATTRIBUTES"] = "key1=value1,key2=value2" -maybe_set_otel_providers([gcp_exporters]) -``` + run_config = RunConfig( + telemetry=TelemetryConfig( + capture_message_content=ContentCapturingMode.SPAN_AND_EVENT, + ), + ) + ``` -### Kotlin programmatic setup +=== "Go" -In Kotlin, ADK uses standard JVM logging facilities (defaulting to Flogger) and OpenTelemetry for structured GenAI logs. + You can enable full prompt logging when initializing telemetry by exporting `OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true`: -#### Capture prompt content + ```go + package main -You can enable full prompt logging by configuring the global `TelemetryConfig`: + import ( + "context" + "os" -```kotlin ---8<-- "examples/kotlin/snippets/observability/LoggingExamples.kt:capture_content" -``` + "google.golang.org/adk/v2/telemetry" + ) -#### Activity logging with Plugins + func main() { + ctx := context.Background() -To get detailed logs of agent activity (user messages, model requests/responses, tool calls) in the console, use the `LoggingPlugin`: + // Enable GenAI message content capture via the OpenTelemetry environment variable + os.Setenv("OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT", "true") -```kotlin ---8<-- "examples/kotlin/snippets/observability/LoggingExamples.kt:logging_plugin" -``` + tp, err := telemetry.New(ctx) + if err != nil { + // handle error + } + defer tp.Shutdown(ctx) + tp.SetGlobalOtelProviders() + } + ``` -#### Full debug capture to a file +=== "Kotlin" -
- Supported in ADKKotlin v0.6.0 -
+ You can enable full prompt logging by configuring the global `TelemetryConfig`: -To record the same activity in full, as YAML appended to `adk_debug.yaml` rather than truncated console output, use the `DebugLoggingPlugin`: + ```kotlin + --8<-- "examples/kotlin/snippets/observability/LoggingExamples.kt:capture_content" + ``` -```kotlin ---8<-- "examples/kotlin/snippets/observability/LoggingExamples.kt:debug_logging_plugin" -``` +### OTLP export -!!! warning - The output file holds raw prompts, tool arguments and session state. Treat it as sensitive. +=== "Python" -### Go programmatic setup + To export logs to an OpenTelemetry Collector (or an OTLP-compatible backend) + programmatically: -In Go, ADK uses the `google.golang.org/adk/v2/telemetry` package for OpenTelemetry -configuration and the standard `log` package for general events. + ```python + from google.adk.telemetry.setup import maybe_set_otel_providers + import os -#### Capture prompt content + os.environ["OTEL_EXPORTER_OTLP_LOGS_ENDPOINT"] = "http://your-collector:4318/v1/logs" + os.environ["OTEL_SERVICE_NAME"] = "your-adk-agent" + os.environ["OTEL_RESOURCE_ATTRIBUTES"] = "key1=value1,key2=value2" + maybe_set_otel_providers() + ``` -You can enable full prompt logging programmatically when initializing telemetry: +=== "Go" -```go -package main + To export logs to an OTLP-compatible backend, configure the standard + OpenTelemetry environment variables, such as `OTEL_EXPORTER_OTLP_ENDPOINT` + or `OTEL_EXPORTER_OTLP_LOGS_ENDPOINT`. The ADK telemetry package uses these + settings automatically when initialized. -import ( - "context" - "google.golang.org/adk/v2/telemetry" -) +=== "Kotlin" -func main() { - ctx := context.Background() - tp, err := telemetry.New(ctx, - telemetry.WithGenAICaptureMessageContent(true), - ) - if err != nil { - // handle error - } - defer tp.Shutdown(ctx) - tp.SetGlobalOtelProviders() -} -``` + ADK Kotlin's OpenTelemetry integration emits **traces only** — it registers no + `LoggerProvider`, so there is no OTLP log export. Application logs go to your JVM + logging backend. To configure trace export, see the [Traces](traces.md) + documentation. -#### OTLP export +### GCP export setup -To export logs to an OTLP-compatible backend, configure the standard -OpenTelemetry environment variables (e.g., `OTEL_EXPORTER_OTLP_ENDPOINT` or -`OTEL_EXPORTER_OTLP_LOGS_ENDPOINT`). The ADK telemetry package will -automatically use these settings when initialized. +=== "Python" -#### GCP export setup + To export logs to Google Cloud Logging programmatically, use the OpenTelemetry + Google Cloud exporter. Here is an example in Python: -To export logs to Google Cloud Logging, use the `WithOtelToCloud` option: + ```python + from google.adk.telemetry.google_cloud import get_gcp_exporters + from google.adk.telemetry.setup import maybe_set_otel_providers + import os -```go -package main + gcp_exporters = get_gcp_exporters( + enable_cloud_logging = True, + ) + os.environ["OTEL_SERVICE_NAME"] = "your-adk-agent" + os.environ["OTEL_RESOURCE_ATTRIBUTES"] = "key1=value1,key2=value2" + maybe_set_otel_providers([gcp_exporters]) + ``` -import ( - "context" - "google.golang.org/adk/v2/telemetry" -) +=== "Go" -func main() { - ctx := context.Background() - tp, err := telemetry.New(ctx, - telemetry.WithOtelToCloud(true), - ) - if err != nil { - // handle error - } - defer tp.Shutdown(ctx) - tp.SetGlobalOtelProviders() -} -``` + To export logs to Google Cloud Logging, use the `WithOtelToCloud` option: -If using the Go launcher, you can also enable GCP export via the CLI flag: + ```go + package main -```bash -go run main.go web -otel_to_cloud -``` + import ( + "context" + "google.golang.org/adk/v2/telemetry" + ) + + func main() { + ctx := context.Background() + tp, err := telemetry.New(ctx, + telemetry.WithOtelToCloud(true), + ) + if err != nil { + // handle error + } + defer tp.Shutdown(ctx) + tp.SetGlobalOtelProviders() + } + ``` + + If using the Go launcher, you can also enable GCP export via the CLI flag: + + ```bash + go run main.go web -otel_to_cloud + ``` + +=== "Kotlin" + + ADK Kotlin emits no OpenTelemetry log records, so there is nothing for Cloud Logging + to receive; application logs go to your JVM logging backend. ADK Kotlin **traces** can + be sent to Google Cloud by pointing a standard OTLP exporter at `telemetry.googleapis.com` + — see [OTLP with Google Cloud](https://cloud.google.com/stackdriver/docs/otlp/overview) + for the required credentials, quota project and `roles/telemetry.writer` grant. + +## Activity logging with plugins -General events (like server startup or HTTP requests) are logged using the -standard Go `log` package. These logs are written to `stderr` by default. +ADK provides built-in plugins that capture agent activity, including user +messages, model requests and responses, tool calls, and (with +`DebugLoggingPlugin`) session state. These plugins require no changes to your +agent logic. + +### Console logging with `LoggingPlugin` + +To print structured activity logs to the console during execution, attach +`LoggingPlugin` to your `App`: + +=== "Python" + + ```python + from google.adk.apps import App + from google.adk.plugins import LoggingPlugin + + app = App( + name="my_app", + root_agent=root_agent, + plugins=[LoggingPlugin()], + ) + ``` + +=== "Go" + + ```go + package main + + import ( + "context" + "log" + "os" + + "google.golang.org/adk/v2/agent" + "google.golang.org/adk/v2/cmd/launcher" + "google.golang.org/adk/v2/cmd/launcher/full" + "google.golang.org/adk/v2/plugin" + "google.golang.org/adk/v2/plugin/loggingplugin" + "google.golang.org/adk/v2/runner" + ) + + func main() { + ctx := context.Background() + logPlugin := loggingplugin.MustNew("logging_plugin") + + config := &launcher.Config{ + AgentLoader: agent.NewSingleLoader(rootAgent), + PluginConfig: runner.PluginConfig{ + Plugins: []*plugin.Plugin{logPlugin}, + }, + } + + l := full.NewLauncher() + if err := l.Execute(ctx, config, os.Args[1:]); err != nil { + log.Fatalf("run failed: %v", err) + } + } + ``` + +=== "Kotlin" + + ```kotlin + --8<-- "examples/kotlin/snippets/observability/LoggingExamples.kt:logging_plugin" + ``` + +### Full debug capture to a file with `DebugLoggingPlugin` + +
+ Supported in ADKPython v1.23.0Kotlin v0.6.0 +
+ +To record complete interaction data as human-readable YAML appended to +`adk_debug.yaml` rather than truncated console output, use +`DebugLoggingPlugin`: + +=== "Python" + + ```python + from google.adk.apps import App + from google.adk.plugins import DebugLoggingPlugin + + app = App( + name="my_app", + root_agent=root_agent, + plugins=[ + DebugLoggingPlugin( + output_path="adk_debug.yaml", + include_session_state=True, + include_system_instruction=True, + ), + ], + ) + ``` + +=== "Kotlin" + + ```kotlin + --8<-- "examples/kotlin/snippets/observability/LoggingExamples.kt:debug_logging_plugin" + ``` + +!!! warning + The output file holds raw prompts, tool arguments, and session state. + Although ADK automatically redacts credentials and `temp:`-scoped state + keys in Python, treat the output file as sensitive. ## Understanding log output