Repository navigation
Conversation
|
|
There was a problem hiding this comment.
🟢 Approval recommended
The narrow fallback preserves existing behavior and has focused regression coverage, with no blocking issues identified.
No open issues 🎉
What changed in this PR
Fixes LangChain instrumentation to record Anthropic cache-write tokens from per-TTL counts when aggregate counts are unavailable.
Changes:
- Adds a TTL-count fallback while preserving existing aggregate-count precedence.
- Adds regression coverage and a changelog entry.
| File | Description |
|---|---|
| instrumentation/opentelemetry-instrumentation-genai-langchain/tests/test_callback_handler.py | Tests TTL-based token extraction and emitted attributes. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/src/opentelemetry/instrumentation/genai/langchain/utils.py | Sums TTL counts as a fallback. |
| instrumentation/opentelemetry-instrumentation-genai-langchain/.changelog/PR_NUMBER.fixed | Documents the fix. |
Review effort: Balanced
c32b18e to
b011d4e
Compare
Pull request dashboard statusWaiting on reviewers · refreshed 2026-10-21 07:29 UTC Review the latest changes. Status above doesn't look right?
|
b011d4e to
a238a35
Compare
langchain-anthropic zeroes `cache_creation` in `input_token_details` when Anthropic reports `usage.cache_creation` per TTL, moving the count into `ephemeral_5m_input_tokens` / `ephemeral_1h_input_tokens`. Fall back to summing those keys so `gen_ai.usage.cache_write.input_tokens` is recorded. Assisted-by: Claude Code
0ab762b to
8307c9d
Compare
| def test_extract_token_details_anthropic_ttl_breakdown(): | ||
| # Shape produced by langchain-anthropic when Anthropic reports | ||
| # ``usage.cache_creation`` per cache TTL. | ||
| usage = { | ||
| "input_tokens": 22, | ||
| "output_tokens": 8, | ||
| "total_tokens": 30, | ||
| "input_token_details": { | ||
| "cache_read": 4, | ||
| "cache_creation": 0, | ||
| "ephemeral_5m_input_tokens": 5, | ||
| "ephemeral_1h_input_tokens": 0, | ||
| }, | ||
| } | ||
| details = extract_token_details(usage) | ||
| assert details == { | ||
| "cache_write_input_tokens": 5, | ||
| "cache_read_input_tokens": 4, | ||
| } |
There was a problem hiding this comment.
Could we cover this through ChatAnthropic instead? The existing test_chat_anthropic_claude_sonnet_cache_token_details cassette predates the TTL breakdown. Re-recording it (or adding "cache_creation": {"ephemeral_5m_input_tokens": 5, "ephemeral_1h_input_tokens": 0} to its usage) makes that test fail on main and pass here, with the existing assertions. It also keeps passing on the oldest langchain-anthropic. Then the handler-level cases here wouldn't be needed.
There was a problem hiding this comment.
Sure, I also recorded new cassette, thanks! The test now makes two calls (cache write, then read), since the old prompt was too short to trigger caching.
Assisted-by: Claude Code
Assisted-by: Claude Code
Description
Anthropic reports
usage.cache_creationbroken down by cache TTL. When it does, langchain-anthropic setsinput_token_details.cache_creationto0and moves the count intoephemeral_5m_input_tokens/ephemeral_1h_input_tokens, sogen_ai.usage.cache_write.input_tokenswas never recorded forChatAnthropic. This change sums those keys when neithercache_writenorcache_creationis set.The existing
test_chat_anthropic_claude_sonnet_cache_token_detailscassette was recorded before Anthropic added the TTL breakdown, so it still passes onmain.Repro: with langchain-anthropic 1.7.6,
_create_usage_metadata(Usage(input_tokens=13, output_tokens=8, cache_read_input_tokens=4, cache_creation_input_tokens=5, cache_creation={"ephemeral_5m_input_tokens": 5, "ephemeral_1h_input_tokens": 0}))gives{"cache_read": 4, "cache_creation": 0, "ephemeral_5m_input_tokens": 5, ...}. Onmain,extract_token_detailsreturns only the cache-read count.Type of change
How has this been tested?
tox -e py312-test-instrumentation-genai-langchain-latest: 943 passed, 34 skipped, 2 xfailedtox -e py310-test-instrumentation-genai-langchain-oldest: passtox -e precommitandtox -e typecheck: passmainand pass with this change.Checklist
See CONTRIBUTING.md
for the style guide, changelog guidance, and more.
AI-assisted (Claude Code), disclosed with an
Assisted-by:commit trailer. I have reviewed the change and can answer questions about it.