Repository navigation
Python: include Anthropic cache tokens in input_token_count - #9859
Open
NingLin Liu (ninglinLiu) wants to merge 13 commits into
Open
NingLin Liu (ninglinLiu) wants to merge 13 commits into
NingLin Liu (ninglinLiu) wants to merge 13 commits into
Conversation
Fixes microsoft#9808 Anthropic reports input_tokens as only the tokens after the last cache breakpoint. Add cache_creation_input_tokens and cache_read_input_tokens into input_token_count so it matches other connectors and the GenAI semconv (cache_read stays a subset of total input).
NingLin Liu (ninglinLiu)
temporarily deployed
to
github-app-auth
October 30, 2026 02:54 — with
GitHub Actions
Inactive
NingLin Liu (ninglinLiu)
requested review from
Darren Cohen (dargilco),
Jose Alvarez (jpalvarezl) and
sophia-ramsey
as code owners
October 30, 2026 02:54
NingLin Liu (ninglinLiu)
temporarily deployed
to
github-app-auth
October 30, 2026 02:54 — with
GitHub Actions
Inactive
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Partial streaming usage updates drop previously reported cached input tokens when cache fields are omitted, producing negative increments and undercounting total usage.
1 open issue
What changed in this PR
This PR addresses #9808 by including Anthropic cache tokens in total input usage, aligning the Python connector with other Agent Framework providers and telemetry expectations.
Changes:
- Add cache-creation and cache-read tokens to
input_token_count, preserving separate cache fields. - Update the cached-usage unit test to expect 150 total input tokens.
| File | Description |
|---|---|
python/packages/anthropic/tests/test_anthropic_client.py |
Update the expected cached-input total. |
python/packages/anthropic/agent_framework_anthropic/_chat_client.py |
Include cache tokens when calculating total input usage. |
Review effort: Balanced
NingLin Liu (ninglinLiu)
temporarily deployed
to
github-app-auth
October 30, 2026 08:40 — with
GitHub Actions
Inactive
NingLin Liu (ninglinLiu)
temporarily deployed
to
github-app-auth
October 30, 2026 08:48 — with
GitHub Actions
Inactive
message_delta often omits cache_* that were only on message_start; preserve last-seen input/cache components so input_token_count stays inclusive after incremental conversion (Copilot review on microsoft#9859).
NingLin Liu (ninglinLiu)
temporarily deployed
to
github-app-auth
October 30, 2026 10:13 — with
GitHub Actions
Inactive
NingLin Liu (ninglinLiu)
temporarily deployed
to
github-app-auth
October 30, 2026 10:15 — with
GitHub Actions
Inactive
NingLin Liu (ninglinLiu)
temporarily deployed
to
github-app-auth
October 30, 2026 11:35 — with
GitHub Actions
Inactive
NingLin Liu (ninglinLiu)
temporarily deployed
to
github-app-auth
October 30, 2026 12:08 — with
GitHub Actions
Inactive
NingLin Liu (ninglinLiu)
temporarily deployed
to
github-app-auth
October 30, 2026 14:05 — with
GitHub Actions
Inactive
NingLin Liu (ninglinLiu)
temporarily deployed
to
github-app-auth
October 30, 2026 16:05 — with
GitHub Actions
Inactive
NingLin Liu (ninglinLiu)
temporarily deployed
to
github-app-auth
October 30, 2026 17:19 — with
GitHub Actions
Inactive
NingLin Liu (ninglinLiu)
temporarily deployed
to
github-app-auth
October 30, 2026 17:22 — with
GitHub Actions
Inactive
NingLin Liu (ninglinLiu)
temporarily deployed
to
github-app-auth
October 30, 2026 17:40 — with
GitHub Actions
Inactive
NingLin Liu (ninglinLiu)
temporarily deployed
to
github-app-auth
October 30, 2026 19:33 — with
GitHub Actions
Inactive
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Motivation & Context
With Anthropic prompt caching,
AnthropicClientsetUsageDetails["input_token_count"]to Anthropic's rawusage.input_tokens, which is only the tokens after the last cache breakpoint. Total input isinput_tokens + cache_creation_input_tokens + cache_read_input_tokens. Everywhere else in Agent Frameworkinput_token_countis total input (withcache_read_input_token_countas a subset), matching OpenAI/Gemini and the GenAI semconv forgen_ai.usage.input_tokens. A cache hit therefore under-reports Claude input on spans and histograms (e.g. 12 instead of 9312).Description & Review Guide
_parse_usage_from_anthropicsums cache creation/read intoinput_token_countwhen present.test_parse_usage_with_cache_tokensexpected total (100+20+30=150).Related Issue
Fixes #9808
AI Assistance
AI assistance details: confirming Anthropic's input_tokens vs cache fields against the GenAI semconv, implementing the total in
_parse_usage_from_anthropic, updating the unit test, and this PR description.Contribution Checklist