Skip to content

Python: include Anthropic cache tokens in input_token_count - #9859

Open
NingLin Liu (ninglinLiu) wants to merge 13 commits into
microsoft:mainfrom
ninglinLiu:fix/9808-anthropic-input-tokens-include-cache
Open

NingLin Liu (ninglinLiu) wants to merge 13 commits into
microsoft:mainfrom
ninglinLiu:fix/9808-anthropic-input-tokens-include-cache

Conversation

@ninglinLiu

Copy link
Copy Markdown
Contributor

Motivation & Context

With Anthropic prompt caching, AnthropicClient set UsageDetails["input_token_count"] to Anthropic's raw usage.input_tokens, which is only the tokens after the last cache breakpoint. Total input is input_tokens + cache_creation_input_tokens + cache_read_input_tokens. Everywhere else in Agent Framework input_token_count is total input (with cache_read_input_token_count as a subset), matching OpenAI/Gemini and the GenAI semconv for gen_ai.usage.input_tokens. A cache hit therefore under-reports Claude input on spans and histograms (e.g. 12 instead of 9312).

Description & Review Guide

  • What are the major changes?
    • _parse_usage_from_anthropic sums cache creation/read into input_token_count when present.
    • Cache-specific fields are unchanged.
    • Update test_parse_usage_with_cache_tokens expected total (100+20+30=150).
  • What is the impact of these changes? Anthropic usage totals align with other connectors; dashboards comparing providers stop undercounting Claude when caching is used.
  • What do you want reviewers to focus on? That cache fields remain separate and that streaming incremental usage still reconstructs the final total.

Related Issue

Fixes #9808

AI Assistance

  • No material AI assistance was used.
  • This is an AI-assisted contribution. I reviewed, understood, and verified all submitted content and accept responsibility for it.

AI assistance details: confirming Anthropic's input_tokens vs cache fields against the GenAI semconv, implementing the total in _parse_usage_from_anthropic, updating the unit test, and this PR description.

Contribution Checklist

  • The code builds clean without any errors or warnings
  • All unit tests pass, and I have added new tests where possible
  • The PR follows the Contribution Guidelines
  • This PR links to an agreed issue with no competing open PR, or the Related Issue section documents a trivial-change or repository-automation exception.
  • This is not a breaking change.

Fixes microsoft#9808

Anthropic reports input_tokens as only the tokens after the last cache
breakpoint. Add cache_creation_input_tokens and cache_read_input_tokens
into input_token_count so it matches other connectors and the GenAI
semconv (cache_read stays a subset of total input).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Partial streaming usage updates drop previously reported cached input tokens when cache fields are omitted, producing negative increments and undercounting total usage.

1 open issue
What changed in this PR

This PR addresses #9808 by including Anthropic cache tokens in total input usage, aligning the Python connector with other Agent Framework providers and telemetry expectations.

Changes:

  • Add cache-creation and cache-read tokens to input_token_count, preserving separate cache fields.
  • Update the cached-usage unit test to expect 150 total input tokens.
File Description
python/​packages/​anthropic/​tests/​test_anthropic_client.py Update the expected cached-input total.
python/​packages/​anthropic/​agent_framework_anthropic/​_chat_client.py Include cache tokens when calculating total input usage.

Review effort: Balanced

message_delta often omits cache_* that were only on message_start;
preserve last-seen input/cache components so input_token_count stays
inclusive after incremental conversion (Copilot review on microsoft#9859).

This branch was previously deployed

1 inactive deployment
github-app-auth — 1afd2b14 Deployed Oct 30, 2026 by ninglinLiu via add_label #26744
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

python Usage: [Issues, PRs], Target: Python

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Python: [Bug]: AnthropicClient reports input_token_count (and gen_ai.usage.input_tokens) without cached prompt tokens

2 participants