Repository navigation
Conversation
…m view traceloop#3129 removed the LLM_OPERATION_DURATION view so that the histogram uses the OpenTelemetry SDK default boundaries. That default is sized for milliseconds, but gen_ai.client.operation.duration is recorded in seconds (unit "s"), so nearly every LLM call lands in the first bucket. Add the view back with the GenAI semconv boundaries (0.01 ... 81.92 s) and add tests that check explicit_bounds for the duration and token usage histograms via InMemoryMetricReader. The token boundaries are unchanged. Fixes traceloop#4702
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fix(traceloop-sdk): ....Summary
#3129 (following #3104) dropped the
LLM_OPERATION_DURATIONView frommetric_views()so that the histogram uses the OpenTelemetry SDK default boundaries. That default,[0, 5, 10, 25, ..., 10000], is sized for milliseconds, whilegen_ai.client.operation.durationis recorded in seconds (every one of the 14 instrumentations creates it withunit="s"). Almost every LLM call therefore lands in the first bucket, and e.g.histogram_quantile(0.95, ...)is a constant ~4.75 s. This PR adds the View back with the GenAI semconv boundaries[0.01, 0.02, ..., 81.92]seconds. Thelelabels of this already exported histogram will change, which is an expected series change.Root cause
Before #3129 the boundaries were swapped between the two histograms:
LLM_TOKEN_USAGEhad the sub-second[0.01 ... 81.92]list andLLM_OPERATION_DURATIONhad the token-sized[1, 4, 16, ...]list (#3104). #3129 put the token boundaries from opentelemetry-python-contrib onLLM_TOKEN_USAGE, and deliberately removed theLLM_OPERATION_DURATIONView to fall back to the SDK default (see the #3129 description and the discussion in #3104). That was a reasonable way to fix the swap, but the SDK default is a millisecond-oriented list, which does not fit a histogram whose unit iss; the GenAI semconv forgen_ai.client.operation.durationrecommends[0.01 ... 81.92]. This PR restores only the duration View; the token boundaries are untouched.Fix
Add a
View(instrument_name=Meters.LLM_OPERATION_DURATION, aggregation=ExplicitBucketHistogramAggregation([0.01, ..., 81.92]))tometric_views()inpackages/traceloop-sdk/traceloop/sdk/metrics/metrics.py. The token-usage boundaries are unchanged. New tests inpackages/traceloop-sdk/tests/test_metric_views.pybuild aMeterProviderwithInMemoryMetricReader+metric_views()and assertexplicit_boundsfor both histograms.How I reproduced it
With only the new test added (no fix), on
main:test_operation_duration_uses_semconv_second_bucketsfails (bounds are the SDK defaults instead of 0.01 ... 81.92);test_token_usage_uses_semconv_token_bucketspasses (it guards the token boundaries against a future swap).How I verified it
In
packages/traceloop-sdk:uv run pytest tests/test_metric_views.py -v: 2 passed (after the fix).uv run pytest tests/: 446 passed.uv run ruff check .: All checks passed.uv run mypy traceloop/sdk: no issues found in 50 source files.Closes #4702
This PR was prepared with AI assistance (Claude Code). I reproduced the bug, reviewed the change, and ran the tests myself.
Summary by CodeRabbit