Skip to content

fix(traceloop-sdk): restore gen_ai.client.operation.duration histogram view - #4706

Open
mccalpink wants to merge 1 commit into
traceloop:mainfrom
mccalpink:fix/4702-operation-duration-buckets
Open

mccalpink wants to merge 1 commit into
traceloop:mainfrom
mccalpink:fix/4702-operation-duration-buckets

Conversation

@mccalpink

@mccalpink mccalpink commented Nov 12, 2026 •

Copy link
Copy Markdown
  • I have added tests that cover my changes.
  • If adding a new instrumentation or changing an existing one, I've added screenshots from some observability platform showing the change. (N/A: SDK metric view, no instrumentation change; see the test output below.)
  • PR name follows conventional commits format: fix(traceloop-sdk): ....
  • (If applicable) I have updated the documentation accordingly. (N/A)

Summary

#3129 (following #3104) dropped the LLM_OPERATION_DURATION View from metric_views() so that the histogram uses the OpenTelemetry SDK default boundaries. That default, [0, 5, 10, 25, ..., 10000], is sized for milliseconds, while gen_ai.client.operation.duration is recorded in seconds (every one of the 14 instrumentations creates it with unit="s"). Almost every LLM call therefore lands in the first bucket, and e.g. histogram_quantile(0.95, ...) is a constant ~4.75 s. This PR adds the View back with the GenAI semconv boundaries [0.01, 0.02, ..., 81.92] seconds. The le labels of this already exported histogram will change, which is an expected series change.

Root cause

Before #3129 the boundaries were swapped between the two histograms: LLM_TOKEN_USAGE had the sub-second [0.01 ... 81.92] list and LLM_OPERATION_DURATION had the token-sized [1, 4, 16, ...] list (#3104). #3129 put the token boundaries from opentelemetry-python-contrib on LLM_TOKEN_USAGE, and deliberately removed the LLM_OPERATION_DURATION View to fall back to the SDK default (see the #3129 description and the discussion in #3104). That was a reasonable way to fix the swap, but the SDK default is a millisecond-oriented list, which does not fit a histogram whose unit is s; the GenAI semconv for gen_ai.client.operation.duration recommends [0.01 ... 81.92]. This PR restores only the duration View; the token boundaries are untouched.

Fix

Add a View(instrument_name=Meters.LLM_OPERATION_DURATION, aggregation=ExplicitBucketHistogramAggregation([0.01, ..., 81.92])) to metric_views() in packages/traceloop-sdk/traceloop/sdk/metrics/metrics.py. The token-usage boundaries are unchanged. New tests in packages/traceloop-sdk/tests/test_metric_views.py build a MeterProvider with InMemoryMetricReader + metric_views() and assert explicit_bounds for both histograms.

How I reproduced it

With only the new test added (no fix), on main:

cd packages/traceloop-sdk
uv run pytest tests/test_metric_views.py -v

test_operation_duration_uses_semconv_second_buckets fails (bounds are the SDK defaults instead of 0.01 ... 81.92); test_token_usage_uses_semconv_token_buckets passes (it guards the token boundaries against a future swap).

How I verified it

In packages/traceloop-sdk:

  • uv run pytest tests/test_metric_views.py -v: 2 passed (after the fix).
  • uv run pytest tests/: 446 passed.
  • uv run ruff check .: All checks passed.
  • uv run mypy traceloop/sdk: no issues found in 50 source files.

Closes #4702

This PR was prepared with AI assistance (Claude Code). I reproduced the bug, reviewed the change, and ran the tests myself.

Summary by CodeRabbit

  • New Features
    • Operation-duration metrics now use explicit histogram boundaries, ranging from 0.01 to 81.92, for more consistent duration reporting.

…m view

traceloop#3129 removed the LLM_OPERATION_DURATION view so that the histogram uses
the OpenTelemetry SDK default boundaries. That default is sized for
milliseconds, but gen_ai.client.operation.duration is recorded in seconds
(unit "s"), so nearly every LLM call lands in the first bucket.

Add the view back with the GenAI semconv boundaries (0.01 ... 81.92 s)
and add tests that check explicit_bounds for the duration and token usage
histograms via InMemoryMetricReader. The token boundaries are unchanged.

Fixes traceloop#4702
@CLAassistant

CLAassistant commented Nov 12, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@coderabbitai

coderabbitai Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

metric_views() now adds explicit histogram boundaries for LLM_OPERATION_DURATION. New tests collect and verify the duration and token-usage histogram boundaries.

Changes: Duration histogram boundaries

Layer / File(s) Summary
Configure and verify histogram boundaries
packages/traceloop-sdk/traceloop/sdk/metrics/metrics.py, packages/traceloop-sdk/tests/test_metric_views.py
metric_views() adds explicit duration histogram boundaries from 0.01 to 81.92. Tests collect the instrument bounds and compare duration and token-usage bounds with their expected tuples.

Estimated code review effort: 2 (Simple) | ~8 minutes

Change: Bug fix · Severity of issue fixed: Medium

Merge Risk: Minimal · up to ecb1b

The change applies GenAI-appropriate histogram boundaries to the operation-duration metric, so duration observations fall into meaningful buckets. The exported bucket labels change, which is intended. No actionable merge-blocking risk remains.

Pre-merge checks: Passed 4

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title names the main change: restoring the gen_ai.client.operation.duration histogram view in the traceloop-sdk. The raw summary confirms metric_views() now adds an explicit-bucket view for Meters…
Linked Issues check Passed Issue #4702 requires the GenAI semconv boundaries [0.01, 0.02, 0.04, ..., 40.96, 81.92] for gen_ai.client.operation.duration. At the reviewed head, packages/traceloop-sdk/traceloop/sdk/metrics/metrics…
Out of Scope Changes check Passed The PR changes two files. metrics.py adds only the LLM_OPERATION_DURATION View, which is the subject of #4702. The other Views (token usage, Pinecone duration, Pinecone scores) are present at the head…
Full details: Title check

Explanation

The title names the main change: restoring the gen_ai.client.operation.duration histogram view in the traceloop-sdk. The raw summary confirms metric_views() now adds an explicit-bucket view for Meters.LLM_OPERATION_DURATION with boundaries from 0.01 to 81.92, matching the objectives. The title is specific and uses a conventional fix() prefix. It omits the bucket-boundary detail, which is acceptable.

Full details: Linked Issues check

Explanation

Issue #4702 requires the GenAI semconv boundaries [0.01, 0.02, 0.04, ..., 40.96, 81.92] for gen_ai.client.operation.duration. At the reviewed head, packages/traceloop-sdk/traceloop/sdk/metrics/metrics.py metric_votes() includes a View for Meters.LLM_OPERATION_DURATION with ExplicitBucketHistogramAggregation using exactly these 14 boundaries, which matches the issue. The token-usage View is unchanged. The new test packages/traceloop-sdk/tests/test_metric_views.py asserts explicit_bounds for both histograms, covering the requirement with automated tests. The author reports the test suite, ruff, and mypy results; I did not run them. The metrics.py diff against the merge base could not be read, so the change was verified from the head file content only.

Full details: Out of Scope Changes check

Explanation

The PR changes two files. metrics.py adds only the LLM_OPERATION_DURATION View, which is the subject of #4702. The other Views (token usage, Pinecone duration, Pinecone scores) are present at the head and are not altered by this PR per the summary. test_metric_views.py is a new test file that directly verifies the new View and the unchanged token-usage View. No unrelated changes were identified.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 7abd367a-38f5-4c29-a2ea-3e78d89c8b32
Commits

Reviewing files that changed from the base of the PR and between e3d651a and ecb1b3c.

Files selected for processing: 2

  • packages/traceloop-sdk/tests/test_metric_views.py
  • packages/traceloop-sdk/traceloop/sdk/metrics/metrics.py

Included review availability: This review used your included allowance. Your plan provides up to 8 included reviews per hour; 7 remain after this review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

🐛 Bug Report: gen_ai.client.operation.duration lost its histogram view in #3129, falls back to default ms buckets

2 participants