Skip to content

fix(inference): log retryable STT recv errors at debug - #7918

Open
muntazir868 wants to merge 1 commit into
livekit:mainfrom
muntazir868:fix/7441-retryable-stt-debug-log
Open

muntazir868 wants to merge 1 commit into
livekit:mainfrom
muntazir868:fix/7441-retryable-stt-debug-log

Conversation

@muntazir868

@muntazir868 muntazir868 commented Oct 18, 2026 •

Copy link
Copy Markdown
Contributor

Split out of #7502 to keep that PR focused on the RoomIO change.

Problem

When the gateway rejects a streaming STT session before input ends — for example a provider refusing the session during init, code=2006 "Streaming ASR unavailable" — the failure is retryable=True.

SpeechStream's retry loop already reports that failure at WARNING and recreates the stream while retries remain, but the failure was also logged twice at ERROR from inside the stream:

  1. recv_task logged "received error from LiveKit Inference STT" at ERROR for every error message, retryable or not.
  2. recv_task was wrapped in utils.log_exceptions, which logged the propagated exception at ERROR a second time.

Net effect: a gateway-level reject that the framework fully recovered from still produced ERROR-level logs and tripped failure alerts, blaming the application for a handled retry.

Change

Receiver failures are now level-aware and emitted once:

  • failures that will be retried, or that belong to a handled handoff, log at DEBUG
  • a terminal failure, an exhausted retry budget, or a non-retryable error logs once at ERROR
  • the gateway error event is converted to an APIError without an additional duplicate ERROR log

Errors that actually end the session remain visible at ERROR without double-reporting.

Testing

  • uv run pytest tests/test_inference_stt.py -q (175 passed)
  • uv run --group typing mypy --untyped-calls-exclude=smithy_aws_core --platform linux -p livekit.agents (238 source files, no issues)
  • uv run ruff format --check livekit-agents/livekit/agents/inference/stt.py tests/test_inference_stt.py
  • uv run ruff check livekit-agents/livekit/agents/inference/stt.py tests/test_inference_stt.py
  • rebased onto current upstream/main (d78f39990)

Closes the logging half of #7441.

devin-ai-integration[bot]

This comment was marked as resolved.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

2 flags not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)

Devin Review

Retryable gateway errors (e.g. provider refusing session during init, code 2006)
are already reported by SpeechStream's retry loop (WARNING) and the session
recreates the stream once retries are exhausted. Logging that same failure as
ERROR double-logs handled retries and triggers false failure alerts on recovered
sessions.

Changes:
- recv_task: wrap in try/except, log retryable APIErrors at DEBUG level
- error message handler: use DEBUG for retryable errors, ERROR for terminal ones
@muntazir868
muntazir868 force-pushed the fix/7441-retryable-stt-debug-log branch from ddd7675 to ad23fd9 Compare November 21, 2026 18:56

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant