Repository navigation
fix(voice): create room IO media streams off the event loop - #7502
muntazir868 wants to merge 6 commits into
Conversation
1b381fd to
fd66a12
Compare
|
Rebased onto current Also split this PR: the inference STT log-level change is unrelated to the RoomIO stream-creation fix and made this hard to review, so it's now on its own PR: #7918. This PR is now only the RoomIO change. One test needed updating: |
There was a problem hiding this comment.
Note
Newer findings are available below. Devin Review posted a newer report on this PR, in addition to the findings presented here.
Devin Review found 1 new potential issue.
β οΈ 1 issue in files not directly in the diff
β οΈ Handled STT rejects emit duplicate errors
When the gateway rejects a session before input ends, recv_task logs the retryable error at ERROR. Its exception decorator logs another ERROR, even when the stream retries successfully.
2 flags not posted on this PR by your GitHub settings β view them in Devin Review. (Configure)
fd66a12 to
5ca0db1
Compare
AudioStream/VideoStream.from_track performs a synchronous FFI request (initializing noise cancellation can block for >100ms) and was called directly from the track_subscribed handler, tripping the loop monitor. Create the stream in a task via asyncio.to_thread, pass the running loop explicitly so frames still dispatch onto it, and guard against track events that supersede an in-flight creation with a generation counter plus a lock around processor ownership.
- Fix aclose() to clean up resources even when cancelled during settle - Add retry mechanism for failed stream creation (don't clear track on failure) - Increment generation counter to allow retry
37bfccf to
193b990
Compare
There was a problem hiding this comment.
Devin Review found 1 new potential issue.
1 flag not posted on this PR by your GitHub settings β view it in Devin Review. (Configure)
| else: | ||
| # The subscription is dead from RoomIO's perspective. Clear | ||
| # it so a later track/publication event can start a fresh | ||
| # generation; don't keep retrying a permanently broken track. | ||
| self._close_stream() |
There was a problem hiding this comment.
π΄ Failed screen share blocks camera fallback
When screen share creation exhausts its retries, _close_stream forgets the selected publication. If the participant stops sharing, the track removal handlers skip the fallback scan. The agent keeps receiving no video even though the camera is still subscribed.
Learn more
RoomIO accepts both camera and screen share for the linked participant. When a new publication takes over, _on_track_available closes the old local stream without unsubscribing from its publication. The track removal handlers scan other subscribed tracks only when the event matches the selected publication. At retry exhaustion, _close_stream clears that publication, so its eventual removal never triggers the scan.
Example: The agent receives camera frames. The same participant starts sharing their screen, but all three local attempts to open that stream fail. They stop sharing without turning off their camera. The agent remains without video because neither removal event selects the camera.
Recommended fix: Keep the failed track and publication associated with the exhausted generation without scheduling more retries. A real removal can then run the existing fallback scan, and a new subscription or replacement track can still start a fresh generation. Add a test that exhausts retries for one publication, removes it, and verifies that another already-subscribed publication starts forwarding.
Was this helpful? React with π or π to provide feedback.
Closes #7441 (RoomIO half).
Rebased onto current
main. The inference STT log-level change that used to be bundled here is now split out into #7918, so this PR only touches the RoomIO stream-creation path.Problem
AudioStream.from_track/VideoStream.from_trackrun a synchronous FFI request, and_on_track_availablecalled it directly from thetrack_subscribedhandler. Initializing noise cancellation (e.g.BVC()) blocked the loop for 106β119ms and tripped the loop monitor.Changes
_create_streamis now async; the audio/video implementations runfrom_trackviaasyncio.to_threadand passloop=explicitly so the stream still dispatches frames onto the job loop._close_streamdefers processor cleanup to that task while it settles.Testing
uv run pytest tests/test_room_io.py -q(80 passed)uv run ruff format --checkandruff checkfor changed filesuv run --group typing mypy --untyped-calls-exclude=smithy_aws_core --platform linux -p livekit.agents(238 source files)upstream/main(d78f39990)