Skip to content

fix(neuphonic): keep speaking sentences released after the earlier ones finished - #8720

Open
MohammadHijjawi97 wants to merge 1 commit into
livekit:mainfrom
MohammadHijjawi97:fix/neuphonic-stream-later-sentences
Open

MohammadHijjawi97 wants to merge 1 commit into
livekit:mainfrom
MohammadHijjawi97:fix/neuphonic-stream-later-sentences

Conversation

@MohammadHijjawi97

Copy link
Copy Markdown
Contributor

The Neuphonic streaming TTS ends a segment as soon as every sentence sent so far has been answered with a stop. The sentence tokenizer holds the last sentence until more text or the end of input arrives, so with a streamed LLM reply the server often finishes one sentence before the next one is released. The segment then ends after that sentence: the following sentences are still sent, but their audio is never read, so the agent stops speaking mid-reply, and the unread audio stays queued on the pooled websocket.

The same check also never ends a segment that sends no sentence (e.g. whitespace-only input), since no stop ever arrives, so the stream hangs.

Changes

  • recv_task only ends the segment once the input has ended and no sentence is pending.
  • While no sentence is pending, it waits for the next sentence (or the end of input) instead of the socket.

Before / after

Pushing "Hello there, this is the first sentence. And here", letting the first sentence finish, then pushing " is the second sentence." (fake websocket returning 0.1 s of audio per sentence):

  • Before: both sentences sent, 0.1 s of audio played, 2 messages left unread on the pooled socket.
  • After: both sentences sent, 0.2 s of audio played, nothing left unread.

Whitespace-only input: before, the stream never finished; after, it ends with no audio.

Tests

Two regression tests in tests/test_plugin_neuphonic_tts.py, using the existing fake websocket. Both fail before the change and pass after; pytest tests/test_plugin_neuphonic_tts.py passes (8 tests). ruff and mypy (pinned versions) are clean for the plugin.

#8035 touches the same recv_task (receive loop monitoring), so whichever lands second may need a small rebase of the other.

…es finished

The streaming recv task ended the segment as soon as every sentence sent so
far had been answered with a stop. The sentence tokenizer holds the last
sentence until more text or the end of input arrives, so with an LLM reply
the server often finishes one sentence before the next is released: the
segment ended after it, the next sentences were still sent but their audio
was never read, and it stayed queued on the pooled socket.

Only end the segment once the input has ended and nothing is pending, and
wait for the next sentence instead of the socket while nothing is. This also
ends a segment that sent no sentence (whitespace-only input), which
previously waited on the socket forever.
@MohammadHijjawi97
MohammadHijjawi97 requested a review from a team as a code owner November 21, 2026 19:55

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔍 Devin Review: 2 flags

Not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)

Devin Review

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant