Repository navigation
fix(neuphonic): keep speaking sentences released after the earlier ones finished - #8720
Open
MohammadHijjawi97 wants to merge 1 commit into
Open
MohammadHijjawi97 wants to merge 1 commit into
MohammadHijjawi97 wants to merge 1 commit into
Conversation
…es finished The streaming recv task ended the segment as soon as every sentence sent so far had been answered with a stop. The sentence tokenizer holds the last sentence until more text or the end of input arrives, so with an LLM reply the server often finishes one sentence before the next is released: the segment ended after it, the next sentences were still sent but their audio was never read, and it stayed queued on the pooled socket. Only end the segment once the input has ended and nothing is pending, and wait for the next sentence instead of the socket while nothing is. This also ends a segment that sent no sentence (whitespace-only input), which previously waited on the socket forever.
Contributor
There was a problem hiding this comment.
🔍 Devin Review: 2 flags
Not posted on this PR by your GitHub settings — view them in Devin Review. (Configure)
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Neuphonic streaming TTS ends a segment as soon as every sentence sent so far has been answered with a
stop. The sentence tokenizer holds the last sentence until more text or the end of input arrives, so with a streamed LLM reply the server often finishes one sentence before the next one is released. The segment then ends after that sentence: the following sentences are still sent, but their audio is never read, so the agent stops speaking mid-reply, and the unread audio stays queued on the pooled websocket.The same check also never ends a segment that sends no sentence (e.g. whitespace-only input), since no
stopever arrives, so the stream hangs.Changes
recv_taskonly ends the segment once the input has ended and no sentence is pending.Before / after
Pushing "Hello there, this is the first sentence. And here", letting the first sentence finish, then pushing " is the second sentence." (fake websocket returning 0.1 s of audio per sentence):
Whitespace-only input: before, the stream never finished; after, it ends with no audio.
Tests
Two regression tests in
tests/test_plugin_neuphonic_tts.py, using the existing fake websocket. Both fail before the change and pass after;pytest tests/test_plugin_neuphonic_tts.pypasses (8 tests). ruff and mypy (pinned versions) are clean for the plugin.#8035 touches the same
recv_task(receive loop monitoring), so whichever lands second may need a small rebase of the other.