Skip to content

fix(ios): make feedStreamingAudio work and move to FluidAudio 0.18 - #5

Merged
Alex-Wengg merged 3 commits into
mainfrom
fix/streaming-feed-fluidaudio-0.17
Nov 1, 2026
Merged

Alex-Wengg merged 3 commits into
mainfrom
fix/streaming-feed-fluidaudio-0.17

Conversation

@Alex-Wengg

@Alex-Wengg Alex-Wengg commented Oct 27, 2026 •

Copy link
Copy Markdown
Member

Fixes #4.

Problem

  • feedStreamingAudio discarded the audio it was given and still resolved success: true. Streaming only worked through the microphone tap that startStreamingAsr installs itself.
  • The podspec depended on FluidAudio ~> 0.7. CocoaPods trunk stops at 0.7.8 (Nov 2025) and goes read-only on December 2, 2026. 0.7.8 predates FluidAudio's current streaming managers, and the StreamingAsrManager class this module used was removed upstream.

Change

  • Dependency: FluidAudio comes from Swift Package Manager through React Native's spm_dependency (FluidAudio 0.18.0 or later, up to the next minor). The React Native peer dependency is now 0.75 or later, and the podspec stops with a clear error on older versions. pod 'FluidAudio' is removed from the example Podfile.
  • Port to FluidAudio 0.18: batch ASR, plus streaming through StreamingModelVariant.createManager(). The default model is parakeet-eou-320ms, the one FluidAudio's own iOS streaming demos use; JS can choose another with the new model option.
  • feedStreamingAudio works: base64 PCM16 is decoded and fed to the manager. A new source: 'manual' starts a session without the microphone. Feeding a microphone session now rejects instead of silently dropping audio. The microphone tap and fed audio go through one ordered queue, and stop drains it before finishing.
  • Example app: a "Stream Sample File" button feeds a bundled 16 kHz WAV (FluidAudio's Apache-2.0 test fixture) in 320 ms chunks. It also adds the missing NSMicrophoneUsageDescription, without which iOS kills the microphone demo on a device.

Breaking changes for JS users

  • React Native 0.75 or later is required.
  • Feeding audio needs start({ source: 'manual' }). source: 'system' is removed.
  • StreamingUpdate.volatile is always empty; the transcript is in confirmed and text.
  • start() and feed now return { success, source, model } and { success, samples }.

Verification

  • npx tsc --noEmit passes. npx jest: 54/54.

  • pod install in example/ios resolves FluidAudio 0.18.0.

  • xcodebuild builds the example for generic/platform=iOS and generic/platform=iOS Simulator.

  • The manual-feed path was driven on macOS with real speech. A harness ran this module's own AudioUtils.pcm16MonoBuffer (copied verbatim), the same ordered AsyncStream → appendAudio / processBufferedAudio → finish() session loop, and parakeet-eou-320ms from the FluidAudio 0.18.0 checkout the iOS build resolved. It fed base64 PCM16 in 320 ms chunks:

Clip This PR main (audio discarded)
example/assets/speech-16k-mono.wav, 16.9 s "diff means comparing what is net new so what is the difference between the old code and the net new code and analyzing if there's any bugs or problems in that difference" (25 partial updates) empty, 0 samples fed
FLEURS en_us_0003, 5.8 s "the cabbage juice changed his color depending on how acidic basic alkaline the chemical is" (reference: "…changes color depending on how acidic or basic alkaline…") empty

Not verified: model download, the transcript and microphone streaming inside the app on a device or simulator. No iOS Simulator runtime is installed on the build machine.

🤖 Generated with Claude Code

Alex-Wengg and others added 2 commits October 27, 2026 18:24
feedStreamingAudio resolved success without using its audio, so a JS-fed
session produced no transcript and no error. The podspec also pinned
FluidAudio ~> 0.7, and CocoaPods trunk stops at 0.7.8, before
StreamingAsrManager became a protocol.

- Pull FluidAudio 0.17.7+ (up to next minor) from GitHub through React
  Native's spm_dependency podspec helper; peer react-native >= 0.75.
  Drop the pod 'FluidAudio' line from the example Podfile.
- Port the module to FluidAudio 0.17: streaming through
  StreamingModelVariant.createManager() (default Parakeet EOU 320 ms,
  any variant by its raw name), AsrManager.loadModels(_:) and an
  explicit TdtDecoderState per transcribe.
- Streaming takes audio from one ordered queue, fed by the microphone
  tap or by feedStreamingAudio; stop drains it before finish(). A new
  'manual' source skips the microphone; feeding a microphone session
  is rejected instead of ignored. Decode base64 PCM16 unaligned.
- The last streaming manager stays loaded for the next session.
- Example: "Stream Sample File" feeds a bundled 16.9 s clip (FluidAudio
  test fixture) in 320 ms chunks; add NSMicrophoneUsageDescription.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The streaming, batch ASR, VAD and diarizer APIs this module calls are unchanged from 0.17.7; the example builds for the iOS Simulator against 0.18.0.
@Alex-Wengg Alex-Wengg changed the title fix(ios): make feedStreamingAudio work and move to FluidAudio 0.17 fix(ios): make feedStreamingAudio work and move to FluidAudio 0.18 Nov 1, 2026
@Alex-Wengg
Alex-Wengg marked this pull request as ready for review November 1, 2026 21:00
@Alex-Wengg
Alex-Wengg merged commit 5583dcb into main Nov 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feedStreamingAudio discards the audio it's given; FluidAudio pin predates the streaming managers

1 participant