[xcv58]

Problem and result

Add an optional Voice Assistant plugin for controlling enabled MacTools features through a spoken conversation. A global shortcut opens a compact floating panel and starts listening; the same shortcut stops and dismisses it. Stop ends audio while keeping the panel visible, and the panel can expand for longer conversations.

This PR includes the voice runtime, floating interface, setup, and fresh state adapters needed for the complete feature. Host capability discovery, session grants, application launching, and native confirmations already landed in #513.

Changes

• Support bidirectional OpenAI audio, separate live user/assistant captions and waveforms, interruption, serial host tool calls, and bounded in-memory transcripts and usage accounting.
• Bundle a pinned Silero VAD v6 Core ML model with a Swift adapter, provenance hashes, and MIT notices. Start listens locally and connects after sustained speech, retaining onset audio. There is no paid warm-up, runtime model download, or file/environment credential source.
• Store personal API keys in host-specific login Keychain entries, with credential operations off the UI thread. Compact settings group connection, conversation, and feature access. Model IDs and billing warnings stay visible; pricing details and the optional separate stop shortcut expand on demand. Feature access supports search, batch selection, and one Cancel/Done commit.
• Preserve host authorization and native confirmations. Stop, dismissal, lock, sleep, and permission loss end audio and revoke the session. Enabling automatic access for future plugins by voice requires native confirmation; access changes apply to future conversations.
• Add bounded fresh-state adapters for Display Brightness, System Status, Power Status, Appearance, Keep Awake, Night Shift, and System Mute. Unknown or unsupported observations remain unavailable. Brightness controls use stable display identities, serialized/coalesced work, and explicit readback verification across write generations.
• Retain existing fixed/dynamic commands in the shared host catalog, including application activation and window layouts. Coverage documentation describes 69 other providers and 136 static action declarations; availability depends on enabled plugins, hardware, and configuration.
• Localize the plugin in all 12 supported languages, preserve stable voice-control identifiers/preferences/shortcuts, declare minimum host 2.2.1, include third-party notices, and regenerate plugin inventories and website data.

Review fixes

• Gate microphone PCM at capture time, attach capture generations, discard invalidated queued/startup audio, and reset converter history across mute changes.
• Ignore failures from a previous session after a new conversation starts.
• Cancel pending automatic-plugin-access changes when the session stops, while preserving native approval and fresh invocation outside a conversation.
• Keep Shade brightness observations on the main actor and skip reads during pending/in-flight writes so a query cannot enqueue stale dimming.

Verification

• Combined head: 9fbc02f5f5e57294fc65cc803c6475347d61ea4f, based on upstream main 192e93e564cb360c313fc725bef9effb5768c237.
• Before the compact-settings-only update, full make ci passed at 979053d7: 3,147 native tests passed, six skipped; all 245 script tests and frozen PluginKit v7 binary compatibility passed. Seven focused regressions cover capture/converter mute safety, buffered input invalidation, old-session failure isolation, policy cancellation, and Shade observation ordering.
• The compact settings plugin builds successfully; all 245 script checks and 16 focused native credential, feature-access, and settings checks pass. The native tests ran in the existing temporary integration checkout after the Documents-path test host stalled before starting tests.
• The runtime/UI, adapter, review fixes, and compact settings update are preserved for review. git diff --check passes. The complete diff has 77 files, 13,614 additions and 28 deletions; most added lines are localization and generated metadata. There are no additional host Core/App or PluginKit source changes.
• The bundled speech model previously passed offline numerical comparison with its pinned upstream model. Local native/physical evidence and hosted CI are separate; GitHub Build CI passed on the updated PR head.
• The live demo below verifies dark appearance, menu-bar hiding, persistent Dock visibility, a fresh disk-space answer, Safari activation, and Chrome/Safari window layouts.
• Bluetooth switching, macOS 14 and Intel runtime behavior, and broader plugin acceptance remain follow-up validation areas. Arbitrary per-app volume percentages and private clipboard/calendar/file/process reads require additional adapters.

UI evidence

Entry point: enable Voice Assistant in Marketplace, save a personal API key in its settings, then invoke its action or an assigned global shortcut. The floating panel provides input/output captions and levels, mute, Stop, settings, allowed features, and expand/dismiss controls.

Settings — light and dark

Native Retina captures from the compact settings plugin at 9fbc02f5, using an isolated first-use profile with no API key and microphone permission ungranted. No audio or API request was needed for these stills. The profile contains only Voice Assistant plus the host's built-in capabilities, so its feature count differs from the fully configured live demo.

Light	Dark

First-use connection and microphone setup

Light	Dark

Live demo — 2:30

MacTools-Voice-Assistant-PR-1080.mp4

AI-generated command prompts are played from an iPhone; MacTools actions and assistant replies are live. The recording preserves action/response timing within each section. It demonstrates:

• 0:00 — Switch to dark mode.
• 0:16 — Hide the menu bar.
• 0:36 — Keep the Dock visible.
• 0:53 — Ask how much free disk space is available.
• 1:13 — Switch to Safari.
• 1:30 — Activate Chrome and Safari, placing them on the left and right halves.

The GitHub attachment is a size-limited copy of the approved recording. Native Retina source footage and full-quality sharing exports are retained separately.

Checklist

• Core behavior and relevant risks have existing test and manual evidence; remaining limitations are recorded.
• Public UI evidence includes the live interaction demo and light/dark setup and settings screenshots.
• User-facing documentation and changelog fragments are included.
• Plugin metadata, runtime contracts, action policies, and minimum-host compatibility agree.
• Licensing and third-party notices are included; no credentials, local configuration, or unrelated generated files are added.

The provider adapters are included here, superseding xcv58#6.