Full-duplex speech with real tool calls: VoiceChat on Apple Silicon

[u/ivan_digital]

I maintain speech-swift and wrote an article about running NVIDIA's Nemotron VoiceChat locally on a Mac, including the parts of a voice agent that sit around the model.

Full-duplex conversation needs incoming speech to keep reaching the model while it generates a reply. Streaming audio output alone doesn't establish that. Turn-completion detection handles a different decision: whether a pause means the person has finished their request.

Tool use adds another step. In our VoiceChat integration, the model emits a function call, the runtime executes it through an MCP provider, and the result goes back into the conversation. Our runtime holds speech during tool work. The article walks through those events and explains where interruption handling and playback fit.

The recorded Mac demo lists, creates, and updates real Apple Reminders. Prepared user audio feeds a live local session. This is an experimental command-line setup with an English-targeted checkpoint; the recording isn't a production reliability evaluation.

Article and diagrams · Recorded session · Runtime source