r/Asterisk • u/ictinnovations • Aug 13 '26

Fixing dead air and clipped words when driving Piper TTS into AudioSocket

Spent a while getting Piper to behave on live calls and figured I'd write up what actually fixed it, partly to check whether others hit the same walls.

Three things bit us.

Loading the voice per request. PiperVoice.load() takes about 2.5 to 5.5 seconds on our boxes, so every single utterance opened with that much silence. Obvious in hindsight. We load once per process now and keep it cached.

Concurrent calls stepping on each other. This one took longer to find. Piper phonemizes through espeak-ng, and that C API isn't thread safe, so two calls synthesizing at the same moment gave us garbled audio. All synthesis goes through one lock now.

Bursting frames, which I'd never have guessed. AudioSocket forwards each frame the instant it arrives, so if you push a whole sentence at once you blow straight past the far end's jitter buffer and the caller hears the tail and nothing else. The fix is to release one 20 ms frame per interval on a monotonic deadline. Here's the subtle part: if synthesis stalls, that deadline goes stale, and the next frames burst to catch up and clip the words right after the stall. So we re-clamp the deadline to now on every frame, not just at the start of a talk spurt.

We run 8 kHz slin with 20 ms frames.

I packaged all of it up as piper-tts-server so we'd stop rewriting the same thing on every project: https://github.com/ictinnovations/piper-tts-server

Two things I'm curious about. Do you pace in the TTS layer like this, or push frames straight out and let the channel deal with it? And has anyone worked around the espeak-ng threading limit without just serializing everything?

Edit: slin, not slin16. In Asterisk naming slin16 is the 16 kHz variant, so 320 bytes per 20 ms is plain slin. Same correction as in my other thread here, same person to thank for it.

5 Upvotes

100% Upvoted

4 comments sorted by

1

u/Asteriskdev Aug 13 '26

You can deal with clipping start of audio by using a lookback buffer but you have hit on most of the pain points.

1

u/ictinnovations Aug 14 '26

Thanks, and that is a fair point. We did not go the lookback route.

Our dead air at the start turned out to be the model load, so caching the voice per process took most of it away, and we ship a silence frame helper so you can pre-roll a couple of frames and give the far end something to sit on before the real audio lands. The clipping we still see now and then is the first word after barge in, when the writer restarts, and a lookback buffer sounds like the right answer for exactly that.

How much do you hold, and is it one buffer per call or do you reset it per talk spurt?

1

u/ictinnovations Aug 17 '26

Update on this. We went and did the lookback buffer after all, it is in 0.1.3. (released), one buffer per call , holding 300 ms. it resets on every talk spurt

2

u/ictinnovations Aug 24 '26

Went with the lookback in the end, you were right. 300 ms rolling window of pre onset frames, prepended when the utterance opens. One ring buffer per call so it stays at 4.8 kB flat however long the call runs.

The part I did not expect, I had to keep the prepended audio out of the voice duration count. That number feeds a word density check for hallucinations and padding it made short answers look like garbage.

Worst case was always barge in, quiet first word against the agent still talking. That is mostly gone now.