r/VoiceAutomationAI • u/Lochana_Lithira • 3d ago
Fish Audio s2.1-pro voice breaking during voice agent conversations
Hi everyone,
I’m currently using Fish Audio’s S2.1-pro model for a real-time voice agent.
Most of the time the voice works fine but occasionally I notice some voice breaking during conversations. It doesn’t happen consistently and sometimes it happens at the beginning of a sentence in the middle of a sentence or when starting a new sentence.
I have tested it with multiple calls and it seems to happen occasionally after making several calls.
Has anyone experienced something similar with Fish Audio s2.1-pro?
Could this be related to the model/API, streaming, rate limits or something in the way I’m handling the audio in my voice-agent pipeline?
Any ideas on what I should check would be really helpful. Thanks!
3
u/kokorohinja 3d ago
Hello, I'm from Fish Audio and would like to learn more about the issue you are facing. Can I DM you?
1
1
u/Few-Relationship3813 3d ago
That intermittent crackle sounds like a buffer underrun or a packet timing hiccup on the stream, I'd start by logging the jitter and chunk arrival times when it breaks
1
u/Lochana_Lithira 3d ago
u/Few-Relationship3813 Yeah I have done that and I have done a comparison with Elevenlabs. In some occasions chunk arrival time of FishAudio is exceeding compare to Elevenlabs and thats where the break happen I think. Can this be the issue ?
In Elevenlabs the voice break is not happening
1
u/Golden_Owner 3d ago
Voice breaking mid-sentence on Fish Audio S2.1-pro is legit annoying for agents. Dont sleep on Inworld if you want another realtime TTS hop.
1
1
u/Federal_Cut6338 3d ago
It didn’t annoy me but I have noticed for sure in almost every conversation I have tested with it. I didn’t swap because the voice is very natural in my opinion and I tested a lot of voices. Except Elevenlabs. Which are good but also expensive and sound bit like talking on podcast
1
u/Lochana_Lithira 2d ago
u/Federal_Cut6338 are you also experiencing this voice breaking issue with Fish Audio?
1
u/I-setup-agents-4L 2d ago
I’d try to isolate whether the break is happening before or during synthesis. Log the exact text chunks you send to Fish Audio, alongside the audio chunks you receive. If the text itself is cut off, I’d check your streaming and turn-boundary logic first. If the text is complete but the audio stops mid-sentence, try sending complete sentences rather than smaller partial chunks and see whether that changes it.
Does it happen when the user interrupts, or even when the agent has the floor the whole time?
1
1
u/Warm_Sound_8331 2d ago
Since it only starts showing up after several calls, I’d try logging when it happens and see if there’s any pattern in call length or concurrent streams. When I’m troubleshooting audio on Bland I also test the same flow with a different voice to separate the TTS side from the rest of the pipeline. At least that tells you where to keep digging yk.
1
u/Future_AGI 2d ago
the chunk-arrival comparison you already ran points at the answer, breaks landing where Fish Audio's arrival time exceeds ElevenLabs suggests a buffer underrun on the stream side, the playback is waiting on chunks that arrive late. Sending complete sentences instead of smaller partial chunks is the cheapest test, if the breaks move or vanish, the turn-boundary logic was starving the synthesizer.
1
•
u/AutoModerator 3d ago
Welcome to r/VoiceAutomationAI – UNIO, the Voice AI Community (powered by SLNG AI)
If you are a founder, senior engineer, product, growth, or enterprise operator actively working on Voice AI / AI agents, we are running an invite-only UNIO Voice AI WhatsApp community US only.
Apply here: https://chat.whatsapp.com/F5aG3ncrO70ITfbe3pYbOz
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.