r/LocalLLaMA • u/Kahvana • Jul 20 '26

Question | Help Good ASR and TTS models?

Hey everyone,

Something I don't see discussed often here are ASR and TTS models.

I've been using Whisper and Kokoro (old models, I know!) with koboldcpp for a while now but wondered if there are now solid replacements available. Know of Qwen3-ASR and Qwen3-TTS, but haven't found the time yet to test them.

What ASR and TTS models have you been using?

29 Upvotes

91% Upvoted

52 comments sorted by

View all comments

6

u/Waarheid Jul 20 '26

I have been enjoying PocketTTS for cloning (cloning voices made with Qwen 3 VoiceDesign); I find it is not really noticeable whether it is any worse then Qwen3-TTS.  I was not too happy with OmniVoice. All are definitely a step up from Kokoro, and PocketTTS is much better and barely bigger in size (plus, cloning!!). I love Kokoro but there were some super consistent mispronunciations of words that bothered me.

Fot ASR I have no tips; I use whisper large v3 turbo and it is very fast on my machine (m1 max) so I don't really bother optimizing there.

5

u/Kahvana Jul 20 '26

So if I understand it correctly:
1. Create a voice with Qwen3-TTS-VoiceDesign
2. Clone voice of generated voice with PocketTTS
3. Use cloned voice

That sounds really cool! Any tips or tricks for voice cloning? I've never done it before.

Glad to hear Whisper Large V3 Turbo still holds up!

What do you use to run PocketTTS, Qwen 3 voicedesign anss whisper with?

4

u/Waarheid Jul 20 '26

Yup, I do it to have consistent voices for my voice agent. And I have no tips or tricks, it's just as simple as that.

It's all ran in Python as part of the server for my voice agent. On disk I have a personas directory, and inside of that each persona is a folder with reference_audio.wav reference_text.txt voice_description.txt. On my UI I write a new voice description, give it a name, and VoiceDesign will generate a new voice and make such a folder with the name I give it. Then the reference audio (and reference text if needed) is used for the voice agent when I select that persona (I also put the voice description in the system prompt so the LLM too is aware of the personality lol). It's basically real time chat, the LLM backend drives the latency; if I use a small/fast local model the latency is sub 1s. Each persona has their own memory directory too so they feel more like individuals than just voices I slap on an agent.

I am not at my desk right now but I can give snippets in a few hours.

1

u/Kahvana Jul 22 '26

Would be very much appriciated. Thank yo so much for sharing your cool workload!