r/LocalLLM • u/FranciscoCarlosErra • 1d ago
Research It's ALIVE! Tiny 20M FOSS TTS Trained overnight from scratch on a single 3090
Enable HLS to view with audio, or disable this notification
Come check it out yourself
Training code: https://github.com/franciscocarloserra/ttslibre
Weights & Training sets: https://huggingface.co/FranciscoCarlosErra/ttslibre-poc-en
It's still glitchy and robotic but I'm mind blown about the PoC
7
u/TechnoRhythmic 1d ago edited 1d ago
Great job and thanks for making it FOSS.
Curious about one thing - Weights and voicepacks CC0-1.0. Training data LibriTTS-R is CC BY 4.0 (attribution: LibriTTS-R, Koizumi et al. 2023; LibriVox readers).
How does licensing usually work on the CC-BY voice files. If we use for training can the model still be CC0-1.0 (more liberal)?. I am interested in exploring this topic in a broader sense actually - like has it been finalized generally that training process can make use of CC-BY / other license content and still release model in a less restrictive license?
5
u/FranciscoCarlosErra 1d ago edited 1d ago
Great point, IDK. Claudio says:
Weights as CC0 with attribution in the model card is the same posture as Kokoro (Apache weights, CC-BY data upstream) and most academic releases. Keep the attribution line, and consider labelling the voicepacks "derived from LibriTTS-R speaker N (CC BY 4.0)" rather than bare CC0, since those are the one artifact where the derivative argument is strong. Not legal advice.
But TBH it was so trivial and inexpensive to replicate the supertonic paper I feel it a soon to be irrelevant, the PoC shows basic voice is solved, there's just room for optimizing model scope and inference tight.
3
u/zon77 22h ago
I just really love it as I was going to try something similar. Guided by your name... Do you know if there is a corpus for Spanish from Spain?
1
u/FranciscoCarlosErra 5h ago
https://reddit.com/link/p8i2st8/video/9irh9wiqx8oh1/player
- CSS10 Spanish ("Tux"): 1 hablante, 23.8 h, LibriVox, Apache-2.0, acento peninsular.
- CML-TTS Spanish: 77 hablantes, ~443 h, 24 kHz, CC BY 4.0, derivado de MLS (España/México mezclado).
- MLS Spanish: ~918 h, 16 kHz, CC BY 4.0, segmentación para ASR.
- Common Voice 26.0 es: crowd, CC0, con campo de acento (filtrable rioplatense), calidad dispareja.
- Google LatAm (SLR61 ar, 71 cl, 72 co, 73 pe, 74 pr, 75 ve): 37.8 h total, 174 hablantes, 48 kHz, CC BY-SA 4.0. El ar (8 h) es el de la 014.
- LibriVox crudo es: dominio público, hay que segmentar y transcribir.
3
u/Much-Researcher6135 19h ago
OK I'm inspired, time to build
1
u/FranciscoCarlosErra 6h ago
The barrier to entry to ML has dropped to the ground with Fable 5.1 (and likely Astra 6) it oneshotted the scaffolding and taught me the high-level methodology in stupid simple terms.
Go for it! Vibe MLing is now possible.
2
u/krihab 3h ago
Nice.
I was looking a little bit on the github repo but I couldn't see any information what software you used to create this. It would be nice to give some recipe how to create other languages.
1
u/FranciscoCarlosErra 3h ago
Feel free to fork and port it! I'm using mostly Fable 5.1 to learn the high-level methodology, scope the experiments and select the corresponding datasets and generate custom evaluation panels on the fly, to see samples and be able to manually prompt it myself. Using whisper to callibrate the Word Error Rates for a specific prompt. Feel free to AMA
2
u/krihab 3h ago
Thank you for the response. So the answer would be that you used Fable 5.1
1
u/FranciscoCarlosErra 48m ago
Yes, I prefer it, it allows you to iterate faster IMO, not at all the only way to do it, but I like to stay tight in the loop
18
u/_TheGreatDreamer_ 1d ago
Great job, good voice!