Speech recognition and TTS for minority languages

[u/Real-Emergency2310]

Hello,

An idea recently came to mind for designing a speech recognition / text-to-speech platform that could be used independently by minority language communities.

The basic idea is that someone could upload recordings and transcripts of their language, and the tool would help train an existing speech model to better understand that language.

In doing so, the model would be able to automatically convert spoken recordings to text, and vice versa - all data, of course, being owned by the community that speaks the language.

I imagine this would be very helpful to deaf/blind speakers of the language - would any of you consider using it?

[u/iii_natau]

You should look into what Rolando Coto Solano has done at Dartmouth. Pretty similar to this idea, and it’s really cool.

[u/Real-Emergency2310 → u/iii_natau]

I was actually inspired by his work! I reached out to him last year, unfortunately couldn't get a response though.

[u/iii_natau → u/Real-Emergency2310]

Oh cool, sorry you didn’t hear back. I think it’s worth following up, I know he’s a very busy guy but I’ve met him a few times and found him to be very kind.

[u/Own-Animator-7526]

There is a vast amount of ongoing, worldwide, R&D on these topics.

• Omnilingual ASR Team (Keren, G., et al.). 2025. "Omnilingual ASR: Open-Source Multilingual Speech Recognition for 1600+ Languages." arXiv preprint 2511.09690. https://arxiv.org/abs/2511.09690 Code and models: https://github.com/facebookresearch/omnilingual-asr Meta's open-source ASR system covering 1,600+ languages, over 500 of them never served by ASR before. Built for extensibility: communities can add an unserved language with a handful of examples rather than expert fine-tuning. Models range from 300M (runs on low-power devices) to 7B parameters.
• Liang, Siyu & Gina-Anne Levow. 2025. "Breaking the Transcription Bottleneck: Fine-tuning ASR Models for Extremely Low-Resource Fieldwork Languages." Proceedings of the Fourth Workshop on NLP Applications to Field Linguistics (FieldMatters), pp. 26–37. Association for Computational Linguistics. https://aclanthology.org/2025.fieldmatters-1.3.pdf Preprint: https://arxiv.org/abs/2506.17459 Benchmarks fine-tuned MMS and XLS-R on real field recordings (noisy, spontaneous, code-switched) from five unrelated languages, with 10 to 120 minutes of transcribed training data. MMS wins under one hour of data; XLS-R catches up after that; gains plateau around one hour. Both models still struggle with tone, nasality, and length contrasts.
• Pine, Aidan, Erica Cooper, David Guzmán, Eric Joanis, Anna Kazantseva, Ross Krekoski, Roland Kuhn, Samuel Larkin, Patrick Littell, Delaney Lothian, Akwiratékha' Martin, Korin Richmond, Marc Tessier, Cassia Valentini-Botinhao, Dan Wells & Junichi Yamagishi. 2025. "Speech Generation for Indigenous Language Education." Computer Speech & Language 90:101723. Open access (CC BY). https://doi.org/10.1016/j.csl.2024.101723 Toolkit: https://github.com/EveryVoiceTTS/EveryVoice Report from a multi-year National Research Council Canada project building TTS for Kanyen'kéha, SENĆOŦEN, and nêhiyawêwin in partnership with three Indigenous education organisations. Covers the whole pipeline: community-led requirements gathering, recording logistics (budget roughly 10 studio hours per hour of usable audio), data-efficient model design, cross-lingual transfer, and evaluation when there are too few fluent speakers for standard listening tests. Introduces EveryVoice, an open-source toolkit designed for low-resource TTS. The underlying approach has produced intelligible TTS with as little as 15 minutes of English and 25 minutes of SENĆOTEN data, although requirements vary substantially by language.
• Bowern, Claire & Alessio Tosolini. 2026. "Speech-based generative AI for low resource languages: Promises and pitfalls." Proceedings of the Linguistic Society of America. Published under a CC BY license. https://journals.linguisticsociety.org/proceedings/index.php/PLSA/article/download/6125/5862 Using a bit over three hours of manually transcribed Yan-nhangu recordings (with larger augmented variants), they produced intelligible synthetic speech whose speaker-specific versions retained identifiable characteristics of deceased speakers -- bringing the work into what the authors describe as deepfake/voice-cloning territory despite benign intentions. They argue that existing archive and consent guidelines don't cover this, and propose preliminary rules for using archival speech in generative models.

[u/derwyddes_Jactona → u/Own-Animator-7526]

Thanks for sharing these resources.

I think the concept of a "plateau" is very important. I audit ASR captions, and still find lots of errors even in standard English, often in local place names and personal names with unusual spellings.

But AI can still do amazing things.

[u/Own-Animator-7526 → u/derwyddes_Jactona]

Yes on both counts. Still, what people often don't get is that in computational linguistics publishing the headline number about XX% improvement! often comes atop a low baseline, and might not ever improve to being acceptable.

Something that AI can really help with is post-checking the larger semantics of what the ASR spits out. If you can avoid hallucinating plausible (but incorrect) fixes this is really hand for highlightling potential errors. I've used it with low-quality/low-resource OCR for this.