2026 · Malegado · AI lead: models, pipeline, voice
Real-time voice conversation in 590 languages
Malegado teaches languages most AI products ignore. I built the AI inside it: text-to-text models for conversation and lessons, text-to-speech voices, and a streaming talk-back loop so a learner can speak and be answered out loud. The pipeline covers all 590 supported languages and new ones are still being added.
- built with
- PyTorchText-to-speechSpeech-to-textStreaming inferenceFastAPIMobile
- 590+
- languages supported
- [to confirm] ms
- talk-back round trip
- 2
- model families
all through one training and serving pipeline
speech in to speech out, representative
text-to-text and text-to-speech
Speak, and be answered out loud.
A replay of a real exchange. The stage indicator shows which part of the loop is active: speech in, speech to text, the text model, text to speech.
ready
replay
The pipeline, in 6 stages.
- 01
Collect
Phrase, lesson and recorded-speech data per language. Many are low-resource, with little clean text anywhere online.
- 02
Clean and align
Normalise scripts, tones and diacritics. Align audio to text so the speech models learn from matched pairs.
- 03
Tokenise
[to confirm]: tokeniser. Shared across scripts so one model family covers hundreds of languages.
- 04
Train
Fine-tune text-to-text models for conversation and lesson generation. Train text-to-speech voices. [to confirm]: base models.
- 05
Evaluate
Held-out sets per language. Native-speaker review for how natural the speech sounds.
- 06
Serve
Streaming speech-to-text, text model, text-to-speech loop, tuned for a conversational round trip on a phone.
The problem
Malegado teaches Hausa, Yoruba, Swahili, Igbo, Amharic, Kikuyu and hundreds more alongside French, Spanish and English. For most of those languages there is no commercial voice AI to buy and very little clean data to train on. If the app was going to talk back to a learner, we had to build the models ourselves.
My role
I led the AI work. I trained the text-to-text models that power the chatbots and lesson generation, trained the text-to-speech voices, and built the real-time talk-back feature, where a learner speaks and the app answers out loud in the language they are studying. The AI department that now reports to me maintains the pipeline and keeps adding languages.
The pipeline
Every one of the 590 supported languages goes through the same six stages. We collect phrases, lessons and recorded speech. We clean and normalise scripts, tones and diacritics, and align audio to text. A shared tokeniser lets one model family cover many scripts ([to confirm]: which tokeniser). We fine-tune the text models and train the voices ([to confirm]: base models), evaluate on held-out sets per language with native speakers reviewing the speech, and serve everything behind a streaming API the mobile app calls.
The reason it is one pipeline and not 590 projects is scale. A new language is a new dataset dropped into the same stages, not a new engineering effort.
Talk-back, how it works
The learner speaks. Speech-to-text turns that into text while they are still talking. The text model drafts a reply in the target language, and text-to-speech starts voicing the reply before the whole sentence is finished. Each stage streams into the next, so the pause before the app answers is short enough to feel like a conversation rather than a form submission. Representative round trip: [to confirm] ms.
The hard part is not the loop, it is the voices. A text-to-speech model for a tonal, low-resource language has to get tone right from very little audio, or it teaches the learner the wrong word. That is where most of the evaluation effort went.
What I watch in production
Per-language quality, because a regression in one small language is invisible in the averages. Latency percentiles for the talk-back loop, not the mean. And data growth per language, which tells us which voices are due for retraining.
Status
Live on iOS and Android. Languages are still being added.