When the original Coqui company shut down in 2024, it left behind something unusual: a full zoo of text-to-speech architectures — fourteen acoustic models, a shelf of vocoders, speaker encoders, voice conversion — with pretrained checkpoints for over a thousand languages. Most corporate deaths end with an archived repo; this one got a second life, because the Idiap Research Institute maintains this fork under the PyPI name coqui-tts, and it has become the working distribution the community actually installs. The reason to read its source is not sentiment but range: it is the one codebase where you can watch the entire history of neural TTS in a single package — the 2017-era Tacotron attention models, flow-based Glow-TTS, adversarial VITS, feed-forward FastSpeech — and then the modern chapter, XTTSv2, a GPT-style transformer that clones a voice from a six-second clip and streams audio with sub-200-millisecond latency. One pip install coqui-tts reaches all of it. As always in this series, what follows is an educational tour of published source code.

The architecture is a public API over a deep model registry. At the surface sits the TTS class in api.py — give it a model name and text, get a waveform — backed by the Synthesizer, which loads any checkpoint and runs inference, and a model manager that resolves names like tts_models/en/ljspeech/tacotron2-DDC against a registry, downloads the files, and caches them. Below that, the models: every acoustic model inherits a common base that owns formatting, batching, and the training loop glue, while XTTS breaks the mold with its own stack — a discrete VAE that compresses a reference voice into latents, a perceiver encoder, a GPT transformer that generates audio code tokens, and a HiFi-GAN decoder that renders them to mel spectrograms. Around the edges live the vocoders that turn spectrograms into waveforms, the speaker encoder that turns voices into vectors, and the training scripts that make the whole thing reproducible. MPL-2.0-licensed, it remains the widest spanning open TTS codebase in active maintenance.

Coqui TTS at a glance: the TTS class drives the Synthesizer and resolves models through the manager, XTTSv2 generates through its GPT core while VITS and the FastSpeech family cover the classic zoo, vocoders and the speaker encoder serve the audio stack with FreeVC, and training scripts plus a demo server round out operations.

Reading the overview from left to right:

• The TTS class is the one-liner API every tutorial starts with.
• The Synthesizer loads checkpoints and runs the text-to-waveform pass.
• The model manager resolves model names, downloads and caches them.
• XTTSv2 clones a voice from seconds of reference audio in 17 languages.
• VITS is the end-to-end adversarial workhorse of the zoo.
• The forward family shares code across FastSpeech and FastPitch variants.
• The GPT core turns text plus voice latents into audio tokens.
• The vocoder family renders spectrograms into waveforms.

Why You Need This

The first reason is XTTSv2, because it is where text-to-speech became language-model-shaped. xtts.py orchestrates the flow: a reference clip is compressed by the discrete VAE into audio latents, the perceiver encoder condenses those into a fixed set of conditioning tokens, and the text goes through the multilingual tokenizer before the GPT transformer generates audio code tokens one at a time — exactly like an LLM generates text, except the vocabulary is sound. The inference wrapper adds KV-cache generation, the stream generator yields tokens as they arrive so playback starts before synthesis finishes, and the HiFi-GAN decoder renders the result. Speaker management and the on-disk clone cache make repeated synthesis with the same cloned voice cheap.

The second reason is the classic zoo, because almost every architecture that mattered is here in trainable form. base_tts.py defines the shared skeleton — batch formatting, sequence sorting, the format-batch contract that every model implements — and on top of it you can read the field’s history: vits.py at fourteen hundred lines is the end-to-end conditional variational approach where alignment, duration, and waveform are learned jointly with adversarial losses; forward_tts.py is the fast feed-forward family where an external alignment predicts durations; glow_tts.py is the flow-based design with monotonic alignment search; and tacotron.py keeps the attention era alive. Every model is paired with a config class in tts/configs, and the recipes directory shows real training runs for real datasets — the reason this fork, not the archived original, is where fine-tuning happens.

The third reason is the audio stack that surrounds the acoustic models, because mel spectrograms are not sound. The GAN vocoder framework wires generators against discriminators — with the HiFi-GAN generator as the default and WaveRNN, WaveGrad, and UnivNet alternatives in the same shelf — so a trained acoustic model snaps onto any trained vocoder. The voice side is served by the speaker encoder, LSTM and ResNet models that map an utterance to a speaker vector used for similarity scoring and multi-speaker conditioning. FreeVC handles voice conversion directly, and the fork’s news log shows the direction: OpenVoice and kNN-VC adapters, fairseq-MMS models covering roughly eleven hundred languages, and the newer voice-cloning cache. The support layer under it all — the audio processor with its mels, normalization, and Griffin-Lim, plus the resampling and silence-trimming CLI tools — is the unglamorous half of why outputs sound clean.

The detail view: the API layer with synthesizer, manager and server; the acoustic zoo from BaseTTS through VITS and Tortoise; XTTS internals from discrete VAE to stream generator; the vocoder and speaker stack; the operations scripts; and the support layer of processors, caches and configs.

The detail view shows how thoroughly the base class pays off. The training entry point is barely sixty lines because everything hard lives in the base model and the external trainer package — adding a new architecture means implementing format-batch and forward, not rewriting the loop. The vocoder trainer mirrors that brevity for the GAN family. On the synthesis side, the CLI tool batch-renders text files through the Synthesizer with speaker and language selection, and the embeddings script precomputes speaker latents for datasets and cloning. The torch.hub entry in hubconf.py loads models without the package, and the small Flask demo server in server.py predates the Gradio era — a reminder of how the ecosystem’s serving conventions moved on while the model code kept its shape.

From Install to a Cloned Voice

The quickstart is two steps. pip install coqui-tts, then either the CLI — tts --text "hello" --model_name tts_models/en/ljspeech/glow-tts --out_path out.wav — or three lines of Python: import TTS, instantiate with a model name, call tts.tts(text) or tts.tts_to_file(). The manager downloads the checkpoint on first use and caches it. Voice cloning takes one more argument: point the XTTS model at a speaker wav, and the latents are computed, cached, and reused for every later call in that language. Fine-tuning means picking a recipe, editing its config, and running the train script against your dataset. One caution: the first XTTS download is heavy, GPU memory matters for the larger models, and phonemization quality for low-resource languages depends on external espeak data that is easy to mis-install. The docs are candid that results vary by language — the model list is long, but the quality tier is not uniform across it.

Honest limits: the zoo’s breadth is also its maintenance burden — the older architectures are kept compiling and usable but see no new research, and some combinations of model, vocoder, and language produce noticeably dated audio. XTTS weights are gated behind a license acceptance and are non-commercial, which matters more than the MPL code license when shipping. Streaming latency claims assume a decent GPU; on CPU, real-time factors are honest but not snappy. The training stack depends on the separate coqui-tts-trainer package, and recipe configs encode dataset quirks that do not transfer untouched to new data. None of that changes the verdict: for open-weight, self-hosted multilingual speech with voice cloning, this is still the library everything else gets compared to.

Enjoyed this post? Never miss out on future posts by following us