ElevenLabs Made Crystal Clear · chapter 2: How AI Voice Works: Core Concepts and the Big Picture
The two-stage neural TTS pipeline
2026-09-01
Text is cleaned and analyzed, then the acoustic model writes a mel spectrogram and the vocoder converts it into a waveform.
Below: the paragraph from the book that builds this idea, then the diagram itself (Figure 2.1), and a recap. About a minute of reading.
Only then does synthesis happen, in two stages. In NVIDIA's words: "a synthesis network transforms the text into time-aligned features, such as a spectrogram", and then "a vocoder network converts the time-aligned features into audio waveforms". The first network is the acoustic model. Its output, the mel spectrogram, is a compact map of pitch, tone, and timing: which sounds happen when, at what frequency, and how loud. The second network, the vocoder, turns that map into the actual waveform your speakers play. Figure 2.1 lays the whole path out.
Recap
- The idea: Text is cleaned and analyzed, then the acoustic model writes a mel spectrogram and the vocoder converts it into a waveform.
- The picture: Figure 2.1, from chapter 2 ("How AI Voice Works: Core Concepts and the Big Picture") of ElevenLabs Made Crystal Clear.
- Go deeper: the chapter builds this step by step, with recipes and sources at the end.
This diagram is one of many in ElevenLabs Made Crystal Clear.
Every chapter opens with the gist, draws the hard ideas, and ends with recipes and sources.
Get the book
