The two-stage neural TTS pipeline

Only then does synthesis happen, in two stages. In NVIDIA's words: "a synthesis network transforms the text into time-aligned features, such as a spectrogram", and then "a vocoder network converts the time-aligned features into audio waveforms". The first network is the acoustic model. Its output, the mel spectrogram, is a compact map of pitch, tone, and timing: which sounds happen when, at what frequency, and how loud. The second network, the vocoder, turns that map into the actual waveform your speakers play. Figure 2.1 lays the whole path out.