AI Made Crystal Clear · chapter 3: How an LLM Works, With No Math
The prediction loop
2026-09-08
The model reads everything so far, ranks every possible next token, picks one likely candidate (the temperature dial shapes that pick), adds it to the text, and starts over. A long answer is this lap run thousands of times.
Below: the paragraph from the book that builds this idea, then the diagram itself (Figure 3.1), and a recap. About a minute of reading.
Watch the odds shift in a real example from Google's introduction. Given the unfinished sentence "When I hear rain on my roof, I ___", the model ranks every possible continuation: "cook soup" sits near the top of the list, "warm up a kettle" a bit behind it. Add an implausible ending of our own, "sell my house", and it lands far down in the noise. The model picks one likely continuation, adds it to the sentence, and then does the whole ranking again for the next chunk. This figure shows that loop, the only moving part in the machine.
Recap
- The idea: The model reads everything so far, ranks every possible next token, picks one likely candidate (the temperature dial shapes that pick), adds it to the text, and starts over.
- The picture: Figure 3.1, from chapter 3 ("How an LLM Works, With No Math") of AI Made Crystal Clear.
- Go deeper: the chapter builds this step by step, with recipes and sources at the end.
This diagram is one of many in AI Made Crystal Clear.
Every chapter opens with the gist, draws the hard ideas, and ends with recipes and sources.
Get the book
