How do LLMs work? One loop: read, rank, pick, append
By Christophe Paka · Facts checked on 23 October 2026
You have heard that ChatGPT is just autocomplete a hundred times. Then it writes a clean report, and the claim stops making sense. Does it search the web and average the results? Does it know facts? Nobody gives a straight answer.
This matters, because a wrong picture leads to the wrong trust. So how do LLMs work? A large language model does one thing: it predicts the next small piece of text, adds it, and repeats. The figure below draws that loop, the only moving part in the machine, and shows what is missing from it.
How to read this figure
Step 1: read the context. The model reads everything in the chat so far. Each word shifts the odds of the next one, the way learning that a stranger plays professional basketball changes your guess of his height.
Step 2: rank the candidates. The model scores every possible next token. A token is the small piece of text a model reads and writes, about three quarters of a word. In the figure, cook soup gets the longest bar and warm up a kettle a shorter one. Sell my house lands far down the list.
Read the bars in the middle. The longest bar is the likeliest ending, not the true one. The model has a ranked hunch, built from patterns in the text it read during training, not a list of facts.
Step 3: pick one token. The model chooses a likely candidate, not always the top one. A setting called temperature acts here. Low temperature means it almost always takes the longest bar; higher temperature means it sometimes takes a shorter one. That is why the same question can get different answers.
Step 4: append and go again. The model adds the token and runs the lap again. A long answer is this lap run thousands of times. Notice what is not in the loop: no plan, no lookup of facts, no step that checks truth.
Key takeaways
- When someone calls it a search engine, explain the loop: read, rank, pick, append, with no lookup and no truth check.
- Paste the facts you need into the chat, because the loop ranks likely text, not true text.
- Ask the same question in a fresh chat before trusting a single answer.
Worked example: the return window the model guessed
A support rep at Crystal Goods, the book's example shop, asks a plain chat app with nothing pasted in: Crystal Goods customers can return items within blank.
- Rank: 30 days sits near the top, because it is the most common return window in public text. 14 days and 60 days follow.
- The true answer, 45 days, sits low on the list, because the shop's own policy is in no public text the model read.
- Pick: the model takes 30 days and writes a fluent, friendly sentence around it.
Result: A confident, generic, wrong answer. The loop worked as designed: its job is likely text. The fix is to paste the policy into the chat, so the right answer becomes the likely one.
Questions readers ask
Is an LLM just autocomplete?
In one sense, yes: it ranks the likely next token and adds it, again and again. But it ranks with patterns learned from a whole library of text, and it runs that step thousands of times per answer. That is why a simple step produces long, fluent writing.
If I ask how long to boil spaghetti, does it search the web and average?
No. A plain model has no lookup step. It writes the likely continuation of cooking text, which usually matches the box. The loop is right when the likely answer and the true answer agree, and wrong when they do not.
Why do I get a different answer when I ask again?
Step 3 picks a likely token, not always the top one. The temperature dial shapes that choice, so two runs can take different paths. Asking twice in fresh chats is a quick test of how sure the answer is.
Sources: Google ML Crash Course: Introduction to Large Language Models; But what is a GPT? (3Blue1Brown).
This figure is from AI Made Crystal Clear, chapter 3.
The chapter builds the idea step by step, with recipes you can follow, a Try it to check yourself, and every source.
Get the book
