Gemini Made Crystal Clear · chapter 2: Which AI, and Which Gemini
What a benchmark measures versus what you need
2026-09-15
One engineer's own structured evaluation of Gemini Pro found 95% accuracy in documentation generation and a large nominal context window, and also found the model struggled to navigate a large, evolving codebase without dedicated IDE tooling. Both findings are true about the same model at the same time.
Below: the paragraph from the book that builds this idea, then the diagram itself (Figure 2.1), and a recap. About a minute of reading.
This is also where a real gap opens up between what a benchmark measures and what a production task actually needs, and it is worth seeing that gap drawn out once, plainly, before you trust a benchmark result again.
Recap
- The idea: One engineer's own structured evaluation of Gemini Pro found 95% accuracy in documentation generation and a large nominal context window, and also found the model struggled to navigate a large, evolving codebase without dedicated IDE tooling.
- The picture: Figure 2.1, from chapter 2 ("Which AI, and Which Gemini") of Gemini Made Crystal Clear.
- Go deeper: the chapter builds this step by step, with recipes and sources at the end.
This diagram is one of many in Gemini Made Crystal Clear.
Every chapter opens with the gist, draws the hard ideas, and ends with recipes and sources.
Get the book
