Reasoning models explained — extra thinking, extra tokens
Reasoning models spend more compute (and often more billed tokens) before they answer. They help on multi-step math, hard coding, and planning. They waste money on “what’s the weather in the policy docs.”
Compare reasoning SKUs to each other and to a strong non-reasoning flagship on cost per correct answer, not cost per million tokens alone. See also o-series vs Claude.
Hidden tokens
Some APIs bill a chain of thought you never see. Your observability must log billed tokens, not just visible output. Otherwise the “cheap” reasoning row is a surprise.
When they win
Problems with a checkable answer: code that must pass tests, math, structured plans. When the answer is taste (“make this landing page friendlier”), extra thinking is a slow thesaurus.
Latency
Users wait for the thinking. For chat, consider a fast draft plus an optional “think harder” button that calls the reasoning SKU.
Arena Elo distortion
Crowd voters may reward long, careful answers — or punish slowness. Don’t overfit quality index when choosing a reasoning default.
Architecture pattern
Classifier (tiny) → solver (reasoning) → narrator (cheap). Most graphs should not be 100% reasoning models. Use the leaderboard to pick each node’s id.
FAQ
Are reasoning models smarter?
- They are better at some multi-step tasks and worse as a universal default. Measure.
Why is the bill so high?
- Thinking tokens and retries. Cap reasoning effort if the API allows it.
Can open models reason?
- Some distilled reasoners exist. Compare their quality index and then run your hard eval.
Should I use them for RAG?
- Only for multi-hop questions. FAQ RAG should stay on a fast model.
Related guides
Also try: LLM leaderboard, compare tool, live arena.