Best LLM for summarization — long docs without losing facts
Summarization looks easy and fails on facts: dropped numbers, invented decisions, wrong speakers. The best LLM for it has enough context to see the source and a price that lets you summarize everything, not just the important docs.
Rank by context and cost together. A slightly “dumber” model that sees the whole transcript beats a flagship that only sees the first 8k tokens. Then check faithfulness with a quiz on the source.
Longest context windows right now
Live OpenRouter pricing and Arena quality, cached about an hour. Not a static blog table.
| Model | Provider | Blend $ / 1M | Quality | Context |
|---|---|---|---|---|
| Grok 4.20 Multi-Agent | xAI | 1.88 | — | 2,000,000 |
| Grok 4.20 | xAI | 1.88 | — | 2,000,000 |
| Auto Router (Beta) | OpenRouter | — | — | 2,000,000 |
| Pareto Code Router | OpenRouter | — | — | 2,000,000 |
| Auto Router | OpenRouter | — | — | 2,000,000 |
Map-reduce vs one shot
If the doc fits in context with room for instructions, one-shot is simpler. If it does not, chunk and merge. Map-reduce uses more output tokens and can drift; one-shot uses more input tokens. Price both patterns with the monthly cost table.
Faithfulness tests
Ask the model to list every dollar amount or every action item and compare to the source. Models with higher quality index still hallucinate under compression. Cheap models plus extractive first-pass (quotes only) can beat abstractive flagships.
Meetings vs documents
Transcripts are noisy. You may want a model that follows a rigid JSON schema for action items. That is instruction following more than “summarization talent.” Test schema validity in the arena.
Multilingual sources
Summarizing a language you don’t read requires a multilingual model. Do not pick from English Elo alone.
Default shortlist
Cheapest model that fits the 95th-percentile document. Escalate to a quality leader only when the summary is customer-facing or legal. The live context table on this page is the starting catalog view.
FAQ
Should I use the longest-context model?
- If it is also cheap enough, yes. If it is expensive, chunk. See the longest-context guide.
Why do summaries invent facts?
- Compression encourages guessing. Require quotes, or summarize extractively first.
Is a small model enough?
- Often, for internal notes. Measure faithfulness before you standardize.
How do I estimate cost?
- Tokens ≈ words × 1.3 for English, then use the model’s $ / 1M on its detail page.
Related guides
Also try: LLM leaderboard, compare tool, live arena.