Do I need a huge context window for RAG?

No. You need retrieval quality. Huge windows are a fallback, not an architecture.

Should the generator be a reasoning model?

Only if questions are multi-hop and you can afford the tokens. Most FAQ RAG should use a fast cheap model.

How do I cut RAG cost?

Fewer chunks, smaller embeddings, cheaper generator, caching. See cost optimization.

Which model cites best?

Measure citation precision on your corpus. Use quality index only to pick candidates.

Best LLM for RAG — retrieval-augmented generation

RAG quality is mostly retrieval. The LLM still has to attend to chunks, refuse when chunks are irrelevant, and cite them. The best LLM for RAG is cheap enough to stuff useful context and disciplined enough not to ignore it.

Input price and context window dominate RAG bills. Rank with cost and context weights, then test “answer only from the passages” prompts. A slightly weaker model that follows that instruction beats a flagship that riffs from pretraining.

Longest context windows right now

Live OpenRouter pricing and Arena quality, cached about an hour. Not a static blog table.

Model	Provider	Blend $ / 1M	Quality	Context
Grok 4.20 Multi-Agent	xAI	1.88	—	2,000,000
Grok 4.20	xAI	1.88	—	2,000,000
Auto Router (Beta)	OpenRouter	—	—	2,000,000
Pareto Code Router	OpenRouter	—	—	2,000,000
Auto Router	OpenRouter	—	—	2,000,000

Lost in the middle

Models skip content in the center of long prompts. Put the question last, keep top chunks first, and do not paste 50 equally “relevant” passages. A 32k model with 8 good chunks often beats a 1M model with 200 noisy ones.

Citations

Ask for passage ids. Score citation precision. Higher quality index helps but is not sufficient. Smaller models can be fine if the prompt is strict.

Rerank vs bigger LLM

Spending money on a reranker and a cheap generator is a classic win. Compare the cost of a flagship generator to rerank + flash model using the $ / 1M tables.

Multimodal RAG

PDFs with figures need vision or a captioning step. See multimodal and vision pricing guides. Text-only Elo will not rank this.

Ops

RAG systems fail on stale indexes more than on model choice. Pick a default model you can swap. This site’s compare URLs are shareable so you can change the generator without rewriting the design doc.

FAQ

Do I need a huge context window for RAG?: No. You need retrieval quality. Huge windows are a fallback, not an architecture.
Should the generator be a reasoning model?: Only if questions are multi-hop and you can afford the tokens. Most FAQ RAG should use a fast cheap model.
How do I cut RAG cost?: Fewer chunks, smaller embeddings, cheaper generator, caching. See cost optimization.
Which model cites best?: Measure citation precision on your corpus. Use quality index only to pick candidates.

best LLM for RAGbest model for retrieval augmented generationRAG LLM comparisonLLM for document QA

Rank RAG-oriented models