LLM GPU RAM Calculator
Calculate VRAM for any LLM at any quantization level — weights + KV-cache. GPU compatibility matrix and rental prices included.
How to Use
Enter model size
Parameter count in billions (e.g. 7, 13, 70). For MoE models, enter active parameter count.
Set context & batch
Context window in K tokens and concurrent batch size affect KV-cache VRAM — often as large as the weights.
Read the full quantization table
Every quant level shows total VRAM (weights + KV-cache) and which GPUs can run it.
Find the cheapest GPU that fits
The compatibility matrix shows rental prices — sorted so the cheapest option that fits your requirements is first.
How VRAM is calculated
Total VRAM = Weight VRAM + KV-cache VRAM. Weights = params_B × bytes_per_param × 1.08 (overhead). KV-cache = 2 × layers × kv_heads × head_dim × context_tokens × batch × 2 bytes (FP16). Layer and head counts are estimated from parameter count — actual values vary by architecture.
Last verified: 2026-08-19. GPU rental prices from RunPod, Vast.ai, Lambda Labs. See methodology · A100 rental prices · H100 rental prices.
How to Cut This Cost
Use Q4_K_M quantization — reduces VRAM by ~75% vs FP16 with <3% quality loss on most tasks. Fits a 70B model on a single A100 80GB instead of two.
Reduce KV-cache size by lowering max context. At 4K context, KV-cache is often 5–10% of weights. At 128K, it can exceed weights by 3×.
Use spot GPU instances. RunPod Spot and Vast.ai interruptible are 40–70% cheaper than on-demand for batch inference.
FAQ
Have more questions? Contact us