LLM VRAM Calculator
How much GPU memory does that model actually need? Enter any size and quantization — we compute weights, KV cache, and overhead, then show which GPUs can run it.
Total VRAM needed
23.1 GB
■ weights 21.2GB ■ KV cache 0.7GB ■ overhead 1.2GB
4 GPUs in our database can run this fully in VRAM.
VRAM questions, answered
How is LLM VRAM usage calculated?+
Three parts: model weights (parameters × bits per weight ÷ 8 — a 8B model at 4-bit quantization is ~4.9GB), the KV cache (grows linearly with context length and depends on the model architecture), and roughly 1–1.5GB of runtime overhead for the CUDA context and compute buffers.
How much VRAM does a 70B model need?+
About 44GB at Q4 quantization with 8k context — more than any single consumer GPU. Practical options are two 24GB cards, a Mac with 64GB+ unified memory, or partial offloading to system RAM at a large speed cost.
What is quantization and how much quality do I lose at Q4?+
Quantization stores weights in fewer bits. Q4_K_M (about 4.85 bits per weight) roughly halves VRAM versus Q8 with quality loss most people cannot detect in normal use. It is the community default; go Q8 only if you have VRAM to spare, and FP16 basically only for fine-tuning.
Why do MoE models need so much VRAM if they are fast?+
Mixture-of-Experts models keep every expert in memory but only route each token through a few of them. VRAM requirements follow TOTAL parameters, while speed follows ACTIVE parameters — a 35B-A3B model needs the memory of a 35B but generates like a 3B.
Does context length really matter for VRAM?+
Yes — the KV cache grows linearly with context. On a typical 8B model, going from 8k to 128k context adds roughly 2GB; on a 70B it adds over 5GB. If you plan to feed long documents or codebases, budget VRAM for the context, not just the weights.
Skip the math
We already computed requirements for every major open model — Llama, Qwen, DeepSeek, Gemma, FLUX, and 30+ more — with per-GPU verdicts and speed estimates.
Browse model requirements →Ready to buy? See every current GPU ranked by VRAM and AI value, from budget 12GB cards to the RTX 5090.
Best GPUs for AI →