Local AI
GPU for AI Buying Advisor
Choose a GPU by checking whether your model, quantization, context window, and sessions fit in VRAM before you spend the budget.
When to use this
Use it before buying a GPU for local models
This advisor is for Ollama, LM Studio, Open WebUI, llama.cpp, local coding assistants, RAG boxes, and homelab AI workstations where VRAM fit matters more than marketing tier names.
Default result
The server-rendered 32B Q4 example needs 24.2 GB and selects NVIDIA GeForce RTX 5090 with status: Fits under budget.
GPU fit table
This table updates from the same inputs as the recommendation. It shows memory fit, current planning price, and estimated monthly power cost at the entered hours and electricity rate.
| GPU | VRAM | Price | Fit | Power cost |
|---|---|---|---|---|
| NVIDIA GeForce RTX 3060 12GB | 12 GB | $250 | Short by 12.2 GB | $3.79 |
| NVIDIA GeForce RTX 5080 | 16 GB | $999 | Short by 8.2 GB | $8.02 |
| NVIDIA GeForce RTX 4090 | 24 GB | $1,600 | Short by 0.2 GB | $10.02 |
| NVIDIA GeForce RTX 5090 | 32 GB | $1,999 | Fits by 7.8 GB | $12.81 |
| Custom or dual-GPU 48GB plan | 48 GB | $3,500 | Fits by 23.8 GB | $13.36 |
How the GPU advisor estimates VRAM
The calculator estimates weight memory from model parameters and quantization, adds a KV cache estimate from context length and active sessions, adds reserve VRAM, then applies a safety margin. The result is a planning estimate, not a runtime guarantee.
Why context length changes the answer
Long context increases KV cache memory. A model that fits at 4K or 8K context can spill at 32K context, especially with multiple active sessions or a display using the same GPU.
What to do next
After the GPU screen, check PSU wattage, case clearance, system RAM, local runtime support, and current quotes. Then test the exact quant you plan to run before the return window closes.
SSR example
Default math: 18 GB weights + 2 GB KV cache + 2 GB reserve, then 10% safety = 24.2 GB.
Worked example: 32B Q4 coding model with 8K context
The default scenario targets a 32B model with Q4 planning at 4.5 effective bits per parameter. Weight memory is 32 x 4.5 / 8 = 18.0 GB.
The 30B to 40B architecture profile uses 64 layers, 8 KV heads, 128 head dimension, FP16 KV cache, 8,192 context tokens, and 1 active session. KV cache is about 2.0 GB.
Adding 2.0 GB reserve gives 22.0 GB before safety. With a 10% safety margin, required VRAM is 22.0 x 1.10 = 24.2 GB.
An RTX 4090 at 24 GB misses that safety-adjusted target by about 0.2 GB, while the RTX 5090 at 32 GB fits with about 7.8 GB headroom. At the $1,999 planning price and a $2,200 budget, the selected headroom is $201.
Reference data used by the defaults
| Topic | Reference value | Source | Date | Note |
|---|---|---|---|---|
| GeForce RTX 5090 | 21,760 CUDA cores, 32 GB GDDR7, 575 W total graphics power. | NVIDIA RTX 5090 specifications | As of June 20, 2026 | NVIDIA says add-in-card specifications and power requirements can vary by manufacturer and system configuration. |
| GeForce RTX 5080 | 10,752 CUDA cores, 16 GB GDDR7, 360 W total graphics power. | NVIDIA RTX 5080 specifications | As of June 20, 2026 | Use exact vendor model specs before purchase. |
| GeForce RTX 4090 | 16,384 CUDA cores, 24 GB GDDR6X, 450 W total graphics power. | NVIDIA RTX 4090 specifications | As of June 20, 2026 | Used and partner-card pricing varies widely; replace the planning price with a live quote. |
| GeForce RTX 3060 | RTX 3060 variants include 12 GB and 8 GB memory configurations; this calculator uses the 12 GB planning row with 170 W GPU power. | NVIDIA RTX 3060 specifications | As of June 20, 2026 | Verify that the exact card is the 12 GB version before buying for local AI. |
| Electricity rate default | $0.1856 per kWh default, matching 18.56 cents per kWh from the U.S. residential average used in the EIA Electric Power Monthly for March 2026. | EIA Electric Power Monthly | As of June 20, 2026 | Power cost is only an operating-cost estimate; it is not a performance benchmark. |
GPU prices are intentionally editable planning assumptions. Replace them with current quotes before buying.
FAQ
Is this a benchmark calculator?
No. It is a buying screen for memory fit, budget, and power. Tokens per second depends on model architecture, backend, drivers, CPU, RAM, batch size, quantization kernel, and thermal limits.
Why can a 24 GB GPU fail a 24.2 GB estimate?
VRAM capacity is a hard boundary. If the estimate is near the card limit, drivers, display use, KV cache growth, or runtime overhead can force CPU/RAM offload. Keep headroom instead of buying at the exact edge.
Should I buy more VRAM or a faster GPU?
For local LLMs, buy enough VRAM first. A faster GPU that cannot hold the model and context will spill to system RAM and can feel much slower than a lower-tier card with enough memory.
How should I treat the prices?
Treat them as editable planning numbers. GPU prices move by region, vendor, stock, used condition, warranty, tax, and bundles. Enter the actual quote before using the recommendation.
Does multi-GPU memory add together automatically?
Only for runtimes and model layouts that shard the model effectively. Many local workflows are simplest on one large-VRAM GPU. Use the custom 48 GB row only when you know your stack can use it.
What else should I check before buying?
Check PSU wattage, power connectors, case clearance, cooling, motherboard slot spacing, system RAM, driver support, return policy, and whether your target model has a known good local quant.
Decision path
What to do next
Check the exact machine
Use the hardware checker for VRAM, RAM, context, quantization, and expected local fit.
Power supplySize the PSU
A GPU that fits the model still needs a power supply, cables, and cooling that fit the build.
Cost tradeoffCompare local and cloud
Add API usage, hardware cost, power, maintenance, and resale before buying the card.
Topic hubOpen local AI
Move from GPU choice into model fit, setup guides, memory planning, and private AI workflow decisions.