Can I run this AI model?
Estimate weights, context cache and overhead against your memory budget. See the assumptions, then verify the exact setup.
Methodology: from inputs to a memory estimate
The calculator runs in your browser. Your device is not scanned and no model is downloaded or tested. You supply usable memory budgets; model presets supply nominal parameter counts and a cited dense-decoder cache profile. All results are calculated estimates.
1. Weight payload and quantization allowance
Weight bytes = parameters in billions × 10⁹ × nominal bits ÷ 8 × (1 + format allowance / 100)
The allowance is shown separately, so four-bit payload is not confused with the complete weight estimate. The Q4 default is 4.5 effective bits per weight, based on a 144-byte Q4_K block for 256 weights; mixed Q4_K_M files can be larger. Q8_0 uses 34 bytes per 32 weights, or 8.5 effective bits. FP16/BF16 uses 16 bits, not full FP32 precision. Higher-precision tensors and actual file contents can differ from this approximation. ggml format definitions.
2. Context and KV cache
FP16 cache bytes = 2 × layers × KV heads × head dimension × tokens per session × 2 bytes × sessions
Use the context allocated by your runtime, including prompt and generated tokens. The first factor of two represents keys and values. Query-head count is not substituted for KV-head count. The Q8_0 and Q4_0 cache scenarios include block scales (34 and 18 bytes per 32 values) and round each row up to a complete block. They do not assert that your backend supports quantized caches. Real allocation can include extra padding, reserved context or different cache layouts. Cache strategies and limitations.
3. Runtime reserve and safety
Total bytes = (weight payload + weight format allowance + active KV cache + runtime reserve) × (1 + safety margin / 100)
The runtime reserve is your allowance for activations, buffers and other allocations. The safety margin is applied once to that subtotal, and its amount appears in the breakdown. Neither is a measured guarantee. OS and other application use should already be excluded from your usable memory input.
4. RAM, VRAM and unified memory
Dedicated mode compares the full estimate with one VRAM budget, then separately with RAM as a full-RAM scenario. It never sums them and does not solve CPU/GPU layer placement, cache offload or model-loading peaks. Unified mode compares against one usable shared pool exactly once. OS/backend GPU allocation limits may be lower than installed unified memory. CPU mode checks RAM only. Multi-GPU pools are not automatically interchangeable either.
A result is “tight” when headroom is non-negative but below the greater of 1 GiB or 10% of the total. Exact equality is tight, not comfortable. This transparent screening rule is separate from the safety allowance and is not an experimentally certified boundary.
Worked example: the editable 8B preset
For 8 billion parameters at nominal 4-bit with 12.5% format allowance, the weight estimate is 4.5 billion bytes: 4.19 GiB, not 4.5 GiB. A Llama 3.1 8B profile at 8,192 tokens and FP16 cache gives 1.00 GiB. Add 1.5 GiB runtime reserve and 10% safety: 7.36 GiB total. This leaves about 0.64 GiB within the example 8 GiB usable budget; it is still a tight estimate, not a successful runtime test.
Profiles and primary sources
Reviewed . These profiles are specific to the named versions, not universal profiles for models with a similar parameter count. A configured context limit is not automatically the runtime's default or proof of support for every extension.
| Profile / source | Layers | Query / KV heads | Head dimension | Context reference |
|---|---|---|---|---|
| Llama 3.1 8B | 32 | 32 / 8 | 128 | 131,072 |
| Llama 3.1 70B | 80 | 64 / 8 | 128 | 131,072 |
| Llama 3.1 405B | 126 | 128 / 8 | 128 | 131,072 |
| Qwen2.5 7B | 28 | 28 / 4 | 128 | 32,768 |
| Qwen2.5 32B | 64 | 40 / 8 | 128 | 32,768 |
| Mistral 7B v0.3 | 32 | 32 / 8 | 128 | 32,768 |
Published downloads are not resident memory
Ollama's tag listing is an independent download reference: Llama 3.1 8B, Llama 3.1 70B, Qwen2.5 7B, Qwen2.5 32B and Mistral 7B. Listed decimal GB can change with the tag. File size excludes the runtime's cache and other allocations; it is not substituted into the weight formula. Modified parameter/profile combinations no longer show a named download-size comparison.
What this cannot establish
No tokens-per-second prediction, training memory, multimodal encoders, speculative-decoding draft model, automatic Mixture-of-Experts residency or hybrid/latent/sliding-window cache estimate is included. The session assumption is one shared loaded model with independent full-context caches, not one separate model process per session. Different backends can have very different peaks. Before buying hardware, test the exact model file, quantization, backend, drivers and workload.
FAQ
Does an estimated fit prove that the model will run?
No. The verdict compares a memory estimate with the usable budget you enter. It does not test your hardware, drivers, model file, runtime or supported operations. Model-loading peaks and runtime-specific memory layouts need a real test.
What is the difference between GB and GiB?
One GB is 1,000,000,000 bytes; one GiB is 1,073,741,824 bytes. This calculator performs its memory arithmetic in bytes and displays GiB. Download-size references stay labeled as decimal GB and are not added directly to GiB values.
Why can increasing context use much more memory?
For the dense full-attention decoder model used here, each session has its own key/value cache. Cache memory scales with the allocated token count, layers, KV heads and head dimension. The weights are counted once for sessions sharing one model instance. Real cache strategies may allocate differently.
Is Q4 exactly four bits for every weight?
No. The nominal payload is four bits per weight. A separate editable allowance accounts approximately for scales, mixed-precision tensors and format costs. The Q4 default corresponds to 4.5 effective bits, not the exact size of every Q4_K_M model. Use the exact file and runtime to check it.
Can RAM and VRAM be added together?
Not as interchangeable memory. Dedicated mode checks the full estimate separately against VRAM and RAM, without solving layer or cache offload placement. Unified mode uses one usable shared-memory pool and counts it only once. Neither mode guarantees backend compatibility.
Why is there no tokens-per-second prediction?
Memory capacity alone does not establish speed. CPU and GPU architecture, memory bandwidth, runtime, quantization, prompt processing and offloading affect performance. Scaling a CPU benchmark by parameter count would give an unsupported prediction for your device.