Home / Tools / AI model memory calculator

AI model memory calculator

Estimate how much memory an open AI model needs to run on your own computer or Raspberry Pi: the weights at your chosen precision, plus the KV cache for your context length.

  • AI tool
  • Runs in your browser
  • Tested 19 November 2026
  • Sources
Presets fill in the architecture from each model's published configuration.
Architecture (for the KV cache)

Where the memory goes

The weights. Every parameter has to be in memory. At 16-bit precision that's 2 bytes each, so an 8-billion-parameter model needs about 16 GB. Quantisation stores the weights in fewer bits. A 4-bit GGUF file such as Q4_K_M averages a little under 5 bits per weight, because it also stores scaling factors, so the same model shrinks to around 5 GB, usually with only a small loss in quality.

The KV cache. As the model reads your prompt and writes its answer, it keeps a key and a value vector for every token in every layer. Its size is:

2 × layers × KV heads × head dimension × context length × bytes per value

For Llama 3.1 8B at an 8,192-token context in FP16, that's 2 × 32 × 8 × 128 × 8,192 × 2 bytes, exactly 1 GiB. It grows in step with the context, so a 128K context needs 16 GiB for the cache alone. Models with fewer KV heads than attention heads, which is called grouped-query attention, keep this much smaller.

What the estimate leaves out

  • The runtime itself, plus working memory for calculations. Leave at least 1 to 2 GB spare, more on a GPU.
  • On a computer without a separate graphics card, the operating system and your other apps share the same RAM. On a Raspberry Pi with 8 GB, a model around 4 to 5 GB is a practical maximum.
  • Exact GGUF sizes vary a little between models, because different layers are quantised differently. The file size on the download page is the best guide to the weights.

Memory decides whether a model runs at all. Speed depends mostly on memory bandwidth, because every generated token reads all the weights. Our guide to running AI on a Raspberry Pi covers that.

Sources

Where the formulas and figures on this page come from. We test every tool against worked examples from these before publishing.