Where every number on this site comes from

Nothing here is measured. Every figure is computed from published specifications, and the ones that are estimates say so on the page they appear on. This is the whole calculation.

Usable memory

A card advertised at 24 GB does not hand a runtime 24 GB. The display buffer, the driver and the window compositor get there first. We subtract the smaller of 1.6 GB or 12% of the card, so a 24 GB card is treated as 22.4 GB and an 8 GB card as 7.0 GB.

Apple Silicon is different: memory is unified, and macOS caps what the GPU may wire down. We use 75% of total (67% at 16 GB and below), which is close to what Metal reports as its recommended working-set size. On a CPU-only configuration we assume 80% of system RAM is available to the model.

Weights

Weight size is the parameter count times the quantisation's effective bits per weight, divided by eight. The effective figures sit above the nominal bit count because k-quants keep the embedding and output tensors at higher precision. Sizes for a 32.8B model:

Expect a computed size to land within a few percent of the GGUF you actually download. MXFP4 is listed separately because for gpt-oss it is the native format — those 4-bit weights are the reference weights, not a lossy conversion of something larger.

The KV cache

This is the part people forget, and the reason a model that "should fit" does not. Every token you keep in context is stored, per layer, as a key and a value vector:

bytes = layers × 2 × kv_heads × head_dim × context × bytes_per_element

At f16 that is 2 bytes per element. Llama 3.1 8B has 32 layers, 8 KV heads and a head dimension of 128, so it spends 128 KB per token — 1.00 GB at 8K context and 16.0 GB at 128K. Qwen3 32B, with 64 layers, spends 32.0 GB at 128K — more than the weights.

Three families escape this. Gemma and gpt-oss interleave sliding-window layers that only keep a few hundred tokens each, so Gemma 4 31B needs 20.8 GB at 128K where a plain model of that shape would need far more. DeepSeek, GLM, Kimi and Mistral Small 4 use multi-head latent attention and cache a compressed latent instead of full keys and values. And the 2026 hybrids — Qwen3.5, 3.6 and 3.8, Nemotron 3, Kimi K3, LFM2.5 — replace most of their attention blocks with linear attention, Mamba or convolution blocks that keep a fixed-size state rather than a per-token cache: Qwen3.8 27B caches in only 16 of its 64 blocks, so 128K costs 8.0 GB instead of 32.0 GB for Qwen3 32B. All three are modelled here (the recurrent state of a hybrid is folded into the overhead), which is why their long-context numbers look implausibly good — they genuinely are. Layer counts and attention patterns are read from each model's published config.json, never guessed.

Overhead

A flat 0.6 GB covers the CUDA or Metal context, the compute buffers and the graph. It is an approximation, and it is the least precise number on the site.

Verdicts

• Runs great — total is at most 85% of usable memory. There is room for a longer prompt than you asked for.
• Runs — tight — it fits, with under 15% spare. A long system prompt can still push it over.
• Needs CPU offload — the cache and overhead stay on the GPU, whatever VRAM is left holds as many layers as it can, and the remainder streams from system RAM.
• Won't fit — the shortfall is larger than the host can lend at any speed worth having.

Tokens per second — an estimate, and only that

Generation at batch 1 is bound by memory bandwidth: the machine reads every active weight once per token. So the ceiling is bandwidth divided by active weight bytes, and the real number is some fraction of that ceiling. We use 0.65 for dense models on a discrete GPU, 0.6 on unified memory and 0.45 on CPU. Mixture-of-experts models get a much lower fraction (0.25 on a discrete GPU) because although only a few billion parameters are active per token, gathering scattered experts wastes most of the bandwidth advantage.

When layers are split across GPU and host, the two bandwidths combine as a harmonic mean weighted by the split, with host RAM assumed at 60 GB/s. That is why an offloaded 70B lands in the low single digits.

These estimates ignore prompt processing entirely, which is compute-bound rather than bandwidth-bound and can dominate on long prompts. Treat every tok/s figure on this site as an order-of-magnitude guide. If you want real numbers, run the model.

Recommendations — editorial, and labelled as such

A hardware page opens with "best models for your device": one pick per use case (general assistant, coding, reasoning, writing, vision, agents, translation, low latency, long context). We still run no benchmark. Each use case has a hand-ordered shortlist — ordered by capability as reported by the vendors' own evaluations and by the local-model community as of the catalogue date, 21 Aug 2026 — and the pick is simply the best-ranked model on that list that fits your device at the quantisation and context you chose. A fit with under 15% headroom counts as three places worse than a fit with room to spare, so a clearly stronger model that just squeezes in still wins over a weaker one, but a near-equal one with headroom does not lose. Long context is judged at 128K regardless of the page setting, because that is the question being asked. The sizes and speeds beside every pick are computed exactly as above; the ordering is a judgement, the lists are public in the source, and they are revisited whenever a new generation ships. "New this week" and "New this month" are measured against the catalogue date, not today's date, so they never drift silently.

Models outside the catalogue

Paste any public Hugging Face id and we size it the same way: the repository’s config.json supplies the layer count, attention and KV heads, head dimension, context window, expert counts, sliding-window or latent-attention shape, and the safetensors index supplies the exact parameter count (when it is public; otherwise the count is computed from the config and labelled estimated). Active parameters for mixture-of-experts models are always computed from the expert sizes, which lands within a few percent of the card. Those pages carry the banner calculated from model configuration — not yet manually verified: nobody has checked the numbers against a real download, the model is on no shortlist. The curated table is what we stand behind; this is the tool.

The reverse question

Every model page has a companion, what hardware do I need?, which runs the same engine the other way: every device in the catalogue at every quantisation the model ships in. A tier is the smallest memory class that clears its bar — minimum practical (fits at all, lowest sane quantisation), recommended (the recommended quantisation with 15% headroom at 8K), high quality (Q8_0 with headroom), long context (128K at the recommended quantisation), plus the same for Apple Silicon and CPU-only — and lists every device in that class, because "a 24 GB card" is the honest answer, not one product.

What we do not do

• We do not benchmark. There is no measurement rig behind this site and no claim that there is.
• We do not score models. Perplexity deltas per quantisation are quoted from published figures; the recommendation shortlists are an ordering, not a number, and say so.
• We do not track you. There is no analytics, no cookie and no account. Device detection runs in your browser and only the device you pick is ever sent.