Best LLM for 12 GB VRAM · 2026

Best LLM for 12 GB VRAM: what fits, and what to run

A 12 GB card hands a runtime about 10.6 GB once the display and driver have taken their share. At Q4_K_M with an 8K context, 37 of 80 models in the catalogue fit — 32 with headroom, 5 tightly. The pick for most people is Gemma 4 12B: The 12 GB generalist: text, image and audio in, 140+ languages. Speeds are estimated for a GeForce RTX 3060 12 GB; the fit verdicts are the same for every card in the class.

Which one should I run?

One pick per job, from the editorial shortlist, checked to fit at Q4_K_M and 8K. How picks are chosen.

The long-context row is judged at 128K: Qwen3.5 4B holds that window on 12 GB at Q4_K_M with 4.00 GB of KV cache — that is why it can differ from the best overall pick.

Everything that fits 12 GB

Largest first. "Max context" is the longest window the card holds at that quantisation with an f16 cache; a q8_0 cache roughly doubles it.

Just out of reach

These run with part of the weights in system RAM (32 GB assumed), at a few tokens per second. Each needs a bigger card to run properly — the link says which.

Every total is quantised weights + f16 KV cache at 8K + 0.6 GB runtime overhead, against 10.6 GB usable. Tokens per second are estimated from memory bandwidth and never measured. The method.