Best LLM for 6 GB VRAM · 2026
Best LLM for 6 GB VRAM: what fits, and what to run
A 6 GB card hands a runtime about 5.3 GB once the display and driver have taken their share. At Q4_K_M with an 8K context, 17 of 80 models in the catalogue fit — 15 with headroom, 2 tightly. The pick for most people is Qwen3.5 4B: The 6–8 GB pick: a real assistant in 3.4 GB of weights. Speeds are estimated for a GeForce RTX 3060 Laptop; the fit verdicts are the same for every card in the class.
Which one should I run?
One pick per job, from the editorial shortlist, checked to fit at Q4_K_M and 8K. Each label links the full ranking for that job. How picks are chosen.
| Recommendation | Model | Why | Quant · total | Tok/s est. | |
|---|---|---|---|---|---|
| Best overall | Qwen3.5 4B Alibaba · Apache 2.0 |
The 6–8 GB pick: a real assistant in 3.4 GB of weights. | Q4_K_M · 3.5 GB | 78 | Runs great |
| Best for coding | Qwen3.5 4B Alibaba · Apache 2.0 |
A coding agent in 3.4 GB — the 6–8 GB answer. | Q4_K_M · 3.5 GB | 78 | Runs great |
| Best reasoning | Qwen3.5 4B Alibaba · Apache 2.0 |
Step-by-step reasoning in 8 GB. | Q4_K_M · 3.5 GB | 78 | Runs great |
| Best for writing | Qwen3.5 4B Alibaba · Apache 2.0 |
Competent drafting in 8 GB. | Q4_K_M · 3.5 GB | 78 | Runs great |
| Best vision | Qwen3.5 4B Alibaba · Apache 2.0 |
Reads screenshots in 8 GB. | Q4_K_M · 3.5 GB | 78 | Runs great |
| Best for agents | Qwen3.5 4B Alibaba · Apache 2.0 |
Function calling in 8 GB. | Q4_K_M · 3.5 GB | 78 | Runs great |
| Best translation | Qwen3.5 4B Alibaba · Apache 2.0 |
Usable translation in 8 GB. | Q4_K_M · 3.5 GB | 78 | Runs great |
| Fastest good model | Ling 3.0 Tiny 7.9B-A1.3B inclusionAI · MIT |
1.3B active with a latent cache on 6 of 24 layers. | Q4_K_M · 5.1 GB | 107 | Tight fit |
| Best long context | Qwen3.5 2B Alibaba · Apache 2.0 |
Long documents on 4 GB. | Q4_K_M · 3.4 GB at 128K context |
159 | Runs great |
The long-context row is judged at 128K: Qwen3.5 2B holds that window on a 6 GB card at Q4_K_M with 1.50 GB of KV cache — that is why it can differ from the best overall pick.
Everything that fits 6 GB
Largest first. "Max context" is the longest window the card holds at that quantisation with an f16 cache; a q8_0 cache roughly doubles it.
| Model | Params | Quant | Weights | +KV 8K | Total | Headroom | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|---|---|
| Gemma 4 E2B | 5.1B | Q4_K_M | 2.9 GB | 0.07 GB | 3.5 GB | 1.8 GB | 128K | 71 | Runs great |
| Qwen3.5 4B | 4.66B | Q4_K_M | 2.6 GB | 0.25 GB | 3.5 GB | 1.8 GB | 66K | 78 | Runs great |
| Gemma 3 4B | 4.3B | Q4_K_M | 2.4 GB | 0.30 GB | 3.3 GB | 2.0 GB | 92K | 84 | Runs great |
| Qwen3 4B | 4.02B | Q4_K_M | 2.3 GB | 1.13 GB | 4.0 GB | 1.3 GB | 17K | 90 | Runs great |
| Ministral 3 3B | 3.85B | Q4_K_M | 2.2 GB | 0.81 GB | 3.6 GB | 1.7 GB | 24K | 94 | Runs great |
| Phi-4-mini 3.8B | 3.84B | Q4_K_M | 2.2 GB | 1.00 GB | 3.8 GB | 1.5 GB | 20K | 94 | Runs great |
| Granite 4.1 3B | 3.4B | Q4_K_M | 1.9 GB | 0.63 GB | 3.1 GB | 2.2 GB | 35K | 106 | Runs great |
| Llama 3.2 3B Instruct | 3.21B | Q4_K_M | 1.8 GB | 0.88 GB | 3.3 GB | 2.0 GB | 26K | 113 | Runs great |
| LFM2.5 2.6B | 2.7B | Q4_K_M | 1.5 GB | 0.13 GB | 2.2 GB | 3.1 GB | 128K | 134 | Runs great |
| Qwen3.5 2B | 2.27B | Q4_K_M | 1.3 GB | 0.09 GB | 2.0 GB | 3.3 GB | 256K | 159 | Runs great |
| Qwen3 1.7B | 1.72B | Q4_K_M | 1.0 GB | 0.88 GB | 2.4 GB | 2.9 GB | 32K | 210 | Runs great |
| Llama 3.2 1B Instruct | 1.24B | Q4_K_M | 0.7 GB | 0.25 GB | 1.5 GB | 3.8 GB | 128K | 292 | Runs great |
| Gemma 3 1B | 1.0B | Q4_K_M | 0.6 GB | 0.04 GB | 1.2 GB | 4.1 GB | 32K | 362 | Runs great |
| Qwen3.5 0.8B | 0.87B | Q4_K_M | 0.5 GB | 0.09 GB | 1.2 GB | 4.1 GB | 256K | 416 | Runs great |
| Qwen3 0.6B | 0.6B | Q4_K_M | 0.3 GB | 0.88 GB | 1.8 GB | 3.5 GB | 32K | 603 | Runs great |
| Gemma 4 E4B | 8.0B | Q4_K_M | 4.5 GB | 0.14 GB | 5.2 GB | 0.1 GB | 12K | 45 | Tight |
| Ling 3.0 Tiny 7.9B-A1.3B | 7.9B MoE | Q4_K_M | 4.4 GB | 0.05 GB | 5.1 GB | 0.2 GB | 39K | 107 | Tight |
Just out of reach
These run with part of the weights in system RAM (32 GB assumed), at a few tokens per second. Each needs a bigger card to run properly — the link says which.
| Model | Params | Total @ Q4_K_M | Over budget | Tok/s est., offloaded | Needs |
|---|---|---|---|---|---|
| Qwen3.6 35B-A3B | 35.9B MoE | 20.9 GB | +15.6 GB | ~9.2 | 32 GB card |
| Ornith 1.5 35B-A3B | 35.9B MoE | 20.9 GB | +15.6 GB | ~9.2 | 32 GB card |
| LLM-jp 4 33B Thinking | 33.2B | 21.3 GB | +16.0 GB | ~2.2 | 32 GB card |
| Qwen3 32B | 32.8B | 21.0 GB | +15.7 GB | ~2.2 | 32 GB card |
| DeepSeek-R1-Distill-Qwen 32B | 32.8B | 21.0 GB | +15.7 GB | ~2.2 | 32 GB card |
| Qwen2.5-Coder 32B | 32.8B | 21.0 GB | +15.7 GB | ~2.2 | 32 GB card |
Every total is quantised weights + f16 KV cache at 8K + 0.6 GB runtime overhead, against 5.3 GB usable. Tokens per second are estimated from memory bandwidth and never measured. The method.