Best LLM for 6 GB VRAM · 2026

Best LLM for 6 GB VRAM: what fits, and what to run

Updated 8 Oct 2026 1 new this month

A 6 GB card hands a runtime about 5.3 GB once the display and driver have taken their share. At Q4_K_M with an 8K context, 17 of 80 models in the catalogue fit — 15 with headroom, 2 tightly. The pick for most people is Qwen3.5 4B: The 6–8 GB pick: a real assistant in 3.4 GB of weights. Speeds are estimated for a GeForce RTX 3060 Laptop; the fit verdicts are the same for every card in the class.

Cards in this class RTX 3060 LaptopGTX 1660 Super

Which one should I run?

One pick per job, from the editorial shortlist, checked to fit at Q4_K_M and 8K. Each label links the full ranking for that job. How picks are chosen.

RecommendationModelWhyQuant · totalTok/s est.
Best overall Qwen3.5 4B
Alibaba · Apache 2.0
The 6–8 GB pick: a real assistant in 3.4 GB of weights. Q4_K_M · 3.5 GB 78 Runs great
Best for coding Qwen3.5 4B
Alibaba · Apache 2.0
A coding agent in 3.4 GB — the 6–8 GB answer. Q4_K_M · 3.5 GB 78 Runs great
Best reasoning Qwen3.5 4B
Alibaba · Apache 2.0
Step-by-step reasoning in 8 GB. Q4_K_M · 3.5 GB 78 Runs great
Best for writing Qwen3.5 4B
Alibaba · Apache 2.0
Competent drafting in 8 GB. Q4_K_M · 3.5 GB 78 Runs great
Best vision Qwen3.5 4B
Alibaba · Apache 2.0
Reads screenshots in 8 GB. Q4_K_M · 3.5 GB 78 Runs great
Best for agents Qwen3.5 4B
Alibaba · Apache 2.0
Function calling in 8 GB. Q4_K_M · 3.5 GB 78 Runs great
Best translation Qwen3.5 4B
Alibaba · Apache 2.0
Usable translation in 8 GB. Q4_K_M · 3.5 GB 78 Runs great
Fastest good model Ling 3.0 Tiny 7.9B-A1.3B
inclusionAI · MIT
1.3B active with a latent cache on 6 of 24 layers. Q4_K_M · 5.1 GB 107 Tight fit
Best long context Qwen3.5 2B
Alibaba · Apache 2.0
Long documents on 4 GB. Q4_K_M · 3.4 GB
at 128K context
159 Runs great

The long-context row is judged at 128K: Qwen3.5 2B holds that window on a 6 GB card at Q4_K_M with 1.50 GB of KV cache — that is why it can differ from the best overall pick.

Everything that fits 6 GB

Largest first. "Max context" is the longest window the card holds at that quantisation with an f16 cache; a q8_0 cache roughly doubles it.

ModelParamsQuantWeights+KV 8KTotalHeadroomMax contextTok/s est.
Gemma 4 E2B 5.1B Q4_K_M 2.9 GB 0.07 GB 3.5 GB 1.8 GB 128K 71 Runs great
Qwen3.5 4B 4.66B Q4_K_M 2.6 GB 0.25 GB 3.5 GB 1.8 GB 66K 78 Runs great
Gemma 3 4B 4.3B Q4_K_M 2.4 GB 0.30 GB 3.3 GB 2.0 GB 92K 84 Runs great
Qwen3 4B 4.02B Q4_K_M 2.3 GB 1.13 GB 4.0 GB 1.3 GB 17K 90 Runs great
Ministral 3 3B 3.85B Q4_K_M 2.2 GB 0.81 GB 3.6 GB 1.7 GB 24K 94 Runs great
Phi-4-mini 3.8B 3.84B Q4_K_M 2.2 GB 1.00 GB 3.8 GB 1.5 GB 20K 94 Runs great
Granite 4.1 3B 3.4B Q4_K_M 1.9 GB 0.63 GB 3.1 GB 2.2 GB 35K 106 Runs great
Llama 3.2 3B Instruct 3.21B Q4_K_M 1.8 GB 0.88 GB 3.3 GB 2.0 GB 26K 113 Runs great
LFM2.5 2.6B 2.7B Q4_K_M 1.5 GB 0.13 GB 2.2 GB 3.1 GB 128K 134 Runs great
Qwen3.5 2B 2.27B Q4_K_M 1.3 GB 0.09 GB 2.0 GB 3.3 GB 256K 159 Runs great
Qwen3 1.7B 1.72B Q4_K_M 1.0 GB 0.88 GB 2.4 GB 2.9 GB 32K 210 Runs great
Llama 3.2 1B Instruct 1.24B Q4_K_M 0.7 GB 0.25 GB 1.5 GB 3.8 GB 128K 292 Runs great
Gemma 3 1B 1.0B Q4_K_M 0.6 GB 0.04 GB 1.2 GB 4.1 GB 32K 362 Runs great
Qwen3.5 0.8B 0.87B Q4_K_M 0.5 GB 0.09 GB 1.2 GB 4.1 GB 256K 416 Runs great
Qwen3 0.6B 0.6B Q4_K_M 0.3 GB 0.88 GB 1.8 GB 3.5 GB 32K 603 Runs great
Gemma 4 E4B 8.0B Q4_K_M 4.5 GB 0.14 GB 5.2 GB 0.1 GB 12K 45 Tight
Ling 3.0 Tiny 7.9B-A1.3B 7.9B MoE Q4_K_M 4.4 GB 0.05 GB 5.1 GB 0.2 GB 39K 107 Tight

Just out of reach

These run with part of the weights in system RAM (32 GB assumed), at a few tokens per second. Each needs a bigger card to run properly — the link says which.

ModelParamsTotal @ Q4_K_MOver budgetTok/s est., offloadedNeeds
Qwen3.6 35B-A3B 35.9B MoE 20.9 GB +15.6 GB ~9.2 32 GB card
Ornith 1.5 35B-A3B 35.9B MoE 20.9 GB +15.6 GB ~9.2 32 GB card
LLM-jp 4 33B Thinking 33.2B 21.3 GB +16.0 GB ~2.2 32 GB card
Qwen3 32B 32.8B 21.0 GB +15.7 GB ~2.2 32 GB card
DeepSeek-R1-Distill-Qwen 32B 32.8B 21.0 GB +15.7 GB ~2.2 32 GB card
Qwen2.5-Coder 32B 32.8B 21.0 GB +15.7 GB ~2.2 32 GB card

All memory classes 8 GB → Adjust context, quant or cache →

Every total is quantised weights + f16 KV cache at 8K + 0.6 GB runtime overhead, against 5.3 GB usable. Tokens per second are estimated from memory bandwidth and never measured. The method.