Your rig

GeForce RTX 5060 · 8 GB

16 GB system RAM · Blackwell · 448 GB/s · 7.0 GB usable VRAM
Edit hardware Export as JSON
Context length
4K8K32K128K

KV cache at f16, 8K tokens

Quality floor
Q8_0 — near-lossless Q6_K Q5_K_M Q4_K_M — recommended Q3_K_M Q2_K — damaged
KV cache type
f16q8_0q4_0
System RAM (GB)
8163264128
Runtime
Ollamallama.cppLM StudiovLLM

4 of 5 runtimes drive this device.

Allow CPU offload Include partial-offload fits

Offloaded layers run at 60 GB/s host bandwidth — expect single-digit tokens per second.

Which one should I actually run?

Best models for your GeForce RTX 5060

Updated 21 Aug 2026 5 new this week, 14 this month

What do you want to do? Each row is the best-ranked model on that use case's shortlist that fits at Q6_K and 8K. The ordering is editorial; the sizes and speeds are computed. How the shortlists work.

All General assistantCodingReasoning & mathsWritingVisionAgents & tool useTranslationLow latencyLong context
RecommendationModelWhyQuant · totalTok/s est.
Best overall Qwen3.5 4B
Alibaba · Apache 2.0
 The 6–8 GB pick: a real assistant in 3.4 GB of weights. Q6_K · 4.4 GB 76 Runs great
Best for coding Qwen3.5 4B
Alibaba · Apache 2.0
 A coding agent in 3.4 GB — the 6–8 GB answer. Q6_K · 4.4 GB 76 Runs great
Best reasoning Qwen3.5 4B
Alibaba · Apache 2.0
 Step-by-step reasoning in 8 GB. Q6_K · 4.4 GB 76 Runs great
Best for writing Qwen3.5 4B
Alibaba · Apache 2.0
 Competent drafting in 8 GB. Q6_K · 4.4 GB 76 Runs great
Best vision Qwen3.5 4B
Alibaba · Apache 2.0
 Reads screenshots in 8 GB. Q6_K · 4.4 GB 76 Runs great
Best for agents Qwen3.5 4B
Alibaba · Apache 2.0
 Function calling in 8 GB. Q6_K · 4.4 GB 76 Runs great
Best translation Qwen3.5 4B
Alibaba · Apache 2.0
 Usable translation in 8 GB. Q6_K · 4.4 GB 76 Runs great
Fastest good model Ling 3.0 Tiny 7.9B-A1.3B New
inclusionAI · MIT
 1.3B active with a latent cache on 6 of 24 layers. Q6_K · 6.7 GB 105 Tight fit
Best long context Qwen3.5 2B
Alibaba · Apache 2.0
 Long documents on 4 GB. Q6_K · 3.8 GB
at 128K context
 156 Runs great

Everything that fits

Runs great

15 of 76 · fits with headroom for a longer prompt
ModelParamsQuantWeights +KV 8KTotal VRAM headroom Tok/sRuntime
Gemma 4 E2B 5.1B Q6_K 3.9 GB 0.07 GB 4.6 GB
2.4 GB free
 70 Ollama Run
Qwen3.5 4B 4.66B Q6_K 3.6 GB 0.25 GB 4.4 GB
2.6 GB free
 76 Ollama Run
Gemma 3 4B 4.3B Q6_K 3.3 GB 0.30 GB 4.2 GB
2.8 GB free
 83 Ollama Run
Qwen3 4B 4.02B Q6_K 3.1 GB 1.13 GB 4.8 GB
2.2 GB free
 88 Ollama Run
Ministral 3 3B 3.85B Q6_K 2.9 GB 0.81 GB 4.4 GB
2.6 GB free
 92 Ollama Run
Phi-4-mini 3.8B 3.84B Q6_K 2.9 GB 1.00 GB 4.5 GB
2.5 GB free
 92 Ollama Run
Granite 4.1 3B 3.4B Q6_K 2.6 GB 0.63 GB 3.8 GB
3.2 GB free
 104 Ollama Run
Llama 3.2 3B Instruct 3.21B Q6_K 2.5 GB 0.88 GB 3.9 GB
3.1 GB free
 111 Ollama Run
LFM2.5 2.6B New 2.7B Q6_K 2.1 GB 0.13 GB 2.8 GB
4.2 GB free
 132 llama.cpp Run
Qwen3.5 2B 2.27B Q6_K 1.7 GB 0.09 GB 2.4 GB
4.6 GB free
 156 Ollama Run
Qwen3 1.7B 1.72B Q6_K 1.3 GB 0.88 GB 2.8 GB
4.2 GB free
 206 Ollama Run
Llama 3.2 1B Instruct 1.24B Q6_K 0.9 GB 0.25 GB 1.8 GB
5.2 GB free
 286 Ollama Run
Gemma 3 1B 1.0B Q6_K 0.8 GB 0.04 GB 1.4 GB
5.6 GB free
 355 Ollama Run
Qwen3.5 0.8B 0.87B Q6_K 0.7 GB 0.09 GB 1.4 GB
5.6 GB free
 408 Ollama Run
Qwen3 0.6B 0.6B Q6_K 0.5 GB 0.88 GB 1.9 GB
5.1 GB free
 592 Ollama Run

Runs — tight

4 models · fits at 8K, drop context or quant to go longer
ModelParamsQuantWeights +KV 8KTotal VRAM headroom Tok/sRuntime
Gemma 4 E4B 8.0B Q6_K 6.1 GB 0.14 GB 6.9 GB
0.1 GB free
 44 llama.cpp Run
Ling 3.0 Tiny 7.9B-A1.3B New 7.9B MoE / 1.3B act Q6_K 6.0 GB 0.05 GB 6.7 GB
0.3 GB free
 105 llama.cpp Run
DeepSeek-R1-Distill-Qwen 7B 7.62B Q6_K 5.8 GB 0.44 GB 6.9 GB
0.1 GB free
 47 llama.cpp Run
Qwen2.5-Coder 7B 7.62B Q6_K 5.8 GB 0.44 GB 6.9 GB
0.1 GB free
 47 llama.cpp Run

Won't fit in VRAM

57 models · some are possible with CPU offload
ModelParamsQuantWeights +KV 8KTotal Over budget Tok/sRuntime
Devstral Small 2 24B 24B Q6_K 18.3 GB 1.25 GB 20.2 GB +13.2 GB ~2.6 llama.cpp Why
Mistral Small 3.2 24B 23.6B Q6_K 18.0 GB 1.25 GB 19.9 GB +12.9 GB ~2.7 llama.cpp Why
gpt-oss 20B 20.9B MoE / 3.6B act MXFP4 10.8 GB 0.19 GB 11.6 GB +4.6 GB ~15 llama.cpp Why
Qwen3 14B 14.8B Q6_K 11.3 GB 1.25 GB 13.2 GB +6.2 GB ~5.3 llama.cpp Why
DeepSeek-R1-Distill-Qwen 14B 14.8B Q6_K 11.3 GB 1.50 GB 13.4 GB +6.4 GB ~5.1 llama.cpp Why
Qwen2.5-Coder 14B 14.8B Q6_K 11.3 GB 1.50 GB 13.4 GB +6.4 GB ~5.1 llama.cpp Why
Phi-4 14B 14.7B Q6_K 11.2 GB 1.56 GB 13.4 GB +6.4 GB ~5.2 llama.cpp Why
Ministral 3 14B 13.9B Q6_K 10.6 GB 1.25 GB 12.5 GB +5.5 GB ~5.9 llama.cpp Why
Gemma 3 12B 12.2B Q6_K 9.3 GB 0.81 GB 10.7 GB +3.7 GB ~8.1 llama.cpp Why
Mistral NeMo 12B 12.2B Q6_K 9.3 GB 1.25 GB 11.2 GB +4.2 GB ~7.5 llama.cpp Why
Gemma 4 12B 12B Q6_K 9.2 GB 0.81 GB 10.6 GB +3.6 GB ~8.4 llama.cpp Why
Qwen3.5 9B 9.65B Q6_K 7.4 GB 0.25 GB 8.2 GB +1.2 GB ~18 llama.cpp Why
Ornith 1.5 9B New this week 9.41B Q6_K 7.2 GB 0.25 GB 8.0 GB +1.0 GB ~20 llama.cpp Why
Ministral 3 8B 8.92B Q6_K 6.8 GB 1.06 GB 8.5 GB +1.5 GB ~17 llama.cpp Why
Granite 4.1 8B 8.79B Q6_K 6.7 GB 1.25 GB 8.6 GB +1.6 GB ~16 llama.cpp Why
LFM2.5 8B-A1B 8.47B MoE / 1.5B act Q6_K 6.5 GB 0.09 GB 7.2 GB +0.2 GB ~78 llama.cpp Why
Fara 7B 8.29B Q6_K 6.3 GB 0.44 GB 7.4 GB +0.4 GB ~31 llama.cpp Why
Qwen3 8B 8.19B Q6_K 6.3 GB 1.13 GB 8.0 GB +1.0 GB ~22 llama.cpp Why
Llama 3.1 8B Instruct 8.03B Q6_K 6.1 GB 1.00 GB 7.7 GB +0.7 GB ~25 llama.cpp Why
Olmo 3 7B Instruct 7.3B Q6_K 5.6 GB 2.50 GB 8.7 GB +1.7 GB ~17 llama.cpp Why
Mistral 7B Instruct v0.3 7.25B Q6_K 5.5 GB 1.00 GB 7.1 GB +0.1 GB ~42 llama.cpp Why
Kimi K3 2.8T-A104B New 2780B MoE / 104B act Q6_K 2123.0 GB 0.21 GB 2123.9 GB +2116.9 GB ~0.2 llama.cpp Why
Qwen3.8 2.4T-A95B New 2446B MoE / 95B act Q6_K 1868.0 GB 0.72 GB 1869.3 GB +1862.3 GB ~0.2 llama.cpp Why
DeepSeek V4 Pro 1.6T-A49B New 1650B MoE / 49B act Q6_K 1260.1 GB 0.54 GB 1261.2 GB +1254.2 GB ~0.4 llama.cpp Why
Kimi K2.6 1T-A32B 1027B MoE / 32B act Q6_K 784.3 GB 0.54 GB 785.4 GB +778.4 GB ~0.6 llama.cpp Why
GLM-5.2 744B-A40B 753B MoE / 40B act Q6_K 575.1 GB 0.69 GB 576.3 GB +569.3 GB ~0.5 llama.cpp Why
DeepSeek-R1 671B 671B MoE / 37B act Q6_K 512.4 GB 0.54 GB 513.6 GB +506.6 GB ~0.5 llama.cpp Why
Qwen3.5 397B-A17B 403B MoE / 17B act Q6_K 307.8 GB 0.23 GB 308.6 GB +301.6 GB ~1.1 llama.cpp Why
Ornith 1.5 397B-A17B New this week 397B MoE / 17B act Q6_K 303.2 GB 0.23 GB 304.0 GB +297.0 GB ~1.1 llama.cpp Why
DeepSeek V4 Flash 284B-A13B New 284B MoE / 13B act Q6_K 216.9 GB 0.38 GB 217.9 GB +210.9 GB ~1.4 llama.cpp Why
Qwen3 235B-A22B 235B MoE / 22B act Q6_K 179.5 GB 1.47 GB 181.5 GB +174.5 GB ~0.9 llama.cpp Why
Qwen3.5 122B-A10B 125B MoE / 10B act Q6_K 95.5 GB 0.19 GB 96.2 GB +89.2 GB ~1.9 llama.cpp Why
Nemotron 3 Super 120B-A12B 124B MoE / 12B act Q6_K 94.7 GB 0.06 GB 95.4 GB +88.4 GB ~1.6 llama.cpp Why
Ling 3.0 Flash 124B-A5B New 124B MoE / 5.1B act Q6_K 94.7 GB 0.06 GB 95.4 GB +88.4 GB ~3.8 llama.cpp Why
Mistral Small 4 119B-A6B 119B MoE / 6.5B act Q6_K 90.9 GB 0.18 GB 91.7 GB +84.7 GB ~3.0 llama.cpp Why
gpt-oss 120B 117B MoE / 5.1B act MXFP4 60.5 GB 0.29 GB 61.4 GB +54.4 GB ~5.8 llama.cpp Why
Llama 4 Scout 109B-A17B 109B MoE / 17B act Q6_K 83.2 GB 1.50 GB 85.3 GB +78.3 GB ~1.1 llama.cpp Why
Llama 3.3 70B Instruct 70.6B Q6_K 53.9 GB 2.50 GB 57.0 GB +50.0 GB ~0.7 llama.cpp Why
DeepSeek-R1-Distill-Llama 70B 70.6B Q6_K 53.9 GB 2.50 GB 57.0 GB +50.0 GB ~0.7 llama.cpp Why
Qwen3.6 35B-A3B 35.9B MoE / 3.3B act Q6_K 27.4 GB 0.16 GB 28.2 GB +21.2 GB ~6.9 llama.cpp Why
Ornith 1.5 35B-A3B New this week 35.9B MoE / 3.3B act Q6_K 27.4 GB 0.16 GB 28.2 GB +21.2 GB ~6.9 llama.cpp Why
LLM-jp 4 33B Thinking New this week 33.2B Q6_K 25.4 GB 2.00 GB 28.0 GB +21.0 GB ~1.7 llama.cpp Why
Qwen3 32B 32.8B Q6_K 25.0 GB 2.00 GB 27.6 GB +20.6 GB ~1.7 llama.cpp Why
DeepSeek-R1-Distill-Qwen 32B 32.8B Q6_K 25.0 GB 2.00 GB 27.6 GB +20.6 GB ~1.7 llama.cpp Why
Qwen2.5-Coder 32B 32.8B Q6_K 25.0 GB 2.00 GB 27.6 GB +20.6 GB ~1.7 llama.cpp Why
Olmo 3.1 32B Instruct 32.2B Q6_K 24.6 GB 1.25 GB 26.4 GB +19.4 GB ~1.8 llama.cpp Why
Nemotron 3.5 Lightning 30B-A3B New 31.6B MoE / 3.2B act Q6_K 24.1 GB 0.05 GB 24.8 GB +17.8 GB ~7.4 llama.cpp Why
Gemma 4 31B 31.3B Q6_K 23.9 GB 2.03 GB 26.5 GB +19.5 GB ~1.8 llama.cpp Why
GLM-4.7-Flash 30B-A3B 31.2B MoE / 3B act Q6_K 23.8 GB 0.41 GB 24.8 GB +17.8 GB ~7.8 llama.cpp Why
Qwen3 30B-A3B 30.5B MoE / 3.3B act Q6_K 23.3 GB 0.75 GB 24.6 GB +17.6 GB ~7.0 llama.cpp Why
Qwen3 Coder 30B-A3B 30.5B MoE / 3.3B act Q6_K 23.3 GB 0.75 GB 24.6 GB +17.6 GB ~7.0 llama.cpp Why
Muse Glimmer 30B New 29.8B Q6_K 22.8 GB 0.18 GB 23.5 GB +16.5 GB ~2.1 llama.cpp Why
Granite 4.1 30B 28.9B Q6_K 22.1 GB 2.00 GB 24.7 GB +17.7 GB ~2.0 llama.cpp Why
Qwen3.8 27B New this week 27.8B Q6_K 21.2 GB 0.50 GB 22.3 GB +15.3 GB ~2.3 llama.cpp Why
Qwen3.6 27B 27.8B Q6_K 21.2 GB 0.50 GB 22.3 GB +15.3 GB ~2.3 llama.cpp Why
Gemma 3 27B 27.4B Q6_K 20.9 GB 1.03 GB 22.6 GB +15.6 GB ~2.2 llama.cpp Why
Gemma 4 26B-A4B 26.5B MoE / 3.8B act Q6_K 20.2 GB 0.51 GB 21.3 GB +14.3 GB ~6.4 llama.cpp Why

Totals = quantised weights + f16 KV cache at 8K tokens + 0.6 GB runtime overhead, against 7.0 GB usable VRAM (8 GB less display and driver reserve). Tokens per second are estimated from 448 GB/s peak memory bandwidth at batch 1 — not measured. Read the method.