Your rig

GeForce RTX 5060 · 8 GB

256 GB system RAM · Blackwell · 448 GB/s · 7.0 GB usable VRAM
Edit hardware Export as JSON

Context length

KV cache at f16, 128K tokens

Quality floor
Q8_0 — near-lossless Q6_K Q5_K_M Q4_K_M — recommended Q3_K_M Q2_K — damaged
KV cache type
System RAM (GB)
Runtime
Ollamallama.cppLM StudiovLLM

4 of 5 runtimes drive this device.

Allow CPU offload Include partial-offload fits

Offloaded layers run at 60 GB/s host bandwidth — expect single-digit tokens per second.

Which one should I actually run?

Best models for your GeForce RTX 5060

Updated 8 Oct 2026 1 new this month

What do you want to do? Each row is the best-ranked model on that use case's shortlist that fits at Q4_K_M and 128K. The ordering is editorial; the sizes and speeds are computed. How the shortlists work.

RecommendationModelWhyQuant · totalTok/s est.
Best overall Qwen3.5 2B
Alibaba · Apache 2.0
Runs on 4 GB, and still multimodal. Q4_K_M · 3.4 GB 212 Runs great
Best for coding Qwen3.5 2B
Alibaba · Apache 2.0
Autocomplete on 4 GB. Q4_K_M · 3.4 GB 212 Runs great
Best reasoning Qwen3.5 2B
Alibaba · Apache 2.0
Reasoning traces on 4 GB; expect long outputs. Q4_K_M · 3.4 GB 212 Runs great
Best for writing Gemma 4 E4B
Google · Apache 2.0
Readable prose on a laptop. Q4_K_M · 6.9 GB 60 Tight fit
Best vision Gemma 4 E2B
Google · Apache 2.0
Phone-class vision. Q4_K_M · 4.4 GB 95 Runs great
Best for agents Ling 3.0 Tiny 7.9B-A1.3B
inclusionAI · MIT
Tool use and reasoning at 1.3B active, sized for edge boxes. Q4_K_M · 5.9 GB 143 Runs great
Best translation Gemma 4 E4B
Google · Apache 2.0
140+ languages on a laptop. Q4_K_M · 6.9 GB 60 Tight fit
Fastest good model Ling 3.0 Tiny 7.9B-A1.3B
inclusionAI · MIT
1.3B active with a latent cache on 6 of 24 layers. Q4_K_M · 5.9 GB 143 Runs great
Best long context Qwen3.5 2B
Alibaba · Apache 2.0
Long documents on 4 GB. Q4_K_M · 3.4 GB
at 128K context
212 Runs great

Everything that fits

Runs great

9 of 80 · fits with headroom for a longer prompt
ModelParamsQuantWeights +KV 128KTotal VRAM headroom Tok/sRuntime
Ling 3.0 Tiny 7.9B-A1.3B 7.9B MoE / 1.3B act Q4_K_M 4.4 GB 0.84 GB 5.9 GB
1.1 GB free
143 llama.cpp Run
Gemma 4 E2B 5.1B Q4_K_M 2.9 GB 0.89 GB 4.4 GB
2.6 GB free
95 Ollama Run
LFM2.5 2.6B 2.7B Q4_K_M 1.5 GB 2.00 GB 4.1 GB
2.9 GB free
179 llama.cpp Run
Qwen3.5 2B 2.27B Q4_K_M 1.3 GB 1.50 GB 3.4 GB
3.6 GB free
212 Ollama Run
Qwen3 1.7B 1.72B Q4_K_M 1.0 GB 3.50 GB 5.1 GB
1.9 GB free
280 Ollama Run
Llama 3.2 1B Instruct 1.24B Q4_K_M 0.7 GB 4.00 GB 5.3 GB
1.7 GB free
389 Ollama Run
Gemma 3 1B 1.0B Q4_K_M 0.6 GB 0.14 GB 1.3 GB
5.7 GB free
482 Ollama Run
Qwen3.5 0.8B 0.87B Q4_K_M 0.5 GB 1.50 GB 2.6 GB
4.4 GB free
554 Ollama Run
Qwen3 0.6B 0.6B Q4_K_M 0.3 GB 3.50 GB 4.4 GB
2.6 GB free
804 Ollama Run

Runs — tight

3 models · fits at 128K, drop context or quant to go longer
ModelParamsQuantWeights +KV 128KTotal VRAM headroom Tok/sRuntime
LFM2.5 8B-A1B 8.47B MoE / 1.5B act Q4_K_M 4.8 GB 1.46 GB 6.8 GB
0.2 GB free
124 llama.cpp Run
Gemma 4 E4B 8.0B Q4_K_M 4.5 GB 1.78 GB 6.9 GB
0.1 GB free
60 llama.cpp Run
Gemma 3 4B 4.3B Q4_K_M 2.4 GB 3.11 GB 6.1 GB
0.9 GB free
112 llama.cpp Run

Won't fit in VRAM

68 models · some are possible with CPU offload
ModelParamsQuantWeights +KV 128KTotal Over budget Tok/sRuntime
GLM-5.3-Flash 320B-A18B 321B MoE / 18B act Q4_K_M 180.5 GB 1.38 GB 182.5 GB +175.5 GB ~1.4 llama.cpp Why
DeepSeek V4 Flash 284B-A13B 284B MoE / 13B act Q4_K_M 159.7 GB 6.05 GB 166.3 GB +159.3 GB ~1.9 llama.cpp Why
Qwen3 235B-A22B 235B MoE / 22B act Q4_K_M 132.1 GB 23.50 GB 156.2 GB +149.2 GB ~1.1 llama.cpp Why
Qwen3.8-Flash-Next 180B-A6B 180B MoE / 6B act Q4_K_M 101.2 GB 3.00 GB 104.8 GB +97.8 GB ~4.3 llama.cpp Why
Qwen3.5 122B-A10B 125B MoE / 10B act Q4_K_M 70.3 GB 3.00 GB 73.9 GB +66.9 GB ~2.6 llama.cpp Why
Nemotron 3 Super 120B-A12B 124B MoE / 12B act Q4_K_M 69.7 GB 1.00 GB 71.3 GB +64.3 GB ~2.2 llama.cpp Why
Ling 3.0 Flash 124B-A5B 124B MoE / 5.1B act Q4_K_M 69.7 GB 0.98 GB 71.3 GB +64.3 GB ~5.2 llama.cpp Why
Mistral Small 4 119B-A6B 119B MoE / 6.5B act Q4_K_M 66.9 GB 2.81 GB 70.3 GB +63.3 GB ~4.0 llama.cpp Why
gpt-oss 120B 117B MoE / 5.1B act MXFP4 60.5 GB 4.50 GB 65.6 GB +58.6 GB ~5.4 llama.cpp Why
Llama 4 Scout 109B-A17B 109B MoE / 17B act Q4_K_M 61.3 GB 7.13 GB 69.0 GB +62.0 GB ~1.5 llama.cpp Why
Llama 3.3 70B Instruct 70.6B Q4_K_M 39.7 GB 40.00 GB 80.3 GB +73.3 GB ~0.9 llama.cpp Why
DeepSeek-R1-Distill-Llama 70B 70.6B Q4_K_M 39.7 GB 40.00 GB 80.3 GB +73.3 GB ~0.9 llama.cpp Why
Qwen3.6 35B-A3B 35.9B MoE / 3.3B act Q4_K_M 20.2 GB 2.50 GB 23.3 GB +16.3 GB ~9.0 llama.cpp Why
Ornith 1.5 35B-A3B 35.9B MoE / 3.3B act Q4_K_M 20.2 GB 2.50 GB 23.3 GB +16.3 GB ~9.0 llama.cpp Why
LLM-jp 4 33B Thinking 33.2B Q4_K_M 18.7 GB 16.00 GB 35.3 GB +28.3 GB ~1.9 llama.cpp Why
Qwen3 32B 32.8B Q4_K_M 18.4 GB 32.00 GB 51.0 GB +44.0 GB ~2.0 llama.cpp Why
DeepSeek-R1-Distill-Qwen 32B 32.8B Q4_K_M 18.4 GB 32.00 GB 51.0 GB +44.0 GB ~2.0 llama.cpp Why
Qwen2.5-Coder 32B 32.8B Q4_K_M 18.4 GB 32.00 GB 51.0 GB +44.0 GB ~2.0 llama.cpp Why
Olmo 3.1 32B Instruct 32.2B Q4_K_M 18.1 GB 4.75 GB 23.5 GB +16.5 GB ~2.2 llama.cpp Why
Nemotron 3.5 Lightning 30B-A3B 31.6B MoE / 3.2B act Q4_K_M 17.8 GB 0.75 GB 19.1 GB +12.1 GB ~11 llama.cpp Why
Gemma 4 31B 31.3B Q4_K_M 17.6 GB 20.78 GB 39.0 GB +32.0 GB ~2.1 llama.cpp Why
GLM-4.7-Flash 30B-A3B 31.2B MoE / 3B act Q4_K_M 17.5 GB 6.61 GB 24.8 GB +17.8 GB ~8.3 llama.cpp Why
Qwen3 30B-A3B 30.5B MoE / 3.3B act Q4_K_M 17.1 GB 12.00 GB 29.7 GB +22.7 GB ~7.5 llama.cpp Why
Qwen3 Coder 30B-A3B 30.5B MoE / 3.3B act Q4_K_M 17.1 GB 12.00 GB 29.7 GB +22.7 GB ~7.5 llama.cpp Why
Muse Glimmer 30B 29.8B Q4_K_M 16.8 GB 1.70 GB 19.1 GB +12.1 GB ~2.9 llama.cpp Why
Granite 4.1 30B 28.9B Q4_K_M 16.3 GB 32.00 GB 48.9 GB +41.9 GB ~2.2 llama.cpp Why
Qwen3.8 27B 27.8B Q4_K_M 15.6 GB 8.00 GB 24.2 GB +17.2 GB ~2.3 llama.cpp Why
Qwen3.6 27B 27.8B Q4_K_M 15.6 GB 8.00 GB 24.2 GB +17.2 GB ~2.3 llama.cpp Why
Gemma 3 27B 27.4B Q4_K_M 15.4 GB 10.41 GB 26.4 GB +19.4 GB ~2.4 llama.cpp Why
Gemma 4 26B-A4B 26.5B MoE / 3.8B act Q4_K_M 14.9 GB 5.20 GB 20.7 GB +13.7 GB ~7.0 llama.cpp Why
Devstral Small 2 24B 24B Q4_K_M 13.5 GB 20.00 GB 34.1 GB +27.1 GB ~2.7 llama.cpp Why
Mistral Small 3.2 24B 23.6B Q4_K_M 13.3 GB 20.00 GB 33.9 GB +26.9 GB ~2.7 llama.cpp Why
gpt-oss 20B 20.9B MoE / 3.6B act MXFP4 10.8 GB 3.00 GB 14.4 GB +7.4 GB ~10 llama.cpp Why
Qwen3 14B 14.8B Q4_K_M 8.3 GB 20.00 GB 28.9 GB +21.9 GB ~4.4 llama.cpp Why
DeepSeek-R1-Distill-Qwen 14B 14.8B Q4_K_M 8.3 GB 24.00 GB 32.9 GB +25.9 GB ~4.4 llama.cpp Why
Qwen2.5-Coder 14B 14.8B Q4_K_M 8.3 GB 24.00 GB 32.9 GB +25.9 GB ~4.4 llama.cpp Why
Phi-4 14B 14.7B Q4_K_M 8.3 GB 3.13 GB 12.0 GB +5.0 GB ~6.7 llama.cpp Why
Ministral 3 14B 13.9B Q4_K_M 7.8 GB 20.00 GB 28.4 GB +21.4 GB ~4.6 llama.cpp Why
Gemma 3 12B 12.2B Q4_K_M 6.9 GB 8.31 GB 15.8 GB +8.8 GB ~5.3 llama.cpp Why
Mistral NeMo 12B 12.2B Q4_K_M 6.9 GB 20.00 GB 27.5 GB +20.5 GB ~5.3 llama.cpp Why
Gemma 4 12B 12B Q4_K_M 6.7 GB 8.31 GB 15.7 GB +8.7 GB ~5.4 llama.cpp Why
Qwen3.5 9B 9.65B Q4_K_M 5.4 GB 4.00 GB 10.0 GB +3.0 GB ~11 llama.cpp Why
Ornith 1.5 9B 9.41B Q4_K_M 5.3 GB 4.00 GB 9.9 GB +2.9 GB ~11 llama.cpp Why
Ministral 3 8B 8.92B Q4_K_M 5.0 GB 17.00 GB 22.6 GB +15.6 GB ~7.2 llama.cpp Why
Granite 4.1 8B 8.79B Q4_K_M 4.9 GB 20.00 GB 25.5 GB +18.5 GB ~7.3 llama.cpp Why
Fara 7B 8.29B Q4_K_M 4.7 GB 6.84 GB 12.1 GB +5.1 GB ~7.8 llama.cpp Why
Qwen3 8B 8.19B Q4_K_M 4.6 GB 18.00 GB 23.2 GB +16.2 GB ~7.9 llama.cpp Why
Llama 3.1 8B Instruct 8.03B Q4_K_M 4.5 GB 16.00 GB 21.1 GB +14.1 GB ~8.0 llama.cpp Why
DeepSeek-R1-Distill-Qwen 7B 7.62B Q4_K_M 4.3 GB 7.00 GB 11.9 GB +4.9 GB ~8.5 llama.cpp Why
Qwen2.5-Coder 7B 7.62B Q4_K_M 4.3 GB 7.00 GB 11.9 GB +4.9 GB ~8.5 llama.cpp Why
Olmo 3 7B Instruct 7.3B Q4_K_M 4.1 GB 9.50 GB 14.2 GB +7.2 GB ~8.8 llama.cpp Why
Mistral 7B Instruct v0.3 7.25B Q4_K_M 4.1 GB 4.00 GB 8.7 GB +1.7 GB ~18 llama.cpp Why
Qwen3.5 4B 4.66B Q4_K_M 2.6 GB 4.00 GB 7.2 GB +0.2 GB ~67 llama.cpp Why
Qwen3 4B 4.02B Q4_K_M 2.3 GB 4.50 GB 7.4 GB +0.4 GB ~59 llama.cpp Why
Ministral 3 3B 3.85B Q4_K_M 2.2 GB 13.00 GB 15.8 GB +8.8 GB ~17 llama.cpp Why
Phi-4-mini 3.8B 3.84B Q4_K_M 2.2 GB 16.00 GB 18.8 GB +11.8 GB ~17 llama.cpp Why
Granite 4.1 3B 3.4B Q4_K_M 1.9 GB 10.00 GB 12.5 GB +5.5 GB ~19 llama.cpp Why
Llama 3.2 3B Instruct 3.21B Q4_K_M 1.8 GB 14.00 GB 16.4 GB +9.4 GB ~20 llama.cpp Why
Kimi K3 2.8T-A104B 2780B MoE / 104B act Q4_K_M 1563.2 GB 3.38 GB 1567.1 GB +1560.1 GB ~0.2 llama.cpp Why
Qwen3.8 2.4T-A95B 2446B MoE / 95B act Q4_K_M 1375.4 GB 11.50 GB 1387.5 GB +1380.5 GB ~0.3 llama.cpp Why
DeepSeek V4 Pro 1.6T-A49B 1650B MoE / 49B act Q4_K_M 927.8 GB 8.58 GB 937.0 GB +930.0 GB ~0.5 llama.cpp Why
Kimi K2.6 1T-A32B 1027B MoE / 32B act Q4_K_M 577.5 GB 8.58 GB 586.6 GB +579.6 GB ~0.8 llama.cpp Why
GLM-5.2 744B-A40B 753B MoE / 40B act Q4_K_M 423.4 GB 10.97 GB 435.0 GB +428.0 GB ~0.6 llama.cpp Why
GLM-5.3 744B-A40B 753B MoE / 40B act Q4_K_M 423.4 GB 10.97 GB 435.0 GB +428.0 GB ~0.6 llama.cpp Why
DeepSeek-R1 671B 671B MoE / 37B act Q4_K_M 377.3 GB 8.58 GB 386.5 GB +379.5 GB ~0.7 llama.cpp Why
DeepSeek V4.1 Flash 552B-A16B New 552B MoE / 16B act Q4_K_M 310.4 GB 0.57 GB 311.5 GB +304.5 GB ~1.6 vLLM Why
Qwen3.5 397B-A17B 403B MoE / 17B act Q4_K_M 226.6 GB 3.75 GB 231.0 GB +224.0 GB ~1.5 llama.cpp Why
Ornith 1.5 397B-A17B 397B MoE / 17B act Q4_K_M 223.2 GB 3.75 GB 227.6 GB +220.6 GB ~1.5 llama.cpp Why

Totals = quantised weights + f16 KV cache at 128K tokens + 0.6 GB runtime overhead, against 7.0 GB usable VRAM (8 GB less display and driver reserve). Tokens per second are estimated from 448 GB/s peak memory bandwidth at batch 1 — not measured. Read the method.


Same memory, different speed

9 of the 12 devices have the same 7.0 GB usable, so they run exactly the models above. Generation speed scales with memory bandwidth.

DeviceBandwidthSpeed vs yours
GeForce RTX 3070 Ti
Ampere · 2021
608 GB/s 1.36×
Arc A750
Alchemist · 2022
512 GB/s 1.14×
GeForce RTX 2080 Super
Turing · 2019
496 GB/s 1.11×
GeForce RTX 3070
Ampere · 2020
448 GB/s same
GeForce RTX 3060 Ti
Ampere · 2020
448 GB/s same
GeForce RTX 4060 Ti
Ada Lovelace · 2023
288 GB/s 0.64×
GeForce RTX 4060
Ada Lovelace · 2023
272 GB/s 0.61×
GeForce RTX 4070 Laptop
Ada Lovelace · 2023
256 GB/s 0.57×
GeForce RTX 4060 Laptop
Ada Lovelace · 2023
256 GB/s 0.57×

Best LLM for 8 GB VRAM, with every card in the class →

What more memory unlocks

At Q4_K_M and 8K context this device fits 28 of 80 models. Each step up, and the largest models it adds:

Every device ranked by what it can run →