Fastest local LLM · 2026
The fastest local LLMs worth running, ranked for your VRAM
At batch 1, generation speed is memory bandwidth divided by the bytes read per token, so the fastest models worth running are mixtures of experts with few active parameters and small dense models. This shortlist is ordered by how much capability each keeps for its speed. For each memory class below we list every model on that list that actually fits at Q4_K_M with an 8K context, so the top row is the pick and the rest are what you give up by choosing it. Numbers are computed, not benchmarked; how the shortlist works.
Tokens per second is not the whole latency. Time to first token is the prompt being read, which is compute-bound and grows with prompt length — a long system prompt or a retrieved document costs time before the first word on any of these.
Fastest good LLM for 6 GB VRAM
5.3 GB usable · speeds for a GeForce RTX 3060 Laptop everything that fits 6 GB →Ling 3.0 Tiny 7.9B-A1.3B is the pick: 1.3B active with a latent cache on 6 of 24 layers. 6 other models on the low-latency shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | Ling 3.0 Tiny 7.9B-A1.3B inclusionAI · MIT |
1.3B active with a latent cache on 6 of 24 layers. | 7.9B MoE | Q4_K_M · 5.1 GB | 39K | 107 | Tight fit |
| 2 | Qwen3.5 4B Alibaba · Apache 2.0 |
Snappy on 8 GB. | 4.66B | Q4_K_M · 3.5 GB | 66K | 78 | Runs great |
| 3 | Gemma 4 E4B Google · Apache 2.0 |
Edge-tuned; a tiny KV cache. | 8.0B | Q4_K_M · 5.2 GB | 12K | 45 | Tight fit |
| 4 | Qwen3.5 2B Alibaba · Apache 2.0 |
Instant on anything. | 2.27B | Q4_K_M · 2.0 GB | 256K | 159 | Runs great |
| 5 | Gemma 4 E2B Google · Apache 2.0 |
Phone-class speed. | 5.1B | Q4_K_M · 3.5 GB | 128K | 71 | Runs great |
| 6 | LFM2.5 2.6B Liquid AI · LFM Open License v1.0 |
Convolution hybrid built for CPUs. | 2.7B | Q4_K_M · 2.2 GB | 128K | 134 | Runs great |
| 7 | Qwen3.5 0.8B Alibaba · Apache 2.0 |
Classifier and draft-model speed. | 0.87B | Q4_K_M · 1.2 GB | 256K | 416 | Runs great |
Fastest good LLM for 8 GB VRAM
7.0 GB usable · speeds for a GeForce RTX 4060 everything that fits 8 GB →LFM2.5 8B-A1B is the pick: 1.5B active; designed for laptops without a GPU. 9 other models on the low-latency shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | LFM2.5 8B-A1B Liquid AI · LFM Open License v1.0 |
1.5B active; designed for laptops without a GPU. | 8.47B MoE | Q4_K_M · 5.5 GB | 125K | 75 | Runs great |
| 2 | Ling 3.0 Tiny 7.9B-A1.3B inclusionAI · MIT |
1.3B active with a latent cache on 6 of 24 layers. | 7.9B MoE | Q4_K_M · 5.1 GB | 128K | 87 | Runs great |
| 3 | Qwen3.5 9B Alibaba · Apache 2.0 |
The fastest dense model that is still a real assistant. | 9.65B | Q4_K_M · 6.3 GB | 31K | 30 | Tight fit |
| 4 | Granite 4.1 8B IBM · Apache 2.0 |
No thinking traces, so answers start immediately. | 8.79B | Q4_K_M · 6.8 GB | 9K | 33 | Tight fit |
| 5 | Qwen3.5 4B Alibaba · Apache 2.0 |
Snappy on 8 GB. | 4.66B | Q4_K_M · 3.5 GB | 120K | 63 | Runs great |
| 6 | Gemma 4 E4B Google · Apache 2.0 |
Edge-tuned; a tiny KV cache. | 8.0B | Q4_K_M · 5.2 GB | 128K | 37 | Runs great |
| 7 | Qwen3.5 2B Alibaba · Apache 2.0 |
Instant on anything. | 2.27B | Q4_K_M · 2.0 GB | 256K | 129 | Runs great |
| 8 | Gemma 4 E2B Google · Apache 2.0 |
Phone-class speed. | 5.1B | Q4_K_M · 3.5 GB | 128K | 57 | Runs great |
| 9 | LFM2.5 2.6B Liquid AI · LFM Open License v1.0 |
Convolution hybrid built for CPUs. | 2.7B | Q4_K_M · 2.2 GB | 128K | 108 | Runs great |
| 10 | Qwen3.5 0.8B Alibaba · Apache 2.0 |
Classifier and draft-model speed. | 0.87B | Q4_K_M · 1.2 GB | 256K | 337 | Runs great |
Fastest good LLM for 12 GB VRAM
10.6 GB usable · speeds for a GeForce RTX 3060 12 GB everything that fits 12 GB →LFM2.5 8B-A1B is the pick: 1.5B active; designed for laptops without a GPU. 9 other models on the low-latency shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | LFM2.5 8B-A1B Liquid AI · LFM Open License v1.0 |
1.5B active; designed for laptops without a GPU. | 8.47B MoE | Q4_K_M · 5.5 GB | 125K | 99 | Runs great |
| 2 | Ling 3.0 Tiny 7.9B-A1.3B inclusionAI · MIT |
1.3B active with a latent cache on 6 of 24 layers. | 7.9B MoE | Q4_K_M · 5.1 GB | 128K | 115 | Runs great |
| 3 | Qwen3.5 9B Alibaba · Apache 2.0 |
The fastest dense model that is still a real assistant. | 9.65B | Q4_K_M · 6.3 GB | 146K | 40 | Runs great |
| 4 | Granite 4.1 8B IBM · Apache 2.0 |
No thinking traces, so answers start immediately. | 8.79B | Q4_K_M · 6.8 GB | 32K | 44 | Runs great |
| 5 | Qwen3.5 4B Alibaba · Apache 2.0 |
Snappy on 8 GB. | 4.66B | Q4_K_M · 3.5 GB | 236K | 83 | Runs great |
| 6 | Gemma 4 E4B Google · Apache 2.0 |
Edge-tuned; a tiny KV cache. | 8.0B | Q4_K_M · 5.2 GB | 128K | 48 | Runs great |
| 7 | Qwen3.5 2B Alibaba · Apache 2.0 |
Instant on anything. | 2.27B | Q4_K_M · 2.0 GB | 256K | 171 | Runs great |
| 8 | Gemma 4 E2B Google · Apache 2.0 |
Phone-class speed. | 5.1B | Q4_K_M · 3.5 GB | 128K | 76 | Runs great |
| 9 | LFM2.5 2.6B Liquid AI · LFM Open License v1.0 |
Convolution hybrid built for CPUs. | 2.7B | Q4_K_M · 2.2 GB | 128K | 144 | Runs great |
| 10 | Qwen3.5 0.8B Alibaba · Apache 2.0 |
Classifier and draft-model speed. | 0.87B | Q4_K_M · 1.2 GB | 256K | 445 | Runs great |
Fastest good LLM for 16 GB VRAM
14.4 GB usable · speeds for a GeForce RTX 5080 everything that fits 16 GB →gpt-oss 20B is the pick: 3.6B active, native 4-bit. 10 other models on the low-latency shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | gpt-oss 20B OpenAI · Apache 2.0 |
3.6B active, native 4-bit. | 20.9B MoE | MXFP4 · 11.6 GB | 126K | 120 | Runs great |
| 2 | LFM2.5 8B-A1B Liquid AI · LFM Open License v1.0 |
1.5B active; designed for laptops without a GPU. | 8.47B MoE | Q4_K_M · 5.5 GB | 125K | 265 | Runs great |
| 3 | Ling 3.0 Tiny 7.9B-A1.3B inclusionAI · MIT |
1.3B active with a latent cache on 6 of 24 layers. | 7.9B MoE | Q4_K_M · 5.1 GB | 128K | 306 | Runs great |
| 4 | Qwen3.5 9B Alibaba · Apache 2.0 |
The fastest dense model that is still a real assistant. | 9.65B | Q4_K_M · 6.3 GB | 256K | 107 | Runs great |
| 5 | Granite 4.1 8B IBM · Apache 2.0 |
No thinking traces, so answers start immediately. | 8.79B | Q4_K_M · 6.8 GB | 56K | 118 | Runs great |
| 6 | Qwen3.5 4B Alibaba · Apache 2.0 |
Snappy on 8 GB. | 4.66B | Q4_K_M · 3.5 GB | 256K | 222 | Runs great |
| 7 | Gemma 4 E4B Google · Apache 2.0 |
Edge-tuned; a tiny KV cache. | 8.0B | Q4_K_M · 5.2 GB | 128K | 129 | Runs great |
| 8 | Qwen3.5 2B Alibaba · Apache 2.0 |
Instant on anything. | 2.27B | Q4_K_M · 2.0 GB | 256K | 455 | Runs great |
| 9 | Gemma 4 E2B Google · Apache 2.0 |
Phone-class speed. | 5.1B | Q4_K_M · 3.5 GB | 128K | 203 | Runs great |
| 10 | LFM2.5 2.6B Liquid AI · LFM Open License v1.0 |
Convolution hybrid built for CPUs. | 2.7B | Q4_K_M · 2.2 GB | 128K | 383 | Runs great |
| 11 | Qwen3.5 0.8B Alibaba · Apache 2.0 |
Classifier and draft-model speed. | 0.87B | Q4_K_M · 1.2 GB | 256K | 1188 | Runs great |
Fastest good LLM for 24 GB VRAM
22.4 GB usable · speeds for a GeForce RTX 4090 everything that fits 24 GB →Qwen3.6 35B-A3B is the pick: 3.3B active per token: the fastest model that still competes with dense 27Bs. 14 other models on the low-latency shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.6 35B-A3B Alibaba · Apache 2.0 |
3.3B active per token: the fastest model that still competes with dense 27Bs. | 35.9B MoE | Q4_K_M · 20.9 GB | 82K | 126 | Tight fit |
| 2 | Gemma 4 26B-A4B Google · Apache 2.0 |
3.8B active and a small file — the quickest Gemma 4 with vision. | 26.5B MoE | Q4_K_M · 16.0 GB | 171K | 110 | Runs great |
| 3 | Nemotron 3.5 Lightning 30B-A3B NVIDIA · OpenMDW-1.1 |
Only 6 attention blocks; NVIDIA claims up to 4× the output speed of its class. | 31.6B MoE | Q4_K_M · 18.4 GB | 256K | 130 | Runs great |
| 4 | GLM-4.7-Flash 30B-A3B Z.ai · MIT |
3B active, 60–80 tok/s reported on a 4090. | 31.2B MoE | Q4_K_M · 18.6 GB | 82K | 139 | Runs great |
| 5 | gpt-oss 20B OpenAI · Apache 2.0 |
3.6B active, native 4-bit. | 20.9B MoE | MXFP4 · 11.6 GB | 128K | 126 | Runs great |
| 6 | LFM2.5 8B-A1B Liquid AI · LFM Open License v1.0 |
1.5B active; designed for laptops without a GPU. | 8.47B MoE | Q4_K_M · 5.5 GB | 125K | 278 | Runs great |
| 7 | Ling 3.0 Tiny 7.9B-A1.3B inclusionAI · MIT |
1.3B active with a latent cache on 6 of 24 layers. | 7.9B MoE | Q4_K_M · 5.1 GB | 128K | 321 | Runs great |
| 8 | Qwen3.5 9B Alibaba · Apache 2.0 |
The fastest dense model that is still a real assistant. | 9.65B | Q4_K_M · 6.3 GB | 256K | 112 | Runs great |
| 9 | Granite 4.1 8B IBM · Apache 2.0 |
No thinking traces, so answers start immediately. | 8.79B | Q4_K_M · 6.8 GB | 107K | 123 | Runs great |
| 10 | Qwen3.5 4B Alibaba · Apache 2.0 |
Snappy on 8 GB. | 4.66B | Q4_K_M · 3.5 GB | 256K | 233 | Runs great |
| 11 | Gemma 4 E4B Google · Apache 2.0 |
Edge-tuned; a tiny KV cache. | 8.0B | Q4_K_M · 5.2 GB | 128K | 136 | Runs great |
| 12 | Qwen3.5 2B Alibaba · Apache 2.0 |
Instant on anything. | 2.27B | Q4_K_M · 2.0 GB | 256K | 478 | Runs great |
| 13 | Gemma 4 E2B Google · Apache 2.0 |
Phone-class speed. | 5.1B | Q4_K_M · 3.5 GB | 128K | 213 | Runs great |
| 14 | LFM2.5 2.6B Liquid AI · LFM Open License v1.0 |
Convolution hybrid built for CPUs. | 2.7B | Q4_K_M · 2.2 GB | 128K | 402 | Runs great |
| 15 | Qwen3.5 0.8B Alibaba · Apache 2.0 |
Classifier and draft-model speed. | 0.87B | Q4_K_M · 1.2 GB | 256K | 1247 | Runs great |
Fastest good LLM for 32 GB VRAM
30.4 GB usable · speeds for a GeForce RTX 5090 everything that fits 32 GB →Qwen3.6 35B-A3B is the pick: 3.3B active per token: the fastest model that still competes with dense 27Bs. 14 other models on the low-latency shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.6 35B-A3B Alibaba · Apache 2.0 |
3.3B active per token: the fastest model that still competes with dense 27Bs. | 35.9B MoE | Q4_K_M · 20.9 GB | 256K | 225 | Runs great |
| 2 | Gemma 4 26B-A4B Google · Apache 2.0 |
3.8B active and a small file — the quickest Gemma 4 with vision. | 26.5B MoE | Q4_K_M · 16.0 GB | 256K | 195 | Runs great |
| 3 | Nemotron 3.5 Lightning 30B-A3B NVIDIA · OpenMDW-1.1 |
Only 6 attention blocks; NVIDIA claims up to 4× the output speed of its class. | 31.6B MoE | Q4_K_M · 18.4 GB | 256K | 232 | Runs great |
| 4 | GLM-4.7-Flash 30B-A3B Z.ai · MIT |
3B active, 60–80 tok/s reported on a 4090. | 31.2B MoE | Q4_K_M · 18.6 GB | 198K | 247 | Runs great |
| 5 | gpt-oss 20B OpenAI · Apache 2.0 |
3.6B active, native 4-bit. | 20.9B MoE | MXFP4 · 11.6 GB | 128K | 224 | Runs great |
| 6 | LFM2.5 8B-A1B Liquid AI · LFM Open License v1.0 |
1.5B active; designed for laptops without a GPU. | 8.47B MoE | Q4_K_M · 5.5 GB | 125K | 495 | Runs great |
| 7 | Ling 3.0 Tiny 7.9B-A1.3B inclusionAI · MIT |
1.3B active with a latent cache on 6 of 24 layers. | 7.9B MoE | Q4_K_M · 5.1 GB | 128K | 571 | Runs great |
| 8 | Qwen3.5 9B Alibaba · Apache 2.0 |
The fastest dense model that is still a real assistant. | 9.65B | Q4_K_M · 6.3 GB | 256K | 200 | Runs great |
| 9 | Granite 4.1 8B IBM · Apache 2.0 |
No thinking traces, so answers start immediately. | 8.79B | Q4_K_M · 6.8 GB | 128K | 219 | Runs great |
| 10 | Qwen3.5 4B Alibaba · Apache 2.0 |
Snappy on 8 GB. | 4.66B | Q4_K_M · 3.5 GB | 256K | 414 | Runs great |
| 11 | Gemma 4 E4B Google · Apache 2.0 |
Edge-tuned; a tiny KV cache. | 8.0B | Q4_K_M · 5.2 GB | 128K | 241 | Runs great |
| 12 | Qwen3.5 2B Alibaba · Apache 2.0 |
Instant on anything. | 2.27B | Q4_K_M · 2.0 GB | 256K | 850 | Runs great |
| 13 | Gemma 4 E2B Google · Apache 2.0 |
Phone-class speed. | 5.1B | Q4_K_M · 3.5 GB | 128K | 378 | Runs great |
| 14 | LFM2.5 2.6B Liquid AI · LFM Open License v1.0 |
Convolution hybrid built for CPUs. | 2.7B | Q4_K_M · 2.2 GB | 128K | 715 | Runs great |
| 15 | Qwen3.5 0.8B Alibaba · Apache 2.0 |
Classifier and draft-model speed. | 0.87B | Q4_K_M · 1.2 GB | 256K | 2218 | Runs great |
Fastest good LLM for 48 GB VRAM
46.4 GB usable · speeds for an RTX 6000 Ada everything that fits 48 GB →Qwen3.6 35B-A3B is the pick: 3.3B active per token: the fastest model that still competes with dense 27Bs. 14 other models on the low-latency shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.6 35B-A3B Alibaba · Apache 2.0 |
3.3B active per token: the fastest model that still competes with dense 27Bs. | 35.9B MoE | Q4_K_M · 20.9 GB | 256K | 120 | Runs great |
| 2 | Gemma 4 26B-A4B Google · Apache 2.0 |
3.8B active and a small file — the quickest Gemma 4 with vision. | 26.5B MoE | Q4_K_M · 16.0 GB | 256K | 105 | Runs great |
| 3 | Nemotron 3.5 Lightning 30B-A3B NVIDIA · OpenMDW-1.1 |
Only 6 attention blocks; NVIDIA claims up to 4× the output speed of its class. | 31.6B MoE | Q4_K_M · 18.4 GB | 256K | 124 | Runs great |
| 4 | GLM-4.7-Flash 30B-A3B Z.ai · MIT |
3B active, 60–80 tok/s reported on a 4090. | 31.2B MoE | Q4_K_M · 18.6 GB | 198K | 133 | Runs great |
| 5 | gpt-oss 20B OpenAI · Apache 2.0 |
3.6B active, native 4-bit. | 20.9B MoE | MXFP4 · 11.6 GB | 128K | 120 | Runs great |
| 6 | LFM2.5 8B-A1B Liquid AI · LFM Open License v1.0 |
1.5B active; designed for laptops without a GPU. | 8.47B MoE | Q4_K_M · 5.5 GB | 125K | 265 | Runs great |
| 7 | Ling 3.0 Tiny 7.9B-A1.3B inclusionAI · MIT |
1.3B active with a latent cache on 6 of 24 layers. | 7.9B MoE | Q4_K_M · 5.1 GB | 128K | 306 | Runs great |
| 8 | Qwen3.5 9B Alibaba · Apache 2.0 |
The fastest dense model that is still a real assistant. | 9.65B | Q4_K_M · 6.3 GB | 256K | 107 | Runs great |
| 9 | Granite 4.1 8B IBM · Apache 2.0 |
No thinking traces, so answers start immediately. | 8.79B | Q4_K_M · 6.8 GB | 128K | 118 | Runs great |
| 10 | Qwen3.5 4B Alibaba · Apache 2.0 |
Snappy on 8 GB. | 4.66B | Q4_K_M · 3.5 GB | 256K | 222 | Runs great |
| 11 | Gemma 4 E4B Google · Apache 2.0 |
Edge-tuned; a tiny KV cache. | 8.0B | Q4_K_M · 5.2 GB | 128K | 129 | Runs great |
| 12 | Qwen3.5 2B Alibaba · Apache 2.0 |
Instant on anything. | 2.27B | Q4_K_M · 2.0 GB | 256K | 455 | Runs great |
| 13 | Gemma 4 E2B Google · Apache 2.0 |
Phone-class speed. | 5.1B | Q4_K_M · 3.5 GB | 128K | 203 | Runs great |
| 14 | LFM2.5 2.6B Liquid AI · LFM Open License v1.0 |
Convolution hybrid built for CPUs. | 2.7B | Q4_K_M · 2.2 GB | 128K | 383 | Runs great |
| 15 | Qwen3.5 0.8B Alibaba · Apache 2.0 |
Classifier and draft-model speed. | 0.87B | Q4_K_M · 1.2 GB | 256K | 1188 | Runs great |
Fastest good LLM for 16 GB unified memory
10.7 GB usable · speeds for an M4 · 16 GB everything that fits 16 GB →LFM2.5 8B-A1B is the pick: 1.5B active; designed for laptops without a GPU. 9 other models on the low-latency shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | LFM2.5 8B-A1B Liquid AI · LFM Open License v1.0 |
1.5B active; designed for laptops without a GPU. | 8.47B MoE | Q4_K_M · 5.5 GB | 125K | 37 | Runs great |
| 2 | Ling 3.0 Tiny 7.9B-A1.3B inclusionAI · MIT |
1.3B active with a latent cache on 6 of 24 layers. | 7.9B MoE | Q4_K_M · 5.1 GB | 128K | 43 | Runs great |
| 3 | Qwen3.5 9B Alibaba · Apache 2.0 |
The fastest dense model that is still a real assistant. | 9.65B | Q4_K_M · 6.3 GB | 149K | 12 | Runs great |
| 4 | Granite 4.1 8B IBM · Apache 2.0 |
No thinking traces, so answers start immediately. | 8.79B | Q4_K_M · 6.8 GB | 33K | 14 | Runs great |
| 5 | Qwen3.5 4B Alibaba · Apache 2.0 |
Snappy on 8 GB. | 4.66B | Q4_K_M · 3.5 GB | 239K | 26 | Runs great |
| 6 | Gemma 4 E4B Google · Apache 2.0 |
Edge-tuned; a tiny KV cache. | 8.0B | Q4_K_M · 5.2 GB | 128K | 15 | Runs great |
| 7 | Qwen3.5 2B Alibaba · Apache 2.0 |
Instant on anything. | 2.27B | Q4_K_M · 2.0 GB | 256K | 53 | Runs great |
| 8 | Gemma 4 E2B Google · Apache 2.0 |
Phone-class speed. | 5.1B | Q4_K_M · 3.5 GB | 128K | 23 | Runs great |
| 9 | LFM2.5 2.6B Liquid AI · LFM Open License v1.0 |
Convolution hybrid built for CPUs. | 2.7B | Q4_K_M · 2.2 GB | 128K | 44 | Runs great |
| 10 | Qwen3.5 0.8B Alibaba · Apache 2.0 |
Classifier and draft-model speed. | 0.87B | Q4_K_M · 1.2 GB | 256K | 137 | Runs great |
Fastest good LLM for 64 GB unified memory
48.0 GB usable · speeds for an M4 Max · 64 GB everything that fits 64 GB →Qwen3.6 35B-A3B is the pick: 3.3B active per token: the fastest model that still competes with dense 27Bs. 14 other models on the low-latency shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.6 35B-A3B Alibaba · Apache 2.0 |
3.3B active per token: the fastest model that still competes with dense 27Bs. | 35.9B MoE | Q4_K_M · 20.9 GB | 256K | 77 | Runs great |
| 2 | Gemma 4 26B-A4B Google · Apache 2.0 |
3.8B active and a small file — the quickest Gemma 4 with vision. | 26.5B MoE | Q4_K_M · 16.0 GB | 256K | 67 | Runs great |
| 3 | Nemotron 3.5 Lightning 30B-A3B NVIDIA · OpenMDW-1.1 |
Only 6 attention blocks; NVIDIA claims up to 4× the output speed of its class. | 31.6B MoE | Q4_K_M · 18.4 GB | 256K | 79 | Runs great |
| 4 | GLM-4.7-Flash 30B-A3B Z.ai · MIT |
3B active, 60–80 tok/s reported on a 4090. | 31.2B MoE | Q4_K_M · 18.6 GB | 198K | 84 | Runs great |
| 5 | gpt-oss 20B OpenAI · Apache 2.0 |
3.6B active, native 4-bit. | 20.9B MoE | MXFP4 · 11.6 GB | 128K | 76 | Runs great |
| 6 | LFM2.5 8B-A1B Liquid AI · LFM Open License v1.0 |
1.5B active; designed for laptops without a GPU. | 8.47B MoE | Q4_K_M · 5.5 GB | 125K | 169 | Runs great |
| 7 | Ling 3.0 Tiny 7.9B-A1.3B inclusionAI · MIT |
1.3B active with a latent cache on 6 of 24 layers. | 7.9B MoE | Q4_K_M · 5.1 GB | 128K | 195 | Runs great |
| 8 | Qwen3.5 9B Alibaba · Apache 2.0 |
The fastest dense model that is still a real assistant. | 9.65B | Q4_K_M · 6.3 GB | 256K | 56 | Runs great |
| 9 | Granite 4.1 8B IBM · Apache 2.0 |
No thinking traces, so answers start immediately. | 8.79B | Q4_K_M · 6.8 GB | 128K | 62 | Runs great |
| 10 | Qwen3.5 4B Alibaba · Apache 2.0 |
Snappy on 8 GB. | 4.66B | Q4_K_M · 3.5 GB | 256K | 116 | Runs great |
| 11 | Gemma 4 E4B Google · Apache 2.0 |
Edge-tuned; a tiny KV cache. | 8.0B | Q4_K_M · 5.2 GB | 128K | 68 | Runs great |
| 12 | Qwen3.5 2B Alibaba · Apache 2.0 |
Instant on anything. | 2.27B | Q4_K_M · 2.0 GB | 256K | 239 | Runs great |
| 13 | Gemma 4 E2B Google · Apache 2.0 |
Phone-class speed. | 5.1B | Q4_K_M · 3.5 GB | 128K | 106 | Runs great |
| 14 | LFM2.5 2.6B Liquid AI · LFM Open License v1.0 |
Convolution hybrid built for CPUs. | 2.7B | Q4_K_M · 2.2 GB | 128K | 201 | Runs great |
| 15 | Qwen3.5 0.8B Alibaba · Apache 2.0 |
Classifier and draft-model speed. | 0.87B | Q4_K_M · 1.2 GB | 256K | 624 | Runs great |
Fastest good LLM for 128 GB unified memory
96.0 GB usable · speeds for an M4 Max · 128 GB everything that fits 128 GB →Qwen3.6 35B-A3B is the pick: 3.3B active per token: the fastest model that still competes with dense 27Bs. 14 other models on the low-latency shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.6 35B-A3B Alibaba · Apache 2.0 |
3.3B active per token: the fastest model that still competes with dense 27Bs. | 35.9B MoE | Q4_K_M · 20.9 GB | 256K | 77 | Runs great |
| 2 | Gemma 4 26B-A4B Google · Apache 2.0 |
3.8B active and a small file — the quickest Gemma 4 with vision. | 26.5B MoE | Q4_K_M · 16.0 GB | 256K | 67 | Runs great |
| 3 | Nemotron 3.5 Lightning 30B-A3B NVIDIA · OpenMDW-1.1 |
Only 6 attention blocks; NVIDIA claims up to 4× the output speed of its class. | 31.6B MoE | Q4_K_M · 18.4 GB | 256K | 79 | Runs great |
| 4 | GLM-4.7-Flash 30B-A3B Z.ai · MIT |
3B active, 60–80 tok/s reported on a 4090. | 31.2B MoE | Q4_K_M · 18.6 GB | 198K | 84 | Runs great |
| 5 | gpt-oss 20B OpenAI · Apache 2.0 |
3.6B active, native 4-bit. | 20.9B MoE | MXFP4 · 11.6 GB | 128K | 76 | Runs great |
| 6 | LFM2.5 8B-A1B Liquid AI · LFM Open License v1.0 |
1.5B active; designed for laptops without a GPU. | 8.47B MoE | Q4_K_M · 5.5 GB | 125K | 169 | Runs great |
| 7 | Ling 3.0 Tiny 7.9B-A1.3B inclusionAI · MIT |
1.3B active with a latent cache on 6 of 24 layers. | 7.9B MoE | Q4_K_M · 5.1 GB | 128K | 195 | Runs great |
| 8 | Qwen3.5 9B Alibaba · Apache 2.0 |
The fastest dense model that is still a real assistant. | 9.65B | Q4_K_M · 6.3 GB | 256K | 56 | Runs great |
| 9 | Granite 4.1 8B IBM · Apache 2.0 |
No thinking traces, so answers start immediately. | 8.79B | Q4_K_M · 6.8 GB | 128K | 62 | Runs great |
| 10 | Qwen3.5 4B Alibaba · Apache 2.0 |
Snappy on 8 GB. | 4.66B | Q4_K_M · 3.5 GB | 256K | 116 | Runs great |
| 11 | Gemma 4 E4B Google · Apache 2.0 |
Edge-tuned; a tiny KV cache. | 8.0B | Q4_K_M · 5.2 GB | 128K | 68 | Runs great |
| 12 | Qwen3.5 2B Alibaba · Apache 2.0 |
Instant on anything. | 2.27B | Q4_K_M · 2.0 GB | 256K | 239 | Runs great |
| 13 | Gemma 4 E2B Google · Apache 2.0 |
Phone-class speed. | 5.1B | Q4_K_M · 3.5 GB | 128K | 106 | Runs great |
| 14 | LFM2.5 2.6B Liquid AI · LFM Open License v1.0 |
Convolution hybrid built for CPUs. | 2.7B | Q4_K_M · 2.2 GB | 128K | 201 | Runs great |
| 15 | Qwen3.5 0.8B Alibaba · Apache 2.0 |
Classifier and draft-model speed. | 0.87B | Q4_K_M · 1.2 GB | 256K | 624 | Runs great |
The whole low-latency shortlist, and what each one needs
Every model on the list, best first, with the smallest discrete card and the smallest Mac that run it with 15% headroom at the recommended quantisation and 8K context. Every class has its own page, Macs from 8 to 512 GB included — the full list.
| # | Model | Params | Weights | Needs (GPU) | Needs (Mac) | Ollama tag |
|---|---|---|---|---|---|---|
| 1 | Qwen3.8-Flash-Next 180B-A6B The open preview of the Qwen4 architecture: a 125B-A6B hybrid (Gated DeltaNet + sparse attention, KV cache on 12 of 48 blocks) plus a 51B n-gram embedding and a 4B draft head — 180B on disk, 6B active. The hosted "Qwen3.8-Flash" is this model with a 1M window. |
180B MoE | 101.2 GB @ Q4_K_M | 141 GB card | 192 GB Mac | qwen3.8-flash-next:125b-a6b-nvfp4 |
| 2 | Qwen3.6 35B-A3B Mixture of experts with ~3B active: the fastest serious model a 24 GB card runs, and the best MoE under 40B on agentic coding. |
35.9B MoE | 20.2 GB @ Q4_K_M | 32 GB card | 36 GB Mac | qwen3.6:35b-a3b |
| 3 | Gemma 4 26B-A4B Mixture of experts with 3.8B active. Slower to think than Qwen3.6 35B-A3B, faster to answer, and it sees images. |
26.5B MoE | 14.9 GB @ Q4_K_M | 24 GB card | 32 GB Mac | gemma4:26b |
| 4 | Nemotron 3.5 Lightning 30B-A3B Mamba-2 + MoE hybrid built for the execution layer of agents: only 6 attention blocks, so the KV cache is almost free. Weights, data and recipe all open. |
31.6B MoE | 17.8 GB @ Q4_K_M | 24 GB card | 32 GB Mac | nemotron-3.5-lightning:30b-a3b |
| 5 | GLM-4.7-Flash 30B-A3B MIT-licensed 30B-A3B tuned for agentic coding, with a DeepSeek-style latent KV cache. 60–80 tok/s reported on a 4090. |
31.2B MoE | 17.5 GB @ Q4_K_M | 24 GB card | 32 GB Mac | glm-4.7-flash:latest |
| 6 | gpt-oss 20B Ships natively in MXFP4, so the 4-bit weights are the reference weights, not a lossy copy. Fits 16 GB. |
20.9B MoE | 10.8 GB @ MXFP4 | 16 GB card | 24 GB Mac | gpt-oss:20b |
| 7 | LFM2.5 8B-A1B An 8B MoE with ~1.5B active, aimed at laptops without a GPU. Licence is permissive below $10M revenue. |
8.47B MoE | 4.8 GB @ Q4_K_M | 8 GB card | 16 GB Mac | lfm2.5:8b |
| 8 | Ling 3.0 Tiny 7.9B-A1.3B An 8B MoE with 1.3B active and a latent KV cache on only 6 of 24 layers — reasoning and tool use sized for Apple Silicon and edge boxes. |
7.9B MoE | 4.4 GB @ Q4_K_M | 8 GB card | 16 GB Mac | — |
| 9 | Qwen3.5 9B The default for 8–12 GB cards in 2026: beats every older 8B on every published benchmark, with vision. |
9.65B | 5.4 GB @ Q4_K_M | 10 GB card | 16 GB Mac | qwen3.5:9b |
| 10 | Granite 4.1 8B Matches the old Granite 4.0 32B MoE at a quarter of the size. Fast, Apache 2.0, no reasoning traces. |
8.79B | 4.9 GB @ Q4_K_M | 10 GB card | 16 GB Mac | granite4.1:8b |
| 11 | Qwen3.5 4B The 8 GB coding agent. Q4 lands near 3.4 GB, leaving room for a real context window. |
4.66B | 2.6 GB @ Q4_K_M | 6 GB card | 8 GB Mac | qwen3.5:4b |
| 12 | Gemma 4 E4B The laptop Gemma. 4.5B effective, 8B on disk; a single KV head per window layer keeps its cache tiny. |
8.0B | 4.5 GB @ Q4_K_M | 8 GB card | 16 GB Mac | gemma4:e4b |
| 13 | Qwen3.5 2B Phone-class, and multimodal. Replaces Llama 3.2 3B as the "it runs on anything" answer. |
2.27B | 1.3 GB @ Q4_K_M | 6 GB card | 8 GB Mac | qwen3.5:2b |
| 14 | Gemma 4 E2B "E2B" is 2.3B effective, but the file holds 5B because of per-layer embeddings — size it as 5B. Text, image and audio in. |
5.1B | 2.9 GB @ Q4_K_M | 6 GB card | 8 GB Mac | gemma4:e2b |
| 15 | LFM2.5 2.6B Convolution-heavy hybrid for CPUs and NPUs: 22 of 30 blocks keep no KV cache at all. |
2.7B | 1.5 GB @ Q4_K_M | 6 GB card | 8 GB Mac | — |
| 16 | Qwen3.5 0.8B Draft model for speculative decoding, or a classifier that fits in 1 GB. |
0.87B | 0.5 GB @ Q4_K_M | 6 GB card | 8 GB Mac | qwen3.5:0.8b |
A fast model that leaves headroom also leaves room for a draft model. Speculative decoding (llama.cpp's --model-draft) lets a small model propose tokens that a larger one checks in a batch, which often speeds up predictable text such as code by half again or more.
Other jobs