Fastest local LLM · 2026

The fastest local LLMs worth running, ranked for your VRAM

Updated 8 Oct 2026 1 new this month

At batch 1, generation speed is memory bandwidth divided by the bytes read per token, so the fastest models worth running are mixtures of experts with few active parameters and small dense models. This shortlist is ordered by how much capability each keeps for its speed. For each memory class below we list every model on that list that actually fits at Q4_K_M with an 8K context, so the top row is the pick and the rest are what you give up by choosing it. Numbers are computed, not benchmarked; how the shortlist works.

Tokens per second is not the whole latency. Time to first token is the prompt being read, which is compute-bound and grows with prompt length — a long system prompt or a retrieved document costs time before the first word on any of these.

Fastest good LLM for 6 GB VRAM

5.3 GB usable · speeds for a GeForce RTX 3060 Laptop everything that fits 6 GB →

Ling 3.0 Tiny 7.9B-A1.3B is the pick: 1.3B active with a latent cache on 6 of 24 layers. 6 other models on the low-latency shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 Ling 3.0 Tiny 7.9B-A1.3B
inclusionAI · MIT
1.3B active with a latent cache on 6 of 24 layers. 7.9B MoE Q4_K_M · 5.1 GB 39K 107 Tight fit
2 Qwen3.5 4B
Alibaba · Apache 2.0
Snappy on 8 GB. 4.66B Q4_K_M · 3.5 GB 66K 78 Runs great
3 Gemma 4 E4B
Google · Apache 2.0
Edge-tuned; a tiny KV cache. 8.0B Q4_K_M · 5.2 GB 12K 45 Tight fit
4 Qwen3.5 2B
Alibaba · Apache 2.0
Instant on anything. 2.27B Q4_K_M · 2.0 GB 256K 159 Runs great
5 Gemma 4 E2B
Google · Apache 2.0
Phone-class speed. 5.1B Q4_K_M · 3.5 GB 128K 71 Runs great
6 LFM2.5 2.6B
Liquid AI · LFM Open License v1.0
Convolution hybrid built for CPUs. 2.7B Q4_K_M · 2.2 GB 128K 134 Runs great
7 Qwen3.5 0.8B
Alibaba · Apache 2.0
Classifier and draft-model speed. 0.87B Q4_K_M · 1.2 GB 256K 416 Runs great

Fastest good LLM for 8 GB VRAM

7.0 GB usable · speeds for a GeForce RTX 4060 everything that fits 8 GB →

LFM2.5 8B-A1B is the pick: 1.5B active; designed for laptops without a GPU. 9 other models on the low-latency shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 LFM2.5 8B-A1B
Liquid AI · LFM Open License v1.0
1.5B active; designed for laptops without a GPU. 8.47B MoE Q4_K_M · 5.5 GB 125K 75 Runs great
2 Ling 3.0 Tiny 7.9B-A1.3B
inclusionAI · MIT
1.3B active with a latent cache on 6 of 24 layers. 7.9B MoE Q4_K_M · 5.1 GB 128K 87 Runs great
3 Qwen3.5 9B
Alibaba · Apache 2.0
The fastest dense model that is still a real assistant. 9.65B Q4_K_M · 6.3 GB 31K 30 Tight fit
4 Granite 4.1 8B
IBM · Apache 2.0
No thinking traces, so answers start immediately. 8.79B Q4_K_M · 6.8 GB 9K 33 Tight fit
5 Qwen3.5 4B
Alibaba · Apache 2.0
Snappy on 8 GB. 4.66B Q4_K_M · 3.5 GB 120K 63 Runs great
6 Gemma 4 E4B
Google · Apache 2.0
Edge-tuned; a tiny KV cache. 8.0B Q4_K_M · 5.2 GB 128K 37 Runs great
7 Qwen3.5 2B
Alibaba · Apache 2.0
Instant on anything. 2.27B Q4_K_M · 2.0 GB 256K 129 Runs great
8 Gemma 4 E2B
Google · Apache 2.0
Phone-class speed. 5.1B Q4_K_M · 3.5 GB 128K 57 Runs great
9 LFM2.5 2.6B
Liquid AI · LFM Open License v1.0
Convolution hybrid built for CPUs. 2.7B Q4_K_M · 2.2 GB 128K 108 Runs great
10 Qwen3.5 0.8B
Alibaba · Apache 2.0
Classifier and draft-model speed. 0.87B Q4_K_M · 1.2 GB 256K 337 Runs great

Fastest good LLM for 12 GB VRAM

10.6 GB usable · speeds for a GeForce RTX 3060 12 GB everything that fits 12 GB →

LFM2.5 8B-A1B is the pick: 1.5B active; designed for laptops without a GPU. 9 other models on the low-latency shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 LFM2.5 8B-A1B
Liquid AI · LFM Open License v1.0
1.5B active; designed for laptops without a GPU. 8.47B MoE Q4_K_M · 5.5 GB 125K 99 Runs great
2 Ling 3.0 Tiny 7.9B-A1.3B
inclusionAI · MIT
1.3B active with a latent cache on 6 of 24 layers. 7.9B MoE Q4_K_M · 5.1 GB 128K 115 Runs great
3 Qwen3.5 9B
Alibaba · Apache 2.0
The fastest dense model that is still a real assistant. 9.65B Q4_K_M · 6.3 GB 146K 40 Runs great
4 Granite 4.1 8B
IBM · Apache 2.0
No thinking traces, so answers start immediately. 8.79B Q4_K_M · 6.8 GB 32K 44 Runs great
5 Qwen3.5 4B
Alibaba · Apache 2.0
Snappy on 8 GB. 4.66B Q4_K_M · 3.5 GB 236K 83 Runs great
6 Gemma 4 E4B
Google · Apache 2.0
Edge-tuned; a tiny KV cache. 8.0B Q4_K_M · 5.2 GB 128K 48 Runs great
7 Qwen3.5 2B
Alibaba · Apache 2.0
Instant on anything. 2.27B Q4_K_M · 2.0 GB 256K 171 Runs great
8 Gemma 4 E2B
Google · Apache 2.0
Phone-class speed. 5.1B Q4_K_M · 3.5 GB 128K 76 Runs great
9 LFM2.5 2.6B
Liquid AI · LFM Open License v1.0
Convolution hybrid built for CPUs. 2.7B Q4_K_M · 2.2 GB 128K 144 Runs great
10 Qwen3.5 0.8B
Alibaba · Apache 2.0
Classifier and draft-model speed. 0.87B Q4_K_M · 1.2 GB 256K 445 Runs great

Fastest good LLM for 16 GB VRAM

14.4 GB usable · speeds for a GeForce RTX 5080 everything that fits 16 GB →

gpt-oss 20B is the pick: 3.6B active, native 4-bit. 10 other models on the low-latency shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 gpt-oss 20B
OpenAI · Apache 2.0
3.6B active, native 4-bit. 20.9B MoE MXFP4 · 11.6 GB 126K 120 Runs great
2 LFM2.5 8B-A1B
Liquid AI · LFM Open License v1.0
1.5B active; designed for laptops without a GPU. 8.47B MoE Q4_K_M · 5.5 GB 125K 265 Runs great
3 Ling 3.0 Tiny 7.9B-A1.3B
inclusionAI · MIT
1.3B active with a latent cache on 6 of 24 layers. 7.9B MoE Q4_K_M · 5.1 GB 128K 306 Runs great
4 Qwen3.5 9B
Alibaba · Apache 2.0
The fastest dense model that is still a real assistant. 9.65B Q4_K_M · 6.3 GB 256K 107 Runs great
5 Granite 4.1 8B
IBM · Apache 2.0
No thinking traces, so answers start immediately. 8.79B Q4_K_M · 6.8 GB 56K 118 Runs great
6 Qwen3.5 4B
Alibaba · Apache 2.0
Snappy on 8 GB. 4.66B Q4_K_M · 3.5 GB 256K 222 Runs great
7 Gemma 4 E4B
Google · Apache 2.0
Edge-tuned; a tiny KV cache. 8.0B Q4_K_M · 5.2 GB 128K 129 Runs great
8 Qwen3.5 2B
Alibaba · Apache 2.0
Instant on anything. 2.27B Q4_K_M · 2.0 GB 256K 455 Runs great
9 Gemma 4 E2B
Google · Apache 2.0
Phone-class speed. 5.1B Q4_K_M · 3.5 GB 128K 203 Runs great
10 LFM2.5 2.6B
Liquid AI · LFM Open License v1.0
Convolution hybrid built for CPUs. 2.7B Q4_K_M · 2.2 GB 128K 383 Runs great
11 Qwen3.5 0.8B
Alibaba · Apache 2.0
Classifier and draft-model speed. 0.87B Q4_K_M · 1.2 GB 256K 1188 Runs great

Fastest good LLM for 24 GB VRAM

22.4 GB usable · speeds for a GeForce RTX 4090 everything that fits 24 GB →

Qwen3.6 35B-A3B is the pick: 3.3B active per token: the fastest model that still competes with dense 27Bs. 14 other models on the low-latency shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 Qwen3.6 35B-A3B
Alibaba · Apache 2.0
3.3B active per token: the fastest model that still competes with dense 27Bs. 35.9B MoE Q4_K_M · 20.9 GB 82K 126 Tight fit
2 Gemma 4 26B-A4B
Google · Apache 2.0
3.8B active and a small file — the quickest Gemma 4 with vision. 26.5B MoE Q4_K_M · 16.0 GB 171K 110 Runs great
3 Nemotron 3.5 Lightning 30B-A3B
NVIDIA · OpenMDW-1.1
Only 6 attention blocks; NVIDIA claims up to 4× the output speed of its class. 31.6B MoE Q4_K_M · 18.4 GB 256K 130 Runs great
4 GLM-4.7-Flash 30B-A3B
Z.ai · MIT
3B active, 60–80 tok/s reported on a 4090. 31.2B MoE Q4_K_M · 18.6 GB 82K 139 Runs great
5 gpt-oss 20B
OpenAI · Apache 2.0
3.6B active, native 4-bit. 20.9B MoE MXFP4 · 11.6 GB 128K 126 Runs great
6 LFM2.5 8B-A1B
Liquid AI · LFM Open License v1.0
1.5B active; designed for laptops without a GPU. 8.47B MoE Q4_K_M · 5.5 GB 125K 278 Runs great
7 Ling 3.0 Tiny 7.9B-A1.3B
inclusionAI · MIT
1.3B active with a latent cache on 6 of 24 layers. 7.9B MoE Q4_K_M · 5.1 GB 128K 321 Runs great
8 Qwen3.5 9B
Alibaba · Apache 2.0
The fastest dense model that is still a real assistant. 9.65B Q4_K_M · 6.3 GB 256K 112 Runs great
9 Granite 4.1 8B
IBM · Apache 2.0
No thinking traces, so answers start immediately. 8.79B Q4_K_M · 6.8 GB 107K 123 Runs great
10 Qwen3.5 4B
Alibaba · Apache 2.0
Snappy on 8 GB. 4.66B Q4_K_M · 3.5 GB 256K 233 Runs great
11 Gemma 4 E4B
Google · Apache 2.0
Edge-tuned; a tiny KV cache. 8.0B Q4_K_M · 5.2 GB 128K 136 Runs great
12 Qwen3.5 2B
Alibaba · Apache 2.0
Instant on anything. 2.27B Q4_K_M · 2.0 GB 256K 478 Runs great
13 Gemma 4 E2B
Google · Apache 2.0
Phone-class speed. 5.1B Q4_K_M · 3.5 GB 128K 213 Runs great
14 LFM2.5 2.6B
Liquid AI · LFM Open License v1.0
Convolution hybrid built for CPUs. 2.7B Q4_K_M · 2.2 GB 128K 402 Runs great
15 Qwen3.5 0.8B
Alibaba · Apache 2.0
Classifier and draft-model speed. 0.87B Q4_K_M · 1.2 GB 256K 1247 Runs great

Fastest good LLM for 32 GB VRAM

30.4 GB usable · speeds for a GeForce RTX 5090 everything that fits 32 GB →

Qwen3.6 35B-A3B is the pick: 3.3B active per token: the fastest model that still competes with dense 27Bs. 14 other models on the low-latency shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 Qwen3.6 35B-A3B
Alibaba · Apache 2.0
3.3B active per token: the fastest model that still competes with dense 27Bs. 35.9B MoE Q4_K_M · 20.9 GB 256K 225 Runs great
2 Gemma 4 26B-A4B
Google · Apache 2.0
3.8B active and a small file — the quickest Gemma 4 with vision. 26.5B MoE Q4_K_M · 16.0 GB 256K 195 Runs great
3 Nemotron 3.5 Lightning 30B-A3B
NVIDIA · OpenMDW-1.1
Only 6 attention blocks; NVIDIA claims up to 4× the output speed of its class. 31.6B MoE Q4_K_M · 18.4 GB 256K 232 Runs great
4 GLM-4.7-Flash 30B-A3B
Z.ai · MIT
3B active, 60–80 tok/s reported on a 4090. 31.2B MoE Q4_K_M · 18.6 GB 198K 247 Runs great
5 gpt-oss 20B
OpenAI · Apache 2.0
3.6B active, native 4-bit. 20.9B MoE MXFP4 · 11.6 GB 128K 224 Runs great
6 LFM2.5 8B-A1B
Liquid AI · LFM Open License v1.0
1.5B active; designed for laptops without a GPU. 8.47B MoE Q4_K_M · 5.5 GB 125K 495 Runs great
7 Ling 3.0 Tiny 7.9B-A1.3B
inclusionAI · MIT
1.3B active with a latent cache on 6 of 24 layers. 7.9B MoE Q4_K_M · 5.1 GB 128K 571 Runs great
8 Qwen3.5 9B
Alibaba · Apache 2.0
The fastest dense model that is still a real assistant. 9.65B Q4_K_M · 6.3 GB 256K 200 Runs great
9 Granite 4.1 8B
IBM · Apache 2.0
No thinking traces, so answers start immediately. 8.79B Q4_K_M · 6.8 GB 128K 219 Runs great
10 Qwen3.5 4B
Alibaba · Apache 2.0
Snappy on 8 GB. 4.66B Q4_K_M · 3.5 GB 256K 414 Runs great
11 Gemma 4 E4B
Google · Apache 2.0
Edge-tuned; a tiny KV cache. 8.0B Q4_K_M · 5.2 GB 128K 241 Runs great
12 Qwen3.5 2B
Alibaba · Apache 2.0
Instant on anything. 2.27B Q4_K_M · 2.0 GB 256K 850 Runs great
13 Gemma 4 E2B
Google · Apache 2.0
Phone-class speed. 5.1B Q4_K_M · 3.5 GB 128K 378 Runs great
14 LFM2.5 2.6B
Liquid AI · LFM Open License v1.0
Convolution hybrid built for CPUs. 2.7B Q4_K_M · 2.2 GB 128K 715 Runs great
15 Qwen3.5 0.8B
Alibaba · Apache 2.0
Classifier and draft-model speed. 0.87B Q4_K_M · 1.2 GB 256K 2218 Runs great

Fastest good LLM for 48 GB VRAM

46.4 GB usable · speeds for an RTX 6000 Ada everything that fits 48 GB →

Qwen3.6 35B-A3B is the pick: 3.3B active per token: the fastest model that still competes with dense 27Bs. 14 other models on the low-latency shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 Qwen3.6 35B-A3B
Alibaba · Apache 2.0
3.3B active per token: the fastest model that still competes with dense 27Bs. 35.9B MoE Q4_K_M · 20.9 GB 256K 120 Runs great
2 Gemma 4 26B-A4B
Google · Apache 2.0
3.8B active and a small file — the quickest Gemma 4 with vision. 26.5B MoE Q4_K_M · 16.0 GB 256K 105 Runs great
3 Nemotron 3.5 Lightning 30B-A3B
NVIDIA · OpenMDW-1.1
Only 6 attention blocks; NVIDIA claims up to 4× the output speed of its class. 31.6B MoE Q4_K_M · 18.4 GB 256K 124 Runs great
4 GLM-4.7-Flash 30B-A3B
Z.ai · MIT
3B active, 60–80 tok/s reported on a 4090. 31.2B MoE Q4_K_M · 18.6 GB 198K 133 Runs great
5 gpt-oss 20B
OpenAI · Apache 2.0
3.6B active, native 4-bit. 20.9B MoE MXFP4 · 11.6 GB 128K 120 Runs great
6 LFM2.5 8B-A1B
Liquid AI · LFM Open License v1.0
1.5B active; designed for laptops without a GPU. 8.47B MoE Q4_K_M · 5.5 GB 125K 265 Runs great
7 Ling 3.0 Tiny 7.9B-A1.3B
inclusionAI · MIT
1.3B active with a latent cache on 6 of 24 layers. 7.9B MoE Q4_K_M · 5.1 GB 128K 306 Runs great
8 Qwen3.5 9B
Alibaba · Apache 2.0
The fastest dense model that is still a real assistant. 9.65B Q4_K_M · 6.3 GB 256K 107 Runs great
9 Granite 4.1 8B
IBM · Apache 2.0
No thinking traces, so answers start immediately. 8.79B Q4_K_M · 6.8 GB 128K 118 Runs great
10 Qwen3.5 4B
Alibaba · Apache 2.0
Snappy on 8 GB. 4.66B Q4_K_M · 3.5 GB 256K 222 Runs great
11 Gemma 4 E4B
Google · Apache 2.0
Edge-tuned; a tiny KV cache. 8.0B Q4_K_M · 5.2 GB 128K 129 Runs great
12 Qwen3.5 2B
Alibaba · Apache 2.0
Instant on anything. 2.27B Q4_K_M · 2.0 GB 256K 455 Runs great
13 Gemma 4 E2B
Google · Apache 2.0
Phone-class speed. 5.1B Q4_K_M · 3.5 GB 128K 203 Runs great
14 LFM2.5 2.6B
Liquid AI · LFM Open License v1.0
Convolution hybrid built for CPUs. 2.7B Q4_K_M · 2.2 GB 128K 383 Runs great
15 Qwen3.5 0.8B
Alibaba · Apache 2.0
Classifier and draft-model speed. 0.87B Q4_K_M · 1.2 GB 256K 1188 Runs great

Fastest good LLM for 16 GB unified memory

10.7 GB usable · speeds for an M4 · 16 GB everything that fits 16 GB →

LFM2.5 8B-A1B is the pick: 1.5B active; designed for laptops without a GPU. 9 other models on the low-latency shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 LFM2.5 8B-A1B
Liquid AI · LFM Open License v1.0
1.5B active; designed for laptops without a GPU. 8.47B MoE Q4_K_M · 5.5 GB 125K 37 Runs great
2 Ling 3.0 Tiny 7.9B-A1.3B
inclusionAI · MIT
1.3B active with a latent cache on 6 of 24 layers. 7.9B MoE Q4_K_M · 5.1 GB 128K 43 Runs great
3 Qwen3.5 9B
Alibaba · Apache 2.0
The fastest dense model that is still a real assistant. 9.65B Q4_K_M · 6.3 GB 149K 12 Runs great
4 Granite 4.1 8B
IBM · Apache 2.0
No thinking traces, so answers start immediately. 8.79B Q4_K_M · 6.8 GB 33K 14 Runs great
5 Qwen3.5 4B
Alibaba · Apache 2.0
Snappy on 8 GB. 4.66B Q4_K_M · 3.5 GB 239K 26 Runs great
6 Gemma 4 E4B
Google · Apache 2.0
Edge-tuned; a tiny KV cache. 8.0B Q4_K_M · 5.2 GB 128K 15 Runs great
7 Qwen3.5 2B
Alibaba · Apache 2.0
Instant on anything. 2.27B Q4_K_M · 2.0 GB 256K 53 Runs great
8 Gemma 4 E2B
Google · Apache 2.0
Phone-class speed. 5.1B Q4_K_M · 3.5 GB 128K 23 Runs great
9 LFM2.5 2.6B
Liquid AI · LFM Open License v1.0
Convolution hybrid built for CPUs. 2.7B Q4_K_M · 2.2 GB 128K 44 Runs great
10 Qwen3.5 0.8B
Alibaba · Apache 2.0
Classifier and draft-model speed. 0.87B Q4_K_M · 1.2 GB 256K 137 Runs great

Fastest good LLM for 64 GB unified memory

48.0 GB usable · speeds for an M4 Max · 64 GB everything that fits 64 GB →

Qwen3.6 35B-A3B is the pick: 3.3B active per token: the fastest model that still competes with dense 27Bs. 14 other models on the low-latency shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 Qwen3.6 35B-A3B
Alibaba · Apache 2.0
3.3B active per token: the fastest model that still competes with dense 27Bs. 35.9B MoE Q4_K_M · 20.9 GB 256K 77 Runs great
2 Gemma 4 26B-A4B
Google · Apache 2.0
3.8B active and a small file — the quickest Gemma 4 with vision. 26.5B MoE Q4_K_M · 16.0 GB 256K 67 Runs great
3 Nemotron 3.5 Lightning 30B-A3B
NVIDIA · OpenMDW-1.1
Only 6 attention blocks; NVIDIA claims up to 4× the output speed of its class. 31.6B MoE Q4_K_M · 18.4 GB 256K 79 Runs great
4 GLM-4.7-Flash 30B-A3B
Z.ai · MIT
3B active, 60–80 tok/s reported on a 4090. 31.2B MoE Q4_K_M · 18.6 GB 198K 84 Runs great
5 gpt-oss 20B
OpenAI · Apache 2.0
3.6B active, native 4-bit. 20.9B MoE MXFP4 · 11.6 GB 128K 76 Runs great
6 LFM2.5 8B-A1B
Liquid AI · LFM Open License v1.0
1.5B active; designed for laptops without a GPU. 8.47B MoE Q4_K_M · 5.5 GB 125K 169 Runs great
7 Ling 3.0 Tiny 7.9B-A1.3B
inclusionAI · MIT
1.3B active with a latent cache on 6 of 24 layers. 7.9B MoE Q4_K_M · 5.1 GB 128K 195 Runs great
8 Qwen3.5 9B
Alibaba · Apache 2.0
The fastest dense model that is still a real assistant. 9.65B Q4_K_M · 6.3 GB 256K 56 Runs great
9 Granite 4.1 8B
IBM · Apache 2.0
No thinking traces, so answers start immediately. 8.79B Q4_K_M · 6.8 GB 128K 62 Runs great
10 Qwen3.5 4B
Alibaba · Apache 2.0
Snappy on 8 GB. 4.66B Q4_K_M · 3.5 GB 256K 116 Runs great
11 Gemma 4 E4B
Google · Apache 2.0
Edge-tuned; a tiny KV cache. 8.0B Q4_K_M · 5.2 GB 128K 68 Runs great
12 Qwen3.5 2B
Alibaba · Apache 2.0
Instant on anything. 2.27B Q4_K_M · 2.0 GB 256K 239 Runs great
13 Gemma 4 E2B
Google · Apache 2.0
Phone-class speed. 5.1B Q4_K_M · 3.5 GB 128K 106 Runs great
14 LFM2.5 2.6B
Liquid AI · LFM Open License v1.0
Convolution hybrid built for CPUs. 2.7B Q4_K_M · 2.2 GB 128K 201 Runs great
15 Qwen3.5 0.8B
Alibaba · Apache 2.0
Classifier and draft-model speed. 0.87B Q4_K_M · 1.2 GB 256K 624 Runs great

Fastest good LLM for 128 GB unified memory

96.0 GB usable · speeds for an M4 Max · 128 GB everything that fits 128 GB →

Qwen3.6 35B-A3B is the pick: 3.3B active per token: the fastest model that still competes with dense 27Bs. 14 other models on the low-latency shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 Qwen3.6 35B-A3B
Alibaba · Apache 2.0
3.3B active per token: the fastest model that still competes with dense 27Bs. 35.9B MoE Q4_K_M · 20.9 GB 256K 77 Runs great
2 Gemma 4 26B-A4B
Google · Apache 2.0
3.8B active and a small file — the quickest Gemma 4 with vision. 26.5B MoE Q4_K_M · 16.0 GB 256K 67 Runs great
3 Nemotron 3.5 Lightning 30B-A3B
NVIDIA · OpenMDW-1.1
Only 6 attention blocks; NVIDIA claims up to 4× the output speed of its class. 31.6B MoE Q4_K_M · 18.4 GB 256K 79 Runs great
4 GLM-4.7-Flash 30B-A3B
Z.ai · MIT
3B active, 60–80 tok/s reported on a 4090. 31.2B MoE Q4_K_M · 18.6 GB 198K 84 Runs great
5 gpt-oss 20B
OpenAI · Apache 2.0
3.6B active, native 4-bit. 20.9B MoE MXFP4 · 11.6 GB 128K 76 Runs great
6 LFM2.5 8B-A1B
Liquid AI · LFM Open License v1.0
1.5B active; designed for laptops without a GPU. 8.47B MoE Q4_K_M · 5.5 GB 125K 169 Runs great
7 Ling 3.0 Tiny 7.9B-A1.3B
inclusionAI · MIT
1.3B active with a latent cache on 6 of 24 layers. 7.9B MoE Q4_K_M · 5.1 GB 128K 195 Runs great
8 Qwen3.5 9B
Alibaba · Apache 2.0
The fastest dense model that is still a real assistant. 9.65B Q4_K_M · 6.3 GB 256K 56 Runs great
9 Granite 4.1 8B
IBM · Apache 2.0
No thinking traces, so answers start immediately. 8.79B Q4_K_M · 6.8 GB 128K 62 Runs great
10 Qwen3.5 4B
Alibaba · Apache 2.0
Snappy on 8 GB. 4.66B Q4_K_M · 3.5 GB 256K 116 Runs great
11 Gemma 4 E4B
Google · Apache 2.0
Edge-tuned; a tiny KV cache. 8.0B Q4_K_M · 5.2 GB 128K 68 Runs great
12 Qwen3.5 2B
Alibaba · Apache 2.0
Instant on anything. 2.27B Q4_K_M · 2.0 GB 256K 239 Runs great
13 Gemma 4 E2B
Google · Apache 2.0
Phone-class speed. 5.1B Q4_K_M · 3.5 GB 128K 106 Runs great
14 LFM2.5 2.6B
Liquid AI · LFM Open License v1.0
Convolution hybrid built for CPUs. 2.7B Q4_K_M · 2.2 GB 128K 201 Runs great
15 Qwen3.5 0.8B
Alibaba · Apache 2.0
Classifier and draft-model speed. 0.87B Q4_K_M · 1.2 GB 256K 624 Runs great

The whole low-latency shortlist, and what each one needs

Every model on the list, best first, with the smallest discrete card and the smallest Mac that run it with 15% headroom at the recommended quantisation and 8K context. Every class has its own page, Macs from 8 to 512 GB included — the full list.

#ModelParamsWeightsNeeds (GPU)Needs (Mac)Ollama tag
1 Qwen3.8-Flash-Next 180B-A6B
The open preview of the Qwen4 architecture: a 125B-A6B hybrid (Gated DeltaNet + sparse attention, KV cache on 12 of 48 blocks) plus a 51B n-gram embedding and a 4B draft head — 180B on disk, 6B active. The hosted "Qwen3.8-Flash" is this model with a 1M window.
180B MoE 101.2 GB @ Q4_K_M 141 GB card 192 GB Mac qwen3.8-flash-next:125b-a6b-nvfp4
2 Qwen3.6 35B-A3B
Mixture of experts with ~3B active: the fastest serious model a 24 GB card runs, and the best MoE under 40B on agentic coding.
35.9B MoE 20.2 GB @ Q4_K_M 32 GB card 36 GB Mac qwen3.6:35b-a3b
3 Gemma 4 26B-A4B
Mixture of experts with 3.8B active. Slower to think than Qwen3.6 35B-A3B, faster to answer, and it sees images.
26.5B MoE 14.9 GB @ Q4_K_M 24 GB card 32 GB Mac gemma4:26b
4 Nemotron 3.5 Lightning 30B-A3B
Mamba-2 + MoE hybrid built for the execution layer of agents: only 6 attention blocks, so the KV cache is almost free. Weights, data and recipe all open.
31.6B MoE 17.8 GB @ Q4_K_M 24 GB card 32 GB Mac nemotron-3.5-lightning:30b-a3b
5 GLM-4.7-Flash 30B-A3B
MIT-licensed 30B-A3B tuned for agentic coding, with a DeepSeek-style latent KV cache. 60–80 tok/s reported on a 4090.
31.2B MoE 17.5 GB @ Q4_K_M 24 GB card 32 GB Mac glm-4.7-flash:latest
6 gpt-oss 20B
Ships natively in MXFP4, so the 4-bit weights are the reference weights, not a lossy copy. Fits 16 GB.
20.9B MoE 10.8 GB @ MXFP4 16 GB card 24 GB Mac gpt-oss:20b
7 LFM2.5 8B-A1B
An 8B MoE with ~1.5B active, aimed at laptops without a GPU. Licence is permissive below $10M revenue.
8.47B MoE 4.8 GB @ Q4_K_M 8 GB card 16 GB Mac lfm2.5:8b
8 Ling 3.0 Tiny 7.9B-A1.3B
An 8B MoE with 1.3B active and a latent KV cache on only 6 of 24 layers — reasoning and tool use sized for Apple Silicon and edge boxes.
7.9B MoE 4.4 GB @ Q4_K_M 8 GB card 16 GB Mac —
9 Qwen3.5 9B
The default for 8–12 GB cards in 2026: beats every older 8B on every published benchmark, with vision.
9.65B 5.4 GB @ Q4_K_M 10 GB card 16 GB Mac qwen3.5:9b
10 Granite 4.1 8B
Matches the old Granite 4.0 32B MoE at a quarter of the size. Fast, Apache 2.0, no reasoning traces.
8.79B 4.9 GB @ Q4_K_M 10 GB card 16 GB Mac granite4.1:8b
11 Qwen3.5 4B
The 8 GB coding agent. Q4 lands near 3.4 GB, leaving room for a real context window.
4.66B 2.6 GB @ Q4_K_M 6 GB card 8 GB Mac qwen3.5:4b
12 Gemma 4 E4B
The laptop Gemma. 4.5B effective, 8B on disk; a single KV head per window layer keeps its cache tiny.
8.0B 4.5 GB @ Q4_K_M 8 GB card 16 GB Mac gemma4:e4b
13 Qwen3.5 2B
Phone-class, and multimodal. Replaces Llama 3.2 3B as the "it runs on anything" answer.
2.27B 1.3 GB @ Q4_K_M 6 GB card 8 GB Mac qwen3.5:2b
14 Gemma 4 E2B
"E2B" is 2.3B effective, but the file holds 5B because of per-layer embeddings — size it as 5B. Text, image and audio in.
5.1B 2.9 GB @ Q4_K_M 6 GB card 8 GB Mac gemma4:e2b
15 LFM2.5 2.6B
Convolution-heavy hybrid for CPUs and NPUs: 22 of 30 blocks keep no KV cache at all.
2.7B 1.5 GB @ Q4_K_M 6 GB card 8 GB Mac —
16 Qwen3.5 0.8B
Draft model for speculative decoding, or a classifier that fits in 1 GB.
0.87B 0.5 GB @ Q4_K_M 6 GB card 8 GB Mac qwen3.5:0.8b

A fast model that leaves headroom also leaves room for a draft model. Speculative decoding (llama.cpp's --model-draft) lets a small model propose tokens that a larger one checks in a batch, which often speeds up predictable text such as code by half again or more.

Other jobs