Best local LLM for long context · 2026
The best local LLM for long context, ranked for your VRAM
Unlike the other shortlists, this one is judged at a 128K-token context (or the model's maximum, if lower), because that is the question. What decides it is not the advertised window but the KV cache: hybrid and sliding-window models keep a per-token cache on only some of their layers, so they hold 128K in a fraction of the memory a plain transformer needs. For each memory class below we list every model on that list that actually fits at Q4_K_M with a 128K context, so the top row is the pick and the rest are what you give up by choosing it. Numbers are computed, not benchmarked; how the shortlist works.
A q8_0 KV cache halves the cache for a negligible loss, and is the first thing to try when a model you want misses by a little. q4_0 quarters it but degrades long-context recall — the one thing you came here for.
Best long-context LLM for 6 GB VRAM
5.3 GB usable · speeds for a GeForce RTX 3060 Laptop everything that fits 6 GB →Qwen3.5 2B is the pick: Long documents on 4 GB. It is the only model on the long-context shortlist that fits.
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.5 2B Alibaba · Apache 2.0 |
Long documents on 4 GB. | 2.27B | Q4_K_M · 3.4 GB | 256K | 159 | Runs great |
Best long-context LLM for 8 GB VRAM
7.0 GB usable · speeds for a GeForce RTX 4060 everything that fits 8 GB →Qwen3.5 2B is the pick: Long documents on 4 GB. It is the only model on the long-context shortlist that fits.
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.5 2B Alibaba · Apache 2.0 |
Long documents on 4 GB. | 2.27B | Q4_K_M · 3.4 GB | 256K | 129 | Runs great |
Best long-context LLM for 12 GB VRAM
10.6 GB usable · speeds for a GeForce RTX 3060 12 GB everything that fits 12 GB →Qwen3.5 9B is the pick: 262K native in 8–12 GB. 2 other models on the long-context shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.5 9B Alibaba · Apache 2.0 |
262K native in 8–12 GB. | 9.65B | Q4_K_M · 10.0 GB | 146K | 40 | Tight fit |
| 2 | Qwen3.5 4B Alibaba · Apache 2.0 |
262K on 8 GB. | 4.66B | Q4_K_M · 7.2 GB | 236K | 83 | Runs great |
| 3 | Qwen3.5 2B Alibaba · Apache 2.0 |
Long documents on 4 GB. | 2.27B | Q4_K_M · 3.4 GB | 256K | 171 | Runs great |
Best long-context LLM for 16 GB VRAM
14.4 GB usable · speeds for a GeForce RTX 5080 everything that fits 16 GB →Qwen3.5 9B is the pick: 262K native in 8–12 GB. 2 other models on the long-context shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.5 9B Alibaba · Apache 2.0 |
262K native in 8–12 GB. | 9.65B | Q4_K_M · 10.0 GB | 256K | 107 | Runs great |
| 2 | Qwen3.5 4B Alibaba · Apache 2.0 |
262K on 8 GB. | 4.66B | Q4_K_M · 7.2 GB | 256K | 222 | Runs great |
| 3 | Qwen3.5 2B Alibaba · Apache 2.0 |
Long documents on 4 GB. | 2.27B | Q4_K_M · 3.4 GB | 256K | 455 | Runs great |
Best long-context LLM for 24 GB VRAM
22.4 GB usable · speeds for a GeForce RTX 4090 everything that fits 24 GB →Nemotron 3.5 Lightning 30B-A3B is the pick: 262K, and a Mamba hybrid whose cache barely grows. 5 other models on the long-context shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | Nemotron 3.5 Lightning 30B-A3B NVIDIA · OpenMDW-1.1 |
262K, and a Mamba hybrid whose cache barely grows. | 31.6B MoE | Q4_K_M · 19.1 GB | 256K | 130 | Tight fit |
| 2 | Gemma 4 26B-A4B Google · Apache 2.0 |
256K at MoE speed. | 26.5B MoE | Q4_K_M · 20.7 GB | 171K | 110 | Tight fit |
| 3 | Gemma 4 12B Google · Apache 2.0 |
256K on a 12 GB card. | 12B | Q4_K_M · 15.7 GB | 235K | 90 | Runs great |
| 4 | Qwen3.5 9B Alibaba · Apache 2.0 |
262K native in 8–12 GB. | 9.65B | Q4_K_M · 10.0 GB | 256K | 112 | Runs great |
| 5 | Qwen3.5 4B Alibaba · Apache 2.0 |
262K on 8 GB. | 4.66B | Q4_K_M · 7.2 GB | 256K | 233 | Runs great |
| 6 | Qwen3.5 2B Alibaba · Apache 2.0 |
Long documents on 4 GB. | 2.27B | Q4_K_M · 3.4 GB | 256K | 478 | Runs great |
Best long-context LLM for 32 GB VRAM
30.4 GB usable · speeds for a GeForce RTX 5090 everything that fits 32 GB →Qwen3.8 27B is the pick: 262K native, and only 16 of 64 blocks keep a KV cache — long context is cheap here. 8 other models on the long-context shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.8 27B Alibaba · Apache 2.0 |
262K native, and only 16 of 64 blocks keep a KV cache — long context is cheap here. | 27.8B | Q4_K_M · 24.2 GB | 226K | 69 | Runs great |
| 2 | Qwen3.6 35B-A3B Alibaba · Apache 2.0 |
262K with just 10 KV-cache layers and 2 KV heads: the cheapest long context at 24 GB. | 35.9B MoE | Q4_K_M · 23.3 GB | 256K | 225 | Runs great |
| 3 | Qwen3.6 27B Alibaba · Apache 2.0 |
262K native, same hybrid attention as 3.8. | 27.8B | Q4_K_M · 24.2 GB | 226K | 69 | Runs great |
| 4 | Nemotron 3.5 Lightning 30B-A3B NVIDIA · OpenMDW-1.1 |
262K, and a Mamba hybrid whose cache barely grows. | 31.6B MoE | Q4_K_M · 19.1 GB | 256K | 232 | Runs great |
| 5 | Gemma 4 26B-A4B Google · Apache 2.0 |
256K at MoE speed. | 26.5B MoE | Q4_K_M · 20.7 GB | 256K | 195 | Runs great |
| 6 | Gemma 4 12B Google · Apache 2.0 |
256K on a 12 GB card. | 12B | Q4_K_M · 15.7 GB | 256K | 161 | Runs great |
| 7 | Qwen3.5 9B Alibaba · Apache 2.0 |
262K native in 8–12 GB. | 9.65B | Q4_K_M · 10.0 GB | 256K | 200 | Runs great |
| 8 | Qwen3.5 4B Alibaba · Apache 2.0 |
262K on 8 GB. | 4.66B | Q4_K_M · 7.2 GB | 256K | 414 | Runs great |
| 9 | Qwen3.5 2B Alibaba · Apache 2.0 |
Long documents on 4 GB. | 2.27B | Q4_K_M · 3.4 GB | 256K | 850 | Runs great |
Best long-context LLM for 48 GB VRAM
46.4 GB usable · speeds for an RTX 6000 Ada everything that fits 48 GB →Qwen3.8 27B is the pick: 262K native, and only 16 of 64 blocks keep a KV cache — long context is cheap here. 9 other models on the long-context shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.8 27B Alibaba · Apache 2.0 |
262K native, and only 16 of 64 blocks keep a KV cache — long context is cheap here. | 27.8B | Q4_K_M · 24.2 GB | 256K | 37 | Runs great |
| 2 | Qwen3.6 35B-A3B Alibaba · Apache 2.0 |
262K with just 10 KV-cache layers and 2 KV heads: the cheapest long context at 24 GB. | 35.9B MoE | Q4_K_M · 23.3 GB | 256K | 120 | Runs great |
| 3 | Qwen3.6 27B Alibaba · Apache 2.0 |
262K native, same hybrid attention as 3.8. | 27.8B | Q4_K_M · 24.2 GB | 256K | 37 | Runs great |
| 4 | Gemma 4 31B Google · Apache 2.0 |
256K with sliding-window layers keeping the cache small. | 31.3B | Q4_K_M · 39.0 GB | 175K | 33 | Runs great |
| 5 | Nemotron 3.5 Lightning 30B-A3B NVIDIA · OpenMDW-1.1 |
262K, and a Mamba hybrid whose cache barely grows. | 31.6B MoE | Q4_K_M · 19.1 GB | 256K | 124 | Runs great |
| 6 | Gemma 4 26B-A4B Google · Apache 2.0 |
256K at MoE speed. | 26.5B MoE | Q4_K_M · 20.7 GB | 256K | 105 | Runs great |
| 7 | Gemma 4 12B Google · Apache 2.0 |
256K on a 12 GB card. | 12B | Q4_K_M · 15.7 GB | 256K | 86 | Runs great |
| 8 | Qwen3.5 9B Alibaba · Apache 2.0 |
262K native in 8–12 GB. | 9.65B | Q4_K_M · 10.0 GB | 256K | 107 | Runs great |
| 9 | Qwen3.5 4B Alibaba · Apache 2.0 |
262K on 8 GB. | 4.66B | Q4_K_M · 7.2 GB | 256K | 222 | Runs great |
| 10 | Qwen3.5 2B Alibaba · Apache 2.0 |
Long documents on 4 GB. | 2.27B | Q4_K_M · 3.4 GB | 256K | 455 | Runs great |
Best long-context LLM for 16 GB unified memory
10.7 GB usable · speeds for an M4 · 16 GB everything that fits 16 GB →Qwen3.5 9B is the pick: 262K native in 8–12 GB. 2 other models on the long-context shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.5 9B Alibaba · Apache 2.0 |
262K native in 8–12 GB. | 9.65B | Q4_K_M · 10.0 GB | 149K | 12 | Tight fit |
| 2 | Qwen3.5 4B Alibaba · Apache 2.0 |
262K on 8 GB. | 4.66B | Q4_K_M · 7.2 GB | 239K | 26 | Runs great |
| 3 | Qwen3.5 2B Alibaba · Apache 2.0 |
Long documents on 4 GB. | 2.27B | Q4_K_M · 3.4 GB | 256K | 53 | Runs great |
Best long-context LLM for 64 GB unified memory
48.0 GB usable · speeds for an M4 Max · 64 GB everything that fits 64 GB →Qwen3.8 27B is the pick: 262K native, and only 16 of 64 blocks keep a KV cache — long context is cheap here. 9 other models on the long-context shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.8 27B Alibaba · Apache 2.0 |
262K native, and only 16 of 64 blocks keep a KV cache — long context is cheap here. | 27.8B | Q4_K_M · 24.2 GB | 256K | 20 | Runs great |
| 2 | Qwen3.6 35B-A3B Alibaba · Apache 2.0 |
262K with just 10 KV-cache layers and 2 KV heads: the cheapest long context at 24 GB. | 35.9B MoE | Q4_K_M · 23.3 GB | 256K | 77 | Runs great |
| 3 | Qwen3.6 27B Alibaba · Apache 2.0 |
262K native, same hybrid attention as 3.8. | 27.8B | Q4_K_M · 24.2 GB | 256K | 20 | Runs great |
| 4 | Gemma 4 31B Google · Apache 2.0 |
256K with sliding-window layers keeping the cache small. | 31.3B | Q4_K_M · 39.0 GB | 185K | 17 | Runs great |
| 5 | Nemotron 3.5 Lightning 30B-A3B NVIDIA · OpenMDW-1.1 |
262K, and a Mamba hybrid whose cache barely grows. | 31.6B MoE | Q4_K_M · 19.1 GB | 256K | 79 | Runs great |
| 6 | Gemma 4 26B-A4B Google · Apache 2.0 |
256K at MoE speed. | 26.5B MoE | Q4_K_M · 20.7 GB | 256K | 67 | Runs great |
| 7 | Gemma 4 12B Google · Apache 2.0 |
256K on a 12 GB card. | 12B | Q4_K_M · 15.7 GB | 256K | 45 | Runs great |
| 8 | Qwen3.5 9B Alibaba · Apache 2.0 |
262K native in 8–12 GB. | 9.65B | Q4_K_M · 10.0 GB | 256K | 56 | Runs great |
| 9 | Qwen3.5 4B Alibaba · Apache 2.0 |
262K on 8 GB. | 4.66B | Q4_K_M · 7.2 GB | 256K | 116 | Runs great |
| 10 | Qwen3.5 2B Alibaba · Apache 2.0 |
Long documents on 4 GB. | 2.27B | Q4_K_M · 3.4 GB | 256K | 239 | Runs great |
Best long-context LLM for 128 GB unified memory
96.0 GB usable · speeds for an M4 Max · 128 GB everything that fits 128 GB →Qwen3.8 27B is the pick: 262K native, and only 16 of 64 blocks keep a KV cache — long context is cheap here. 12 other models on the long-context shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | Qwen3.8 27B Alibaba · Apache 2.0 |
262K native, and only 16 of 64 blocks keep a KV cache — long context is cheap here. | 27.8B | Q4_K_M · 24.2 GB | 256K | 20 | Runs great |
| 2 | Qwen3.6 35B-A3B Alibaba · Apache 2.0 |
262K with just 10 KV-cache layers and 2 KV heads: the cheapest long context at 24 GB. | 35.9B MoE | Q4_K_M · 23.3 GB | 256K | 77 | Runs great |
| 3 | Qwen3.6 27B Alibaba · Apache 2.0 |
262K native, same hybrid attention as 3.8. | 27.8B | Q4_K_M · 24.2 GB | 256K | 20 | Runs great |
| 4 | Gemma 4 31B Google · Apache 2.0 |
256K with sliding-window layers keeping the cache small. | 31.3B | Q4_K_M · 39.0 GB | 256K | 17 | Runs great |
| 5 | Nemotron 3.5 Lightning 30B-A3B NVIDIA · OpenMDW-1.1 |
262K, and a Mamba hybrid whose cache barely grows. | 31.6B MoE | Q4_K_M · 19.1 GB | 256K | 79 | Runs great |
| 6 | Gemma 4 26B-A4B Google · Apache 2.0 |
256K at MoE speed. | 26.5B MoE | Q4_K_M · 20.7 GB | 256K | 67 | Runs great |
| 7 | Qwen3.5 122B-A10B Alibaba · Apache 2.0 |
262K for unified-memory machines. | 125B MoE | Q4_K_M · 73.9 GB | 256K | 25 | Runs great |
| 8 | Ling 3.0 Flash 124B-A5B inclusionAI · MIT |
262K with MLA on only 7 of 42 layers. | 124B MoE | Q4_K_M · 71.3 GB | 256K | 50 | Runs great |
| 9 | Gemma 4 12B Google · Apache 2.0 |
256K on a 12 GB card. | 12B | Q4_K_M · 15.7 GB | 256K | 45 | Runs great |
| 10 | Qwen3.5 9B Alibaba · Apache 2.0 |
262K native in 8–12 GB. | 9.65B | Q4_K_M · 10.0 GB | 256K | 56 | Runs great |
| 11 | Llama 4 Scout 109B-A17B Meta · Llama 4 Community |
A 10M window on paper; chunked attention keeps it affordable. | 109B MoE | Q4_K_M · 69.0 GB | 703K | 15 | Runs great |
| 12 | Qwen3.5 4B Alibaba · Apache 2.0 |
262K on 8 GB. | 4.66B | Q4_K_M · 7.2 GB | 256K | 116 | Runs great |
| 13 | Qwen3.5 2B Alibaba · Apache 2.0 |
Long documents on 4 GB. | 2.27B | Q4_K_M · 3.4 GB | 256K | 239 | Runs great |
The whole long-context shortlist, and what each one needs
Every model on the list, best first, with the smallest discrete card and the smallest Mac that run it with 15% headroom at the recommended quantisation and 128K context. Every class has its own page, Macs from 8 to 512 GB included — the full list.
| # | Model | Params | Weights | Needs (GPU) | Needs (Mac) | Ollama tag |
|---|---|---|---|---|---|---|
| 1 | DeepSeek V4.1 Flash 552B-A16B New Twice V4 Flash's backbone (552B, 16B active per generated token) plus 196B of Engram lookup tables (189 GiB at FP8) that DwarfStar streams from SSD, so they are not counted here. A 512 GB Mac model, and as of October 2026 no mainline llama.cpp or Ollama build: DwarfStar on a Mac, vLLM across four GPUs. DeepSeek puts the cache at 890 bytes/token; modelled conservatively as 4 latent layers plus a 128-token window. |
552B MoE | 310.4 GB @ Q4_K_M | more than one card | 512 GB Mac | — |
| 2 | DeepSeek V4 Flash 284B-A13B The V4 for 192–256 GB machines: Q3 squeezes into 192 GB, Q4 wants 256 GB, and 128 GB falls short even at Q2. Cache is modelled as a 576-wide latent; V4 compresses it further at long context, so this is conservative. |
284B MoE | 159.7 GB @ Q4_K_M | more than one card | 512 GB Mac | — |
| 3 | GLM-5.3-Flash 320B-A18B The first natively multimodal GLM-5 and the first hybrid: 34 linear-attention blocks and 11 sparse-attention blocks with a 512-wide latent cache, so a 1M window stays affordable. Z.ai says it beats GLM-5.2 at 18B active; MIT. |
321B MoE | 180.5 GB @ Q4_K_M | more than one card | 512 GB Mac | — |
| 4 | Qwen3.8-Flash-Next 180B-A6B The open preview of the Qwen4 architecture: a 125B-A6B hybrid (Gated DeltaNet + sparse attention, KV cache on 12 of 48 blocks) plus a 51B n-gram embedding and a 4B draft head — 180B on disk, 6B active. The hosted "Qwen3.8-Flash" is this model with a 1M window. |
180B MoE | 101.2 GB @ Q4_K_M | 141 GB card | 192 GB Mac | qwen3.8-flash-next:125b-a6b-nvfp4 |
| 5 | Qwen3.8 27B The current default local Qwen: dense 27B, text + image + video, 262K context. Only 16 of its 64 blocks keep a KV cache, so long context is cheap. |
27.8B | 15.6 GB @ Q4_K_M | 32 GB card | 48 GB Mac | qwen3.8:27b |
| 6 | Qwen3.6 35B-A3B Mixture of experts with ~3B active: the fastest serious model a 24 GB card runs, and the best MoE under 40B on agentic coding. |
35.9B MoE | 20.2 GB @ Q4_K_M | 32 GB card | 48 GB Mac | qwen3.6:35b-a3b |
| 7 | Qwen3.6 27B The 24 GB coding pick of spring 2026 (77.2 SWE-bench Verified). Same shape as 3.8, one generation behind. |
27.8B | 15.6 GB @ Q4_K_M | 32 GB card | 48 GB Mac | qwen3.6:27b |
| 8 | Gemma 4 31B The dense flagship: strongest maths of the 24–32 GB class (89% AIME), clean prose, vision. Q4 is a tight 24 GB fit. |
31.3B | 17.6 GB @ Q4_K_M | 48 GB card | 64 GB Mac | gemma4:31b |
| 9 | Nemotron 3.5 Lightning 30B-A3B Mamba-2 + MoE hybrid built for the execution layer of agents: only 6 attention blocks, so the KV cache is almost free. Weights, data and recipe all open. |
31.6B MoE | 17.8 GB @ Q4_K_M | 32 GB card | 32 GB Mac | nemotron-3.5-lightning:30b-a3b |
| 10 | Gemma 4 26B-A4B Mixture of experts with 3.8B active. Slower to think than Qwen3.6 35B-A3B, faster to answer, and it sees images. |
26.5B MoE | 14.9 GB @ Q4_K_M | 32 GB card | 36 GB Mac | gemma4:26b |
| 11 | Qwen3.5 122B-A10B The 96–128 GB unified-memory model: 122B of knowledge at 10B-active speed. |
125B MoE | 70.3 GB @ Q4_K_M | 141 GB card | 128 GB Mac | qwen3.5:122b-a10b |
| 12 | Ling 3.0 Flash 124B-A5B A 124B hybrid (5 linear-attention layers per MLA layer) with 5.1B active: SWE-bench Pro 56.6 and AIME 93 claimed. Built for 96–128 GB machines. |
124B MoE | 69.7 GB @ Q4_K_M | 141 GB card | 128 GB Mac | — |
| 13 | Gemma 4 12B The "unified" Gemma 4: text, image and audio in one 12B that fits a 12 GB card at Q4. 140+ languages. |
12B | 6.7 GB @ Q4_K_M | 24 GB card | 32 GB Mac | gemma4:12b |
| 14 | Qwen3.5 9B The default for 8–12 GB cards in 2026: beats every older 8B on every published benchmark, with vision. |
9.65B | 5.4 GB @ Q4_K_M | 16 GB card | 24 GB Mac | qwen3.5:9b |
| 15 | Llama 4 Scout 109B-A17B A 10M-token window on paper, chunked attention in practice (8K chunks on 3 of 4 layers). Needs 64 GB+ at Q4. |
109B MoE | 61.3 GB @ Q4_K_M | 141 GB card | 128 GB Mac | llama4:scout |
| 16 | Qwen3.5 4B The 8 GB coding agent. Q4 lands near 3.4 GB, leaving room for a real context window. |
4.66B | 2.6 GB @ Q4_K_M | 10 GB card | 16 GB Mac | qwen3.5:4b |
| 17 | Qwen3.5 2B Phone-class, and multimodal. Replaces Llama 3.2 3B as the "it runs on anything" answer. |
2.27B | 1.3 GB @ Q4_K_M | 6 GB card | 8 GB Mac | qwen3.5:2b |
Holding 128K is not the same as using it well. Recall sags toward the middle of very long prompts on most models: put the question after the document, and test retrieval on your own material before you rely on it. The KV cache calculator shows what each model's cache costs at 4K to 128K.
Other jobs