Best local LLM for writing · 2026
The best local LLM for writing, ranked for your VRAM
Prose is the least benchmarkable job on this site, so this shortlist leans on community consensus — creative-writing comparisons and r/LocalLLaMA — about voice, how well a model takes direction and how little it pads, as of the catalogue date. For each memory class below we list every model on that list that actually fits at Q4_K_M with an 8K context, so the top row is the pick and the rest are what you give up by choosing it. Numbers are computed, not benchmarked; how the shortlist works.
For long-form work the draft lives in the context. A 4,000-word chapter is roughly 5–6K tokens, so 8K is enough to revise one chapter, not a manuscript; the "max context" column says how much of a book each model can hold on your class.
Best writing LLM for 6 GB VRAM
5.3 GB usable · speeds for a GeForce RTX 3060 Laptop everything that fits 6 GB →Gemma 4 E4B is the pick: Readable prose on a laptop. 1 other model on the writing shortlist also fits, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | Gemma 4 E4B Google · Apache 2.0 |
Readable prose on a laptop. | 8.0B | Q4_K_M · 5.2 GB | 12K | 45 | Tight fit |
| 2 | Qwen3.5 4B Alibaba · Apache 2.0 |
Competent drafting in 8 GB. | 4.66B | Q4_K_M · 3.5 GB | 66K | 78 | Runs great |
Best writing LLM for 8 GB VRAM
7.0 GB usable · speeds for a GeForce RTX 4060 everything that fits 8 GB →Ministral 3 8B is the pick: Mistral's voice in 8B, Apache 2.0. 3 other models on the writing shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | Ministral 3 8B Mistral AI · Apache 2.0 |
Mistral's voice in 8B, Apache 2.0. | 8.92B | Q4_K_M · 6.7 GB | 10K | 33 | Tight fit |
| 2 | Gemma 4 E4B Google · Apache 2.0 |
Readable prose on a laptop. | 8.0B | Q4_K_M · 5.2 GB | 128K | 37 | Runs great |
| 3 | Llama 3.1 8B Instruct Meta · Llama 3.1 Community |
Largest fine-tune ecosystem if you want a specific style. | 8.03B | Q4_K_M · 6.1 GB | 15K | 36 | Tight fit |
| 4 | Qwen3.5 4B Alibaba · Apache 2.0 |
Competent drafting in 8 GB. | 4.66B | Q4_K_M · 3.5 GB | 120K | 63 | Runs great |
Best writing LLM for 12 GB VRAM
10.6 GB usable · speeds for a GeForce RTX 3060 12 GB everything that fits 12 GB →Gemma 4 12B is the pick: The 12 GB writing pick; 140+ languages. 5 other models on the writing shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | Gemma 4 12B Google · Apache 2.0 |
The 12 GB writing pick; 140+ languages. | 12B | Q4_K_M · 8.2 GB | 47K | 32 | Runs great |
| 2 | Mistral NeMo 12B Mistral AI · Apache 2.0 |
A long-standing roleplay and fiction favourite with a 128K window. | 12.2B | Q4_K_M · 8.7 GB | 20K | 32 | Runs great |
| 3 | Ministral 3 8B Mistral AI · Apache 2.0 |
Mistral's voice in 8B, Apache 2.0. | 8.92B | Q4_K_M · 6.7 GB | 37K | 43 | Runs great |
| 4 | Gemma 4 E4B Google · Apache 2.0 |
Readable prose on a laptop. | 8.0B | Q4_K_M · 5.2 GB | 128K | 48 | Runs great |
| 5 | Llama 3.1 8B Instruct Meta · Llama 3.1 Community |
Largest fine-tune ecosystem if you want a specific style. | 8.03B | Q4_K_M · 6.1 GB | 43K | 48 | Runs great |
| 6 | Qwen3.5 4B Alibaba · Apache 2.0 |
Competent drafting in 8 GB. | 4.66B | Q4_K_M · 3.5 GB | 236K | 83 | Runs great |
Best writing LLM for 16 GB VRAM
14.4 GB usable · speeds for a GeForce RTX 5080 everything that fits 16 GB →Gemma 4 12B is the pick: The 12 GB writing pick; 140+ languages. 5 other models on the writing shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | Gemma 4 12B Google · Apache 2.0 |
The 12 GB writing pick; 140+ languages. | 12B | Q4_K_M · 8.2 GB | 107K | 86 | Runs great |
| 2 | Mistral NeMo 12B Mistral AI · Apache 2.0 |
A long-standing roleplay and fiction favourite with a 128K window. | 12.2B | Q4_K_M · 8.7 GB | 44K | 85 | Runs great |
| 3 | Ministral 3 8B Mistral AI · Apache 2.0 |
Mistral's voice in 8B, Apache 2.0. | 8.92B | Q4_K_M · 6.7 GB | 66K | 116 | Runs great |
| 4 | Gemma 4 E4B Google · Apache 2.0 |
Readable prose on a laptop. | 8.0B | Q4_K_M · 5.2 GB | 128K | 129 | Runs great |
| 5 | Llama 3.1 8B Instruct Meta · Llama 3.1 Community |
Largest fine-tune ecosystem if you want a specific style. | 8.03B | Q4_K_M · 6.1 GB | 74K | 129 | Runs great |
| 6 | Qwen3.5 4B Alibaba · Apache 2.0 |
Competent drafting in 8 GB. | 4.66B | Q4_K_M · 3.5 GB | 256K | 222 | Runs great |
Best writing LLM for 24 GB VRAM
22.4 GB usable · speeds for a GeForce RTX 4090 everything that fits 24 GB →Gemma 4 31B is the pick: The cleanest, least verbose prose of the 2026 24–32 GB class. 8 other models on the writing shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | Gemma 4 31B Google · Apache 2.0 |
The cleanest, least verbose prose of the 2026 24–32 GB class. | 31.3B | Q4_K_M · 20.2 GB | 21K | 35 | Tight fit |
| 2 | Gemma 4 26B-A4B Google · Apache 2.0 |
Gemma 4 prose at MoE speed, with room for a long draft in context. | 26.5B MoE | Q4_K_M · 16.0 GB | 171K | 110 | Runs great |
| 3 | Mistral Small 3.2 24B Mistral AI · Apache 2.0 |
Natural dialogue and a light hand with instructions. | 23.6B | Q4_K_M · 15.1 GB | 54K | 46 | Runs great |
| 4 | Gemma 4 12B Google · Apache 2.0 |
The 12 GB writing pick; 140+ languages. | 12B | Q4_K_M · 8.2 GB | 235K | 90 | Runs great |
| 5 | Mistral NeMo 12B Mistral AI · Apache 2.0 |
A long-standing roleplay and fiction favourite with a 128K window. | 12.2B | Q4_K_M · 8.7 GB | 95K | 89 | Runs great |
| 6 | Ministral 3 8B Mistral AI · Apache 2.0 |
Mistral's voice in 8B, Apache 2.0. | 8.92B | Q4_K_M · 6.7 GB | 126K | 122 | Runs great |
| 7 | Gemma 4 E4B Google · Apache 2.0 |
Readable prose on a laptop. | 8.0B | Q4_K_M · 5.2 GB | 128K | 136 | Runs great |
| 8 | Llama 3.1 8B Instruct Meta · Llama 3.1 Community |
Largest fine-tune ecosystem if you want a specific style. | 8.03B | Q4_K_M · 6.1 GB | 128K | 135 | Runs great |
| 9 | Qwen3.5 4B Alibaba · Apache 2.0 |
Competent drafting in 8 GB. | 4.66B | Q4_K_M · 3.5 GB | 256K | 233 | Runs great |
Best writing LLM for 32 GB VRAM
30.4 GB usable · speeds for a GeForce RTX 5090 everything that fits 32 GB →Gemma 4 31B is the pick: The cleanest, least verbose prose of the 2026 24–32 GB class. 8 other models on the writing shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | Gemma 4 31B Google · Apache 2.0 |
The cleanest, least verbose prose of the 2026 24–32 GB class. | 31.3B | Q4_K_M · 20.2 GB | 73K | 62 | Runs great |
| 2 | Gemma 4 26B-A4B Google · Apache 2.0 |
Gemma 4 prose at MoE speed, with room for a long draft in context. | 26.5B MoE | Q4_K_M · 16.0 GB | 256K | 195 | Runs great |
| 3 | Mistral Small 3.2 24B Mistral AI · Apache 2.0 |
Natural dialogue and a light hand with instructions. | 23.6B | Q4_K_M · 15.1 GB | 105K | 82 | Runs great |
| 4 | Gemma 4 12B Google · Apache 2.0 |
The 12 GB writing pick; 140+ languages. | 12B | Q4_K_M · 8.2 GB | 256K | 161 | Runs great |
| 5 | Mistral NeMo 12B Mistral AI · Apache 2.0 |
A long-standing roleplay and fiction favourite with a 128K window. | 12.2B | Q4_K_M · 8.7 GB | 128K | 158 | Runs great |
| 6 | Ministral 3 8B Mistral AI · Apache 2.0 |
Mistral's voice in 8B, Apache 2.0. | 8.92B | Q4_K_M · 6.7 GB | 186K | 216 | Runs great |
| 7 | Gemma 4 E4B Google · Apache 2.0 |
Readable prose on a laptop. | 8.0B | Q4_K_M · 5.2 GB | 128K | 241 | Runs great |
| 8 | Llama 3.1 8B Instruct Meta · Llama 3.1 Community |
Largest fine-tune ecosystem if you want a specific style. | 8.03B | Q4_K_M · 6.1 GB | 128K | 240 | Runs great |
| 9 | Qwen3.5 4B Alibaba · Apache 2.0 |
Competent drafting in 8 GB. | 4.66B | Q4_K_M · 3.5 GB | 256K | 414 | Runs great |
Best writing LLM for 48 GB VRAM
46.4 GB usable · speeds for an RTX 6000 Ada everything that fits 48 GB →Llama 3.3 70B Instruct is the pick: The creative-writing favourite: consistent voice, takes direction, handles fiction framing. 9 other models on the writing shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | Llama 3.3 70B Instruct Meta · Llama 3.3 Community |
The creative-writing favourite: consistent voice, takes direction, handles fiction framing. | 70.6B | Q4_K_M · 42.8 GB | 19K | 15 | Tight fit |
| 2 | Gemma 4 31B Google · Apache 2.0 |
The cleanest, least verbose prose of the 2026 24–32 GB class. | 31.3B | Q4_K_M · 20.2 GB | 175K | 33 | Runs great |
| 3 | Gemma 4 26B-A4B Google · Apache 2.0 |
Gemma 4 prose at MoE speed, with room for a long draft in context. | 26.5B MoE | Q4_K_M · 16.0 GB | 256K | 105 | Runs great |
| 4 | Mistral Small 3.2 24B Mistral AI · Apache 2.0 |
Natural dialogue and a light hand with instructions. | 23.6B | Q4_K_M · 15.1 GB | 128K | 44 | Runs great |
| 5 | Gemma 4 12B Google · Apache 2.0 |
The 12 GB writing pick; 140+ languages. | 12B | Q4_K_M · 8.2 GB | 256K | 86 | Runs great |
| 6 | Mistral NeMo 12B Mistral AI · Apache 2.0 |
A long-standing roleplay and fiction favourite with a 128K window. | 12.2B | Q4_K_M · 8.7 GB | 128K | 85 | Runs great |
| 7 | Ministral 3 8B Mistral AI · Apache 2.0 |
Mistral's voice in 8B, Apache 2.0. | 8.92B | Q4_K_M · 6.7 GB | 256K | 116 | Runs great |
| 8 | Gemma 4 E4B Google · Apache 2.0 |
Readable prose on a laptop. | 8.0B | Q4_K_M · 5.2 GB | 128K | 129 | Runs great |
| 9 | Llama 3.1 8B Instruct Meta · Llama 3.1 Community |
Largest fine-tune ecosystem if you want a specific style. | 8.03B | Q4_K_M · 6.1 GB | 128K | 129 | Runs great |
| 10 | Qwen3.5 4B Alibaba · Apache 2.0 |
Competent drafting in 8 GB. | 4.66B | Q4_K_M · 3.5 GB | 256K | 222 | Runs great |
Best writing LLM for 16 GB unified memory
10.7 GB usable · speeds for an M4 · 16 GB everything that fits 16 GB →Gemma 4 12B is the pick: The 12 GB writing pick; 140+ languages. 5 other models on the writing shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | Gemma 4 12B Google · Apache 2.0 |
The 12 GB writing pick; 140+ languages. | 12B | Q4_K_M · 8.2 GB | 48K | 9.9 | Runs great |
| 2 | Mistral NeMo 12B Mistral AI · Apache 2.0 |
A long-standing roleplay and fiction favourite with a 128K window. | 12.2B | Q4_K_M · 8.7 GB | 20K | 9.8 | Runs great |
| 3 | Ministral 3 8B Mistral AI · Apache 2.0 |
Mistral's voice in 8B, Apache 2.0. | 8.92B | Q4_K_M · 6.7 GB | 38K | 13 | Runs great |
| 4 | Gemma 4 E4B Google · Apache 2.0 |
Readable prose on a laptop. | 8.0B | Q4_K_M · 5.2 GB | 128K | 15 | Runs great |
| 5 | Llama 3.1 8B Instruct Meta · Llama 3.1 Community |
Largest fine-tune ecosystem if you want a specific style. | 8.03B | Q4_K_M · 6.1 GB | 44K | 15 | Runs great |
| 6 | Qwen3.5 4B Alibaba · Apache 2.0 |
Competent drafting in 8 GB. | 4.66B | Q4_K_M · 3.5 GB | 239K | 26 | Runs great |
Best writing LLM for 64 GB unified memory
48.0 GB usable · speeds for an M4 Max · 64 GB everything that fits 64 GB →Llama 3.3 70B Instruct is the pick: The creative-writing favourite: consistent voice, takes direction, handles fiction framing. 9 other models on the writing shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | Llama 3.3 70B Instruct Meta · Llama 3.3 Community |
The creative-writing favourite: consistent voice, takes direction, handles fiction framing. | 70.6B | Q4_K_M · 42.8 GB | 24K | 7.7 | Tight fit |
| 2 | Gemma 4 31B Google · Apache 2.0 |
The cleanest, least verbose prose of the 2026 24–32 GB class. | 31.3B | Q4_K_M · 20.2 GB | 185K | 17 | Runs great |
| 3 | Gemma 4 26B-A4B Google · Apache 2.0 |
Gemma 4 prose at MoE speed, with room for a long draft in context. | 26.5B MoE | Q4_K_M · 16.0 GB | 256K | 67 | Runs great |
| 4 | Mistral Small 3.2 24B Mistral AI · Apache 2.0 |
Natural dialogue and a light hand with instructions. | 23.6B | Q4_K_M · 15.1 GB | 128K | 23 | Runs great |
| 5 | Gemma 4 12B Google · Apache 2.0 |
The 12 GB writing pick; 140+ languages. | 12B | Q4_K_M · 8.2 GB | 256K | 45 | Runs great |
| 6 | Mistral NeMo 12B Mistral AI · Apache 2.0 |
A long-standing roleplay and fiction favourite with a 128K window. | 12.2B | Q4_K_M · 8.7 GB | 128K | 44 | Runs great |
| 7 | Ministral 3 8B Mistral AI · Apache 2.0 |
Mistral's voice in 8B, Apache 2.0. | 8.92B | Q4_K_M · 6.7 GB | 256K | 61 | Runs great |
| 8 | Gemma 4 E4B Google · Apache 2.0 |
Readable prose on a laptop. | 8.0B | Q4_K_M · 5.2 GB | 128K | 68 | Runs great |
| 9 | Llama 3.1 8B Instruct Meta · Llama 3.1 Community |
Largest fine-tune ecosystem if you want a specific style. | 8.03B | Q4_K_M · 6.1 GB | 128K | 68 | Runs great |
| 10 | Qwen3.5 4B Alibaba · Apache 2.0 |
Competent drafting in 8 GB. | 4.66B | Q4_K_M · 3.5 GB | 256K | 116 | Runs great |
Best writing LLM for 128 GB unified memory
96.0 GB usable · speeds for an M4 Max · 128 GB everything that fits 128 GB →Llama 3.3 70B Instruct is the pick: The creative-writing favourite: consistent voice, takes direction, handles fiction framing. 9 other models on the writing shortlist also fit, in order of capability:
| # | Model | Why | Params | Quant · total | Max context | Tok/s est. | |
|---|---|---|---|---|---|---|---|
| 1 | Llama 3.3 70B Instruct Meta · Llama 3.3 Community |
The creative-writing favourite: consistent voice, takes direction, handles fiction framing. | 70.6B | Q4_K_M · 42.8 GB | 128K | 7.7 | Runs great |
| 2 | Gemma 4 31B Google · Apache 2.0 |
The cleanest, least verbose prose of the 2026 24–32 GB class. | 31.3B | Q4_K_M · 20.2 GB | 256K | 17 | Runs great |
| 3 | Gemma 4 26B-A4B Google · Apache 2.0 |
Gemma 4 prose at MoE speed, with room for a long draft in context. | 26.5B MoE | Q4_K_M · 16.0 GB | 256K | 67 | Runs great |
| 4 | Mistral Small 3.2 24B Mistral AI · Apache 2.0 |
Natural dialogue and a light hand with instructions. | 23.6B | Q4_K_M · 15.1 GB | 128K | 23 | Runs great |
| 5 | Gemma 4 12B Google · Apache 2.0 |
The 12 GB writing pick; 140+ languages. | 12B | Q4_K_M · 8.2 GB | 256K | 45 | Runs great |
| 6 | Mistral NeMo 12B Mistral AI · Apache 2.0 |
A long-standing roleplay and fiction favourite with a 128K window. | 12.2B | Q4_K_M · 8.7 GB | 128K | 44 | Runs great |
| 7 | Ministral 3 8B Mistral AI · Apache 2.0 |
Mistral's voice in 8B, Apache 2.0. | 8.92B | Q4_K_M · 6.7 GB | 256K | 61 | Runs great |
| 8 | Gemma 4 E4B Google · Apache 2.0 |
Readable prose on a laptop. | 8.0B | Q4_K_M · 5.2 GB | 128K | 68 | Runs great |
| 9 | Llama 3.1 8B Instruct Meta · Llama 3.1 Community |
Largest fine-tune ecosystem if you want a specific style. | 8.03B | Q4_K_M · 6.1 GB | 128K | 68 | Runs great |
| 10 | Qwen3.5 4B Alibaba · Apache 2.0 |
Competent drafting in 8 GB. | 4.66B | Q4_K_M · 3.5 GB | 256K | 116 | Runs great |
The whole writing shortlist, and what each one needs
Every model on the list, best first, with the smallest discrete card and the smallest Mac that run it with 15% headroom at the recommended quantisation and 8K context. Every class has its own page, Macs from 8 to 512 GB included — the full list.
| # | Model | Params | Weights | Needs (GPU) | Needs (Mac) | Ollama tag |
|---|---|---|---|---|---|---|
| 1 | Llama 3.3 70B Instruct Still the creative-writing favourite: consistent voice, takes direction. Needs 48 GB to sit comfortably on GPU at Q4. |
70.6B | 39.7 GB @ Q4_K_M | 64 GB card | 96 GB Mac | llama3.3:70b |
| 2 | Gemma 4 31B The dense flagship: strongest maths of the 24–32 GB class (89% AIME), clean prose, vision. Q4 is a tight 24 GB fit. |
31.3B | 17.6 GB @ Q4_K_M | 32 GB card | 32 GB Mac | gemma4:31b |
| 3 | Gemma 4 26B-A4B Mixture of experts with 3.8B active. Slower to think than Qwen3.6 35B-A3B, faster to answer, and it sees images. |
26.5B MoE | 14.9 GB @ Q4_K_M | 24 GB card | 32 GB Mac | gemma4:26b |
| 4 | Mistral Small 3.2 24B Apache-2.0, vision-capable, and the most 24 GB-friendly of the 2025 generalists. |
23.6B | 13.3 GB @ Q4_K_M | 20 GB card | 24 GB Mac | mistral-small:24b |
| 5 | Gemma 4 12B The "unified" Gemma 4: text, image and audio in one 12B that fits a 12 GB card at Q4. 140+ languages. |
12B | 6.7 GB @ Q4_K_M | 11 GB card | 16 GB Mac | gemma4:12b |
| 6 | Mistral NeMo 12B Multilingual 12B with a 128K window, built with NVIDIA. A roleplay and fiction favourite that refuses to die. |
12.2B | 6.9 GB @ Q4_K_M | 12 GB card | 16 GB Mac | mistral-nemo:12b |
| 7 | Ministral 3 8B Mistral's 8B with images in. Plain GQA, so budget more KV cache than Qwen3.5 9B at the same context. |
8.92B | 5.0 GB @ Q4_K_M | 10 GB card | 16 GB Mac | ministral-3:8b |
| 8 | Gemma 4 E4B The laptop Gemma. 4.5B effective, 8B on disk; a single KV head per window layer keeps its cache tiny. |
8.0B | 4.5 GB @ Q4_K_M | 8 GB card | 16 GB Mac | gemma4:e4b |
| 9 | Llama 3.1 8B Instruct Still the most widely deployed local model, with the largest fine-tune ecosystem. Not the strongest 8B any more. |
8.03B | 4.5 GB @ Q4_K_M | 10 GB card | 16 GB Mac | llama3.1:8b |
| 10 | Qwen3.5 4B The 8 GB coding agent. Q4 lands near 3.4 GB, leaving room for a real context window. |
4.66B | 2.6 GB @ Q4_K_M | 6 GB card | 8 GB Mac | qwen3.5:4b |
Instruction-tuned models are trained to be helpful, which reads as bland in fiction. If the pick feels flat, a community fine-tune of the same base — Llama 3.1 8B and Mistral NeMo have the most — is the usual next step, and needs exactly the memory the base model does.
Other jobs