Best local LLM for writing · 2026

The best local LLM for writing, ranked for your VRAM

Updated 8 Oct 2026 1 new this month

Prose is the least benchmarkable job on this site, so this shortlist leans on community consensus — creative-writing comparisons and r/LocalLLaMA — about voice, how well a model takes direction and how little it pads, as of the catalogue date. For each memory class below we list every model on that list that actually fits at Q4_K_M with an 8K context, so the top row is the pick and the rest are what you give up by choosing it. Numbers are computed, not benchmarked; how the shortlist works.

For long-form work the draft lives in the context. A 4,000-word chapter is roughly 5–6K tokens, so 8K is enough to revise one chapter, not a manuscript; the "max context" column says how much of a book each model can hold on your class.

Best writing LLM for 6 GB VRAM

5.3 GB usable · speeds for a GeForce RTX 3060 Laptop everything that fits 6 GB →

Gemma 4 E4B is the pick: Readable prose on a laptop. 1 other model on the writing shortlist also fits, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 Gemma 4 E4B
Google · Apache 2.0
Readable prose on a laptop. 8.0B Q4_K_M · 5.2 GB 12K 45 Tight fit
2 Qwen3.5 4B
Alibaba · Apache 2.0
Competent drafting in 8 GB. 4.66B Q4_K_M · 3.5 GB 66K 78 Runs great

Best writing LLM for 8 GB VRAM

7.0 GB usable · speeds for a GeForce RTX 4060 everything that fits 8 GB →

Ministral 3 8B is the pick: Mistral's voice in 8B, Apache 2.0. 3 other models on the writing shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 Ministral 3 8B
Mistral AI · Apache 2.0
Mistral's voice in 8B, Apache 2.0. 8.92B Q4_K_M · 6.7 GB 10K 33 Tight fit
2 Gemma 4 E4B
Google · Apache 2.0
Readable prose on a laptop. 8.0B Q4_K_M · 5.2 GB 128K 37 Runs great
3 Llama 3.1 8B Instruct
Meta · Llama 3.1 Community
Largest fine-tune ecosystem if you want a specific style. 8.03B Q4_K_M · 6.1 GB 15K 36 Tight fit
4 Qwen3.5 4B
Alibaba · Apache 2.0
Competent drafting in 8 GB. 4.66B Q4_K_M · 3.5 GB 120K 63 Runs great

Best writing LLM for 12 GB VRAM

10.6 GB usable · speeds for a GeForce RTX 3060 12 GB everything that fits 12 GB →

Gemma 4 12B is the pick: The 12 GB writing pick; 140+ languages. 5 other models on the writing shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 Gemma 4 12B
Google · Apache 2.0
The 12 GB writing pick; 140+ languages. 12B Q4_K_M · 8.2 GB 47K 32 Runs great
2 Mistral NeMo 12B
Mistral AI · Apache 2.0
A long-standing roleplay and fiction favourite with a 128K window. 12.2B Q4_K_M · 8.7 GB 20K 32 Runs great
3 Ministral 3 8B
Mistral AI · Apache 2.0
Mistral's voice in 8B, Apache 2.0. 8.92B Q4_K_M · 6.7 GB 37K 43 Runs great
4 Gemma 4 E4B
Google · Apache 2.0
Readable prose on a laptop. 8.0B Q4_K_M · 5.2 GB 128K 48 Runs great
5 Llama 3.1 8B Instruct
Meta · Llama 3.1 Community
Largest fine-tune ecosystem if you want a specific style. 8.03B Q4_K_M · 6.1 GB 43K 48 Runs great
6 Qwen3.5 4B
Alibaba · Apache 2.0
Competent drafting in 8 GB. 4.66B Q4_K_M · 3.5 GB 236K 83 Runs great

Best writing LLM for 16 GB VRAM

14.4 GB usable · speeds for a GeForce RTX 5080 everything that fits 16 GB →

Gemma 4 12B is the pick: The 12 GB writing pick; 140+ languages. 5 other models on the writing shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 Gemma 4 12B
Google · Apache 2.0
The 12 GB writing pick; 140+ languages. 12B Q4_K_M · 8.2 GB 107K 86 Runs great
2 Mistral NeMo 12B
Mistral AI · Apache 2.0
A long-standing roleplay and fiction favourite with a 128K window. 12.2B Q4_K_M · 8.7 GB 44K 85 Runs great
3 Ministral 3 8B
Mistral AI · Apache 2.0
Mistral's voice in 8B, Apache 2.0. 8.92B Q4_K_M · 6.7 GB 66K 116 Runs great
4 Gemma 4 E4B
Google · Apache 2.0
Readable prose on a laptop. 8.0B Q4_K_M · 5.2 GB 128K 129 Runs great
5 Llama 3.1 8B Instruct
Meta · Llama 3.1 Community
Largest fine-tune ecosystem if you want a specific style. 8.03B Q4_K_M · 6.1 GB 74K 129 Runs great
6 Qwen3.5 4B
Alibaba · Apache 2.0
Competent drafting in 8 GB. 4.66B Q4_K_M · 3.5 GB 256K 222 Runs great

Best writing LLM for 24 GB VRAM

22.4 GB usable · speeds for a GeForce RTX 4090 everything that fits 24 GB →

Gemma 4 31B is the pick: The cleanest, least verbose prose of the 2026 24–32 GB class. 8 other models on the writing shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 Gemma 4 31B
Google · Apache 2.0
The cleanest, least verbose prose of the 2026 24–32 GB class. 31.3B Q4_K_M · 20.2 GB 21K 35 Tight fit
2 Gemma 4 26B-A4B
Google · Apache 2.0
Gemma 4 prose at MoE speed, with room for a long draft in context. 26.5B MoE Q4_K_M · 16.0 GB 171K 110 Runs great
3 Mistral Small 3.2 24B
Mistral AI · Apache 2.0
Natural dialogue and a light hand with instructions. 23.6B Q4_K_M · 15.1 GB 54K 46 Runs great
4 Gemma 4 12B
Google · Apache 2.0
The 12 GB writing pick; 140+ languages. 12B Q4_K_M · 8.2 GB 235K 90 Runs great
5 Mistral NeMo 12B
Mistral AI · Apache 2.0
A long-standing roleplay and fiction favourite with a 128K window. 12.2B Q4_K_M · 8.7 GB 95K 89 Runs great
6 Ministral 3 8B
Mistral AI · Apache 2.0
Mistral's voice in 8B, Apache 2.0. 8.92B Q4_K_M · 6.7 GB 126K 122 Runs great
7 Gemma 4 E4B
Google · Apache 2.0
Readable prose on a laptop. 8.0B Q4_K_M · 5.2 GB 128K 136 Runs great
8 Llama 3.1 8B Instruct
Meta · Llama 3.1 Community
Largest fine-tune ecosystem if you want a specific style. 8.03B Q4_K_M · 6.1 GB 128K 135 Runs great
9 Qwen3.5 4B
Alibaba · Apache 2.0
Competent drafting in 8 GB. 4.66B Q4_K_M · 3.5 GB 256K 233 Runs great

Best writing LLM for 32 GB VRAM

30.4 GB usable · speeds for a GeForce RTX 5090 everything that fits 32 GB →

Gemma 4 31B is the pick: The cleanest, least verbose prose of the 2026 24–32 GB class. 8 other models on the writing shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 Gemma 4 31B
Google · Apache 2.0
The cleanest, least verbose prose of the 2026 24–32 GB class. 31.3B Q4_K_M · 20.2 GB 73K 62 Runs great
2 Gemma 4 26B-A4B
Google · Apache 2.0
Gemma 4 prose at MoE speed, with room for a long draft in context. 26.5B MoE Q4_K_M · 16.0 GB 256K 195 Runs great
3 Mistral Small 3.2 24B
Mistral AI · Apache 2.0
Natural dialogue and a light hand with instructions. 23.6B Q4_K_M · 15.1 GB 105K 82 Runs great
4 Gemma 4 12B
Google · Apache 2.0
The 12 GB writing pick; 140+ languages. 12B Q4_K_M · 8.2 GB 256K 161 Runs great
5 Mistral NeMo 12B
Mistral AI · Apache 2.0
A long-standing roleplay and fiction favourite with a 128K window. 12.2B Q4_K_M · 8.7 GB 128K 158 Runs great
6 Ministral 3 8B
Mistral AI · Apache 2.0
Mistral's voice in 8B, Apache 2.0. 8.92B Q4_K_M · 6.7 GB 186K 216 Runs great
7 Gemma 4 E4B
Google · Apache 2.0
Readable prose on a laptop. 8.0B Q4_K_M · 5.2 GB 128K 241 Runs great
8 Llama 3.1 8B Instruct
Meta · Llama 3.1 Community
Largest fine-tune ecosystem if you want a specific style. 8.03B Q4_K_M · 6.1 GB 128K 240 Runs great
9 Qwen3.5 4B
Alibaba · Apache 2.0
Competent drafting in 8 GB. 4.66B Q4_K_M · 3.5 GB 256K 414 Runs great

Best writing LLM for 48 GB VRAM

46.4 GB usable · speeds for an RTX 6000 Ada everything that fits 48 GB →

Llama 3.3 70B Instruct is the pick: The creative-writing favourite: consistent voice, takes direction, handles fiction framing. 9 other models on the writing shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 Llama 3.3 70B Instruct
Meta · Llama 3.3 Community
The creative-writing favourite: consistent voice, takes direction, handles fiction framing. 70.6B Q4_K_M · 42.8 GB 19K 15 Tight fit
2 Gemma 4 31B
Google · Apache 2.0
The cleanest, least verbose prose of the 2026 24–32 GB class. 31.3B Q4_K_M · 20.2 GB 175K 33 Runs great
3 Gemma 4 26B-A4B
Google · Apache 2.0
Gemma 4 prose at MoE speed, with room for a long draft in context. 26.5B MoE Q4_K_M · 16.0 GB 256K 105 Runs great
4 Mistral Small 3.2 24B
Mistral AI · Apache 2.0
Natural dialogue and a light hand with instructions. 23.6B Q4_K_M · 15.1 GB 128K 44 Runs great
5 Gemma 4 12B
Google · Apache 2.0
The 12 GB writing pick; 140+ languages. 12B Q4_K_M · 8.2 GB 256K 86 Runs great
6 Mistral NeMo 12B
Mistral AI · Apache 2.0
A long-standing roleplay and fiction favourite with a 128K window. 12.2B Q4_K_M · 8.7 GB 128K 85 Runs great
7 Ministral 3 8B
Mistral AI · Apache 2.0
Mistral's voice in 8B, Apache 2.0. 8.92B Q4_K_M · 6.7 GB 256K 116 Runs great
8 Gemma 4 E4B
Google · Apache 2.0
Readable prose on a laptop. 8.0B Q4_K_M · 5.2 GB 128K 129 Runs great
9 Llama 3.1 8B Instruct
Meta · Llama 3.1 Community
Largest fine-tune ecosystem if you want a specific style. 8.03B Q4_K_M · 6.1 GB 128K 129 Runs great
10 Qwen3.5 4B
Alibaba · Apache 2.0
Competent drafting in 8 GB. 4.66B Q4_K_M · 3.5 GB 256K 222 Runs great

Best writing LLM for 16 GB unified memory

10.7 GB usable · speeds for an M4 · 16 GB everything that fits 16 GB →

Gemma 4 12B is the pick: The 12 GB writing pick; 140+ languages. 5 other models on the writing shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 Gemma 4 12B
Google · Apache 2.0
The 12 GB writing pick; 140+ languages. 12B Q4_K_M · 8.2 GB 48K 9.9 Runs great
2 Mistral NeMo 12B
Mistral AI · Apache 2.0
A long-standing roleplay and fiction favourite with a 128K window. 12.2B Q4_K_M · 8.7 GB 20K 9.8 Runs great
3 Ministral 3 8B
Mistral AI · Apache 2.0
Mistral's voice in 8B, Apache 2.0. 8.92B Q4_K_M · 6.7 GB 38K 13 Runs great
4 Gemma 4 E4B
Google · Apache 2.0
Readable prose on a laptop. 8.0B Q4_K_M · 5.2 GB 128K 15 Runs great
5 Llama 3.1 8B Instruct
Meta · Llama 3.1 Community
Largest fine-tune ecosystem if you want a specific style. 8.03B Q4_K_M · 6.1 GB 44K 15 Runs great
6 Qwen3.5 4B
Alibaba · Apache 2.0
Competent drafting in 8 GB. 4.66B Q4_K_M · 3.5 GB 239K 26 Runs great

Best writing LLM for 64 GB unified memory

48.0 GB usable · speeds for an M4 Max · 64 GB everything that fits 64 GB →

Llama 3.3 70B Instruct is the pick: The creative-writing favourite: consistent voice, takes direction, handles fiction framing. 9 other models on the writing shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 Llama 3.3 70B Instruct
Meta · Llama 3.3 Community
The creative-writing favourite: consistent voice, takes direction, handles fiction framing. 70.6B Q4_K_M · 42.8 GB 24K 7.7 Tight fit
2 Gemma 4 31B
Google · Apache 2.0
The cleanest, least verbose prose of the 2026 24–32 GB class. 31.3B Q4_K_M · 20.2 GB 185K 17 Runs great
3 Gemma 4 26B-A4B
Google · Apache 2.0
Gemma 4 prose at MoE speed, with room for a long draft in context. 26.5B MoE Q4_K_M · 16.0 GB 256K 67 Runs great
4 Mistral Small 3.2 24B
Mistral AI · Apache 2.0
Natural dialogue and a light hand with instructions. 23.6B Q4_K_M · 15.1 GB 128K 23 Runs great
5 Gemma 4 12B
Google · Apache 2.0
The 12 GB writing pick; 140+ languages. 12B Q4_K_M · 8.2 GB 256K 45 Runs great
6 Mistral NeMo 12B
Mistral AI · Apache 2.0
A long-standing roleplay and fiction favourite with a 128K window. 12.2B Q4_K_M · 8.7 GB 128K 44 Runs great
7 Ministral 3 8B
Mistral AI · Apache 2.0
Mistral's voice in 8B, Apache 2.0. 8.92B Q4_K_M · 6.7 GB 256K 61 Runs great
8 Gemma 4 E4B
Google · Apache 2.0
Readable prose on a laptop. 8.0B Q4_K_M · 5.2 GB 128K 68 Runs great
9 Llama 3.1 8B Instruct
Meta · Llama 3.1 Community
Largest fine-tune ecosystem if you want a specific style. 8.03B Q4_K_M · 6.1 GB 128K 68 Runs great
10 Qwen3.5 4B
Alibaba · Apache 2.0
Competent drafting in 8 GB. 4.66B Q4_K_M · 3.5 GB 256K 116 Runs great

Best writing LLM for 128 GB unified memory

96.0 GB usable · speeds for an M4 Max · 128 GB everything that fits 128 GB →

Llama 3.3 70B Instruct is the pick: The creative-writing favourite: consistent voice, takes direction, handles fiction framing. 9 other models on the writing shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 Llama 3.3 70B Instruct
Meta · Llama 3.3 Community
The creative-writing favourite: consistent voice, takes direction, handles fiction framing. 70.6B Q4_K_M · 42.8 GB 128K 7.7 Runs great
2 Gemma 4 31B
Google · Apache 2.0
The cleanest, least verbose prose of the 2026 24–32 GB class. 31.3B Q4_K_M · 20.2 GB 256K 17 Runs great
3 Gemma 4 26B-A4B
Google · Apache 2.0
Gemma 4 prose at MoE speed, with room for a long draft in context. 26.5B MoE Q4_K_M · 16.0 GB 256K 67 Runs great
4 Mistral Small 3.2 24B
Mistral AI · Apache 2.0
Natural dialogue and a light hand with instructions. 23.6B Q4_K_M · 15.1 GB 128K 23 Runs great
5 Gemma 4 12B
Google · Apache 2.0
The 12 GB writing pick; 140+ languages. 12B Q4_K_M · 8.2 GB 256K 45 Runs great
6 Mistral NeMo 12B
Mistral AI · Apache 2.0
A long-standing roleplay and fiction favourite with a 128K window. 12.2B Q4_K_M · 8.7 GB 128K 44 Runs great
7 Ministral 3 8B
Mistral AI · Apache 2.0
Mistral's voice in 8B, Apache 2.0. 8.92B Q4_K_M · 6.7 GB 256K 61 Runs great
8 Gemma 4 E4B
Google · Apache 2.0
Readable prose on a laptop. 8.0B Q4_K_M · 5.2 GB 128K 68 Runs great
9 Llama 3.1 8B Instruct
Meta · Llama 3.1 Community
Largest fine-tune ecosystem if you want a specific style. 8.03B Q4_K_M · 6.1 GB 128K 68 Runs great
10 Qwen3.5 4B
Alibaba · Apache 2.0
Competent drafting in 8 GB. 4.66B Q4_K_M · 3.5 GB 256K 116 Runs great

The whole writing shortlist, and what each one needs

Every model on the list, best first, with the smallest discrete card and the smallest Mac that run it with 15% headroom at the recommended quantisation and 8K context. Every class has its own page, Macs from 8 to 512 GB included — the full list.

#ModelParamsWeightsNeeds (GPU)Needs (Mac)Ollama tag
1 Llama 3.3 70B Instruct
Still the creative-writing favourite: consistent voice, takes direction. Needs 48 GB to sit comfortably on GPU at Q4.
70.6B 39.7 GB @ Q4_K_M 64 GB card 96 GB Mac llama3.3:70b
2 Gemma 4 31B
The dense flagship: strongest maths of the 24–32 GB class (89% AIME), clean prose, vision. Q4 is a tight 24 GB fit.
31.3B 17.6 GB @ Q4_K_M 32 GB card 32 GB Mac gemma4:31b
3 Gemma 4 26B-A4B
Mixture of experts with 3.8B active. Slower to think than Qwen3.6 35B-A3B, faster to answer, and it sees images.
26.5B MoE 14.9 GB @ Q4_K_M 24 GB card 32 GB Mac gemma4:26b
4 Mistral Small 3.2 24B
Apache-2.0, vision-capable, and the most 24 GB-friendly of the 2025 generalists.
23.6B 13.3 GB @ Q4_K_M 20 GB card 24 GB Mac mistral-small:24b
5 Gemma 4 12B
The "unified" Gemma 4: text, image and audio in one 12B that fits a 12 GB card at Q4. 140+ languages.
12B 6.7 GB @ Q4_K_M 11 GB card 16 GB Mac gemma4:12b
6 Mistral NeMo 12B
Multilingual 12B with a 128K window, built with NVIDIA. A roleplay and fiction favourite that refuses to die.
12.2B 6.9 GB @ Q4_K_M 12 GB card 16 GB Mac mistral-nemo:12b
7 Ministral 3 8B
Mistral's 8B with images in. Plain GQA, so budget more KV cache than Qwen3.5 9B at the same context.
8.92B 5.0 GB @ Q4_K_M 10 GB card 16 GB Mac ministral-3:8b
8 Gemma 4 E4B
The laptop Gemma. 4.5B effective, 8B on disk; a single KV head per window layer keeps its cache tiny.
8.0B 4.5 GB @ Q4_K_M 8 GB card 16 GB Mac gemma4:e4b
9 Llama 3.1 8B Instruct
Still the most widely deployed local model, with the largest fine-tune ecosystem. Not the strongest 8B any more.
8.03B 4.5 GB @ Q4_K_M 10 GB card 16 GB Mac llama3.1:8b
10 Qwen3.5 4B
The 8 GB coding agent. Q4 lands near 3.4 GB, leaving room for a real context window.
4.66B 2.6 GB @ Q4_K_M 6 GB card 8 GB Mac qwen3.5:4b

Instruction-tuned models are trained to be helpful, which reads as bland in fiction. If the pick feels flat, a community fine-tune of the same base — Llama 3.1 8B and Mistral NeMo have the most — is the usual next step, and needs exactly the memory the base model does.

Other jobs