No — not on this device
Qwen2.5-Coder 32B at Q4_K_M needs 21.0 GB against 5.4 GB usable, and the shortfall of 15.6 GB is more than 128 GB of system RAM can cover at a tolerable speed. A smaller sibling or a lower quantisation is the honest answer here.
The first local code model that felt competitive with hosted assistants. Qwen3.8 27B is smaller and far ahead on agentic work.
Hardware requirements What hardware do I need to run Qwen2.5-Coder 32B? →At Q4_K_M with an 8K context: a 32 GB GPU (RTX 5090) or a 36 GB Mac (M4 Max). Minimum, recommended, high-quality and long-context tiers, with all 114 devices checked at every quantisation.
Fits instead: Qwen2.5-Coder 7B (5.3 GB)
The VRAM budget
Quantisation ladder
| Quant | Weights | Total @ 8K | Max context | Tok/s | Quality | Fit |
|---|---|---|---|---|---|---|
| Q8_0 | 32.5 GB | 35.1 GB | — | ~1.7 | −0.1% ppl | 29.7 GB over |
| Q6_K | 25.0 GB | 27.6 GB | — | ~2.2 | −0.4% ppl | 22.2 GB over |
| Q5_K_M | 21.7 GB | 24.3 GB | — | ~2.6 | −0.8% ppl | 18.9 GB over |
| Q4_K_M | 18.4 GB | 21.0 GB | — | ~3.0 | −1.9% ppl | 15.6 GB over |
| Q3_K_M | 14.9 GB | 17.5 GB | — | ~3.7 | −5.4% ppl | 12.1 GB over |
| Q2_K | 12.8 GB | 15.4 GB | — | ~4.4 | −15% ppl | 10.0 GB over |
Quality is the published perplexity delta against f16 weights. Max context assumes an f16 KV cache; q8_0 roughly doubles it.
How to run it
$ pip install mlx-lm $ mlx_lm.generate --model mlx-community/Qwen2.5-Coder-32B-Instruct-4bit \ --max-tokens 512 --prompt "Hello"
Apple's own array framework. The fastest path on Apple Silicon. More on MLX.
Related models on an M2 · 8 GB
| Model | Total | Tok/s est. | |
|---|---|---|---|
| Qwen2.5-Coder 14B 14.8B · same family |
Q4_K_M · 10.4 GB | — | Won't fit |
| Qwen2.5-Coder 7B 7.62B · same family |
Q4_K_M · 5.3 GB | 13 | Tight fit |
| Qwen3 32B 32.8B · similar size |
Q4_K_M · 21.0 GB | — | Won't fit |
| DeepSeek-R1-Distill-Qwen 32B 32.8B · similar size |
Q4_K_M · 21.0 GB | — | Won't fit |
| LLM-jp 4 33B Thinking 33.2B · similar size |
Q4_K_M · 21.3 GB | — | Won't fit |
| Olmo 3.1 32B Instruct 32.2B · similar size |
Q4_K_M · 20.0 GB | — | Won't fit |
| Nemotron 3.5 Lightning 30B-A3B 31.6B MoE · similar size |
Q4_K_M · 18.4 GB | — | Won't fit |
Qwen2.5-Coder 32B on other hardware
| Device | Total | Tok/s est. | |
|---|---|---|---|
| GeForce RTX 4060 Laptop 7.0 GB usable · 256 GB/s |
21.0 GB | ~2.4 | CPU offload |
| GeForce RTX 3060 12 GB 10.6 GB usable · 360 GB/s |
21.0 GB | ~3.1 | CPU offload |
| M4 · 16 GB 10.7 GB usable · 120 GB/s |
21.0 GB | — | Won't fit |
| GeForce RTX 4090 22.4 GB usable · 1008 GB/s |
21.0 GB | 33 | Tight fit |
| Radeon RX 7900 XTX 22.4 GB usable · 960 GB/s |
21.0 GB | 32 | Tight fit |
| GeForce RTX 5090 30.4 GB usable · 1792 GB/s |
21.0 GB | 59 | Runs great |
| M4 Max · 36 GB 27.0 GB usable · 410 GB/s |
21.0 GB | 12 | Runs great |
| M3 Max · 64 GB 48.0 GB usable · 400 GB/s |
21.0 GB | 12 | Runs great |