Hardware requirements

What hardware do I need to run Qwen3.8 27B?

27.8B 256K context Apache 2.0 New this week

At Q4_K_M the weights are 15.6 GB; with an 8K KV cache and runtime overhead that is 16.7 GB, so it wants a 24 GB card to run with headroom — in practice 24 GB VRAM: GeForce RTX 4090 and 8 others. Every figure below is computed from the model’s published shape against each device’s usable memory; tokens per second are estimates. The method.

Minimum practical 16 GB VRAM

GeForce RTX 5080 — Q3_K_M, 13.8 GB of 14.4 GB, ~46 tok/s, up to 18K context.

Also: GeForce RTX 5070 Ti, GeForce RTX 4080 Super, GeForce RTX 4080, GeForce RTX 4070 Ti Super, Radeon RX 9070 XT +12 more
the smallest card that holds it at all, at Q3_K_M — the lowest quantisation worth running
Recommended 24 GB VRAM

GeForce RTX 4090 — Q4_K_M, 16.7 GB of 22.4 GB, ~39 tok/s, up to 98K context.

Also: Radeon RX 7900 XTX, GeForce RTX 3090 Ti, GeForce RTX 3090, GeForce RTX 5090 Laptop, RTX A5000 +3 more
runs the recommended quantisation with headroom at 8K context
High quality 40 GB VRAM

A100 40 GB — Q8_0, 28.6 GB of 38.4 GB, ~34 tok/s, up to 164K context.

near-lossless Q8_0 with headroom
Long context 32 GB VRAM

GeForce RTX 5090 — Q4_K_M, 24.2 GB of 30.4 GB, ~69 tok/s at 128K.

Also: RTX 5000 Ada, Radeon Pro W7800
128K tokens (or the model’s maximum) at the recommended quantisation
Apple Silicon 32 GB unified

M1 Pro · 32 GB — Q4_K_M, 16.7 GB of 24.0 GB, ~7.1 tok/s, up to 124K context.

Also: M2 Pro · 32 GB, M4 · 32 GB
smallest Mac that runs the recommended quantisation with headroom
CPU only 32 GB RAM

CPU only · DDR4 dual-channel — Q6_K, 22.3 GB of 25.6 GB, ~1.0 tok/s, up to 60K context.

no GPU — weights stream from system RAM

How much VRAM at each quantisation

QuantWeights+KV 8K+KV 32K+KV 128KTotal @ 8KNeedsQuality
Q8_027.5 GB0.50 GB2.00 GB8.0 GB 28.6 GB40 GB card−0.1% ppl
Q6_K21.2 GB0.50 GB2.00 GB8.0 GB 22.3 GB32 GB card−0.4% ppl
Q5_K_M18.4 GB0.50 GB2.00 GB8.0 GB 19.5 GB32 GB card−0.8% ppl
Q4_K_M15.6 GB0.50 GB2.00 GB8.0 GB 16.7 GB24 GB card−1.9% ppl
Q3_K_M12.7 GB0.50 GB2.00 GB8.0 GB 13.8 GB20 GB card−5.4% ppl
Q2_K10.8 GB0.50 GB2.00 GB8.0 GB 11.9 GB16 GB card−15% ppl

Totals add 0.6 GB runtime overhead and an f16 KV cache; "needs" is the smallest discrete card class whose usable memory (after display and driver reserve) leaves 15% headroom. A q8_0 cache roughly halves the KV columns.

Every device, checked

System RAM for offload:
163264128256

NVIDIA · GeForce

DeviceMemoryBest quant with headroomFits at allMax context @ Q4_K_MTok/s est.128K
GeForce RTX 5090 32 GB Q6_K Q8_0 226K 69 yes Check
GeForce RTX 5080 16 GB — Q3_K_M — — no Check
GeForce RTX 5070 Ti 16 GB — Q3_K_M — — no Check
GeForce RTX 5070 12 GB — offload — ~9.1 no Check
GeForce RTX 5060 Ti 16 GB 16 GB — Q3_K_M — — no Check
GeForce RTX 5060 8 GB — offload — ~4.8 no Check
GeForce RTX 4090 24 GB Q4_K_M Q6_K 98K 39 no Check
GeForce RTX 4080 Super 16 GB — Q3_K_M — — no Check
GeForce RTX 4080 16 GB — Q3_K_M — — no Check
GeForce RTX 4070 Ti Super 16 GB — Q3_K_M — — no Check
GeForce RTX 4070 Ti 12 GB — offload — ~8.5 no Check
GeForce RTX 4070 Super 12 GB — offload — ~8.5 no Check
GeForce RTX 4070 12 GB — offload — ~8.5 no Check
GeForce RTX 4060 Ti 16 GB 16 GB — Q3_K_M — — no Check
GeForce RTX 4060 Ti 8 GB — offload — ~4.5 no Check
GeForce RTX 4060 8 GB — offload — ~4.5 no Check
GeForce RTX 3090 Ti 24 GB Q4_K_M Q6_K 98K 39 no Check
GeForce RTX 3090 24 GB Q4_K_M Q6_K 98K 36 no Check
GeForce RTX 3080 Ti 12 GB — offload — ~9.6 no Check
GeForce RTX 3080 12 GB 12 GB — offload — ~9.6 no Check
GeForce RTX 3080 10 GB — offload — ~6.5 no Check
GeForce RTX 3070 Ti 8 GB — offload — ~5.0 no Check
GeForce RTX 3070 8 GB — offload — ~4.8 no Check
GeForce RTX 3060 Ti 8 GB — offload — ~4.8 no Check
GeForce RTX 3060 12 GB 12 GB — offload — ~7.7 no Check
GeForce RTX 2080 Ti 11 GB — offload — ~7.4 no Check
GeForce RTX 2060 12 GB 12 GB — offload — ~7.5 no Check
GeForce GTX 1080 Ti 11 GB — offload — ~7.1 no Check

NVIDIA · GeForce laptop

DeviceMemoryBest quant with headroomFits at allMax context @ Q4_K_MTok/s est.128K
GeForce RTX 5090 Laptop 24 GB Q4_K_M Q6_K 98K 35 no Check
GeForce RTX 4090 Laptop 16 GB — Q3_K_M — — no Check
GeForce RTX 4080 Laptop 12 GB — offload — ~8.1 no Check
GeForce RTX 4070 Laptop 8 GB — offload — ~4.5 no Check

NVIDIA · Workstation

DeviceMemoryBest quant with headroomFits at allMax context @ Q4_K_MTok/s est.128K
RTX 6000 Ada 48 GB Q8_0 Q8_0 256K 37 yes Check
RTX 5000 Ada 32 GB Q6_K Q8_0 226K 22 yes Check
RTX 4000 Ada 20 GB Q3_K_M Q4_K_M 34K 14 no Check
RTX 2000 Ada 16 GB — Q3_K_M — — no Check
RTX A6000 48 GB Q8_0 Q8_0 256K 30 yes Check
RTX A5000 24 GB Q4_K_M Q6_K 98K 30 no Check
RTX A4000 16 GB — Q3_K_M — — no Check

NVIDIA · Data centre

DeviceMemoryBest quant with headroomFits at allMax context @ Q4_K_MTok/s est.128K
L40S 48 GB Q8_0 Q8_0 256K 33 yes Check
L4 24 GB Q4_K_M Q6_K 98K 12 no Check
A10 24 GB Q4_K_M Q6_K 98K 23 no Check
A100 40 GB 40 GB Q8_0 Q8_0 256K 60 yes Check
A100 80 GB 80 GB Q8_0 Q8_0 256K 79 yes Check
H100 PCIe 80 GB Q8_0 Q8_0 256K 77 yes Check
H100 SXM 80 GB Q8_0 Q8_0 256K 130 yes Check
H200 SXM 141 GB Q8_0 Q8_0 256K 186 yes Check
Tesla P40 24 GB Q4_K_M Q6_K 98K 13 no Check

AMD · Radeon

DeviceMemoryBest quant with headroomFits at allMax context @ Q4_K_MTok/s est.128K
Radeon RX 9070 XT 16 GB — Q3_K_M — — no Check
Radeon RX 9070 16 GB — Q3_K_M — — no Check
Radeon RX 7900 XTX 24 GB Q4_K_M Q6_K 98K 37 no Check
Radeon RX 7900 XT 20 GB Q3_K_M Q4_K_M 34K 31 no Check
Radeon RX 7900 GRE 16 GB — Q3_K_M — — no Check
Radeon RX 7800 XT 16 GB — Q3_K_M — — no Check
Radeon RX 7700 XT 12 GB — offload — ~8.1 no Check
Radeon RX 7600 XT 16 GB — Q3_K_M — — no Check
Radeon RX 6900 XT 16 GB — Q3_K_M — — no Check
Radeon RX 6800 XT 16 GB — Q3_K_M — — no Check
Radeon RX 6700 XT 12 GB — offload — ~7.8 no Check

AMD · Radeon Pro

DeviceMemoryBest quant with headroomFits at allMax context @ Q4_K_MTok/s est.128K
Radeon Pro W7900 48 GB Q8_0 Q8_0 256K 33 yes Check
Radeon Pro W7800 32 GB Q6_K Q8_0 226K 22 yes Check

AMD · Instinct

DeviceMemoryBest quant with headroomFits at allMax context @ Q4_K_MTok/s est.128K
Instinct MI210 64 GB Q8_0 Q8_0 256K 63 yes Check
Instinct MI300X 192 GB Q8_0 Q8_0 256K 205 yes Check

AMD · Ryzen AI Max

DeviceMemoryBest quant with headroomFits at allMax context @ Q4_K_MTok/s est.128K
Ryzen AI Max+ 395 · 128 GB 128 GB Q8_0 Q8_0 256K 9.2 yes Check
Ryzen AI Max+ 395 · 64 GB 64 GB Q8_0 Q8_0 256K 9.2 yes Check

Apple · Apple Silicon

DeviceMemoryBest quant with headroomFits at allMax context @ Q4_K_MTok/s est.128K
M1 · 16 GB 16 GB — no — — no Check
M1 Pro · 32 GB 32 GB Q5_K_M Q6_K 124K 7.1 no Check
M1 Max · 64 GB 64 GB Q8_0 Q8_0 256K 14 yes Check
M1 Ultra · 128 GB 128 GB Q8_0 Q8_0 256K 29 yes Check
M2 · 24 GB 24 GB Q3_K_M Q4_K_M 28K 3.6 no Check
M2 Pro · 32 GB 32 GB Q5_K_M Q6_K 124K 7.1 no Check
M2 Max · 96 GB 96 GB Q8_0 Q8_0 256K 14 yes Check
M2 Ultra · 192 GB 192 GB Q8_0 Q8_0 256K 29 yes Check
M3 · 24 GB 24 GB Q3_K_M Q4_K_M 28K 3.6 no Check
M3 Pro · 36 GB 36 GB Q6_K Q6_K 172K 5.4 tight Check
M3 Max · 64 GB 64 GB Q8_0 Q8_0 256K 14 yes Check
M3 Max · 128 GB 128 GB Q8_0 Q8_0 256K 14 yes Check
M3 Ultra · 256 GB 256 GB Q8_0 Q8_0 256K 29 yes Check
M3 Ultra · 512 GB 512 GB Q8_0 Q8_0 256K 29 yes Check
M4 · 32 GB 32 GB Q5_K_M Q6_K 124K 4.3 no Check
M4 Pro · 48 GB 48 GB Q8_0 Q8_0 256K 9.8 yes Check
M4 Pro · 64 GB 64 GB Q8_0 Q8_0 256K 9.8 yes Check
M4 Max · 64 GB 64 GB Q8_0 Q8_0 256K 15 yes Check
M4 Max · 128 GB 128 GB Q8_0 Q8_0 256K 20 yes Check

Intel · Arc

DeviceMemoryBest quant with headroomFits at allMax context @ Q4_K_MTok/s est.128K
Arc B580 12 GB — offload — ~8.2 no Check
Arc B570 10 GB — offload — ~5.9 no Check
Arc A770 16 GB 16 GB — Q3_K_M — — no Check
Arc A750 8 GB — offload — ~4.9 no Check

CPU · CPU + system RAM

DeviceMemoryBest quant with headroomFits at allMax context @ Q4_K_MTok/s est.128K
CPU only · DDR4 dual-channel 32 GB Q5_K_M Q6_K 149K 1.4 tight Check
CPU only · DDR5 dual-channel 64 GB Q8_0 Q8_0 256K 2.4 yes Check
CPU only · DDR5 8-channel server 256 GB Q8_0 Q8_0 256K 8.2 yes Check