Hardware requirements

What hardware do I need to run Gemma 3 4B?

4.3B 128K context Gemma Terms of Use

At Q4_K_M the weights are 2.4 GB; with an 8K KV cache and runtime overhead that is 3.3 GB, so it wants a 6 GB card to run with headroom — in practice 6 GB VRAM: GeForce RTX 3060 Laptop and 1 other. Every figure below is computed from the model’s published shape against each device’s usable memory; tokens per second are estimates. The method.

Minimum practical 6 GB VRAM

GeForce RTX 3060 Laptop — Q8_0, 5.2 GB of 5.3 GB, ~48 tok/s, up to 14K context.

the smallest card that holds it at all, at Q3_K_M — the lowest quantisation worth running
Recommended 6 GB VRAM

GeForce RTX 3060 Laptop — Q4_K_M, 3.3 GB of 5.3 GB, ~84 tok/s, up to 92K context.

runs the recommended quantisation with headroom at 8K context
High quality 8 GB VRAM

GeForce RTX 4060 Laptop — Q8_0, 5.2 GB of 7.0 GB, ~36 tok/s, up to 86K context.

near-lossless Q8_0 with headroom
Long context 10 GB VRAM

GeForce RTX 3080 — Q4_K_M, 6.1 GB of 8.8 GB, ~190 tok/s at 128K.

Also: Arc B570
128K tokens (or the model’s maximum) at the recommended quantisation
Apple Silicon 8 GB unified

M2 · 8 GB — Q4_K_M, 3.3 GB of 5.4 GB, ~23 tok/s, up to 96K context.

smallest Mac that runs the recommended quantisation with headroom
CPU only 32 GB RAM

CPU only · DDR4 dual-channel — Q8_0, 5.2 GB of 25.6 GB, ~5.0 tok/s, up to 128K context.

no GPU — weights stream from system RAM

How much VRAM at each quantisation

QuantWeights+KV 8K+KV 32K+KV 128KTotal @ 8KNeedsQuality
F168.0 GB0.30 GB0.86 GB3.1 GB 8.9 GB12 GB cardReference
Q8_04.3 GB0.30 GB0.86 GB3.1 GB 5.2 GB8 GB card−0.1% ppl
Q6_K3.3 GB0.30 GB0.86 GB3.1 GB 4.2 GB6 GB card−0.4% ppl
Q5_K_M2.8 GB0.30 GB0.86 GB3.1 GB 3.7 GB6 GB card−0.8% ppl
Q4_K_M2.4 GB0.30 GB0.86 GB3.1 GB 3.3 GB6 GB card−1.9% ppl
Q3_K_M2.0 GB0.30 GB0.86 GB3.1 GB 2.9 GB6 GB card−5.4% ppl
Q2_K1.7 GB0.30 GB0.86 GB3.1 GB 2.6 GB6 GB card−15% ppl

Totals add 0.6 GB runtime overhead and an f16 KV cache; "needs" is the smallest discrete card class whose usable memory (after display and driver reserve) leaves 15% headroom. A q8_0 cache roughly halves the KV columns.

Every device, checked

System RAM for offload:

NVIDIA · GeForce

DeviceMemoryBest quant with headroomFits at allMax context @ Q4_K_MTok/s est.128K
GeForce RTX 5090 32 GB Q8_0 Q8_0 128K 449 yes Check
GeForce RTX 5080 16 GB Q8_0 Q8_0 128K 240 yes Check
GeForce RTX 5070 Ti 16 GB Q8_0 Q8_0 128K 224 yes Check
GeForce RTX 5070 12 GB Q8_0 Q8_0 128K 168 yes Check
GeForce RTX 5060 Ti 16 GB 16 GB Q8_0 Q8_0 128K 112 yes Check
GeForce RTX 5060 8 GB Q8_0 Q8_0 128K 112 tight Check
GeForce RTX 4090 24 GB Q8_0 Q8_0 128K 252 yes Check
GeForce RTX 4080 Super 16 GB Q8_0 Q8_0 128K 184 yes Check
GeForce RTX 4080 16 GB Q8_0 Q8_0 128K 180 yes Check
GeForce RTX 4070 Ti Super 16 GB Q8_0 Q8_0 128K 168 yes Check
GeForce RTX 4070 Ti 12 GB Q8_0 Q8_0 128K 126 yes Check
GeForce RTX 4070 Super 12 GB Q8_0 Q8_0 128K 126 yes Check
GeForce RTX 4070 12 GB Q8_0 Q8_0 128K 126 yes Check
GeForce RTX 4060 Ti 16 GB 16 GB Q8_0 Q8_0 128K 72 yes Check
GeForce RTX 4060 Ti 8 GB Q8_0 Q8_0 128K 72 tight Check
GeForce RTX 4060 8 GB Q8_0 Q8_0 128K 68 tight Check
GeForce RTX 3090 Ti 24 GB Q8_0 Q8_0 128K 252 yes Check
GeForce RTX 3090 24 GB Q8_0 Q8_0 128K 234 yes Check
GeForce RTX 3080 Ti 12 GB Q8_0 Q8_0 128K 228 yes Check
GeForce RTX 3080 12 GB 12 GB Q8_0 Q8_0 128K 228 yes Check
GeForce RTX 3080 10 GB Q8_0 Q8_0 128K 190 yes Check
GeForce RTX 3070 Ti 8 GB Q8_0 Q8_0 128K 152 tight Check
GeForce RTX 3070 8 GB Q8_0 Q8_0 128K 112 tight Check
GeForce RTX 3060 Ti 8 GB Q8_0 Q8_0 128K 112 tight Check
GeForce RTX 3060 12 GB 12 GB Q8_0 Q8_0 128K 90 yes Check
GeForce RTX 3060 8 GB 8 GB Q8_0 Q8_0 128K 60 tight Check
GeForce RTX 2080 Ti 11 GB Q8_0 Q8_0 128K 154 yes Check
GeForce RTX 2080 Super 8 GB Q8_0 Q8_0 128K 124 tight Check
GeForce RTX 2060 12 GB 12 GB Q8_0 Q8_0 128K 84 yes Check
GeForce GTX 1660 Super 6 GB Q6_K Q8_0 92K 84 no Check
GeForce GTX 1080 Ti 11 GB Q8_0 Q8_0 128K 121 yes Check

NVIDIA · GeForce laptop

DeviceMemoryBest quant with headroomFits at allMax context @ Q4_K_MTok/s est.128K
GeForce RTX 5090 Laptop 24 GB Q8_0 Q8_0 128K 224 yes Check
GeForce RTX 4090 Laptop 16 GB Q8_0 Q8_0 128K 144 yes Check
GeForce RTX 4080 Laptop 12 GB Q8_0 Q8_0 128K 108 yes Check
GeForce RTX 4070 Laptop 8 GB Q8_0 Q8_0 128K 64 tight Check
GeForce RTX 4060 Laptop 8 GB Q8_0 Q8_0 128K 64 tight Check
GeForce RTX 3060 Laptop 6 GB Q6_K Q8_0 92K 84 no Check

NVIDIA · Workstation

DeviceMemoryBest quant with headroomFits at allMax context @ Q4_K_MTok/s est.128K
RTX 6000 Ada 48 GB Q8_0 Q8_0 128K 240 yes Check
RTX 5000 Ada 32 GB Q8_0 Q8_0 128K 144 yes Check
RTX 4000 Ada 20 GB Q8_0 Q8_0 128K 90 yes Check
RTX 2000 Ada 16 GB Q8_0 Q8_0 128K 56 yes Check
RTX A6000 48 GB Q8_0 Q8_0 128K 192 yes Check
RTX A5000 24 GB Q8_0 Q8_0 128K 192 yes Check
RTX A4000 16 GB Q8_0 Q8_0 128K 112 yes Check

NVIDIA · Data centre

DeviceMemoryBest quant with headroomFits at allMax context @ Q4_K_MTok/s est.128K
L40S 48 GB Q8_0 Q8_0 128K 216 yes Check
L4 24 GB Q8_0 Q8_0 128K 75 yes Check
A10 24 GB Q8_0 Q8_0 128K 150 yes Check
A100 40 GB 40 GB Q8_0 Q8_0 128K 389 yes Check
A100 80 GB 80 GB Q8_0 Q8_0 128K 511 yes Check
H100 PCIe 80 GB Q8_0 Q8_0 128K 501 yes Check
H100 SXM 80 GB Q8_0 Q8_0 128K 839 yes Check
H200 SXM 141 GB Q8_0 Q8_0 128K 1202 yes Check
Tesla P40 24 GB Q8_0 Q8_0 128K 87 yes Check
Tesla P100 16 GB 16 GB Q8_0 Q8_0 128K 183 yes Check
Tesla M40 24 GB 24 GB Q8_0 Q8_0 128K 72 yes Check
CMP 100-210 16 GB Q8_0 Q8_0 128K 208 yes Check

NVIDIA · DGX Spark

DeviceMemoryBest quant with headroomFits at allMax context @ Q4_K_MTok/s est.128K
DGX Spark · 128 GB 128 GB Q8_0 Q8_0 128K 63 yes Check

NVIDIA · Multi-GPU

DeviceMemoryBest quant with headroomFits at allMax context @ Q4_K_MTok/s est.128K
2× GeForce RTX 3090 48 GB Q8_0 Q8_0 128K 234 yes Check
2× GeForce RTX 4090 48 GB Q8_0 Q8_0 128K 252 yes Check
2× GeForce RTX 3060 12 GB 24 GB Q8_0 Q8_0 128K 90 yes Check

AMD · Radeon

DeviceMemoryBest quant with headroomFits at allMax context @ Q4_K_MTok/s est.128K
Radeon RX 9070 XT 16 GB Q8_0 Q8_0 128K 161 yes Check
Radeon RX 9070 16 GB Q8_0 Q8_0 128K 161 yes Check
Radeon RX 7900 XTX 24 GB Q8_0 Q8_0 128K 240 yes Check
Radeon RX 7900 XT 20 GB Q8_0 Q8_0 128K 200 yes Check
Radeon RX 7900 GRE 16 GB Q8_0 Q8_0 128K 144 yes Check
Radeon RX 7800 XT 16 GB Q8_0 Q8_0 128K 156 yes Check
Radeon RX 7700 XT 12 GB Q8_0 Q8_0 128K 108 yes Check
Radeon RX 7600 XT 16 GB Q8_0 Q8_0 128K 72 yes Check
Radeon RX 6900 XT 16 GB Q8_0 Q8_0 128K 128 yes Check
Radeon RX 6800 XT 16 GB Q8_0 Q8_0 128K 128 yes Check
Radeon RX 6700 XT 12 GB Q8_0 Q8_0 128K 96 yes Check
Radeon RX 6750 XT 12 GB Q8_0 Q8_0 128K 108 yes Check
Radeon RX 6600 XT 8 GB Q8_0 Q8_0 128K 64 tight Check
Radeon RX 6600 8 GB Q8_0 Q8_0 128K 56 tight Check

AMD · Radeon Pro

DeviceMemoryBest quant with headroomFits at allMax context @ Q4_K_MTok/s est.128K
Radeon Pro W7900 48 GB Q8_0 Q8_0 128K 216 yes Check
Radeon Pro W7800 32 GB Q8_0 Q8_0 128K 144 yes Check

AMD · Instinct

DeviceMemoryBest quant with headroomFits at allMax context @ Q4_K_MTok/s est.128K
Instinct MI210 64 GB Q8_0 Q8_0 128K 410 yes Check
Instinct MI300X 192 GB Q8_0 Q8_0 128K 1327 yes Check

AMD · Ryzen AI Max

DeviceMemoryBest quant with headroomFits at allMax context @ Q4_K_MTok/s est.128K
Ryzen AI Max+ 395 · 128 GB 128 GB Q8_0 Q8_0 128K 59 yes Check
Ryzen AI Max+ 395 · 64 GB 64 GB Q8_0 Q8_0 128K 59 yes Check

Apple · Apple Silicon

DeviceMemoryBest quant with headroomFits at allMax context @ Q4_K_MTok/s est.128K
M1 · 8 GB 8 GB Q6_K Q8_0 96K 16 no Check
M1 · 16 GB 16 GB Q8_0 Q8_0 128K 16 yes Check
M1 Pro · 32 GB 32 GB Q8_0 Q8_0 128K 46 yes Check
M1 Max · 64 GB 64 GB Q8_0 Q8_0 128K 92 yes Check
M1 Ultra · 128 GB 128 GB Q8_0 Q8_0 128K 185 yes Check
M2 · 8 GB 8 GB Q6_K Q8_0 96K 23 no Check
M2 · 24 GB 24 GB Q8_0 Q8_0 128K 23 yes Check
M2 Pro · 32 GB 32 GB Q8_0 Q8_0 128K 46 yes Check
M2 Max · 64 GB 64 GB Q8_0 Q8_0 128K 92 yes Check
M2 Max · 96 GB 96 GB Q8_0 Q8_0 128K 92 yes Check
M2 Ultra · 192 GB 192 GB Q8_0 Q8_0 128K 185 yes Check
M3 · 8 GB 8 GB Q6_K Q8_0 96K 23 no Check
M3 · 24 GB 24 GB Q8_0 Q8_0 128K 23 yes Check
M3 Pro · 36 GB 36 GB Q8_0 Q8_0 128K 35 yes Check
M3 Max · 64 GB 64 GB Q8_0 Q8_0 128K 92 yes Check
M3 Max · 128 GB 128 GB Q8_0 Q8_0 128K 92 yes Check
M3 Ultra · 256 GB 256 GB Q8_0 Q8_0 128K 185 yes Check
M3 Ultra · 512 GB 512 GB Q8_0 Q8_0 128K 185 yes Check
M4 · 16 GB 16 GB Q8_0 Q8_0 128K 28 yes Check
M4 · 32 GB 32 GB Q8_0 Q8_0 128K 28 yes Check
M4 Pro · 24 GB 24 GB Q8_0 Q8_0 128K 63 yes Check
M4 Pro · 48 GB 48 GB Q8_0 Q8_0 128K 63 yes Check
M4 Pro · 64 GB 64 GB Q8_0 Q8_0 128K 63 yes Check
M4 Max · 36 GB 36 GB Q8_0 Q8_0 128K 95 yes Check
M4 Max · 48 GB 48 GB Q8_0 Q8_0 128K 126 yes Check
M4 Max · 64 GB 64 GB Q8_0 Q8_0 128K 126 yes Check
M4 Max · 128 GB 128 GB Q8_0 Q8_0 128K 126 yes Check

Intel · Arc

DeviceMemoryBest quant with headroomFits at allMax context @ Q4_K_MTok/s est.128K
Arc B580 12 GB Q8_0 Q8_0 128K 114 yes Check
Arc B570 10 GB Q8_0 Q8_0 128K 95 yes Check
Arc A770 16 GB 16 GB Q8_0 Q8_0 128K 140 yes Check
Arc A750 8 GB Q8_0 Q8_0 128K 128 tight Check

CPU · CPU + system RAM

DeviceMemoryBest quant with headroomFits at allMax context @ Q4_K_MTok/s est.128K
CPU only · DDR4 dual-channel 32 GB Q8_0 Q8_0 128K 8.8 yes Check
CPU only · DDR5 dual-channel 64 GB Q8_0 Q8_0 128K 16 yes Check
CPU only · DDR5 8-channel server 256 GB Q8_0 Q8_0 128K 53 yes Check

Buying for this is a break-even question, not a spec question: the card above is paid once, a hosted model is paid per token forever. Price the other side of it with the AI article cost calculator — what one finished article costs on 40 hosted models — and compare that against the volume you expect before you spend on hardware.