How much VRAM does Ling 3.0 Tiny 7.9B-A1.3B need?

At Q4_K_M the weights are 4.4 GB; with an 8K context KV cache and runtime overhead it needs 5.1 GB, so a 8 GB card runs it with headroom.

Can a GeForce RTX 4070 run Ling 3.0 Tiny 7.9B-A1.3B?

Yes — Q8_0 fits with headroom at 8K context, about 91 tokens per second estimated.

Can a GeForce RTX 4090 run Ling 3.0 Tiny 7.9B-A1.3B?

Yes — Q8_0 fits with headroom at 8K context, about 182 tokens per second estimated.

Can a GeForce RTX 3060 12 GB run Ling 3.0 Tiny 7.9B-A1.3B?

Yes — Q8_0 fits with headroom at 8K context, about 65 tokens per second estimated.

Can a M3 Max · 64 GB run Ling 3.0 Tiny 7.9B-A1.3B?

Yes — Q8_0 fits with headroom at 8K context, about 81 tokens per second estimated.

Hardware requirements

What hardware do I need to run Ling 3.0 Tiny 7.9B-A1.3B?

7.9B MoE · 1.3B active 128K context MIT New this month

At Q4_K_M the weights are 4.4 GB; with an 8K KV cache and runtime overhead that is 5.1 GB, so it wants a 8 GB card to run with headroom — in practice 8 GB VRAM: GeForce RTX 3070 Ti and 7 others. Every figure below is computed from the model’s published shape against each device’s usable memory; tokens per second are estimates. The method.

Minimum practical 8 GB VRAM

GeForce RTX 3070 Ti — Q6_K, 6.7 GB of 7.0 GB, ~143 tok/s, up to 55K context.

Also: Arc A750, GeForce RTX 5060, GeForce RTX 3070, GeForce RTX 3060 Ti, GeForce RTX 4060 Ti +2 more

the smallest card that holds it at all, at Q3_K_M — the lowest quantisation worth running

Recommended 8 GB VRAM

GeForce RTX 3070 Ti — Q4_K_M, 5.1 GB of 7.0 GB, ~194 tok/s, up to 128K context.

Also: Arc A750, GeForce RTX 5060, GeForce RTX 3070, GeForce RTX 3060 Ti, GeForce RTX 4060 Ti +2 more

runs the recommended quantisation with headroom at 8K context

High quality 12 GB VRAM

GeForce RTX 3060 12 GB — Q8_0, 8.5 GB of 10.6 GB, ~65 tok/s, up to 128K context.

Also: GeForce RTX 3080 Ti, GeForce RTX 3080 12 GB, GeForce RTX 5070, GeForce RTX 4070 Ti, GeForce RTX 4070 Super +6 more

near-lossless Q8_0 with headroom

Long context 8 GB VRAM

GeForce RTX 3070 Ti — Q4_K_M, 5.9 GB of 7.0 GB, ~194 tok/s at 128K.

Also: Arc A750, GeForce RTX 5060, GeForce RTX 3070, GeForce RTX 3060 Ti, GeForce RTX 4060 Ti +2 more

128K tokens (or the model’s maximum) at the recommended quantisation

Apple Silicon 16 GB unified

M1 · 16 GB — Q4_K_M, 5.1 GB of 10.7 GB, ~24 tok/s, up to 128K context.

smallest Mac that runs the recommended quantisation with headroom

CPU only 32 GB RAM

CPU only · DDR4 dual-channel — Q8_0, 8.5 GB of 25.6 GB, ~7.4 tok/s, up to 128K context.

no GPU — weights stream from system RAM

How much VRAM at each quantisation

Quant	Weights	+KV 8K	+KV 32K	+KV 128K	Total @ 8K	Needs	Quality
F16	14.7 GB	0.05 GB	0.21 GB	0.8 GB	15.4 GB	20 GB card	Reference
Q8_0	7.8 GB	0.05 GB	0.21 GB	0.8 GB	8.5 GB	12 GB card	−0.1% ppl
Q6_K	6.0 GB	0.05 GB	0.21 GB	0.8 GB	6.7 GB	10 GB card	−0.4% ppl
Q5_K_M	5.2 GB	0.05 GB	0.21 GB	0.8 GB	5.9 GB	8 GB card	−0.8% ppl
Q4_K_M	4.4 GB	0.05 GB	0.21 GB	0.8 GB	5.1 GB	8 GB card	−1.9% ppl
Q3_K_M	3.6 GB	0.05 GB	0.21 GB	0.8 GB	4.2 GB	8 GB card	−5.4% ppl
Q2_K	3.1 GB	0.05 GB	0.21 GB	0.8 GB	3.7 GB	8 GB card	−15% ppl

Totals add 0.6 GB runtime overhead and an f16 KV cache; "needs" is the smallest discrete card class whose usable memory (after display and driver reserve) leaves 15% headroom. A q8_0 cache roughly halves the KV columns.

Every device, checked

System RAM for offload:

16 32 64 128 256

NVIDIA · GeForce

Device	Memory	Best quant with headroom	Fits at all	Max context @ Q4_K_M	Tok/s est.	128K
GeForce RTX 5090	32 GB	Q8_0	Q8_0	128K	571	yes	Check
GeForce RTX 5080	16 GB	Q8_0	Q8_0	128K	306	yes	Check
GeForce RTX 5070 Ti	16 GB	Q8_0	Q8_0	128K	285	yes	Check
GeForce RTX 5070	12 GB	Q8_0	Q8_0	128K	214	yes	Check
GeForce RTX 5060 Ti 16 GB	16 GB	Q8_0	Q8_0	128K	143	yes	Check
GeForce RTX 5060	8 GB	Q5_K_M	Q6_K	128K	143	yes	Check
GeForce RTX 4090	24 GB	Q8_0	Q8_0	128K	321	yes	Check
GeForce RTX 4080 Super	16 GB	Q8_0	Q8_0	128K	234	yes	Check
GeForce RTX 4080	16 GB	Q8_0	Q8_0	128K	228	yes	Check
GeForce RTX 4070 Ti Super	16 GB	Q8_0	Q8_0	128K	214	yes	Check
GeForce RTX 4070 Ti	12 GB	Q8_0	Q8_0	128K	161	yes	Check
GeForce RTX 4070 Super	12 GB	Q8_0	Q8_0	128K	161	yes	Check
GeForce RTX 4070	12 GB	Q8_0	Q8_0	128K	161	yes	Check
GeForce RTX 4060 Ti 16 GB	16 GB	Q8_0	Q8_0	128K	92	yes	Check
GeForce RTX 4060 Ti	8 GB	Q5_K_M	Q6_K	128K	92	yes	Check
GeForce RTX 4060	8 GB	Q5_K_M	Q6_K	128K	87	yes	Check
GeForce RTX 3090 Ti	24 GB	Q8_0	Q8_0	128K	321	yes	Check
GeForce RTX 3090	24 GB	Q8_0	Q8_0	128K	298	yes	Check
GeForce RTX 3080 Ti	12 GB	Q8_0	Q8_0	128K	290	yes	Check
GeForce RTX 3080 12 GB	12 GB	Q8_0	Q8_0	128K	290	yes	Check
GeForce RTX 3080	10 GB	Q6_K	Q8_0	128K	242	yes	Check
GeForce RTX 3070 Ti	8 GB	Q5_K_M	Q6_K	128K	194	yes	Check
GeForce RTX 3070	8 GB	Q5_K_M	Q6_K	128K	143	yes	Check
GeForce RTX 3060 Ti	8 GB	Q5_K_M	Q6_K	128K	143	yes	Check
GeForce RTX 3060 12 GB	12 GB	Q8_0	Q8_0	128K	115	yes	Check
GeForce RTX 2080 Ti	11 GB	Q6_K	Q8_0	128K	196	yes	Check
GeForce RTX 2060 12 GB	12 GB	Q8_0	Q8_0	128K	107	yes	Check
GeForce GTX 1080 Ti	11 GB	Q6_K	Q8_0	128K	154	yes	Check

NVIDIA · GeForce laptop

Device	Memory	Best quant with headroom	Fits at all	Max context @ Q4_K_M	Tok/s est.	128K
GeForce RTX 5090 Laptop	24 GB	Q8_0	Q8_0	128K	285	yes	Check
GeForce RTX 4090 Laptop	16 GB	Q8_0	Q8_0	128K	183	yes	Check
GeForce RTX 4080 Laptop	12 GB	Q8_0	Q8_0	128K	138	yes	Check
GeForce RTX 4070 Laptop	8 GB	Q5_K_M	Q6_K	128K	82	yes	Check

NVIDIA · Workstation

Device	Memory	Best quant with headroom	Fits at all	Max context @ Q4_K_M	Tok/s est.	128K
RTX 6000 Ada	48 GB	Q8_0	Q8_0	128K	306	yes	Check
RTX 5000 Ada	32 GB	Q8_0	Q8_0	128K	183	yes	Check
RTX 4000 Ada	20 GB	Q8_0	Q8_0	128K	115	yes	Check
RTX 2000 Ada	16 GB	Q8_0	Q8_0	128K	71	yes	Check
RTX A6000	48 GB	Q8_0	Q8_0	128K	245	yes	Check
RTX A5000	24 GB	Q8_0	Q8_0	128K	245	yes	Check
RTX A4000	16 GB	Q8_0	Q8_0	128K	143	yes	Check

NVIDIA · Data centre

Device	Memory	Best quant with headroom	Fits at all	Max context @ Q4_K_M	Tok/s est.	128K
L40S	48 GB	Q8_0	Q8_0	128K	275	yes	Check
L4	24 GB	Q8_0	Q8_0	128K	96	yes	Check
A10	24 GB	Q8_0	Q8_0	128K	191	yes	Check
A100 40 GB	40 GB	Q8_0	Q8_0	128K	495	yes	Check
A100 80 GB	80 GB	Q8_0	Q8_0	128K	649	yes	Check
H100 PCIe	80 GB	Q8_0	Q8_0	128K	637	yes	Check
H100 SXM	80 GB	Q8_0	Q8_0	128K	1067	yes	Check
H200 SXM	141 GB	Q8_0	Q8_0	128K	1529	yes	Check
Tesla P40	24 GB	Q8_0	Q8_0	128K	110	yes	Check

AMD · Radeon

Device	Memory	Best quant with headroom	Fits at all	Max context @ Q4_K_M	Tok/s est.	128K
Radeon RX 9070 XT	16 GB	Q8_0	Q8_0	128K	205	yes	Check
Radeon RX 9070	16 GB	Q8_0	Q8_0	128K	205	yes	Check
Radeon RX 7900 XTX	24 GB	Q8_0	Q8_0	128K	306	yes	Check
Radeon RX 7900 XT	20 GB	Q8_0	Q8_0	128K	255	yes	Check
Radeon RX 7900 GRE	16 GB	Q8_0	Q8_0	128K	183	yes	Check
Radeon RX 7800 XT	16 GB	Q8_0	Q8_0	128K	199	yes	Check
Radeon RX 7700 XT	12 GB	Q8_0	Q8_0	128K	138	yes	Check
Radeon RX 7600 XT	16 GB	Q8_0	Q8_0	128K	92	yes	Check
Radeon RX 6900 XT	16 GB	Q8_0	Q8_0	128K	163	yes	Check
Radeon RX 6800 XT	16 GB	Q8_0	Q8_0	128K	163	yes	Check
Radeon RX 6700 XT	12 GB	Q8_0	Q8_0	128K	122	yes	Check

AMD · Radeon Pro

Device	Memory	Best quant with headroom	Fits at all	Max context @ Q4_K_M	Tok/s est.	128K
Radeon Pro W7900	48 GB	Q8_0	Q8_0	128K	275	yes	Check
Radeon Pro W7800	32 GB	Q8_0	Q8_0	128K	183	yes	Check

AMD · Instinct

Device	Memory	Best quant with headroom	Fits at all	Max context @ Q4_K_M	Tok/s est.	128K
Instinct MI210	64 GB	Q8_0	Q8_0	128K	522	yes	Check
Instinct MI300X	192 GB	Q8_0	Q8_0	128K	1688	yes	Check

AMD · Ryzen AI Max

Device	Memory	Best quant with headroom	Fits at all	Max context @ Q4_K_M	Tok/s est.	128K
Ryzen AI Max+ 395 · 128 GB	128 GB	Q8_0	Q8_0	128K	91	yes	Check
Ryzen AI Max+ 395 · 64 GB	64 GB	Q8_0	Q8_0	128K	91	yes	Check

Apple · Apple Silicon

Device	Memory	Best quant with headroom	Fits at all	Max context @ Q4_K_M	Tok/s est.	128K
M1 · 16 GB	16 GB	Q8_0	Q8_0	128K	24	yes	Check
M1 Pro · 32 GB	32 GB	Q8_0	Q8_0	128K	71	yes	Check
M1 Max · 64 GB	64 GB	Q8_0	Q8_0	128K	143	yes	Check
M1 Ultra · 128 GB	128 GB	Q8_0	Q8_0	128K	285	yes	Check
M2 · 24 GB	24 GB	Q8_0	Q8_0	128K	36	yes	Check
M2 Pro · 32 GB	32 GB	Q8_0	Q8_0	128K	71	yes	Check
M2 Max · 96 GB	96 GB	Q8_0	Q8_0	128K	143	yes	Check
M2 Ultra · 192 GB	192 GB	Q8_0	Q8_0	128K	285	yes	Check
M3 · 24 GB	24 GB	Q8_0	Q8_0	128K	36	yes	Check
M3 Pro · 36 GB	36 GB	Q8_0	Q8_0	128K	54	yes	Check
M3 Max · 64 GB	64 GB	Q8_0	Q8_0	128K	143	yes	Check
M3 Max · 128 GB	128 GB	Q8_0	Q8_0	128K	143	yes	Check
M3 Ultra · 256 GB	256 GB	Q8_0	Q8_0	128K	285	yes	Check
M3 Ultra · 512 GB	512 GB	Q8_0	Q8_0	128K	285	yes	Check
M4 · 32 GB	32 GB	Q8_0	Q8_0	128K	43	yes	Check
M4 Pro · 48 GB	48 GB	Q8_0	Q8_0	128K	97	yes	Check
M4 Pro · 64 GB	64 GB	Q8_0	Q8_0	128K	97	yes	Check
M4 Max · 64 GB	64 GB	Q8_0	Q8_0	128K	146	yes	Check
M4 Max · 128 GB	128 GB	Q8_0	Q8_0	128K	195	yes	Check

Intel · Arc

Device	Memory	Best quant with headroom	Fits at all	Max context @ Q4_K_M	Tok/s est.	128K
Arc B580	12 GB	Q8_0	Q8_0	128K	145	yes	Check
Arc B570	10 GB	Q6_K	Q8_0	128K	121	yes	Check
Arc A770 16 GB	16 GB	Q8_0	Q8_0	128K	178	yes	Check
Arc A750	8 GB	Q5_K_M	Q6_K	128K	163	yes	Check

CPU · CPU + system RAM

Device	Memory	Best quant with headroom	Fits at all	Max context @ Q4_K_M	Tok/s est.	128K
CPU only · DDR4 dual-channel	32 GB	Q8_0	Q8_0	128K	13	yes	Check
CPU only · DDR5 dual-channel	64 GB	Q8_0	Q8_0	128K	23	yes	Check
CPU only · DDR5 8-channel server	256 GB	Q8_0	Q8_0	128K	78	yes	Check