Yes — with 88.4 GB to spare

Qwen3.8-Flash-Next 180B-A6B at Q4_K_M fits your Instinct MI300X entirely on the GPU at 8K context, at an estimated 366 tokens per second. There is room for its full 256K window.

Fully on GPU 8K context Q4_K_M · 101.2 GB Qwen Community 1.0 Released 24 Aug 2026 Vision

The open preview of the Qwen4 architecture: a 125B-A6B hybrid (Gated DeltaNet + sparse attention, KV cache on 12 of 48 blocks) plus a 51B n-gram embedding and a 4B draft head — 180B on disk, 6B active. The hosted "Qwen3.8-Flash" is this model with a 1M window.

Hardware requirements What hardware do I need to run Qwen3.8-Flash-Next 180B-A6B? →

At Q4_K_M with an 8K context: a 141 GB GPU (H200 SXM) or a 192 GB Mac (M2 Ultra). Minimum, recommended, high-quality and long-context tiers, with all 114 devices checked at every quantisation.

The VRAM budget

weights 101.2 GB
Weights 101.2 GB KV cache @ 8K 0.19 GB Runtime overhead 0.6 GB Free 88.4 GB of 190.4 GB

Quantisation ladder

QuantWeightsTotal @ 8KMax contextTok/sQualityFit
Q8_0 178.1 GB 178.9 GB 256K 208 −0.1% ppl Long context
Q6_K 137.5 GB 138.3 GB 256K 269 −0.4% ppl Long context
Q5_K_M 118.8 GB 119.6 GB 256K 312 −0.8% ppl Long context
Q4_K_M 101.2 GB 102.0 GB 256K 366 −1.9% ppl Recommended
Q3_K_M 81.9 GB 82.7 GB 256K 452 −5.4% ppl Long context
Q2_K 70.2 GB 71.0 GB 256K 527 −15% ppl Long context

Quality is the published perplexity delta against f16 weights. Max context assumes an f16 KV cache; q8_0 roughly doubles it. Only 12 of its 48 blocks keep a per-token KV cache; the rest are linear-attention, Mamba or convolution blocks with a fixed-size state.

How to run it

terminal
$ ollama pull qwen3.8-flash-next:125b-a6b-nvfp4
$ OLLAMA_CONTEXT_LENGTH=8192 \
    ollama run qwen3.8-flash-next:125b-a6b-nvfp4

The default. One binary, a model registry, an OpenAI-compatible port. More on Ollama.

01Download is 101.2 GB. Keep it on an SSD — a first load off a spinning disk takes minutes.
02Close anything else holding VRAM. A browser with hardware acceleration can sit on 1–2 GB.
03There is room to go to the model's full 256K context on this card.
See all models for this rig Compare with another model

Related models on an Instinct MI300X

ModelTotalTok/s est.
Qwen3.8 27B
27.8B · same family
Q4_K_M · 16.7 GB 205 Runs great
Qwen3.8 2.4T-A95B
2446B MoE · same family
Q4_K_M · 1376.7 GB — Won't fit
Qwen3 235B-A22B
235B MoE · similar size
Q4_K_M · 134.2 GB 100 Runs great
Qwen3.5 122B-A10B
125B MoE · similar size
Q4_K_M · 71.1 GB 219 Runs great
Nemotron 3 Super 120B-A12B
124B MoE · similar size
Q4_K_M · 70.4 GB 183 Runs great
Ling 3.0 Flash 124B-A5B
124B MoE · similar size
Q4_K_M · 70.4 GB 430 Runs great
Mistral Small 4 119B-A6B
119B MoE · similar size
Q4_K_M · 67.7 GB 338 Runs great

Every model against the Instinct MI300X →

Qwen3.8-Flash-Next 180B-A6B on other hardware

DeviceTotalTok/s est.
GeForce RTX 4060 Laptop
7.0 GB usable · 256 GB/s
102.0 GB — Won't fit
GeForce RTX 3060 12 GB
10.6 GB usable · 360 GB/s
102.0 GB — Won't fit
M4 · 16 GB
10.7 GB usable · 120 GB/s
102.0 GB — Won't fit
GeForce RTX 4090
22.4 GB usable · 1008 GB/s
102.0 GB — Won't fit
Radeon RX 7900 XTX
22.4 GB usable · 960 GB/s
102.0 GB — Won't fit
M3 Max · 64 GB
48.0 GB usable · 400 GB/s
102.0 GB — Won't fit
H200 SXM
139.4 GB usable · 4800 GB/s
102.0 GB 331 Runs great
M2 Ultra · 192 GB
144.0 GB usable · 800 GB/s
102.0 GB 62 Runs great

All 114 devices, by tier →