No — not on this device
gpt-oss 120B at MXFP4 needs 61.4 GB against 48.0 GB usable, and the shortfall of 13.4 GB is more than 256 GB of system RAM can cover at a tolerable speed. A smaller sibling or a lower quantisation is the honest answer here.
Designed to land on one 80 GB card. Only ~5B parameters are active per token.
What hardware do I need for gpt-oss 120B? →
Fits instead: gpt-oss 20B (11.6 GB)
The VRAM budget
Quantisation ladder
| Quant | Weights | Total @ 8K | Max context | Tok/s | Quality | Fit |
|---|---|---|---|---|---|---|
| MXFP4 | 60.5 GB | 61.4 GB | — | ~27 | Reference | 13.4 GB over |
Quality is the published perplexity delta against f16 weights. Max context assumes an f16 KV cache; q8_0 roughly doubles it. This model interleaves sliding-window layers (128 tokens, 1 global in 2), which is why its cache barely grows with context.
How to run it
$ pip install mlx-lm $ mlx_lm.generate --model mlx-community/gpt-oss-120b-4bit \ --max-tokens 512 --prompt "Hello"
Apple's own array framework. The fastest path on Apple Silicon. More on MLX.