Runtimes
Five ways to actually run the weights
They mostly wrap the same two engines. What differs is how much they decide for you — and on a machine that is short on memory, deciding for yourself is the difference between 30 tokens a second and 3.
| Runtime | Format | Discrete GPU | Apple Silicon | CPU only | Pick it when |
|---|---|---|---|---|---|
| Ollama | GGUF | yes | yes | yes | You want it working in five minutes |
| llama.cpp | GGUF | yes | yes | yes | You need a flag the others hide |
| LM Studio | GGUF / MLX | yes | yes | yes | You would rather click than type |
| MLX | MLX safetensors | — | yes | — | You are on a Mac and want the speed |
| vLLM | safetensors / AWQ / GPTQ | yes | — | — | You are serving more than one person |
The default. One binary, a model registry, an OpenAI-compatible port.
Good at
Easiest to install, keeps models loaded, works the same on all three platforms.
Costs you
Its own registry lags Hugging Face, and per-model settings are hidden behind Modelfiles.
Runs on
Discrete GPUs, Unified memory, CPU only
The engine underneath most of the others. Every knob is exposed.
Good at
Fastest to get new quantisation formats, full control over offload and cache type.
Costs you
You manage the build, the flags and the files yourself.
Runs on
Discrete GPUs, Unified memory, CPU only
A desktop app over llama.cpp and MLX, with a CLI if you want one.
Good at
Shows you the fit estimate before downloading, and swaps engines per model.
Costs you
Closed source, and the GUI hides which engine actually ran.
Runs on
Discrete GPUs, Unified memory, CPU only
Apple's own array framework. The fastest path on Apple Silicon.
Good at
Noticeably faster than GGUF on the same Mac, and unified memory means no offload split.
Costs you
Apple Silicon only, and the community conversions lag the largest releases.
Runs on
Unified memory
A serving engine, not a chat app. Built for many requests at once.
Good at
Paged attention and continuous batching — throughput scales with concurrency.
Costs you
Wants unquantised or AWQ/GPTQ weights and a lot of VRAM. Overkill for one user.
Runs on
Discrete GPUs
Every model page renders the exact command for whichever of these can drive your device, with the context flag set to a size we have already checked will fit.