Best LLM for 24 GB VRAM · 2026

Best LLM for 24 GB VRAM: what fits, and what to run

A 24 GB card hands a runtime about 22.4 GB once the display and driver have taken their share. At Q4_K_M with an 8K context, 58 of 80 models in the catalogue fit — 50 with headroom, 8 tightly. The pick for most people is Qwen3.8 27B: Best capability per gigabyte: a current-generation dense 27B with vision and 262K context. Speeds are estimated for a GeForce RTX 4090; the fit verdicts are the same for every card in the class.

Which one should I run?

One pick per job, from the editorial shortlist, checked to fit at Q4_K_M and 8K. How picks are chosen.

The long-context row is judged at 128K: Nemotron 3.5 Lightning 30B-A3B holds that window on 24 GB at Q4_K_M with 0.75 GB of KV cache — that is why it can differ from the best overall pick.

Everything that fits 24 GB

Largest first. "Max context" is the longest window the card holds at that quantisation with an f16 cache; a q8_0 cache roughly doubles it.

Just out of reach

These run with part of the weights in system RAM (32 GB assumed), at a few tokens per second. Each needs a bigger card to run properly — the link says which.

Every total is quantised weights + f16 KV cache at 8K + 0.6 GB runtime overhead, against 22.4 GB usable. Tokens per second are estimated from memory bandwidth and never measured. The method.