Models
80 open-weight models, sized
Updated 8 Oct 2026 1 new this month
Parameter counts, layer counts and attention shapes are read from each model's own config, because the KV-cache arithmetic depends on them exactly. Sizes below are at Q4_K_M and 8K context against a GeForce RTX 4090. Missing a model that shipped this week? Open an issue — the table is hand-maintained.
| Model | Released | Params | Quant | Weights | Max context | Licence | On this rig | |
|---|---|---|---|---|---|---|---|---|
| Llama 3.2 3B Instruct The 2024 "it just runs" model for 8 GB laptops. Qwen3.5 4B does the same job better now. |
Sep 2024 | 3.21B | Q4_K_M | 1.8 GB | 128K | Llama 3.2 Community | 338 tok/s | Check · Hardware |
| Phi-4-mini 3.8B MIT-licensed, dense, and unusually strong on instruction following for its size. |
Feb 2025 | 3.84B | Q4_K_M | 2.2 GB | 128K | MIT | 283 tok/s | Check · Hardware |
| Qwen3 4B The 2025 sweet spot for 8 GB cards with reasoning traces. Qwen3.5 4B adds vision and 8× the context. |
Apr 2025 | 4.02B | Q4_K_M | 2.3 GB | 32K | Apache 2.0 | 270 tok/s | Check · Hardware |
| Gemma 3 4B Vision-capable at 4B. Superseded by Gemma 4 E4B, still everywhere. |
Mar 2025 | 4.3B | Q4_K_M | 2.4 GB | 128K | Gemma Terms of Use | 252 tok/s | Check · Hardware |
| Qwen3.5 4B The 8 GB coding agent. Q4 lands near 3.4 GB, leaving room for a real context window. |
28 Feb 2026 | 4.66B | Q4_K_M | 2.6 GB | 256K | Apache 2.0 | 233 tok/s | Check · Hardware |
| Mistral 7B Instruct v0.3 Old but extremely well behaved, and permissively licensed for commercial use. |
May 2024 | 7.25B | Q4_K_M | 4.1 GB | 32K | Apache 2.0 | 150 tok/s | Check · Hardware |
| Olmo 3 7B Instruct Fully open — training data and code included. Full multi-head attention, so its cache is 4× a GQA 7B at the same context. |
20 Nov 2025 | 7.3B | Q4_K_M | 4.1 GB | 64K | Apache 2.0 | 149 tok/s | Check · Hardware |
| Gemma 4 E4B The laptop Gemma. 4.5B effective, 8B on disk; a single KV head per window layer keeps its cache tiny. |
2 Apr 2026 | 8.0B | Q4_K_M | 4.5 GB | 128K | Apache 2.0 | 136 tok/s | Check · Hardware |
| Llama 3.1 8B Instruct Still the most widely deployed local model, with the largest fine-tune ecosystem. Not the strongest 8B any more. |
Jul 2024 | 8.03B | Q4_K_M | 4.5 GB | 128K | Llama 3.1 Community | 135 tok/s | Check · Hardware |
| Qwen3 8B Apache-2.0 alternative to Llama 3.1 8B, with a switchable thinking mode. |
Apr 2025 | 8.19B | Q4_K_M | 4.6 GB | 128K | Apache 2.0 | 133 tok/s | Check · Hardware |
| LFM2.5 8B-A1B An 8B MoE with ~1.5B active, aimed at laptops without a GPU. Licence is permissive below $10M revenue. |
28 May 2026 | 8.47B MoE | Q4_K_M | 4.8 GB | 125K | LFM Open License v1.0 | 278 tok/s | Check · Hardware |
| Granite 4.1 8B Matches the old Granite 4.0 32B MoE at a quarter of the size. Fast, Apache 2.0, no reasoning traces. |
29 Apr 2026 | 8.79B | Q4_K_M | 4.9 GB | 128K | Apache 2.0 | 123 tok/s | Check · Hardware |
| Ministral 3 8B Mistral's 8B with images in. Plain GQA, so budget more KV cache than Qwen3.5 9B at the same context. |
Dec 2025 | 8.92B | Q4_K_M | 5.0 GB | 256K | Apache 2.0 | 122 tok/s | Check · Hardware |
| Qwen3.5 9B The default for 8–12 GB cards in 2026: beats every older 8B on every published benchmark, with vision. |
28 Feb 2026 | 9.65B | Q4_K_M | 5.4 GB | 256K | Apache 2.0 | 112 tok/s | Check · Hardware |
| Gemma 4 12B The "unified" Gemma 4: text, image and audio in one 12B that fits a 12 GB card at Q4. 140+ languages. |
29 May 2026 | 12B | Q4_K_M | 6.7 GB | 256K | Apache 2.0 | 90 tok/s | Check · Hardware |
| Gemma 3 12B Strong multilingual chat with images, sized for 12–16 GB cards. Gemma 4 12B is the same size and better. |
Mar 2025 | 12.2B | Q4_K_M | 6.9 GB | 128K | Gemma Terms of Use | 89 tok/s | Check · Hardware |
| Mistral NeMo 12B Multilingual 12B with a 128K window, built with NVIDIA. A roleplay and fiction favourite that refuses to die. |
Jul 2024 | 12.2B | Q4_K_M | 6.9 GB | 128K | Apache 2.0 | 89 tok/s | Check · Hardware |
| Ministral 3 14B The largest Ministral. A 12 GB card runs it at Q4 with a few gigabytes to spare. |
Dec 2025 | 13.9B | Q4_K_M | 7.8 GB | 256K | Apache 2.0 | 78 tok/s | Check · Hardware |
| Qwen3 14B The largest Qwen3 that fits a 12 GB card at Q4 with room for context. |
Apr 2025 | 14.8B | Q4_K_M | 8.3 GB | 128K | Apache 2.0 | 73 tok/s | Check · Hardware |
| gpt-oss 20B Ships natively in MXFP4, so the 4-bit weights are the reference weights, not a lossy copy. Fits 16 GB. |
Aug 2025 | 20.9B MoE | MXFP4 | 10.8 GB | 128K | Apache 2.0 | 126 tok/s | Check · Hardware |
| Mistral Small 3.2 24B Apache-2.0, vision-capable, and the most 24 GB-friendly of the 2025 generalists. |
Jun 2025 | 23.6B | Q4_K_M | 13.3 GB | 128K | Apache 2.0 | 46 tok/s | Check · Hardware |
| Gemma 4 26B-A4B Mixture of experts with 3.8B active. Slower to think than Qwen3.6 35B-A3B, faster to answer, and it sees images. |
2 Apr 2026 | 26.5B MoE | Q4_K_M | 14.9 GB | 256K | Apache 2.0 | 110 tok/s | Check · Hardware |
| Gemma 3 27B The 2025 single-GPU generalist with vision. Its Gemma-licence terms are the reason to prefer Gemma 4 now. |
Mar 2025 | 27.4B | Q4_K_M | 15.4 GB | 128K | Gemma Terms of Use | 40 tok/s | Check · Hardware |
| Qwen3.8 27B The current default local Qwen: dense 27B, text + image + video, 262K context. Only 16 of its 64 blocks keep a KV cache, so long context is cheap. |
14 Aug 2026 | 27.8B | Q4_K_M | 15.6 GB | 256K | Apache 2.0 | 39 tok/s | Check · Hardware |
| Qwen3.6 27B The 24 GB coding pick of spring 2026 (77.2 SWE-bench Verified). Same shape as 3.8, one generation behind. |
22 Apr 2026 | 27.8B | Q4_K_M | 15.6 GB | 256K | Apache 2.0 | 39 tok/s | Check · Hardware |
| Granite 4.1 30B The largest Granite. Dense 29B at Q4 is a comfortable 24 GB fit. |
29 Apr 2026 | 28.9B | Q4_K_M | 16.3 GB | 128K | Apache 2.0 | 38 tok/s | Check · Hardware |
| Muse Glimmer 30B Meta's first open weights since Llama 4: a dense 30B distilled from Muse Spark for always-on local agents. Two KV heads keep the cache small. |
10 Aug 2026 | 29.8B | Q4_K_M | 16.8 GB | 128K | Apache 2.0 | 36 tok/s | Check · Hardware |
| Qwen3 30B-A3B The MoE that made "3B active" a category. Qwen3.6 35B-A3B is its direct replacement. |
Apr 2025 | 30.5B MoE | Q4_K_M | 17.1 GB | 128K | Apache 2.0 | 126 tok/s | Check · Hardware |
| GLM-4.7-Flash 30B-A3B MIT-licensed 30B-A3B tuned for agentic coding, with a DeepSeek-style latent KV cache. 60–80 tok/s reported on a 4090. |
20 Jan 2026 | 31.2B MoE | Q4_K_M | 17.5 GB | 198K | MIT | 139 tok/s | Check · Hardware |
| Gemma 4 31B The dense flagship: strongest maths of the 24–32 GB class (89% AIME), clean prose, vision. Q4 is a tight 24 GB fit. |
2 Apr 2026 | 31.3B | Q4_K_M | 17.6 GB | 256K | Apache 2.0 | 35 tok/s | Check · Hardware |
| Nemotron 3.5 Lightning 30B-A3B Mamba-2 + MoE hybrid built for the execution layer of agents: only 6 attention blocks, so the KV cache is almost free. Weights, data and recipe all open. |
11 Aug 2026 | 31.6B MoE | Q4_K_M | 17.8 GB | 256K | OpenMDW-1.1 | 130 tok/s | Check · Hardware |
| Olmo 3.1 32B Instruct The largest fully open model you can audit end to end. Q4 fits 24 GB, tightly. |
10 Dec 2025 | 32.2B | Q4_K_M | 18.1 GB | 64K | Apache 2.0 | 34 tok/s | Check · Hardware |
| Qwen3 32B The classic 24 GB target, and still the strongest local translator under 70B. Qwen3.8 27B is smaller and better at everything else. |
Apr 2025 | 32.8B | Q4_K_M | 18.4 GB | 128K | Apache 2.0 | 33 tok/s | Check · Hardware |
| LLM-jp 4 33B Thinking Japan’s national-institute reasoning model, Japanese and English. A plain dense Llama-style 33B: Q4 is a tight 24 GB fit. |
14 Aug 2026 | 33.2B | Q4_K_M | 18.7 GB | 64K | Apache 2.0 | 33 tok/s | Check · Hardware |
| Qwen3.6 35B-A3B Mixture of experts with ~3B active: the fastest serious model a 24 GB card runs, and the best MoE under 40B on agentic coding. |
16 Apr 2026 | 35.9B MoE | Q4_K_M | 20.2 GB | 256K | Apache 2.0 | 126 tok/s | Check · Hardware |
| Llama 3.3 70B Instruct Still the creative-writing favourite: consistent voice, takes direction. Needs 48 GB to sit comfortably on GPU at Q4. |
Dec 2024 | 70.6B | Q4_K_M | 39.7 GB | 128K | Llama 3.3 Community | offload | Check · Hardware |
| Llama 4 Scout 109B-A17B A 10M-token window on paper, chunked attention in practice (8K chunks on 3 of 4 layers). Needs 64 GB+ at Q4. |
5 Apr 2025 | 109B MoE | Q4_K_M | 61.3 GB | 1024K | Llama 4 Community | won't fit | Check · Hardware |
| Mistral Small 4 119B-A6B Instruct, reasoning, vision and code in one 119B MoE. A latent KV cache (320 wide) keeps context cheap; the weights still want 64 GB+. |
17 Mar 2026 | 119B MoE | Q4_K_M | 66.9 GB | 256K | Apache 2.0 | won't fit | Check · Hardware |
| Nemotron 3 Super 120B-A12B The open-training-data 120B. Same hybrid layout as Lightning, so 128K context costs under a gigabyte. |
Mar 2026 | 124B MoE | Q4_K_M | 69.7 GB | 256K | NVIDIA Open Model | won't fit | Check · Hardware |
| Qwen3.5 122B-A10B The 96–128 GB unified-memory model: 122B of knowledge at 10B-active speed. |
24 Feb 2026 | 125B MoE | Q4_K_M | 70.3 GB | 256K | Apache 2.0 | won't fit | Check · Hardware |
| Qwen3.8-Flash-Next 180B-A6B The open preview of the Qwen4 architecture: a 125B-A6B hybrid (Gated DeltaNet + sparse attention, KV cache on 12 of 48 blocks) plus a 51B n-gram embedding and a 4B draft head — 180B on disk, 6B active. The hosted "Qwen3.8-Flash" is this model with a 1M window. |
24 Aug 2026 | 180B MoE | Q4_K_M | 101.2 GB | 256K | Qwen Community 1.0 | won't fit | Check · Hardware |
| Qwen3 235B-A22B Workstation class. Realistically a 192 GB unified-memory or multi-GPU model. |
Apr 2025 | 235B MoE | Q4_K_M | 132.1 GB | 128K | Apache 2.0 | won't fit | Check · Hardware |
| GLM-5.3-Flash 320B-A18B The first natively multimodal GLM-5 and the first hybrid: 34 linear-attention blocks and 11 sparse-attention blocks with a 512-wide latent cache, so a 1M window stays affordable. Z.ai says it beats GLM-5.2 at 18B active; MIT. |
25 Aug 2026 | 321B MoE | Q4_K_M | 180.5 GB | 1024K | MIT | won't fit | Check · Hardware |
| DeepSeek V4.1 Flash 552B-A16B New Twice V4 Flash's backbone (552B, 16B active per generated token) plus 196B of Engram lookup tables (189 GiB at FP8) that DwarfStar streams from SSD, so they are not counted here. A 512 GB Mac model, and as of October 2026 no mainline llama.cpp or Ollama build: DwarfStar on a Mac, vLLM across four GPUs. DeepSeek puts the cache at 890 bytes/token; modelled conservatively as 4 latent layers plus a 128-token window. |
10 Sep 2026 | 552B MoE | Q4_K_M | 310.4 GB | 1024K | MIT | won't fit | Check · Hardware |
Weight sizes are computed from the parameter count and the quantisation's effective bits per weight, not read off a file listing — expect them to land within a few percent of the GGUF you actually download. The method, in full.