Models
76 open-weight models, sized
Updated 21 Aug 2026 5 new this week, 14 this month
Parameter counts, layer counts and attention shapes are read from each model's own config, because the KV-cache arithmetic depends on them exactly. Sizes below are at Q4_K_M and 8K context against a GeForce RTX 3060 12 GB. Missing a model that shipped this week? Open an issue — the table is hand-maintained.
| Model | Released | Params | Quant | Weights | Max context | Licence | On this rig | |
|---|---|---|---|---|---|---|---|---|
| Llama 3.2 3B Instruct The 2024 "it just runs" model for 8 GB laptops. Qwen3.5 4B does the same job better now. |
Sep 2024 | 3.21B | Q4_K_M | 1.8 GB | 128K | Llama 3.2 Community | 121 tok/s | Check · Hardware |
| Phi-4-mini 3.8B MIT-licensed, dense, and unusually strong on instruction following for its size. |
Feb 2025 | 3.84B | Q4_K_M | 2.2 GB | 128K | MIT | 101 tok/s | Check · Hardware |
| Qwen3 4B The 2025 sweet spot for 8 GB cards with reasoning traces. Qwen3.5 4B adds vision and 8× the context. |
Apr 2025 | 4.02B | Q4_K_M | 2.3 GB | 32K | Apache 2.0 | 96 tok/s | Check · Hardware |
| Gemma 3 4B Vision-capable at 4B. Superseded by Gemma 4 E4B, still everywhere. |
Mar 2025 | 4.3B | Q4_K_M | 2.4 GB | 128K | Gemma Terms of Use | 90 tok/s | Check · Hardware |
| Qwen3.5 4B The 8 GB coding agent. Q4 lands near 3.4 GB, leaving room for a real context window. |
28 Feb 2026 | 4.66B | Q4_K_M | 2.6 GB | 256K | Apache 2.0 | 83 tok/s | Check · Hardware |
| Mistral 7B Instruct v0.3 Old but extremely well behaved, and permissively licensed for commercial use. |
May 2024 | 7.25B | Q4_K_M | 4.1 GB | 32K | Apache 2.0 | 53 tok/s | Check · Hardware |
| Olmo 3 7B Instruct Fully open — training data and code included. Full multi-head attention, so its cache is 4× a GQA 7B at the same context. |
20 Nov 2025 | 7.3B | Q4_K_M | 4.1 GB | 64K | Apache 2.0 | 53 tok/s | Check · Hardware |
| Gemma 4 E4B The laptop Gemma. 4.5B effective, 8B on disk; a single KV head per window layer keeps its cache tiny. |
2 Apr 2026 | 8.0B | Q4_K_M | 4.5 GB | 128K | Apache 2.0 | 48 tok/s | Check · Hardware |
| Llama 3.1 8B Instruct Still the most widely deployed local model, with the largest fine-tune ecosystem. Not the strongest 8B any more. |
Jul 2024 | 8.03B | Q4_K_M | 4.5 GB | 128K | Llama 3.1 Community | 48 tok/s | Check · Hardware |
| Qwen3 8B Apache-2.0 alternative to Llama 3.1 8B, with a switchable thinking mode. |
Apr 2025 | 8.19B | Q4_K_M | 4.6 GB | 128K | Apache 2.0 | 47 tok/s | Check · Hardware |
| LFM2.5 8B-A1B An 8B MoE with ~1.5B active, aimed at laptops without a GPU. Licence is permissive below $10M revenue. |
28 May 2026 | 8.47B MoE | Q4_K_M | 4.8 GB | 125K | LFM Open License v1.0 | 99 tok/s | Check · Hardware |
| Granite 4.1 8B Matches the old Granite 4.0 32B MoE at a quarter of the size. Fast, Apache 2.0, no reasoning traces. |
29 Apr 2026 | 8.79B | Q4_K_M | 4.9 GB | 128K | Apache 2.0 | 44 tok/s | Check · Hardware |
| Ministral 3 8B Mistral's 8B with images in. Plain GQA, so budget more KV cache than Qwen3.5 9B at the same context. |
Dec 2025 | 8.92B | Q4_K_M | 5.0 GB | 256K | Apache 2.0 | 43 tok/s | Check · Hardware |
| Qwen3.5 9B The default for 8–12 GB cards in 2026: beats every older 8B on every published benchmark, with vision. |
28 Feb 2026 | 9.65B | Q4_K_M | 5.4 GB | 256K | Apache 2.0 | 40 tok/s | Check · Hardware |
| Gemma 4 12B The "unified" Gemma 4: text, image and audio in one 12B that fits a 12 GB card at Q4. 140+ languages. |
29 May 2026 | 12B | Q4_K_M | 6.7 GB | 256K | Apache 2.0 | 32 tok/s | Check · Hardware |
| Gemma 3 12B Strong multilingual chat with images, sized for 12–16 GB cards. Gemma 4 12B is the same size and better. |
Mar 2025 | 12.2B | Q4_K_M | 6.9 GB | 128K | Gemma Terms of Use | 32 tok/s | Check · Hardware |
| Mistral NeMo 12B Multilingual 12B with a 128K window, built with NVIDIA. A roleplay and fiction favourite that refuses to die. |
Jul 2024 | 12.2B | Q4_K_M | 6.9 GB | 128K | Apache 2.0 | 32 tok/s | Check · Hardware |
| Ministral 3 14B The largest Ministral. A 12 GB card runs it at Q4 with a few gigabytes to spare. |
Dec 2025 | 13.9B | Q4_K_M | 7.8 GB | 256K | Apache 2.0 | 28 tok/s | Check · Hardware |
| Qwen3 14B The largest Qwen3 that fits a 12 GB card at Q4 with room for context. |
Apr 2025 | 14.8B | Q4_K_M | 8.3 GB | 128K | Apache 2.0 | 26 tok/s | Check · Hardware |
| gpt-oss 20B Ships natively in MXFP4, so the 4-bit weights are the reference weights, not a lossy copy. Fits 16 GB. |
Aug 2025 | 20.9B MoE | MXFP4 | 10.8 GB | 128K | Apache 2.0 | offload | Check · Hardware |
| Mistral Small 3.2 24B Apache-2.0, vision-capable, and the most 24 GB-friendly of the 2025 generalists. |
Jun 2025 | 23.6B | Q4_K_M | 13.3 GB | 128K | Apache 2.0 | offload | Check · Hardware |
| Gemma 4 26B-A4B Mixture of experts with 3.8B active. Slower to think than Qwen3.6 35B-A3B, faster to answer, and it sees images. |
2 Apr 2026 | 26.5B MoE | Q4_K_M | 14.9 GB | 256K | Apache 2.0 | offload | Check · Hardware |
| Gemma 3 27B The 2025 single-GPU generalist with vision. Its Gemma-licence terms are the reason to prefer Gemma 4 now. |
Mar 2025 | 27.4B | Q4_K_M | 15.4 GB | 128K | Gemma Terms of Use | offload | Check · Hardware |
| Qwen3.8 27B New this week The current default local Qwen: dense 27B, text + image + video, 262K context. Only 16 of its 64 blocks keep a KV cache, so long context is cheap. |
14 Aug 2026 | 27.8B | Q4_K_M | 15.6 GB | 256K | Apache 2.0 | offload | Check · Hardware |
| Qwen3.6 27B The 24 GB coding pick of spring 2026 (77.2 SWE-bench Verified). Same shape as 3.8, one generation behind. |
22 Apr 2026 | 27.8B | Q4_K_M | 15.6 GB | 256K | Apache 2.0 | offload | Check · Hardware |
| Granite 4.1 30B The largest Granite. Dense 29B at Q4 is a comfortable 24 GB fit. |
29 Apr 2026 | 28.9B | Q4_K_M | 16.3 GB | 128K | Apache 2.0 | offload | Check · Hardware |
| Muse Glimmer 30B New Meta's first open weights since Llama 4: a dense 30B distilled from Muse Spark for always-on local agents. Two KV heads keep the cache small. |
10 Aug 2026 | 29.8B | Q4_K_M | 16.8 GB | 128K | Apache 2.0 | offload | Check · Hardware |
| Qwen3 30B-A3B The MoE that made "3B active" a category. Qwen3.6 35B-A3B is its direct replacement. |
Apr 2025 | 30.5B MoE | Q4_K_M | 17.1 GB | 128K | Apache 2.0 | offload | Check · Hardware |
| GLM-4.7-Flash 30B-A3B MIT-licensed 30B-A3B tuned for agentic coding, with a DeepSeek-style latent KV cache. 60–80 tok/s reported on a 4090. |
20 Jan 2026 | 31.2B MoE | Q4_K_M | 17.5 GB | 198K | MIT | offload | Check · Hardware |
| Gemma 4 31B The dense flagship: strongest maths of the 24–32 GB class (89% AIME), clean prose, vision. Q4 is a tight 24 GB fit. |
2 Apr 2026 | 31.3B | Q4_K_M | 17.6 GB | 256K | Apache 2.0 | offload | Check · Hardware |
| Nemotron 3.5 Lightning 30B-A3B New Mamba-2 + MoE hybrid built for the execution layer of agents: only 6 attention blocks, so the KV cache is almost free. Weights, data and recipe all open. |
11 Aug 2026 | 31.6B MoE | Q4_K_M | 17.8 GB | 256K | OpenMDW-1.1 | offload | Check · Hardware |
| Olmo 3.1 32B Instruct The largest fully open model you can audit end to end. Q4 fits 24 GB, tightly. |
10 Dec 2025 | 32.2B | Q4_K_M | 18.1 GB | 64K | Apache 2.0 | offload | Check · Hardware |
| Qwen3 32B The classic 24 GB target, and still the strongest local translator under 70B. Qwen3.8 27B is smaller and better at everything else. |
Apr 2025 | 32.8B | Q4_K_M | 18.4 GB | 128K | Apache 2.0 | offload | Check · Hardware |
| LLM-jp 4 33B Thinking New this week Japan’s national-institute reasoning model, Japanese and English. A plain dense Llama-style 33B: Q4 is a tight 24 GB fit. |
14 Aug 2026 | 33.2B | Q4_K_M | 18.7 GB | 64K | Apache 2.0 | offload | Check · Hardware |
| Qwen3.6 35B-A3B Mixture of experts with ~3B active: the fastest serious model a 24 GB card runs, and the best MoE under 40B on agentic coding. |
16 Apr 2026 | 35.9B MoE | Q4_K_M | 20.2 GB | 256K | Apache 2.0 | offload | Check · Hardware |
| Llama 3.3 70B Instruct Still the creative-writing favourite: consistent voice, takes direction. Needs 48 GB to sit comfortably on GPU at Q4. |
Dec 2024 | 70.6B | Q4_K_M | 39.7 GB | 128K | Llama 3.3 Community | won't fit | Check · Hardware |
| Llama 4 Scout 109B-A17B A 10M-token window on paper, chunked attention in practice (8K chunks on 3 of 4 layers). Needs 64 GB+ at Q4. |
5 Apr 2025 | 109B MoE | Q4_K_M | 61.3 GB | 1024K | Llama 4 Community | won't fit | Check · Hardware |
| Mistral Small 4 119B-A6B Instruct, reasoning, vision and code in one 119B MoE. A latent KV cache (320 wide) keeps context cheap; the weights still want 64 GB+. |
17 Mar 2026 | 119B MoE | Q4_K_M | 66.9 GB | 256K | Apache 2.0 | won't fit | Check · Hardware |
| Nemotron 3 Super 120B-A12B The open-training-data 120B. Same hybrid layout as Lightning, so 128K context costs under a gigabyte. |
Mar 2026 | 124B MoE | Q4_K_M | 69.7 GB | 256K | NVIDIA Open Model | won't fit | Check · Hardware |
| Qwen3.5 122B-A10B The 96–128 GB unified-memory model: 122B of knowledge at 10B-active speed. |
24 Feb 2026 | 125B MoE | Q4_K_M | 70.3 GB | 256K | Apache 2.0 | won't fit | Check · Hardware |
| Qwen3 235B-A22B Workstation class. Realistically a 192 GB unified-memory or multi-GPU model. |
Apr 2025 | 235B MoE | Q4_K_M | 132.1 GB | 128K | Apache 2.0 | won't fit | Check · Hardware |
Weight sizes are computed from the parameter count and the quantisation's effective bits per weight, not read off a file listing — expect them to land within a few percent of the GGUF you actually download. The method, in full.