Models
80 open-weight models, sized
Updated 8 Oct 2026 1 new this month
Parameter counts, layer counts and attention shapes are read from each model's own config, because the KV-cache arithmetic depends on them exactly. Sizes below are at Q4_K_M and 8K context against a GeForce RTX 4060 Laptop. Missing a model that shipped this week? Open an issue — the table is hand-maintained.
| Model | Released | Params | Quant | Weights | Max context | Licence | On this rig | |
|---|---|---|---|---|---|---|---|---|
| DeepSeek-R1-Distill-Qwen 7B Reasoning traces on a 7B budget. Expect long outputs — budget context accordingly. |
Jan 2025 | 7.62B | Q4_K_M | 4.3 GB | 128K | MIT | 36 tok/s | Check · Hardware |
| Ling 3.0 Tiny 7.9B-A1.3B An 8B MoE with 1.3B active and a latent KV cache on only 6 of 24 layers — reasoning and tool use sized for Apple Silicon and edge boxes. |
10 Aug 2026 | 7.9B MoE | Q4_K_M | 4.4 GB | 128K | MIT | 82 tok/s | Check · Hardware |
| Ornith 1.5 9B The small Ornith: a coding-agent reasoning build on the Qwen3.5 9B architecture. Same VRAM as its base, thinks before every answer. |
19 Aug 2026 | 9.41B | Q4_K_M | 5.3 GB | 256K | MIT | 29 tok/s | Check · Hardware |
| Qwen3.5 9B The default for 8–12 GB cards in 2026: beats every older 8B on every published benchmark, with vision. |
28 Feb 2026 | 9.65B | Q4_K_M | 5.4 GB | 256K | Apache 2.0 | 29 tok/s | Check · Hardware |
| Phi-4 14B Trained heavily on synthetic reasoning data. Short 16K window is its main limitation. |
Dec 2024 | 14.7B | Q4_K_M | 8.3 GB | 16K | MIT | offload | Check · Hardware |
| Qwen3 14B The largest Qwen3 that fits a 12 GB card at Q4 with room for context. |
Apr 2025 | 14.8B | Q4_K_M | 8.3 GB | 128K | Apache 2.0 | offload | Check · Hardware |
| DeepSeek-R1-Distill-Qwen 14B The 2025 reasoning-per-gigabyte pick for a 12 GB card. Qwen3.5 9B in thinking mode has since overtaken it. |
Jan 2025 | 14.8B | Q4_K_M | 8.3 GB | 128K | MIT | offload | Check · Hardware |
| gpt-oss 20B Ships natively in MXFP4, so the 4-bit weights are the reference weights, not a lossy copy. Fits 16 GB. |
Aug 2025 | 20.9B MoE | MXFP4 | 10.8 GB | 128K | Apache 2.0 | offload | Check · Hardware |
| Qwen3.8 27B The current default local Qwen: dense 27B, text + image + video, 262K context. Only 16 of its 64 blocks keep a KV cache, so long context is cheap. |
14 Aug 2026 | 27.8B | Q4_K_M | 15.6 GB | 256K | Apache 2.0 | offload | Check · Hardware |
| Qwen3.6 27B The 24 GB coding pick of spring 2026 (77.2 SWE-bench Verified). Same shape as 3.8, one generation behind. |
22 Apr 2026 | 27.8B | Q4_K_M | 15.6 GB | 256K | Apache 2.0 | offload | Check · Hardware |
| Gemma 4 31B The dense flagship: strongest maths of the 24–32 GB class (89% AIME), clean prose, vision. Q4 is a tight 24 GB fit. |
2 Apr 2026 | 31.3B | Q4_K_M | 17.6 GB | 256K | Apache 2.0 | offload | Check · Hardware |
| Olmo 3.1 32B Instruct The largest fully open model you can audit end to end. Q4 fits 24 GB, tightly. |
10 Dec 2025 | 32.2B | Q4_K_M | 18.1 GB | 64K | Apache 2.0 | offload | Check · Hardware |
| Qwen3 32B The classic 24 GB target, and still the strongest local translator under 70B. Qwen3.8 27B is smaller and better at everything else. |
Apr 2025 | 32.8B | Q4_K_M | 18.4 GB | 128K | Apache 2.0 | offload | Check · Hardware |
| DeepSeek-R1-Distill-Qwen 32B MIT-licensed and close to the 70B distill on maths. A 24 GB card handles it at Q4. |
Jan 2025 | 32.8B | Q4_K_M | 18.4 GB | 128K | MIT | offload | Check · Hardware |
| LLM-jp 4 33B Thinking Japan’s national-institute reasoning model, Japanese and English. A plain dense Llama-style 33B: Q4 is a tight 24 GB fit. |
14 Aug 2026 | 33.2B | Q4_K_M | 18.7 GB | 64K | Apache 2.0 | offload | Check · Hardware |
| Qwen3.6 35B-A3B Mixture of experts with ~3B active: the fastest serious model a 24 GB card runs, and the best MoE under 40B on agentic coding. |
16 Apr 2026 | 35.9B MoE | Q4_K_M | 20.2 GB | 256K | Apache 2.0 | offload | Check · Hardware |
| Ornith 1.5 35B-A3B A reasoning-first MIT build on the Qwen3.6 35B-A3B architecture (thinks before every answer). Same VRAM as its base. |
19 Aug 2026 | 35.9B MoE | Q4_K_M | 20.2 GB | 256K | MIT | offload | Check · Hardware |
| Llama 3.3 70B Instruct Still the creative-writing favourite: consistent voice, takes direction. Needs 48 GB to sit comfortably on GPU at Q4. |
Dec 2024 | 70.6B | Q4_K_M | 39.7 GB | 128K | Llama 3.3 Community | won't fit | Check · Hardware |
| DeepSeek-R1-Distill-Llama 70B The strongest of the R1 distills, and the one that most needs 48 GB or more. |
Jan 2025 | 70.6B | Q4_K_M | 39.7 GB | 128K | MIT / Llama 3.3 Community | won't fit | Check · Hardware |
| gpt-oss 120B Designed to land on one 80 GB card. Only ~5B parameters are active per token. |
Aug 2025 | 117B MoE | MXFP4 | 60.5 GB | 128K | Apache 2.0 | won't fit | Check · Hardware |
| Mistral Small 4 119B-A6B Instruct, reasoning, vision and code in one 119B MoE. A latent KV cache (320 wide) keeps context cheap; the weights still want 64 GB+. |
17 Mar 2026 | 119B MoE | Q4_K_M | 66.9 GB | 256K | Apache 2.0 | won't fit | Check · Hardware |
| Nemotron 3 Super 120B-A12B The open-training-data 120B. Same hybrid layout as Lightning, so 128K context costs under a gigabyte. |
Mar 2026 | 124B MoE | Q4_K_M | 69.7 GB | 256K | NVIDIA Open Model | won't fit | Check · Hardware |
| Ling 3.0 Flash 124B-A5B A 124B hybrid (5 linear-attention layers per MLA layer) with 5.1B active: SWE-bench Pro 56.6 and AIME 93 claimed. Built for 96–128 GB machines. |
2 Aug 2026 | 124B MoE | Q4_K_M | 69.7 GB | 256K | MIT | won't fit | Check · Hardware |
| Qwen3.5 122B-A10B The 96–128 GB unified-memory model: 122B of knowledge at 10B-active speed. |
24 Feb 2026 | 125B MoE | Q4_K_M | 70.3 GB | 256K | Apache 2.0 | won't fit | Check · Hardware |
| Qwen3.8-Flash-Next 180B-A6B The open preview of the Qwen4 architecture: a 125B-A6B hybrid (Gated DeltaNet + sparse attention, KV cache on 12 of 48 blocks) plus a 51B n-gram embedding and a 4B draft head — 180B on disk, 6B active. The hosted "Qwen3.8-Flash" is this model with a 1M window. |
24 Aug 2026 | 180B MoE | Q4_K_M | 101.2 GB | 256K | Qwen Community 1.0 | won't fit | Check · Hardware |
| Qwen3 235B-A22B Workstation class. Realistically a 192 GB unified-memory or multi-GPU model. |
Apr 2025 | 235B MoE | Q4_K_M | 132.1 GB | 128K | Apache 2.0 | won't fit | Check · Hardware |
| DeepSeek V4 Flash 284B-A13B The V4 for 192–256 GB machines: Q3 squeezes into 192 GB, Q4 wants 256 GB, and 128 GB falls short even at Q2. Cache is modelled as a 576-wide latent; V4 compresses it further at long context, so this is conservative. |
31 Jul 2026 | 284B MoE | Q4_K_M | 159.7 GB | 1024K | MIT | won't fit | Check · Hardware |
| GLM-5.3-Flash 320B-A18B The first natively multimodal GLM-5 and the first hybrid: 34 linear-attention blocks and 11 sparse-attention blocks with a 512-wide latent cache, so a 1M window stays affordable. Z.ai says it beats GLM-5.2 at 18B active; MIT. |
25 Aug 2026 | 321B MoE | Q4_K_M | 180.5 GB | 1024K | MIT | won't fit | Check · Hardware |
| Ornith 1.5 397B-A17B The flagship Ornith on the Qwen3.5 397B-A17B architecture, MIT-licensed. A 256 GB Mac Studio at Q4, and it is in the Ollama library. |
19 Aug 2026 | 397B MoE | Q4_K_M | 223.2 GB | 256K | MIT | won't fit | Check · Hardware |
| Qwen3.5 397B-A17B Flagship-class at 17B active. A 256 GB Mac Studio or a multi-GPU box at Q4. |
16 Feb 2026 | 403B MoE | Q4_K_M | 226.6 GB | 256K | Apache 2.0 | won't fit | Check · Hardware |
| DeepSeek V4.1 Flash 552B-A16B New Twice V4 Flash's backbone (552B, 16B active per generated token) plus 196B of Engram lookup tables (189 GiB at FP8) that DwarfStar streams from SSD, so they are not counted here. A 512 GB Mac model, and as of October 2026 no mainline llama.cpp or Ollama build: DwarfStar on a Mac, vLLM across four GPUs. DeepSeek puts the cache at 890 bytes/token; modelled conservatively as 4 latent layers plus a 128-token window. |
10 Sep 2026 | 552B MoE | Q4_K_M | 310.4 GB | 1024K | MIT | won't fit | Check · Hardware |
| DeepSeek-R1 671B The January 2025 moment. Multi-head latent attention keeps its KV cache tiny; the weights do not. |
Jan 2025 | 671B MoE | Q4_K_M | 377.3 GB | 128K | MIT | won't fit | Check · Hardware |
| GLM-5.2 744B-A40B The strongest all-round open-weight model of mid-2026 on most public boards. Listed as a ceiling: 512 GB of unified memory at Q4. |
13 Jun 2026 | 753B MoE | Q4_K_M | 423.4 GB | 1024K | MIT | won't fit | Check · Hardware |
| GLM-5.3 744B-A40B Same base as GLM-5.2, new post-training: Z.ai's most capable open-weights coder (Terminal-Bench 3.0 28.3, DeepSWE 66.9). Not MIT any more — its own GLM-5.3 licence. Same ceiling as 5.2: 512 GB of unified memory at Q4. |
25 Aug 2026 | 753B MoE | Q4_K_M | 423.4 GB | 1024K | GLM-5.3 License | won't fit | Check · Hardware |
| Kimi K2.6 1T-A32B The open coding-agent benchmark leader of spring 2026 (80.2 SWE-bench). A 512 GB Mac Studio pair, or a ceiling. |
20 Apr 2026 | 1027B MoE | Q4_K_M | 577.5 GB | 256K | Modified MIT | won't fit | Check · Hardware |
| DeepSeek V4 Pro 1.6T-A49B The 0813 refresh of the V4 flagship. Included as the honest ceiling; a terabyte of weights at Q4. |
13 Aug 2026 | 1650B MoE | Q4_K_M | 927.8 GB | 1024K | MIT | won't fit | Check · Hardware |
| Qwen3.8 2.4T-A95B The first open Qwen-Max-class flagship. Listed as the honest ceiling: nothing short of a rack runs it. |
12 Aug 2026 | 2446B MoE | Q4_K_M | 1375.4 GB | 256K | Qwen3.8-Max License | won't fit | Check · Hardware |
| Kimi K3 2.8T-A104B The largest open-weight model ever published. 69 of 93 blocks are linear attention, so its cache is tiny; its 2.8T weights are the problem. |
27 Jul 2026 | 2780B MoE | Q4_K_M | 1563.2 GB | 1024K | Modified MIT | won't fit | Check · Hardware |
Weight sizes are computed from the parameter count and the quantisation's effective bits per weight, not read off a file listing — expect them to land within a few percent of the GGUF you actually download. The method, in full.