Models

80 open-weight models, sized

Updated 8 Oct 2026 1 new this month

Parameter counts, layer counts and attention shapes are read from each model's own config, because the KV-cache arithmetic depends on them exactly. Sizes below are at Q4_K_M and 8K context against a GeForce RTX 4060 Laptop. Missing a model that shipped this week? Open an issue — the table is hand-maintained.

ModelReleasedParamsQuantWeightsMax contextLicenceOn this rig
Qwen3.5 2B
Phone-class, and multimodal. Replaces Llama 3.2 3B as the "it runs on anything" answer.
28 Feb 2026 2.27B Q4_K_M 1.3 GB 256K Apache 2.0 121 tok/s Check · Hardware
Ministral 3 3B
Edge model with a vision encoder and a 256K window. Apache 2.0.
Dec 2025 3.85B Q4_K_M 2.2 GB 256K Apache 2.0 72 tok/s Check · Hardware
Gemma 3 4B
Vision-capable at 4B. Superseded by Gemma 4 E4B, still everywhere.
Mar 2025 4.3B Q4_K_M 2.4 GB 128K Gemma Terms of Use 64 tok/s Check · Hardware
Qwen3.5 4B
The 8 GB coding agent. Q4 lands near 3.4 GB, leaving room for a real context window.
28 Feb 2026 4.66B Q4_K_M 2.6 GB 256K Apache 2.0 59 tok/s Check · Hardware
Gemma 4 E2B
"E2B" is 2.3B effective, but the file holds 5B because of per-layer embeddings — size it as 5B. Text, image and audio in.
2 Apr 2026 5.1B Q4_K_M 2.9 GB 128K Apache 2.0 54 tok/s Check · Hardware
Gemma 4 E4B
The laptop Gemma. 4.5B effective, 8B on disk; a single KV head per window layer keeps its cache tiny.
2 Apr 2026 8.0B Q4_K_M 4.5 GB 128K Apache 2.0 34 tok/s Check · Hardware
Fara 7B
A web computer-use agent on a Qwen2.5-VL base — it clicks, fills forms and stops for permission. Not a chat model; size it like an 8B with vision.
24 Nov 2025 8.29B Q4_K_M 4.7 GB 125K MIT 33 tok/s Check · Hardware
Ministral 3 8B
Mistral's 8B with images in. Plain GQA, so budget more KV cache than Qwen3.5 9B at the same context.
Dec 2025 8.92B Q4_K_M 5.0 GB 256K Apache 2.0 31 tok/s Check · Hardware
Qwen3.5 9B
The default for 8–12 GB cards in 2026: beats every older 8B on every published benchmark, with vision.
28 Feb 2026 9.65B Q4_K_M 5.4 GB 256K Apache 2.0 29 tok/s Check · Hardware
Gemma 4 12B
The "unified" Gemma 4: text, image and audio in one 12B that fits a 12 GB card at Q4. 140+ languages.
29 May 2026 12B Q4_K_M 6.7 GB 256K Apache 2.0 offload Check · Hardware
Gemma 3 12B
Strong multilingual chat with images, sized for 12–16 GB cards. Gemma 4 12B is the same size and better.
Mar 2025 12.2B Q4_K_M 6.9 GB 128K Gemma Terms of Use offload Check · Hardware
Ministral 3 14B
The largest Ministral. A 12 GB card runs it at Q4 with a few gigabytes to spare.
Dec 2025 13.9B Q4_K_M 7.8 GB 256K Apache 2.0 offload Check · Hardware
Mistral Small 3.2 24B
Apache-2.0, vision-capable, and the most 24 GB-friendly of the 2025 generalists.
Jun 2025 23.6B Q4_K_M 13.3 GB 128K Apache 2.0 offload Check · Hardware
Gemma 4 26B-A4B
Mixture of experts with 3.8B active. Slower to think than Qwen3.6 35B-A3B, faster to answer, and it sees images.
2 Apr 2026 26.5B MoE Q4_K_M 14.9 GB 256K Apache 2.0 offload Check · Hardware
Gemma 3 27B
The 2025 single-GPU generalist with vision. Its Gemma-licence terms are the reason to prefer Gemma 4 now.
Mar 2025 27.4B Q4_K_M 15.4 GB 128K Gemma Terms of Use offload Check · Hardware
Qwen3.8 27B
The current default local Qwen: dense 27B, text + image + video, 262K context. Only 16 of its 64 blocks keep a KV cache, so long context is cheap.
14 Aug 2026 27.8B Q4_K_M 15.6 GB 256K Apache 2.0 offload Check · Hardware
Qwen3.6 27B
The 24 GB coding pick of spring 2026 (77.2 SWE-bench Verified). Same shape as 3.8, one generation behind.
22 Apr 2026 27.8B Q4_K_M 15.6 GB 256K Apache 2.0 offload Check · Hardware
Muse Glimmer 30B
Meta's first open weights since Llama 4: a dense 30B distilled from Muse Spark for always-on local agents. Two KV heads keep the cache small.
10 Aug 2026 29.8B Q4_K_M 16.8 GB 128K Apache 2.0 offload Check · Hardware
Gemma 4 31B
The dense flagship: strongest maths of the 24–32 GB class (89% AIME), clean prose, vision. Q4 is a tight 24 GB fit.
2 Apr 2026 31.3B Q4_K_M 17.6 GB 256K Apache 2.0 offload Check · Hardware
Qwen3.6 35B-A3B
Mixture of experts with ~3B active: the fastest serious model a 24 GB card runs, and the best MoE under 40B on agentic coding.
16 Apr 2026 35.9B MoE Q4_K_M 20.2 GB 256K Apache 2.0 offload Check · Hardware
Llama 4 Scout 109B-A17B
A 10M-token window on paper, chunked attention in practice (8K chunks on 3 of 4 layers). Needs 64 GB+ at Q4.
5 Apr 2025 109B MoE Q4_K_M 61.3 GB 1024K Llama 4 Community won't fit Check · Hardware
Mistral Small 4 119B-A6B
Instruct, reasoning, vision and code in one 119B MoE. A latent KV cache (320 wide) keeps context cheap; the weights still want 64 GB+.
17 Mar 2026 119B MoE Q4_K_M 66.9 GB 256K Apache 2.0 won't fit Check · Hardware
Qwen3.5 122B-A10B
The 96–128 GB unified-memory model: 122B of knowledge at 10B-active speed.
24 Feb 2026 125B MoE Q4_K_M 70.3 GB 256K Apache 2.0 won't fit Check · Hardware
Qwen3.8-Flash-Next 180B-A6B
The open preview of the Qwen4 architecture: a 125B-A6B hybrid (Gated DeltaNet + sparse attention, KV cache on 12 of 48 blocks) plus a 51B n-gram embedding and a 4B draft head — 180B on disk, 6B active. The hosted "Qwen3.8-Flash" is this model with a 1M window.
24 Aug 2026 180B MoE Q4_K_M 101.2 GB 256K Qwen Community 1.0 won't fit Check · Hardware
GLM-5.3-Flash 320B-A18B
The first natively multimodal GLM-5 and the first hybrid: 34 linear-attention blocks and 11 sparse-attention blocks with a 512-wide latent cache, so a 1M window stays affordable. Z.ai says it beats GLM-5.2 at 18B active; MIT.
25 Aug 2026 321B MoE Q4_K_M 180.5 GB 1024K MIT won't fit Check · Hardware
Qwen3.5 397B-A17B
Flagship-class at 17B active. A 256 GB Mac Studio or a multi-GPU box at Q4.
16 Feb 2026 403B MoE Q4_K_M 226.6 GB 256K Apache 2.0 won't fit Check · Hardware
DeepSeek V4.1 Flash 552B-A16B New
Twice V4 Flash's backbone (552B, 16B active per generated token) plus 196B of Engram lookup tables (189 GiB at FP8) that DwarfStar streams from SSD, so they are not counted here. A 512 GB Mac model, and as of October 2026 no mainline llama.cpp or Ollama build: DwarfStar on a Mac, vLLM across four GPUs. DeepSeek puts the cache at 890 bytes/token; modelled conservatively as 4 latent layers plus a 128-token window.
10 Sep 2026 552B MoE Q4_K_M 310.4 GB 1024K MIT won't fit Check · Hardware
Kimi K3 2.8T-A104B
The largest open-weight model ever published. 69 of 93 blocks are linear attention, so its cache is tiny; its 2.8T weights are the problem.
27 Jul 2026 2780B MoE Q4_K_M 1563.2 GB 1024K Modified MIT won't fit Check · Hardware

Weight sizes are computed from the parameter count and the quantisation's effective bits per weight, not read off a file listing — expect them to land within a few percent of the GGUF you actually download. The method, in full.