Models

76 open-weight models, sized

Updated 21 Aug 2026 5 new this week, 14 this month

Parameter counts, layer counts and attention shapes are read from each model's own config, because the KV-cache arithmetic depends on them exactly. Sizes below are at Q4_K_M and 8K context against a Radeon RX 7900 XTX. Missing a model that shipped this week? Open an issue — the table is hand-maintained.

All General assistant Coding Reasoning & maths Writing Vision Agents & tool use Translation Low latency Long context

By size Newest first

RTX 4090 RTX 3060 12 GB RX 7900 XTX M3 Max · 64 GB

Model	Released	Params	Quant	Weights	Max context	Licence	On this rig
Qwen3.5 2B Phone-class, and multimodal. Replaces Llama 3.2 3B as the "it runs on anything" answer.	28 Feb 2026	2.27B	Q4_K_M	1.3 GB	256K	Apache 2.0	455 tok/s	Check · Hardware
Ministral 3 3B Edge model with a vision encoder and a 256K window. Apache 2.0.	Dec 2025	3.85B	Q4_K_M	2.2 GB	256K	Apache 2.0	268 tok/s	Check · Hardware
Gemma 3 4B Vision-capable at 4B. Superseded by Gemma 4 E4B, still everywhere.	Mar 2025	4.3B	Q4_K_M	2.4 GB	128K	Gemma Terms of Use	240 tok/s	Check · Hardware
Qwen3.5 4B The 8 GB coding agent. Q4 lands near 3.4 GB, leaving room for a real context window.	28 Feb 2026	4.66B	Q4_K_M	2.6 GB	256K	Apache 2.0	222 tok/s	Check · Hardware
Gemma 4 E2B "E2B" is 2.3B effective, but the file holds 5B because of per-layer embeddings — size it as 5B. Text, image and audio in.	2 Apr 2026	5.1B	Q4_K_M	2.9 GB	128K	Apache 2.0	203 tok/s	Check · Hardware
Gemma 4 E4B The laptop Gemma. 4.5B effective, 8B on disk; a single KV head per window layer keeps its cache tiny.	2 Apr 2026	8.0B	Q4_K_M	4.5 GB	128K	Apache 2.0	129 tok/s	Check · Hardware
Fara 7B A web computer-use agent on a Qwen2.5-VL base — it clicks, fills forms and stops for permission. Not a chat model; size it like an 8B with vision.	24 Nov 2025	8.29B	Q4_K_M	4.7 GB	125K	MIT	125 tok/s	Check · Hardware
Ministral 3 8B Mistral's 8B with images in. Plain GQA, so budget more KV cache than Qwen3.5 9B at the same context.	Dec 2025	8.92B	Q4_K_M	5.0 GB	256K	Apache 2.0	116 tok/s	Check · Hardware
Qwen3.5 9B The default for 8–12 GB cards in 2026: beats every older 8B on every published benchmark, with vision.	28 Feb 2026	9.65B	Q4_K_M	5.4 GB	256K	Apache 2.0	107 tok/s	Check · Hardware
Gemma 4 12B The "unified" Gemma 4: text, image and audio in one 12B that fits a 12 GB card at Q4. 140+ languages.	29 May 2026	12B	Q4_K_M	6.7 GB	256K	Apache 2.0	86 tok/s	Check · Hardware
Gemma 3 12B Strong multilingual chat with images, sized for 12–16 GB cards. Gemma 4 12B is the same size and better.	Mar 2025	12.2B	Q4_K_M	6.9 GB	128K	Gemma Terms of Use	85 tok/s	Check · Hardware
Ministral 3 14B The largest Ministral. A 12 GB card runs it at Q4 with a few gigabytes to spare.	Dec 2025	13.9B	Q4_K_M	7.8 GB	256K	Apache 2.0	74 tok/s	Check · Hardware
Mistral Small 3.2 24B Apache-2.0, vision-capable, and the most 24 GB-friendly of the 2025 generalists.	Jun 2025	23.6B	Q4_K_M	13.3 GB	128K	Apache 2.0	44 tok/s	Check · Hardware
Gemma 4 26B-A4B Mixture of experts with 3.8B active. Slower to think than Qwen3.6 35B-A3B, faster to answer, and it sees images.	2 Apr 2026	26.5B MoE	Q4_K_M	14.9 GB	256K	Apache 2.0	105 tok/s	Check · Hardware
Gemma 3 27B The 2025 single-GPU generalist with vision. Its Gemma-licence terms are the reason to prefer Gemma 4 now.	Mar 2025	27.4B	Q4_K_M	15.4 GB	128K	Gemma Terms of Use	38 tok/s	Check · Hardware
Qwen3.8 27B New this week The current default local Qwen: dense 27B, text + image + video, 262K context. Only 16 of its 64 blocks keep a KV cache, so long context is cheap.	14 Aug 2026	27.8B	Q4_K_M	15.6 GB	256K	Apache 2.0	37 tok/s	Check · Hardware
Qwen3.6 27B The 24 GB coding pick of spring 2026 (77.2 SWE-bench Verified). Same shape as 3.8, one generation behind.	22 Apr 2026	27.8B	Q4_K_M	15.6 GB	256K	Apache 2.0	37 tok/s	Check · Hardware
Muse Glimmer 30B New Meta's first open weights since Llama 4: a dense 30B distilled from Muse Spark for always-on local agents. Two KV heads keep the cache small.	10 Aug 2026	29.8B	Q4_K_M	16.8 GB	128K	Apache 2.0	35 tok/s	Check · Hardware
Gemma 4 31B The dense flagship: strongest maths of the 24–32 GB class (89% AIME), clean prose, vision. Q4 is a tight 24 GB fit.	2 Apr 2026	31.3B	Q4_K_M	17.6 GB	256K	Apache 2.0	33 tok/s	Check · Hardware
Qwen3.6 35B-A3B Mixture of experts with ~3B active: the fastest serious model a 24 GB card runs, and the best MoE under 40B on agentic coding.	16 Apr 2026	35.9B MoE	Q4_K_M	20.2 GB	256K	Apache 2.0	120 tok/s	Check · Hardware
Llama 4 Scout 109B-A17B A 10M-token window on paper, chunked attention in practice (8K chunks on 3 of 4 layers). Needs 64 GB+ at Q4.	5 Apr 2025	109B MoE	Q4_K_M	61.3 GB	1024K	Llama 4 Community	won't fit	Check · Hardware
Mistral Small 4 119B-A6B Instruct, reasoning, vision and code in one 119B MoE. A latent KV cache (320 wide) keeps context cheap; the weights still want 64 GB+.	17 Mar 2026	119B MoE	Q4_K_M	66.9 GB	256K	Apache 2.0	won't fit	Check · Hardware
Qwen3.5 122B-A10B The 96–128 GB unified-memory model: 122B of knowledge at 10B-active speed.	24 Feb 2026	125B MoE	Q4_K_M	70.3 GB	256K	Apache 2.0	won't fit	Check · Hardware
Qwen3.5 397B-A17B Flagship-class at 17B active. A 256 GB Mac Studio or a multi-GPU box at Q4.	16 Feb 2026	403B MoE	Q4_K_M	226.6 GB	256K	Apache 2.0	won't fit	Check · Hardware
Kimi K3 2.8T-A104B New The largest open-weight model ever published. 69 of 93 blocks are linear attention, so its cache is tiny; its 2.8T weights are the problem.	27 Jul 2026	2780B MoE	Q4_K_M	1563.2 GB	1024K	Modified MIT	won't fit	Check · Hardware

Weight sizes are computed from the parameter count and the quantisation's effective bits per weight, not read off a file listing — expect them to land within a few percent of the GGUF you actually download. The method, in full.