Models

76 open-weight models, sized

Updated 21 Aug 2026 5 new this week, 14 this month

Parameter counts, layer counts and attention shapes are read from each model's own config, because the KV-cache arithmetic depends on them exactly. Sizes below are at Q4_K_M and 8K context against a M3 Max · 64 GB. Missing a model that shipped this week? Open an issue — the table is hand-maintained.

AllGeneral assistantCodingReasoning & mathsWritingVisionAgents & tool useTranslationLow latencyLong context
By sizeNewest first
RTX 4090RTX 3060 12 GBRX 7900 XTXM3 Max · 64 GB
ModelReleasedParamsQuantWeightsMax contextLicenceOn this rig
Qwen3 0.6B
Useful mostly as a speculative-decoding draft model for its larger siblings.
 Apr 2025 0.6B Q4_K_M 0.3 GB 32K Apache 2.0 663 tok/s Check · Hardware
Qwen3.5 0.8B
Draft model for speculative decoding, or a classifier that fits in 1 GB.
 28 Feb 2026 0.87B Q4_K_M 0.5 GB 256K Apache 2.0 457 tok/s Check · Hardware
Gemma 3 1B
Text-only. Single KV head makes its cache almost free at long context.
 Mar 2025 1.0B Q4_K_M 0.6 GB 32K Gemma Terms of Use 398 tok/s Check · Hardware
Llama 3.2 1B Instruct
The smallest Llama worth running. Fits anywhere, including phones and 4 GB cards.
 Sep 2024 1.24B Q4_K_M 0.7 GB 128K Llama 3.2 Community 321 tok/s Check · Hardware
Qwen3 1.7B
Punches above its size on structured tasks, with optional thinking mode.
 Apr 2025 1.72B Q4_K_M 1.0 GB 32K Apache 2.0 231 tok/s Check · Hardware
Qwen3.5 2B
Phone-class, and multimodal. Replaces Llama 3.2 3B as the "it runs on anything" answer.
 28 Feb 2026 2.27B Q4_K_M 1.3 GB 256K Apache 2.0 175 tok/s Check · Hardware
LFM2.5 2.6B New
Convolution-heavy hybrid for CPUs and NPUs: 22 of 30 blocks keep no KV cache at all.
 28 Jul 2026 2.7B Q4_K_M 1.5 GB 128K LFM Open License v1.0 147 tok/s Check · Hardware
Llama 3.2 3B Instruct
The 2024 "it just runs" model for 8 GB laptops. Qwen3.5 4B does the same job better now.
 Sep 2024 3.21B Q4_K_M 1.8 GB 128K Llama 3.2 Community 124 tok/s Check · Hardware
Granite 4.1 3B
Dense, small, enterprise-flavoured: tool calling and instruction following, no thinking mode.
 29 Apr 2026 3.4B Q4_K_M 1.9 GB 128K Apache 2.0 117 tok/s Check · Hardware
Phi-4-mini 3.8B
MIT-licensed, dense, and unusually strong on instruction following for its size.
 Feb 2025 3.84B Q4_K_M 2.2 GB 128K MIT 104 tok/s Check · Hardware
Ministral 3 3B
Edge model with a vision encoder and a 256K window. Apache 2.0.
 Dec 2025 3.85B Q4_K_M 2.2 GB 256K Apache 2.0 103 tok/s Check · Hardware
Qwen3 4B
The 2025 sweet spot for 8 GB cards with reasoning traces. Qwen3.5 4B adds vision and 8× the context.
 Apr 2025 4.02B Q4_K_M 2.3 GB 32K Apache 2.0 99 tok/s Check · Hardware
Gemma 3 4B
Vision-capable at 4B. Superseded by Gemma 4 E4B, still everywhere.
 Mar 2025 4.3B Q4_K_M 2.4 GB 128K Gemma Terms of Use 92 tok/s Check · Hardware
Qwen3.5 4B
The 8 GB coding agent. Q4 lands near 3.4 GB, leaving room for a real context window.
 28 Feb 2026 4.66B Q4_K_M 2.6 GB 256K Apache 2.0 85 tok/s Check · Hardware
Gemma 4 E2B
"E2B" is 2.3B effective, but the file holds 5B because of per-layer embeddings — size it as 5B. Text, image and audio in.
 2 Apr 2026 5.1B Q4_K_M 2.9 GB 128K Apache 2.0 78 tok/s Check · Hardware
Mistral 7B Instruct v0.3
Old but extremely well behaved, and permissively licensed for commercial use.
 May 2024 7.25B Q4_K_M 4.1 GB 32K Apache 2.0 55 tok/s Check · Hardware
Olmo 3 7B Instruct
Fully open — training data and code included. Full multi-head attention, so its cache is 4× a GQA 7B at the same context.
 20 Nov 2025 7.3B Q4_K_M 4.1 GB 64K Apache 2.0 54 tok/s Check · Hardware
DeepSeek-R1-Distill-Qwen 7B
Reasoning traces on a 7B budget. Expect long outputs — budget context accordingly.
 Jan 2025 7.62B Q4_K_M 4.3 GB 128K MIT 52 tok/s Check · Hardware
Qwen2.5-Coder 7B
The standard local autocomplete model — small enough to keep resident all day, and still the best FIM model under 8B.
 Nov 2024 7.62B Q4_K_M 4.3 GB 128K Apache 2.0 52 tok/s Check · Hardware
Ling 3.0 Tiny 7.9B-A1.3B New
An 8B MoE with 1.3B active and a latent KV cache on only 6 of 24 layers — reasoning and tool use sized for Apple Silicon and edge boxes.
 10 Aug 2026 7.9B MoE Q4_K_M 4.4 GB 128K MIT 143 tok/s Check · Hardware
Gemma 4 E4B
The laptop Gemma. 4.5B effective, 8B on disk; a single KV head per window layer keeps its cache tiny.
 2 Apr 2026 8.0B Q4_K_M 4.5 GB 128K Apache 2.0 50 tok/s Check · Hardware
Llama 3.1 8B Instruct
Still the most widely deployed local model, with the largest fine-tune ecosystem. Not the strongest 8B any more.
 Jul 2024 8.03B Q4_K_M 4.5 GB 128K Llama 3.1 Community 50 tok/s Check · Hardware
Qwen3 8B
Apache-2.0 alternative to Llama 3.1 8B, with a switchable thinking mode.
 Apr 2025 8.19B Q4_K_M 4.6 GB 128K Apache 2.0 49 tok/s Check · Hardware
Fara 7B
A web computer-use agent on a Qwen2.5-VL base — it clicks, fills forms and stops for permission. Not a chat model; size it like an 8B with vision.
 24 Nov 2025 8.29B Q4_K_M 4.7 GB 125K MIT 48 tok/s Check · Hardware
LFM2.5 8B-A1B
An 8B MoE with ~1.5B active, aimed at laptops without a GPU. Licence is permissive below $10M revenue.
 28 May 2026 8.47B MoE Q4_K_M 4.8 GB 125K LFM Open License v1.0 124 tok/s Check · Hardware
Granite 4.1 8B
Matches the old Granite 4.0 32B MoE at a quarter of the size. Fast, Apache 2.0, no reasoning traces.
 29 Apr 2026 8.79B Q4_K_M 4.9 GB 128K Apache 2.0 45 tok/s Check · Hardware
Ministral 3 8B
Mistral's 8B with images in. Plain GQA, so budget more KV cache than Qwen3.5 9B at the same context.
 Dec 2025 8.92B Q4_K_M 5.0 GB 256K Apache 2.0 45 tok/s Check · Hardware
Ornith 1.5 9B New this week
The small Ornith: a coding-agent reasoning build on the Qwen3.5 9B architecture. Same VRAM as its base, thinks before every answer.
 19 Aug 2026 9.41B Q4_K_M 5.3 GB 256K MIT 42 tok/s Check · Hardware
Qwen3.5 9B
The default for 8–12 GB cards in 2026: beats every older 8B on every published benchmark, with vision.
 28 Feb 2026 9.65B Q4_K_M 5.4 GB 256K Apache 2.0 41 tok/s Check · Hardware
Gemma 4 12B
The "unified" Gemma 4: text, image and audio in one 12B that fits a 12 GB card at Q4. 140+ languages.
 29 May 2026 12B Q4_K_M 6.7 GB 256K Apache 2.0 33 tok/s Check · Hardware
Gemma 3 12B
Strong multilingual chat with images, sized for 12–16 GB cards. Gemma 4 12B is the same size and better.
 Mar 2025 12.2B Q4_K_M 6.9 GB 128K Gemma Terms of Use 33 tok/s Check · Hardware
Mistral NeMo 12B
Multilingual 12B with a 128K window, built with NVIDIA. A roleplay and fiction favourite that refuses to die.
 Jul 2024 12.2B Q4_K_M 6.9 GB 128K Apache 2.0 33 tok/s Check · Hardware
Ministral 3 14B
The largest Ministral. A 12 GB card runs it at Q4 with a few gigabytes to spare.
 Dec 2025 13.9B Q4_K_M 7.8 GB 256K Apache 2.0 29 tok/s Check · Hardware
Phi-4 14B
Trained heavily on synthetic reasoning data. Short 16K window is its main limitation.
 Dec 2024 14.7B Q4_K_M 8.3 GB 16K MIT 27 tok/s Check · Hardware
Qwen3 14B
The largest Qwen3 that fits a 12 GB card at Q4 with room for context.
 Apr 2025 14.8B Q4_K_M 8.3 GB 128K Apache 2.0 27 tok/s Check · Hardware
DeepSeek-R1-Distill-Qwen 14B
The 2025 reasoning-per-gigabyte pick for a 12 GB card. Qwen3.5 9B in thinking mode has since overtaken it.
 Jan 2025 14.8B Q4_K_M 8.3 GB 128K MIT 27 tok/s Check · Hardware
Qwen2.5-Coder 14B
Noticeably better at whole-file edits than the 7B, still comfortable on 12 GB.
 Nov 2024 14.8B Q4_K_M 8.3 GB 128K Apache 2.0 27 tok/s Check · Hardware
gpt-oss 20B
Ships natively in MXFP4, so the 4-bit weights are the reference weights, not a lossy copy. Fits 16 GB.
 Aug 2025 20.9B MoE MXFP4 10.8 GB 128K Apache 2.0 56 tok/s Check · Hardware
Mistral Small 3.2 24B
Apache-2.0, vision-capable, and the most 24 GB-friendly of the 2025 generalists.
 Jun 2025 23.6B Q4_K_M 13.3 GB 128K Apache 2.0 17 tok/s Check · Hardware
Devstral Small 2 24B
Built for software-engineering agents (OpenHands, Cline). Dense 24B, 384K window. Not in the Ollama library.
 Dec 2025 24B Q4_K_M 13.5 GB 384K Apache 2.0 17 tok/s Check · Hardware
Gemma 4 26B-A4B
Mixture of experts with 3.8B active. Slower to think than Qwen3.6 35B-A3B, faster to answer, and it sees images.
 2 Apr 2026 26.5B MoE Q4_K_M 14.9 GB 256K Apache 2.0 49 tok/s Check · Hardware
Gemma 3 27B
The 2025 single-GPU generalist with vision. Its Gemma-licence terms are the reason to prefer Gemma 4 now.
 Mar 2025 27.4B Q4_K_M 15.4 GB 128K Gemma Terms of Use 15 tok/s Check · Hardware
Qwen3.8 27B New this week
The current default local Qwen: dense 27B, text + image + video, 262K context. Only 16 of its 64 blocks keep a KV cache, so long context is cheap.
 14 Aug 2026 27.8B Q4_K_M 15.6 GB 256K Apache 2.0 14 tok/s Check · Hardware
Qwen3.6 27B
The 24 GB coding pick of spring 2026 (77.2 SWE-bench Verified). Same shape as 3.8, one generation behind.
 22 Apr 2026 27.8B Q4_K_M 15.6 GB 256K Apache 2.0 14 tok/s Check · Hardware
Granite 4.1 30B
The largest Granite. Dense 29B at Q4 is a comfortable 24 GB fit.
 29 Apr 2026 28.9B Q4_K_M 16.3 GB 128K Apache 2.0 14 tok/s Check · Hardware
Muse Glimmer 30B New
Meta's first open weights since Llama 4: a dense 30B distilled from Muse Spark for always-on local agents. Two KV heads keep the cache small.
 10 Aug 2026 29.8B Q4_K_M 16.8 GB 128K Apache 2.0 13 tok/s Check · Hardware
Qwen3 30B-A3B
The MoE that made "3B active" a category. Qwen3.6 35B-A3B is its direct replacement.
 Apr 2025 30.5B MoE Q4_K_M 17.1 GB 128K Apache 2.0 56 tok/s Check · Hardware
Qwen3 Coder 30B-A3B
Agentic coding MoE with a 256K native window. Still the most-downloaded local code model.
 Jul 2025 30.5B MoE Q4_K_M 17.1 GB 256K Apache 2.0 56 tok/s Check · Hardware
GLM-4.7-Flash 30B-A3B
MIT-licensed 30B-A3B tuned for agentic coding, with a DeepSeek-style latent KV cache. 60–80 tok/s reported on a 4090.
 20 Jan 2026 31.2B MoE Q4_K_M 17.5 GB 198K MIT 62 tok/s Check · Hardware
Gemma 4 31B
The dense flagship: strongest maths of the 24–32 GB class (89% AIME), clean prose, vision. Q4 is a tight 24 GB fit.
 2 Apr 2026 31.3B Q4_K_M 17.6 GB 256K Apache 2.0 13 tok/s Check · Hardware
Nemotron 3.5 Lightning 30B-A3B New
Mamba-2 + MoE hybrid built for the execution layer of agents: only 6 attention blocks, so the KV cache is almost free. Weights, data and recipe all open.
 11 Aug 2026 31.6B MoE Q4_K_M 17.8 GB 256K OpenMDW-1.1 58 tok/s Check · Hardware
Olmo 3.1 32B Instruct
The largest fully open model you can audit end to end. Q4 fits 24 GB, tightly.
 10 Dec 2025 32.2B Q4_K_M 18.1 GB 64K Apache 2.0 12 tok/s Check · Hardware
Qwen3 32B
The classic 24 GB target, and still the strongest local translator under 70B. Qwen3.8 27B is smaller and better at everything else.
 Apr 2025 32.8B Q4_K_M 18.4 GB 128K Apache 2.0 12 tok/s Check · Hardware
DeepSeek-R1-Distill-Qwen 32B
MIT-licensed and close to the 70B distill on maths. A 24 GB card handles it at Q4.
 Jan 2025 32.8B Q4_K_M 18.4 GB 128K MIT 12 tok/s Check · Hardware
Qwen2.5-Coder 32B
The first local code model that felt competitive with hosted assistants. Qwen3.8 27B is smaller and far ahead on agentic work.
 Nov 2024 32.8B Q4_K_M 18.4 GB 128K Apache 2.0 12 tok/s Check · Hardware
LLM-jp 4 33B Thinking New this week
Japan’s national-institute reasoning model, Japanese and English. A plain dense Llama-style 33B: Q4 is a tight 24 GB fit.
 14 Aug 2026 33.2B Q4_K_M 18.7 GB 64K Apache 2.0 12 tok/s Check · Hardware
Qwen3.6 35B-A3B
Mixture of experts with ~3B active: the fastest serious model a 24 GB card runs, and the best MoE under 40B on agentic coding.
 16 Apr 2026 35.9B MoE Q4_K_M 20.2 GB 256K Apache 2.0 56 tok/s Check · Hardware
Ornith 1.5 35B-A3B New this week
A reasoning-first MIT build on the Qwen3.6 35B-A3B architecture (thinks before every answer). Same VRAM as its base.
 19 Aug 2026 35.9B MoE Q4_K_M 20.2 GB 256K MIT 56 tok/s Check · Hardware
Llama 3.3 70B Instruct
Still the creative-writing favourite: consistent voice, takes direction. Needs 48 GB to sit comfortably on GPU at Q4.
 Dec 2024 70.6B Q4_K_M 39.7 GB 128K Llama 3.3 Community 5.6 tok/s Check · Hardware
DeepSeek-R1-Distill-Llama 70B
The strongest of the R1 distills, and the one that most needs 48 GB or more.
 Jan 2025 70.6B Q4_K_M 39.7 GB 128K MIT / Llama 3.3 Community 5.6 tok/s Check · Hardware
Llama 4 Scout 109B-A17B
A 10M-token window on paper, chunked attention in practice (8K chunks on 3 of 4 layers). Needs 64 GB+ at Q4.
 5 Apr 2025 109B MoE Q4_K_M 61.3 GB 1024K Llama 4 Community won't fit Check · Hardware
gpt-oss 120B
Designed to land on one 80 GB card. Only ~5B parameters are active per token.
 Aug 2025 117B MoE MXFP4 60.5 GB 128K Apache 2.0 won't fit Check · Hardware
Mistral Small 4 119B-A6B
Instruct, reasoning, vision and code in one 119B MoE. A latent KV cache (320 wide) keeps context cheap; the weights still want 64 GB+.
 17 Mar 2026 119B MoE Q4_K_M 66.9 GB 256K Apache 2.0 won't fit Check · Hardware
Nemotron 3 Super 120B-A12B
The open-training-data 120B. Same hybrid layout as Lightning, so 128K context costs under a gigabyte.
 Mar 2026 124B MoE Q4_K_M 69.7 GB 256K NVIDIA Open Model won't fit Check · Hardware
Ling 3.0 Flash 124B-A5B New
A 124B hybrid (5 linear-attention layers per MLA layer) with 5.1B active: SWE-bench Pro 56.6 and AIME 93 claimed. Built for 96–128 GB machines.
 2 Aug 2026 124B MoE Q4_K_M 69.7 GB 256K MIT won't fit Check · Hardware
Qwen3.5 122B-A10B
The 96–128 GB unified-memory model: 122B of knowledge at 10B-active speed.
 24 Feb 2026 125B MoE Q4_K_M 70.3 GB 256K Apache 2.0 won't fit Check · Hardware
Qwen3 235B-A22B
Workstation class. Realistically a 192 GB unified-memory or multi-GPU model.
 Apr 2025 235B MoE Q4_K_M 132.1 GB 128K Apache 2.0 won't fit Check · Hardware
DeepSeek V4 Flash 284B-A13B New
The V4 that 128 GB machines can actually run at Q3. Cache is modelled as a 576-wide latent; V4 compresses it further at long context, so this is conservative.
 31 Jul 2026 284B MoE Q4_K_M 159.7 GB 1024K MIT won't fit Check · Hardware
Ornith 1.5 397B-A17B New this week
The flagship Ornith on the Qwen3.5 397B-A17B architecture, MIT-licensed. A 256 GB Mac Studio at Q4, and it is in the Ollama library.
 19 Aug 2026 397B MoE Q4_K_M 223.2 GB 256K MIT won't fit Check · Hardware
Qwen3.5 397B-A17B
Flagship-class at 17B active. A 256 GB Mac Studio or a multi-GPU box at Q4.
 16 Feb 2026 403B MoE Q4_K_M 226.6 GB 256K Apache 2.0 won't fit Check · Hardware
DeepSeek-R1 671B
The January 2025 moment. Multi-head latent attention keeps its KV cache tiny; the weights do not.
 Jan 2025 671B MoE Q4_K_M 377.3 GB 128K MIT won't fit Check · Hardware
GLM-5.2 744B-A40B
The strongest all-round open-weight model of mid-2026 on most public boards. Listed as a ceiling: 512 GB of unified memory at Q4.
 13 Jun 2026 753B MoE Q4_K_M 423.4 GB 1024K MIT won't fit Check · Hardware
Kimi K2.6 1T-A32B
The open coding-agent benchmark leader of spring 2026 (80.2 SWE-bench). A 512 GB Mac Studio pair, or a ceiling.
 20 Apr 2026 1027B MoE Q4_K_M 577.5 GB 256K Modified MIT won't fit Check · Hardware
DeepSeek V4 Pro 1.6T-A49B New
The 0813 refresh of the V4 flagship. Included as the honest ceiling; a terabyte of weights at Q4.
 13 Aug 2026 1650B MoE Q4_K_M 927.8 GB 1024K MIT won't fit Check · Hardware
Qwen3.8 2.4T-A95B New
The first open Qwen-Max-class flagship. Listed as the honest ceiling: nothing short of a rack runs it.
 12 Aug 2026 2446B MoE Q4_K_M 1375.4 GB 256K Qwen3.8-Max License won't fit Check · Hardware
Kimi K3 2.8T-A104B New
The largest open-weight model ever published. 69 of 93 blocks are linear attention, so its cache is tiny; its 2.8T weights are the problem.
 27 Jul 2026 2780B MoE Q4_K_M 1563.2 GB 1024K Modified MIT won't fit Check · Hardware

Weight sizes are computed from the parameter count and the quantisation's effective bits per weight, not read off a file listing — expect them to land within a few percent of the GGUF you actually download. The method, in full.