Models

80 open-weight models, sized

Updated 8 Oct 2026 1 new this month

Parameter counts, layer counts and attention shapes are read from each model's own config, because the KV-cache arithmetic depends on them exactly. Sizes below are at Q4_K_M and 8K context against an M3 Max · 64 GB. Missing a model that shipped this week? Open an issue — the table is hand-maintained.

ModelReleasedParamsQuantWeightsMax contextLicenceOn this rig
Qwen3 0.6B
Useful mostly as a speculative-decoding draft model for its larger siblings.
Apr 2025 0.6B Q4_K_M 0.3 GB 32K Apache 2.0 663 tok/s Check · Hardware
Qwen3.5 0.8B
Draft model for speculative decoding, or a classifier that fits in 1 GB.
28 Feb 2026 0.87B Q4_K_M 0.5 GB 256K Apache 2.0 457 tok/s Check · Hardware
Gemma 3 1B
Text-only. Single KV head makes its cache almost free at long context.
Mar 2025 1.0B Q4_K_M 0.6 GB 32K Gemma Terms of Use 398 tok/s Check · Hardware
Llama 3.2 1B Instruct
The smallest Llama worth running. Fits anywhere, including phones and 4 GB cards.
Sep 2024 1.24B Q4_K_M 0.7 GB 128K Llama 3.2 Community 321 tok/s Check · Hardware
Qwen3 1.7B
Punches above its size on structured tasks, with optional thinking mode.
Apr 2025 1.72B Q4_K_M 1.0 GB 32K Apache 2.0 231 tok/s Check · Hardware
Qwen3.5 2B
Phone-class, and multimodal. Replaces Llama 3.2 3B as the "it runs on anything" answer.
28 Feb 2026 2.27B Q4_K_M 1.3 GB 256K Apache 2.0 175 tok/s Check · Hardware
LFM2.5 2.6B
Convolution-heavy hybrid for CPUs and NPUs: 22 of 30 blocks keep no KV cache at all.
28 Jul 2026 2.7B Q4_K_M 1.5 GB 128K LFM Open License v1.0 147 tok/s Check · Hardware
Llama 3.2 3B Instruct
The 2024 "it just runs" model for 8 GB laptops. Qwen3.5 4B does the same job better now.
Sep 2024 3.21B Q4_K_M 1.8 GB 128K Llama 3.2 Community 124 tok/s Check · Hardware
Granite 4.1 3B
Dense, small, enterprise-flavoured: tool calling and instruction following, no thinking mode.
29 Apr 2026 3.4B Q4_K_M 1.9 GB 128K Apache 2.0 117 tok/s Check · Hardware
Phi-4-mini 3.8B
MIT-licensed, dense, and unusually strong on instruction following for its size.
Feb 2025 3.84B Q4_K_M 2.2 GB 128K MIT 104 tok/s Check · Hardware
Ministral 3 3B
Edge model with a vision encoder and a 256K window. Apache 2.0.
Dec 2025 3.85B Q4_K_M 2.2 GB 256K Apache 2.0 103 tok/s Check · Hardware
Qwen3 4B
The 2025 sweet spot for 8 GB cards with reasoning traces. Qwen3.5 4B adds vision and 8× the context.
Apr 2025 4.02B Q4_K_M 2.3 GB 32K Apache 2.0 99 tok/s Check · Hardware
Gemma 3 4B
Vision-capable at 4B. Superseded by Gemma 4 E4B, still everywhere.
Mar 2025 4.3B Q4_K_M 2.4 GB 128K Gemma Terms of Use 92 tok/s Check · Hardware
Qwen3.5 4B
The 8 GB coding agent. Q4 lands near 3.4 GB, leaving room for a real context window.
28 Feb 2026 4.66B Q4_K_M 2.6 GB 256K Apache 2.0 85 tok/s Check · Hardware
Gemma 4 E2B
"E2B" is 2.3B effective, but the file holds 5B because of per-layer embeddings — size it as 5B. Text, image and audio in.
2 Apr 2026 5.1B Q4_K_M 2.9 GB 128K Apache 2.0 78 tok/s Check · Hardware
Mistral 7B Instruct v0.3
Old but extremely well behaved, and permissively licensed for commercial use.
May 2024 7.25B Q4_K_M 4.1 GB 32K Apache 2.0 55 tok/s Check · Hardware
Olmo 3 7B Instruct
Fully open — training data and code included. Full multi-head attention, so its cache is 4× a GQA 7B at the same context.
20 Nov 2025 7.3B Q4_K_M 4.1 GB 64K Apache 2.0 54 tok/s Check · Hardware
DeepSeek-R1-Distill-Qwen 7B
Reasoning traces on a 7B budget. Expect long outputs — budget context accordingly.
Jan 2025 7.62B Q4_K_M 4.3 GB 128K MIT 52 tok/s Check · Hardware
Qwen2.5-Coder 7B
The standard local autocomplete model — small enough to keep resident all day, and still the best FIM model under 8B.
Nov 2024 7.62B Q4_K_M 4.3 GB 128K Apache 2.0 52 tok/s Check · Hardware
Ling 3.0 Tiny 7.9B-A1.3B
An 8B MoE with 1.3B active and a latent KV cache on only 6 of 24 layers — reasoning and tool use sized for Apple Silicon and edge boxes.
10 Aug 2026 7.9B MoE Q4_K_M 4.4 GB 128K MIT 143 tok/s Check · Hardware
Gemma 4 E4B
The laptop Gemma. 4.5B effective, 8B on disk; a single KV head per window layer keeps its cache tiny.
2 Apr 2026 8.0B Q4_K_M 4.5 GB 128K Apache 2.0 50 tok/s Check · Hardware
Llama 3.1 8B Instruct
Still the most widely deployed local model, with the largest fine-tune ecosystem. Not the strongest 8B any more.
Jul 2024 8.03B Q4_K_M 4.5 GB 128K Llama 3.1 Community 50 tok/s Check · Hardware
Qwen3 8B
Apache-2.0 alternative to Llama 3.1 8B, with a switchable thinking mode.
Apr 2025 8.19B Q4_K_M 4.6 GB 128K Apache 2.0 49 tok/s Check · Hardware
Fara 7B
A web computer-use agent on a Qwen2.5-VL base — it clicks, fills forms and stops for permission. Not a chat model; size it like an 8B with vision.
24 Nov 2025 8.29B Q4_K_M 4.7 GB 125K MIT 48 tok/s Check · Hardware
LFM2.5 8B-A1B
An 8B MoE with ~1.5B active, aimed at laptops without a GPU. Licence is permissive below $10M revenue.
28 May 2026 8.47B MoE Q4_K_M 4.8 GB 125K LFM Open License v1.0 124 tok/s Check · Hardware
Granite 4.1 8B
Matches the old Granite 4.0 32B MoE at a quarter of the size. Fast, Apache 2.0, no reasoning traces.
29 Apr 2026 8.79B Q4_K_M 4.9 GB 128K Apache 2.0 45 tok/s Check · Hardware
Ministral 3 8B
Mistral's 8B with images in. Plain GQA, so budget more KV cache than Qwen3.5 9B at the same context.
Dec 2025 8.92B Q4_K_M 5.0 GB 256K Apache 2.0 45 tok/s Check · Hardware
Ornith 1.5 9B
The small Ornith: a coding-agent reasoning build on the Qwen3.5 9B architecture. Same VRAM as its base, thinks before every answer.
19 Aug 2026 9.41B Q4_K_M 5.3 GB 256K MIT 42 tok/s Check · Hardware
Qwen3.5 9B
The default for 8–12 GB cards in 2026: beats every older 8B on every published benchmark, with vision.
28 Feb 2026 9.65B Q4_K_M 5.4 GB 256K Apache 2.0 41 tok/s Check · Hardware
Gemma 4 12B
The "unified" Gemma 4: text, image and audio in one 12B that fits a 12 GB card at Q4. 140+ languages.
29 May 2026 12B Q4_K_M 6.7 GB 256K Apache 2.0 33 tok/s Check · Hardware
Gemma 3 12B
Strong multilingual chat with images, sized for 12–16 GB cards. Gemma 4 12B is the same size and better.
Mar 2025 12.2B Q4_K_M 6.9 GB 128K Gemma Terms of Use 33 tok/s Check · Hardware
Mistral NeMo 12B
Multilingual 12B with a 128K window, built with NVIDIA. A roleplay and fiction favourite that refuses to die.
Jul 2024 12.2B Q4_K_M 6.9 GB 128K Apache 2.0 33 tok/s Check · Hardware
Ministral 3 14B
The largest Ministral. A 12 GB card runs it at Q4 with a few gigabytes to spare.
Dec 2025 13.9B Q4_K_M 7.8 GB 256K Apache 2.0 29 tok/s Check · Hardware
Phi-4 14B
Trained heavily on synthetic reasoning data. Short 16K window is its main limitation.
Dec 2024 14.7B Q4_K_M 8.3 GB 16K MIT 27 tok/s Check · Hardware
Qwen3 14B
The largest Qwen3 that fits a 12 GB card at Q4 with room for context.
Apr 2025 14.8B Q4_K_M 8.3 GB 128K Apache 2.0 27 tok/s Check · Hardware
DeepSeek-R1-Distill-Qwen 14B
The 2025 reasoning-per-gigabyte pick for a 12 GB card. Qwen3.5 9B in thinking mode has since overtaken it.
Jan 2025 14.8B Q4_K_M 8.3 GB 128K MIT 27 tok/s Check · Hardware
Qwen2.5-Coder 14B
Noticeably better at whole-file edits than the 7B, still comfortable on 12 GB.
Nov 2024 14.8B Q4_K_M 8.3 GB 128K Apache 2.0 27 tok/s Check · Hardware
gpt-oss 20B
Ships natively in MXFP4, so the 4-bit weights are the reference weights, not a lossy copy. Fits 16 GB.
Aug 2025 20.9B MoE MXFP4 10.8 GB 128K Apache 2.0 56 tok/s Check · Hardware
Mistral Small 3.2 24B
Apache-2.0, vision-capable, and the most 24 GB-friendly of the 2025 generalists.
Jun 2025 23.6B Q4_K_M 13.3 GB 128K Apache 2.0 17 tok/s Check · Hardware
Devstral Small 2 24B
Built for software-engineering agents (OpenHands, Cline). Dense 24B, 384K window. Not in the Ollama library.
Dec 2025 24B Q4_K_M 13.5 GB 384K Apache 2.0 17 tok/s Check · Hardware
Gemma 4 26B-A4B
Mixture of experts with 3.8B active. Slower to think than Qwen3.6 35B-A3B, faster to answer, and it sees images.
2 Apr 2026 26.5B MoE Q4_K_M 14.9 GB 256K Apache 2.0 49 tok/s Check · Hardware
Gemma 3 27B
The 2025 single-GPU generalist with vision. Its Gemma-licence terms are the reason to prefer Gemma 4 now.
Mar 2025 27.4B Q4_K_M 15.4 GB 128K Gemma Terms of Use 15 tok/s Check · Hardware
Qwen3.8 27B
The current default local Qwen: dense 27B, text + image + video, 262K context. Only 16 of its 64 blocks keep a KV cache, so long context is cheap.
14 Aug 2026 27.8B Q4_K_M 15.6 GB 256K Apache 2.0 14 tok/s Check · Hardware
Qwen3.6 27B
The 24 GB coding pick of spring 2026 (77.2 SWE-bench Verified). Same shape as 3.8, one generation behind.
22 Apr 2026 27.8B Q4_K_M 15.6 GB 256K Apache 2.0 14 tok/s Check · Hardware
Granite 4.1 30B
The largest Granite. Dense 29B at Q4 is a comfortable 24 GB fit.
29 Apr 2026 28.9B Q4_K_M 16.3 GB 128K Apache 2.0 14 tok/s Check · Hardware
Muse Glimmer 30B
Meta's first open weights since Llama 4: a dense 30B distilled from Muse Spark for always-on local agents. Two KV heads keep the cache small.
10 Aug 2026 29.8B Q4_K_M 16.8 GB 128K Apache 2.0 13 tok/s Check · Hardware
Qwen3 30B-A3B
The MoE that made "3B active" a category. Qwen3.6 35B-A3B is its direct replacement.
Apr 2025 30.5B MoE Q4_K_M 17.1 GB 128K Apache 2.0 56 tok/s Check · Hardware
Qwen3 Coder 30B-A3B
Agentic coding MoE with a 256K native window. Still the most-downloaded local code model.
Jul 2025 30.5B MoE Q4_K_M 17.1 GB 256K Apache 2.0 56 tok/s Check · Hardware
GLM-4.7-Flash 30B-A3B
MIT-licensed 30B-A3B tuned for agentic coding, with a DeepSeek-style latent KV cache. 60–80 tok/s reported on a 4090.
20 Jan 2026 31.2B MoE Q4_K_M 17.5 GB 198K MIT 62 tok/s Check · Hardware
Gemma 4 31B
The dense flagship: strongest maths of the 24–32 GB class (89% AIME), clean prose, vision. Q4 is a tight 24 GB fit.
2 Apr 2026 31.3B Q4_K_M 17.6 GB 256K Apache 2.0 13 tok/s Check · Hardware
Nemotron 3.5 Lightning 30B-A3B
Mamba-2 + MoE hybrid built for the execution layer of agents: only 6 attention blocks, so the KV cache is almost free. Weights, data and recipe all open.
11 Aug 2026 31.6B MoE Q4_K_M 17.8 GB 256K OpenMDW-1.1 58 tok/s Check · Hardware
Olmo 3.1 32B Instruct
The largest fully open model you can audit end to end. Q4 fits 24 GB, tightly.
10 Dec 2025 32.2B Q4_K_M 18.1 GB 64K Apache 2.0 12 tok/s Check · Hardware
Qwen3 32B
The classic 24 GB target, and still the strongest local translator under 70B. Qwen3.8 27B is smaller and better at everything else.
Apr 2025 32.8B Q4_K_M 18.4 GB 128K Apache 2.0 12 tok/s Check · Hardware
DeepSeek-R1-Distill-Qwen 32B
MIT-licensed and close to the 70B distill on maths. A 24 GB card handles it at Q4.
Jan 2025 32.8B Q4_K_M 18.4 GB 128K MIT 12 tok/s Check · Hardware
Qwen2.5-Coder 32B
The first local code model that felt competitive with hosted assistants. Qwen3.8 27B is smaller and far ahead on agentic work.
Nov 2024 32.8B Q4_K_M 18.4 GB 128K Apache 2.0 12 tok/s Check · Hardware
LLM-jp 4 33B Thinking
Japan’s national-institute reasoning model, Japanese and English. A plain dense Llama-style 33B: Q4 is a tight 24 GB fit.
14 Aug 2026 33.2B Q4_K_M 18.7 GB 64K Apache 2.0 12 tok/s Check · Hardware
Qwen3.6 35B-A3B
Mixture of experts with ~3B active: the fastest serious model a 24 GB card runs, and the best MoE under 40B on agentic coding.
16 Apr 2026 35.9B MoE Q4_K_M 20.2 GB 256K Apache 2.0 56 tok/s Check · Hardware
Ornith 1.5 35B-A3B
A reasoning-first MIT build on the Qwen3.6 35B-A3B architecture (thinks before every answer). Same VRAM as its base.
19 Aug 2026 35.9B MoE Q4_K_M 20.2 GB 256K MIT 56 tok/s Check · Hardware
Llama 3.3 70B Instruct
Still the creative-writing favourite: consistent voice, takes direction. Needs 48 GB to sit comfortably on GPU at Q4.
Dec 2024 70.6B Q4_K_M 39.7 GB 128K Llama 3.3 Community 5.6 tok/s Check · Hardware
DeepSeek-R1-Distill-Llama 70B
The strongest of the R1 distills, and the one that most needs 48 GB or more.
Jan 2025 70.6B Q4_K_M 39.7 GB 128K MIT / Llama 3.3 Community 5.6 tok/s Check · Hardware
Llama 4 Scout 109B-A17B
A 10M-token window on paper, chunked attention in practice (8K chunks on 3 of 4 layers). Needs 64 GB+ at Q4.
5 Apr 2025 109B MoE Q4_K_M 61.3 GB 1024K Llama 4 Community won't fit Check · Hardware
gpt-oss 120B
Designed to land on one 80 GB card. Only ~5B parameters are active per token.
Aug 2025 117B MoE MXFP4 60.5 GB 128K Apache 2.0 won't fit Check · Hardware
Mistral Small 4 119B-A6B
Instruct, reasoning, vision and code in one 119B MoE. A latent KV cache (320 wide) keeps context cheap; the weights still want 64 GB+.
17 Mar 2026 119B MoE Q4_K_M 66.9 GB 256K Apache 2.0 won't fit Check · Hardware
Nemotron 3 Super 120B-A12B
The open-training-data 120B. Same hybrid layout as Lightning, so 128K context costs under a gigabyte.
Mar 2026 124B MoE Q4_K_M 69.7 GB 256K NVIDIA Open Model won't fit Check · Hardware
Ling 3.0 Flash 124B-A5B
A 124B hybrid (5 linear-attention layers per MLA layer) with 5.1B active: SWE-bench Pro 56.6 and AIME 93 claimed. Built for 96–128 GB machines.
2 Aug 2026 124B MoE Q4_K_M 69.7 GB 256K MIT won't fit Check · Hardware
Qwen3.5 122B-A10B
The 96–128 GB unified-memory model: 122B of knowledge at 10B-active speed.
24 Feb 2026 125B MoE Q4_K_M 70.3 GB 256K Apache 2.0 won't fit Check · Hardware
Qwen3.8-Flash-Next 180B-A6B
The open preview of the Qwen4 architecture: a 125B-A6B hybrid (Gated DeltaNet + sparse attention, KV cache on 12 of 48 blocks) plus a 51B n-gram embedding and a 4B draft head — 180B on disk, 6B active. The hosted "Qwen3.8-Flash" is this model with a 1M window.
24 Aug 2026 180B MoE Q4_K_M 101.2 GB 256K Qwen Community 1.0 won't fit Check · Hardware
Qwen3 235B-A22B
Workstation class. Realistically a 192 GB unified-memory or multi-GPU model.
Apr 2025 235B MoE Q4_K_M 132.1 GB 128K Apache 2.0 won't fit Check · Hardware
DeepSeek V4 Flash 284B-A13B
The V4 for 192–256 GB machines: Q3 squeezes into 192 GB, Q4 wants 256 GB, and 128 GB falls short even at Q2. Cache is modelled as a 576-wide latent; V4 compresses it further at long context, so this is conservative.
31 Jul 2026 284B MoE Q4_K_M 159.7 GB 1024K MIT won't fit Check · Hardware
GLM-5.3-Flash 320B-A18B
The first natively multimodal GLM-5 and the first hybrid: 34 linear-attention blocks and 11 sparse-attention blocks with a 512-wide latent cache, so a 1M window stays affordable. Z.ai says it beats GLM-5.2 at 18B active; MIT.
25 Aug 2026 321B MoE Q4_K_M 180.5 GB 1024K MIT won't fit Check · Hardware
Ornith 1.5 397B-A17B
The flagship Ornith on the Qwen3.5 397B-A17B architecture, MIT-licensed. A 256 GB Mac Studio at Q4, and it is in the Ollama library.
19 Aug 2026 397B MoE Q4_K_M 223.2 GB 256K MIT won't fit Check · Hardware
Qwen3.5 397B-A17B
Flagship-class at 17B active. A 256 GB Mac Studio or a multi-GPU box at Q4.
16 Feb 2026 403B MoE Q4_K_M 226.6 GB 256K Apache 2.0 won't fit Check · Hardware
DeepSeek V4.1 Flash 552B-A16B New
Twice V4 Flash's backbone (552B, 16B active per generated token) plus 196B of Engram lookup tables (189 GiB at FP8) that DwarfStar streams from SSD, so they are not counted here. A 512 GB Mac model, and as of October 2026 no mainline llama.cpp or Ollama build: DwarfStar on a Mac, vLLM across four GPUs. DeepSeek puts the cache at 890 bytes/token; modelled conservatively as 4 latent layers plus a 128-token window.
10 Sep 2026 552B MoE Q4_K_M 310.4 GB 1024K MIT won't fit Check · Hardware
DeepSeek-R1 671B
The January 2025 moment. Multi-head latent attention keeps its KV cache tiny; the weights do not.
Jan 2025 671B MoE Q4_K_M 377.3 GB 128K MIT won't fit Check · Hardware
GLM-5.2 744B-A40B
The strongest all-round open-weight model of mid-2026 on most public boards. Listed as a ceiling: 512 GB of unified memory at Q4.
13 Jun 2026 753B MoE Q4_K_M 423.4 GB 1024K MIT won't fit Check · Hardware
GLM-5.3 744B-A40B
Same base as GLM-5.2, new post-training: Z.ai's most capable open-weights coder (Terminal-Bench 3.0 28.3, DeepSWE 66.9). Not MIT any more — its own GLM-5.3 licence. Same ceiling as 5.2: 512 GB of unified memory at Q4.
25 Aug 2026 753B MoE Q4_K_M 423.4 GB 1024K GLM-5.3 License won't fit Check · Hardware
Kimi K2.6 1T-A32B
The open coding-agent benchmark leader of spring 2026 (80.2 SWE-bench). A 512 GB Mac Studio pair, or a ceiling.
20 Apr 2026 1027B MoE Q4_K_M 577.5 GB 256K Modified MIT won't fit Check · Hardware
DeepSeek V4 Pro 1.6T-A49B
The 0813 refresh of the V4 flagship. Included as the honest ceiling; a terabyte of weights at Q4.
13 Aug 2026 1650B MoE Q4_K_M 927.8 GB 1024K MIT won't fit Check · Hardware
Qwen3.8 2.4T-A95B
The first open Qwen-Max-class flagship. Listed as the honest ceiling: nothing short of a rack runs it.
12 Aug 2026 2446B MoE Q4_K_M 1375.4 GB 256K Qwen3.8-Max License won't fit Check · Hardware
Kimi K3 2.8T-A104B
The largest open-weight model ever published. 69 of 93 blocks are linear attention, so its cache is tiny; its 2.8T weights are the problem.
27 Jul 2026 2780B MoE Q4_K_M 1563.2 GB 1024K Modified MIT won't fit Check · Hardware

Weight sizes are computed from the parameter count and the quantisation's effective bits per weight, not read off a file listing — expect them to land within a few percent of the GGUF you actually download. The method, in full.