Models

76 open-weight models, sized

Updated 21 Aug 2026 5 new this week, 14 this month

Parameter counts, layer counts and attention shapes are read from each model's own config, because the KV-cache arithmetic depends on them exactly. Sizes below are at Q4_K_M and 8K context against a GeForce RTX 3060 12 GB. Missing a model that shipped this week? Open an issue — the table is hand-maintained.

AllGeneral assistantCodingReasoning & mathsWritingVisionAgents & tool useTranslationLow latencyLong context
By sizeNewest first
RTX 4090RTX 3060 12 GBRX 7900 XTXM3 Max · 64 GB
ModelReleasedParamsQuantWeightsMax contextLicenceOn this rig
DeepSeek-R1-Distill-Qwen 7B
Reasoning traces on a 7B budget. Expect long outputs — budget context accordingly.
 Jan 2025 7.62B Q4_K_M 4.3 GB 128K MIT 51 tok/s Check · Hardware
Ling 3.0 Tiny 7.9B-A1.3B New
An 8B MoE with 1.3B active and a latent KV cache on only 6 of 24 layers — reasoning and tool use sized for Apple Silicon and edge boxes.
 10 Aug 2026 7.9B MoE Q4_K_M 4.4 GB 128K MIT 115 tok/s Check · Hardware
Ornith 1.5 9B New this week
The small Ornith: a coding-agent reasoning build on the Qwen3.5 9B architecture. Same VRAM as its base, thinks before every answer.
 19 Aug 2026 9.41B Q4_K_M 5.3 GB 256K MIT 41 tok/s Check · Hardware
Qwen3.5 9B
The default for 8–12 GB cards in 2026: beats every older 8B on every published benchmark, with vision.
 28 Feb 2026 9.65B Q4_K_M 5.4 GB 256K Apache 2.0 40 tok/s Check · Hardware
Phi-4 14B
Trained heavily on synthetic reasoning data. Short 16K window is its main limitation.
 Dec 2024 14.7B Q4_K_M 8.3 GB 16K MIT 26 tok/s Check · Hardware
Qwen3 14B
The largest Qwen3 that fits a 12 GB card at Q4 with room for context.
 Apr 2025 14.8B Q4_K_M 8.3 GB 128K Apache 2.0 26 tok/s Check · Hardware
DeepSeek-R1-Distill-Qwen 14B
The 2025 reasoning-per-gigabyte pick for a 12 GB card. Qwen3.5 9B in thinking mode has since overtaken it.
 Jan 2025 14.8B Q4_K_M 8.3 GB 128K MIT 26 tok/s Check · Hardware
gpt-oss 20B
Ships natively in MXFP4, so the 4-bit weights are the reference weights, not a lossy copy. Fits 16 GB.
 Aug 2025 20.9B MoE MXFP4 10.8 GB 128K Apache 2.0 offload Check · Hardware
Qwen3.8 27B New this week
The current default local Qwen: dense 27B, text + image + video, 262K context. Only 16 of its 64 blocks keep a KV cache, so long context is cheap.
 14 Aug 2026 27.8B Q4_K_M 15.6 GB 256K Apache 2.0 offload Check · Hardware
Qwen3.6 27B
The 24 GB coding pick of spring 2026 (77.2 SWE-bench Verified). Same shape as 3.8, one generation behind.
 22 Apr 2026 27.8B Q4_K_M 15.6 GB 256K Apache 2.0 offload Check · Hardware
Gemma 4 31B
The dense flagship: strongest maths of the 24–32 GB class (89% AIME), clean prose, vision. Q4 is a tight 24 GB fit.
 2 Apr 2026 31.3B Q4_K_M 17.6 GB 256K Apache 2.0 offload Check · Hardware
Olmo 3.1 32B Instruct
The largest fully open model you can audit end to end. Q4 fits 24 GB, tightly.
 10 Dec 2025 32.2B Q4_K_M 18.1 GB 64K Apache 2.0 offload Check · Hardware
Qwen3 32B
The classic 24 GB target, and still the strongest local translator under 70B. Qwen3.8 27B is smaller and better at everything else.
 Apr 2025 32.8B Q4_K_M 18.4 GB 128K Apache 2.0 offload Check · Hardware
DeepSeek-R1-Distill-Qwen 32B
MIT-licensed and close to the 70B distill on maths. A 24 GB card handles it at Q4.
 Jan 2025 32.8B Q4_K_M 18.4 GB 128K MIT offload Check · Hardware
LLM-jp 4 33B Thinking New this week
Japan’s national-institute reasoning model, Japanese and English. A plain dense Llama-style 33B: Q4 is a tight 24 GB fit.
 14 Aug 2026 33.2B Q4_K_M 18.7 GB 64K Apache 2.0 offload Check · Hardware
Qwen3.6 35B-A3B
Mixture of experts with ~3B active: the fastest serious model a 24 GB card runs, and the best MoE under 40B on agentic coding.
 16 Apr 2026 35.9B MoE Q4_K_M 20.2 GB 256K Apache 2.0 offload Check · Hardware
Ornith 1.5 35B-A3B New this week
A reasoning-first MIT build on the Qwen3.6 35B-A3B architecture (thinks before every answer). Same VRAM as its base.
 19 Aug 2026 35.9B MoE Q4_K_M 20.2 GB 256K MIT offload Check · Hardware
Llama 3.3 70B Instruct
Still the creative-writing favourite: consistent voice, takes direction. Needs 48 GB to sit comfortably on GPU at Q4.
 Dec 2024 70.6B Q4_K_M 39.7 GB 128K Llama 3.3 Community won't fit Check · Hardware
DeepSeek-R1-Distill-Llama 70B
The strongest of the R1 distills, and the one that most needs 48 GB or more.
 Jan 2025 70.6B Q4_K_M 39.7 GB 128K MIT / Llama 3.3 Community won't fit Check · Hardware
gpt-oss 120B
Designed to land on one 80 GB card. Only ~5B parameters are active per token.
 Aug 2025 117B MoE MXFP4 60.5 GB 128K Apache 2.0 won't fit Check · Hardware
Mistral Small 4 119B-A6B
Instruct, reasoning, vision and code in one 119B MoE. A latent KV cache (320 wide) keeps context cheap; the weights still want 64 GB+.
 17 Mar 2026 119B MoE Q4_K_M 66.9 GB 256K Apache 2.0 won't fit Check · Hardware
Nemotron 3 Super 120B-A12B
The open-training-data 120B. Same hybrid layout as Lightning, so 128K context costs under a gigabyte.
 Mar 2026 124B MoE Q4_K_M 69.7 GB 256K NVIDIA Open Model won't fit Check · Hardware
Ling 3.0 Flash 124B-A5B New
A 124B hybrid (5 linear-attention layers per MLA layer) with 5.1B active: SWE-bench Pro 56.6 and AIME 93 claimed. Built for 96–128 GB machines.
 2 Aug 2026 124B MoE Q4_K_M 69.7 GB 256K MIT won't fit Check · Hardware
Qwen3.5 122B-A10B
The 96–128 GB unified-memory model: 122B of knowledge at 10B-active speed.
 24 Feb 2026 125B MoE Q4_K_M 70.3 GB 256K Apache 2.0 won't fit Check · Hardware
Qwen3 235B-A22B
Workstation class. Realistically a 192 GB unified-memory or multi-GPU model.
 Apr 2025 235B MoE Q4_K_M 132.1 GB 128K Apache 2.0 won't fit Check · Hardware
DeepSeek V4 Flash 284B-A13B New
The V4 that 128 GB machines can actually run at Q3. Cache is modelled as a 576-wide latent; V4 compresses it further at long context, so this is conservative.
 31 Jul 2026 284B MoE Q4_K_M 159.7 GB 1024K MIT won't fit Check · Hardware
Ornith 1.5 397B-A17B New this week
The flagship Ornith on the Qwen3.5 397B-A17B architecture, MIT-licensed. A 256 GB Mac Studio at Q4, and it is in the Ollama library.
 19 Aug 2026 397B MoE Q4_K_M 223.2 GB 256K MIT won't fit Check · Hardware
Qwen3.5 397B-A17B
Flagship-class at 17B active. A 256 GB Mac Studio or a multi-GPU box at Q4.
 16 Feb 2026 403B MoE Q4_K_M 226.6 GB 256K Apache 2.0 won't fit Check · Hardware
DeepSeek-R1 671B
The January 2025 moment. Multi-head latent attention keeps its KV cache tiny; the weights do not.
 Jan 2025 671B MoE Q4_K_M 377.3 GB 128K MIT won't fit Check · Hardware
GLM-5.2 744B-A40B
The strongest all-round open-weight model of mid-2026 on most public boards. Listed as a ceiling: 512 GB of unified memory at Q4.
 13 Jun 2026 753B MoE Q4_K_M 423.4 GB 1024K MIT won't fit Check · Hardware
Kimi K2.6 1T-A32B
The open coding-agent benchmark leader of spring 2026 (80.2 SWE-bench). A 512 GB Mac Studio pair, or a ceiling.
 20 Apr 2026 1027B MoE Q4_K_M 577.5 GB 256K Modified MIT won't fit Check · Hardware
DeepSeek V4 Pro 1.6T-A49B New
The 0813 refresh of the V4 flagship. Included as the honest ceiling; a terabyte of weights at Q4.
 13 Aug 2026 1650B MoE Q4_K_M 927.8 GB 1024K MIT won't fit Check · Hardware
Qwen3.8 2.4T-A95B New
The first open Qwen-Max-class flagship. Listed as the honest ceiling: nothing short of a rack runs it.
 12 Aug 2026 2446B MoE Q4_K_M 1375.4 GB 256K Qwen3.8-Max License won't fit Check · Hardware
Kimi K3 2.8T-A104B New
The largest open-weight model ever published. 69 of 93 blocks are linear attention, so its cache is tiny; its 2.8T weights are the problem.
 27 Jul 2026 2780B MoE Q4_K_M 1563.2 GB 1024K Modified MIT won't fit Check · Hardware

Weight sizes are computed from the parameter count and the quantisation's effective bits per weight, not read off a file listing — expect them to land within a few percent of the GGUF you actually download. The method, in full.