Best local LLM for agents · 2026

The best local LLM for agents and tool use, ranked for your VRAM

Updated 8 Oct 2026 1 new this month

The agents shortlist is ordered by what the vendors and the community report for function calling, multi-step tool loops and long-horizon tasks — Terminal-Bench, OSWorld, Agents' Last Exam — as of the catalogue date. For each memory class below we list every model on that list that actually fits at Q4_K_M with an 8K context, so the top row is the pick and the rest are what you give up by choosing it. Numbers are computed, not benchmarked; how the shortlist works.

An agent re-reads its whole history every step: the tool schemas, every earlier call and its output. 8K runs out within a few turns of real work, so plan for 32K or more and check the "max context" column before you wire a model into a loop.

Best agent LLM for 6 GB VRAM

5.3 GB usable · speeds for a GeForce RTX 3060 Laptop everything that fits 6 GB →

Qwen3.5 4B is the pick: Function calling in 8 GB. 2 other models on the agents shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 Qwen3.5 4B
Alibaba · Apache 2.0
Function calling in 8 GB. 4.66B Q4_K_M · 3.5 GB 66K 78 Runs great
2 Ling 3.0 Tiny 7.9B-A1.3B
inclusionAI · MIT
Tool use and reasoning at 1.3B active, sized for edge boxes. 7.9B MoE Q4_K_M · 5.1 GB 39K 107 Tight fit
3 Granite 4.1 3B
IBM · Apache 2.0
Small, dense, follows tool schemas. 3.4B Q4_K_M · 3.1 GB 35K 106 Runs great

Best agent LLM for 8 GB VRAM

7.0 GB usable · speeds for a GeForce RTX 4060 everything that fits 8 GB →

Granite 4.1 8B is the pick: Enterprise-grade tool calling in a dense 8B, no thinking overhead. 5 other models on the agents shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 Granite 4.1 8B
IBM · Apache 2.0
Enterprise-grade tool calling in a dense 8B, no thinking overhead. 8.79B Q4_K_M · 6.8 GB 9K 33 Tight fit
2 Fara 7B
Microsoft · MIT
Web computer-use only — clicks and fills forms from screenshots, and stops to ask before anything irreversible. 8.29B Q4_K_M · 5.7 GB 31K 35 Runs great
3 Qwen3.5 9B
Alibaba · Apache 2.0
Tool use on 8–12 GB. 9.65B Q4_K_M · 6.3 GB 31K 30 Tight fit
4 Qwen3.5 4B
Alibaba · Apache 2.0
Function calling in 8 GB. 4.66B Q4_K_M · 3.5 GB 120K 63 Runs great
5 Ling 3.0 Tiny 7.9B-A1.3B
inclusionAI · MIT
Tool use and reasoning at 1.3B active, sized for edge boxes. 7.9B MoE Q4_K_M · 5.1 GB 128K 87 Runs great
6 Granite 4.1 3B
IBM · Apache 2.0
Small, dense, follows tool schemas. 3.4B Q4_K_M · 3.1 GB 57K 86 Runs great

Best agent LLM for 12 GB VRAM

10.6 GB usable · speeds for a GeForce RTX 3060 12 GB everything that fits 12 GB →

Granite 4.1 8B is the pick: Enterprise-grade tool calling in a dense 8B, no thinking overhead. 5 other models on the agents shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 Granite 4.1 8B
IBM · Apache 2.0
Enterprise-grade tool calling in a dense 8B, no thinking overhead. 8.79B Q4_K_M · 6.8 GB 32K 44 Runs great
2 Fara 7B
Microsoft · MIT
Web computer-use only — clicks and fills forms from screenshots, and stops to ask before anything irreversible. 8.29B Q4_K_M · 5.7 GB 97K 47 Runs great
3 Qwen3.5 9B
Alibaba · Apache 2.0
Tool use on 8–12 GB. 9.65B Q4_K_M · 6.3 GB 146K 40 Runs great
4 Qwen3.5 4B
Alibaba · Apache 2.0
Function calling in 8 GB. 4.66B Q4_K_M · 3.5 GB 236K 83 Runs great
5 Ling 3.0 Tiny 7.9B-A1.3B
inclusionAI · MIT
Tool use and reasoning at 1.3B active, sized for edge boxes. 7.9B MoE Q4_K_M · 5.1 GB 128K 115 Runs great
6 Granite 4.1 3B
IBM · Apache 2.0
Small, dense, follows tool schemas. 3.4B Q4_K_M · 3.1 GB 103K 114 Runs great

Best agent LLM for 16 GB VRAM

14.4 GB usable · speeds for a GeForce RTX 5080 everything that fits 16 GB →

gpt-oss 20B is the pick: Harmony-format tool calling, native 4-bit. 6 other models on the agents shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 gpt-oss 20B
OpenAI · Apache 2.0
Harmony-format tool calling, native 4-bit. 20.9B MoE MXFP4 · 11.6 GB 126K 120 Runs great
2 Granite 4.1 8B
IBM · Apache 2.0
Enterprise-grade tool calling in a dense 8B, no thinking overhead. 8.79B Q4_K_M · 6.8 GB 56K 118 Runs great
3 Fara 7B
Microsoft · MIT
Web computer-use only — clicks and fills forms from screenshots, and stops to ask before anything irreversible. 8.29B Q4_K_M · 5.7 GB 125K 125 Runs great
4 Qwen3.5 9B
Alibaba · Apache 2.0
Tool use on 8–12 GB. 9.65B Q4_K_M · 6.3 GB 256K 107 Runs great
5 Qwen3.5 4B
Alibaba · Apache 2.0
Function calling in 8 GB. 4.66B Q4_K_M · 3.5 GB 256K 222 Runs great
6 Ling 3.0 Tiny 7.9B-A1.3B
inclusionAI · MIT
Tool use and reasoning at 1.3B active, sized for edge boxes. 7.9B MoE Q4_K_M · 5.1 GB 128K 306 Runs great
7 Granite 4.1 3B
IBM · Apache 2.0
Small, dense, follows tool schemas. 3.4B Q4_K_M · 3.1 GB 128K 304 Runs great

Best agent LLM for 24 GB VRAM

22.4 GB usable · speeds for a GeForce RTX 4090 everything that fits 24 GB →

Qwen3.8 27B is the pick: Computer use and tool calling are what 3.8 was trained for (OSWorld 84). 11 other models on the agents shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 Qwen3.8 27B
Alibaba · Apache 2.0
Computer use and tool calling are what 3.8 was trained for (OSWorld 84). 27.8B Q4_K_M · 16.7 GB 98K 39 Runs great
2 Muse Glimmer 30B
Meta · Apache 2.0
Distilled specifically for always-on local agent workflows. 29.8B Q4_K_M · 17.5 GB 128K 36 Runs great
3 Qwen3.6 35B-A3B
Alibaba · Apache 2.0
Reliable function calling at MoE speed. 35.9B MoE Q4_K_M · 20.9 GB 82K 126 Tight fit
4 Nemotron 3.5 Lightning 30B-A3B
NVIDIA · OpenMDW-1.1
Made for the execution layer: high-volume tool calls at up to 4× the speed. 31.6B MoE Q4_K_M · 18.4 GB 256K 130 Runs great
5 GLM-4.7-Flash 30B-A3B
Z.ai · MIT
Agentic coding and tool use in a 30B-A3B, MIT. 31.2B MoE Q4_K_M · 18.6 GB 82K 139 Runs great
6 gpt-oss 20B
OpenAI · Apache 2.0
Harmony-format tool calling, native 4-bit. 20.9B MoE MXFP4 · 11.6 GB 128K 126 Runs great
7 Granite 4.1 8B
IBM · Apache 2.0
Enterprise-grade tool calling in a dense 8B, no thinking overhead. 8.79B Q4_K_M · 6.8 GB 107K 123 Runs great
8 Fara 7B
Microsoft · MIT
Web computer-use only — clicks and fills forms from screenshots, and stops to ask before anything irreversible. 8.29B Q4_K_M · 5.7 GB 125K 131 Runs great
9 Qwen3.5 9B
Alibaba · Apache 2.0
Tool use on 8–12 GB. 9.65B Q4_K_M · 6.3 GB 256K 112 Runs great
10 Qwen3.5 4B
Alibaba · Apache 2.0
Function calling in 8 GB. 4.66B Q4_K_M · 3.5 GB 256K 233 Runs great
11 Ling 3.0 Tiny 7.9B-A1.3B
inclusionAI · MIT
Tool use and reasoning at 1.3B active, sized for edge boxes. 7.9B MoE Q4_K_M · 5.1 GB 128K 321 Runs great
12 Granite 4.1 3B
IBM · Apache 2.0
Small, dense, follows tool schemas. 3.4B Q4_K_M · 3.1 GB 128K 319 Runs great

Best agent LLM for 32 GB VRAM

30.4 GB usable · speeds for a GeForce RTX 5090 everything that fits 32 GB →

Qwen3.8 27B is the pick: Computer use and tool calling are what 3.8 was trained for (OSWorld 84). 11 other models on the agents shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 Qwen3.8 27B
Alibaba · Apache 2.0
Computer use and tool calling are what 3.8 was trained for (OSWorld 84). 27.8B Q4_K_M · 16.7 GB 226K 69 Runs great
2 Muse Glimmer 30B
Meta · Apache 2.0
Distilled specifically for always-on local agent workflows. 29.8B Q4_K_M · 17.5 GB 128K 65 Runs great
3 Qwen3.6 35B-A3B
Alibaba · Apache 2.0
Reliable function calling at MoE speed. 35.9B MoE Q4_K_M · 20.9 GB 256K 225 Runs great
4 Nemotron 3.5 Lightning 30B-A3B
NVIDIA · OpenMDW-1.1
Made for the execution layer: high-volume tool calls at up to 4× the speed. 31.6B MoE Q4_K_M · 18.4 GB 256K 232 Runs great
5 GLM-4.7-Flash 30B-A3B
Z.ai · MIT
Agentic coding and tool use in a 30B-A3B, MIT. 31.2B MoE Q4_K_M · 18.6 GB 198K 247 Runs great
6 gpt-oss 20B
OpenAI · Apache 2.0
Harmony-format tool calling, native 4-bit. 20.9B MoE MXFP4 · 11.6 GB 128K 224 Runs great
7 Granite 4.1 8B
IBM · Apache 2.0
Enterprise-grade tool calling in a dense 8B, no thinking overhead. 8.79B Q4_K_M · 6.8 GB 128K 219 Runs great
8 Fara 7B
Microsoft · MIT
Web computer-use only — clicks and fills forms from screenshots, and stops to ask before anything irreversible. 8.29B Q4_K_M · 5.7 GB 125K 233 Runs great
9 Qwen3.5 9B
Alibaba · Apache 2.0
Tool use on 8–12 GB. 9.65B Q4_K_M · 6.3 GB 256K 200 Runs great
10 Qwen3.5 4B
Alibaba · Apache 2.0
Function calling in 8 GB. 4.66B Q4_K_M · 3.5 GB 256K 414 Runs great
11 Ling 3.0 Tiny 7.9B-A1.3B
inclusionAI · MIT
Tool use and reasoning at 1.3B active, sized for edge boxes. 7.9B MoE Q4_K_M · 5.1 GB 128K 571 Runs great
12 Granite 4.1 3B
IBM · Apache 2.0
Small, dense, follows tool schemas. 3.4B Q4_K_M · 3.1 GB 128K 567 Runs great

Best agent LLM for 48 GB VRAM

46.4 GB usable · speeds for an RTX 6000 Ada everything that fits 48 GB →

Qwen3.8 27B is the pick: Computer use and tool calling are what 3.8 was trained for (OSWorld 84). 11 other models on the agents shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 Qwen3.8 27B
Alibaba · Apache 2.0
Computer use and tool calling are what 3.8 was trained for (OSWorld 84). 27.8B Q4_K_M · 16.7 GB 256K 37 Runs great
2 Muse Glimmer 30B
Meta · Apache 2.0
Distilled specifically for always-on local agent workflows. 29.8B Q4_K_M · 17.5 GB 128K 35 Runs great
3 Qwen3.6 35B-A3B
Alibaba · Apache 2.0
Reliable function calling at MoE speed. 35.9B MoE Q4_K_M · 20.9 GB 256K 120 Runs great
4 Nemotron 3.5 Lightning 30B-A3B
NVIDIA · OpenMDW-1.1
Made for the execution layer: high-volume tool calls at up to 4× the speed. 31.6B MoE Q4_K_M · 18.4 GB 256K 124 Runs great
5 GLM-4.7-Flash 30B-A3B
Z.ai · MIT
Agentic coding and tool use in a 30B-A3B, MIT. 31.2B MoE Q4_K_M · 18.6 GB 198K 133 Runs great
6 gpt-oss 20B
OpenAI · Apache 2.0
Harmony-format tool calling, native 4-bit. 20.9B MoE MXFP4 · 11.6 GB 128K 120 Runs great
7 Granite 4.1 8B
IBM · Apache 2.0
Enterprise-grade tool calling in a dense 8B, no thinking overhead. 8.79B Q4_K_M · 6.8 GB 128K 118 Runs great
8 Fara 7B
Microsoft · MIT
Web computer-use only — clicks and fills forms from screenshots, and stops to ask before anything irreversible. 8.29B Q4_K_M · 5.7 GB 125K 125 Runs great
9 Qwen3.5 9B
Alibaba · Apache 2.0
Tool use on 8–12 GB. 9.65B Q4_K_M · 6.3 GB 256K 107 Runs great
10 Qwen3.5 4B
Alibaba · Apache 2.0
Function calling in 8 GB. 4.66B Q4_K_M · 3.5 GB 256K 222 Runs great
11 Ling 3.0 Tiny 7.9B-A1.3B
inclusionAI · MIT
Tool use and reasoning at 1.3B active, sized for edge boxes. 7.9B MoE Q4_K_M · 5.1 GB 128K 306 Runs great
12 Granite 4.1 3B
IBM · Apache 2.0
Small, dense, follows tool schemas. 3.4B Q4_K_M · 3.1 GB 128K 304 Runs great

Best agent LLM for 16 GB unified memory

10.7 GB usable · speeds for an M4 · 16 GB everything that fits 16 GB →

Granite 4.1 8B is the pick: Enterprise-grade tool calling in a dense 8B, no thinking overhead. 5 other models on the agents shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 Granite 4.1 8B
IBM · Apache 2.0
Enterprise-grade tool calling in a dense 8B, no thinking overhead. 8.79B Q4_K_M · 6.8 GB 33K 14 Runs great
2 Fara 7B
Microsoft · MIT
Web computer-use only — clicks and fills forms from screenshots, and stops to ask before anything irreversible. 8.29B Q4_K_M · 5.7 GB 99K 14 Runs great
3 Qwen3.5 9B
Alibaba · Apache 2.0
Tool use on 8–12 GB. 9.65B Q4_K_M · 6.3 GB 149K 12 Runs great
4 Qwen3.5 4B
Alibaba · Apache 2.0
Function calling in 8 GB. 4.66B Q4_K_M · 3.5 GB 239K 26 Runs great
5 Ling 3.0 Tiny 7.9B-A1.3B
inclusionAI · MIT
Tool use and reasoning at 1.3B active, sized for edge boxes. 7.9B MoE Q4_K_M · 5.1 GB 128K 43 Runs great
6 Granite 4.1 3B
IBM · Apache 2.0
Small, dense, follows tool schemas. 3.4B Q4_K_M · 3.1 GB 104K 35 Runs great

Best agent LLM for 64 GB unified memory

48.0 GB usable · speeds for an M4 Max · 64 GB everything that fits 64 GB →

Qwen3.8 27B is the pick: Computer use and tool calling are what 3.8 was trained for (OSWorld 84). 11 other models on the agents shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 Qwen3.8 27B
Alibaba · Apache 2.0
Computer use and tool calling are what 3.8 was trained for (OSWorld 84). 27.8B Q4_K_M · 16.7 GB 256K 20 Runs great
2 Muse Glimmer 30B
Meta · Apache 2.0
Distilled specifically for always-on local agent workflows. 29.8B Q4_K_M · 17.5 GB 128K 18 Runs great
3 Qwen3.6 35B-A3B
Alibaba · Apache 2.0
Reliable function calling at MoE speed. 35.9B MoE Q4_K_M · 20.9 GB 256K 77 Runs great
4 Nemotron 3.5 Lightning 30B-A3B
NVIDIA · OpenMDW-1.1
Made for the execution layer: high-volume tool calls at up to 4× the speed. 31.6B MoE Q4_K_M · 18.4 GB 256K 79 Runs great
5 GLM-4.7-Flash 30B-A3B
Z.ai · MIT
Agentic coding and tool use in a 30B-A3B, MIT. 31.2B MoE Q4_K_M · 18.6 GB 198K 84 Runs great
6 gpt-oss 20B
OpenAI · Apache 2.0
Harmony-format tool calling, native 4-bit. 20.9B MoE MXFP4 · 11.6 GB 128K 76 Runs great
7 Granite 4.1 8B
IBM · Apache 2.0
Enterprise-grade tool calling in a dense 8B, no thinking overhead. 8.79B Q4_K_M · 6.8 GB 128K 62 Runs great
8 Fara 7B
Microsoft · MIT
Web computer-use only — clicks and fills forms from screenshots, and stops to ask before anything irreversible. 8.29B Q4_K_M · 5.7 GB 125K 65 Runs great
9 Qwen3.5 9B
Alibaba · Apache 2.0
Tool use on 8–12 GB. 9.65B Q4_K_M · 6.3 GB 256K 56 Runs great
10 Qwen3.5 4B
Alibaba · Apache 2.0
Function calling in 8 GB. 4.66B Q4_K_M · 3.5 GB 256K 116 Runs great
11 Ling 3.0 Tiny 7.9B-A1.3B
inclusionAI · MIT
Tool use and reasoning at 1.3B active, sized for edge boxes. 7.9B MoE Q4_K_M · 5.1 GB 128K 195 Runs great
12 Granite 4.1 3B
IBM · Apache 2.0
Small, dense, follows tool schemas. 3.4B Q4_K_M · 3.1 GB 128K 160 Runs great

Best agent LLM for 128 GB unified memory

96.0 GB usable · speeds for an M4 Max · 128 GB everything that fits 128 GB →

Ling 3.0 Flash 124B-A5B is the pick: Agentic workflows are what Ling 3.0 was trained for; MIT, 5B active. 12 other models on the agents shortlist also fit, in order of capability:

#ModelWhyParamsQuant · totalMax contextTok/s est.
1 Ling 3.0 Flash 124B-A5B
inclusionAI · MIT
Agentic workflows are what Ling 3.0 was trained for; MIT, 5B active. 124B MoE Q4_K_M · 70.4 GB 256K 50 Runs great
2 Qwen3.8 27B
Alibaba · Apache 2.0
Computer use and tool calling are what 3.8 was trained for (OSWorld 84). 27.8B Q4_K_M · 16.7 GB 256K 20 Runs great
3 Muse Glimmer 30B
Meta · Apache 2.0
Distilled specifically for always-on local agent workflows. 29.8B Q4_K_M · 17.5 GB 128K 18 Runs great
4 Qwen3.6 35B-A3B
Alibaba · Apache 2.0
Reliable function calling at MoE speed. 35.9B MoE Q4_K_M · 20.9 GB 256K 77 Runs great
5 Nemotron 3.5 Lightning 30B-A3B
NVIDIA · OpenMDW-1.1
Made for the execution layer: high-volume tool calls at up to 4× the speed. 31.6B MoE Q4_K_M · 18.4 GB 256K 79 Runs great
6 GLM-4.7-Flash 30B-A3B
Z.ai · MIT
Agentic coding and tool use in a 30B-A3B, MIT. 31.2B MoE Q4_K_M · 18.6 GB 198K 84 Runs great
7 gpt-oss 20B
OpenAI · Apache 2.0
Harmony-format tool calling, native 4-bit. 20.9B MoE MXFP4 · 11.6 GB 128K 76 Runs great
8 Granite 4.1 8B
IBM · Apache 2.0
Enterprise-grade tool calling in a dense 8B, no thinking overhead. 8.79B Q4_K_M · 6.8 GB 128K 62 Runs great
9 Fara 7B
Microsoft · MIT
Web computer-use only — clicks and fills forms from screenshots, and stops to ask before anything irreversible. 8.29B Q4_K_M · 5.7 GB 125K 65 Runs great
10 Qwen3.5 9B
Alibaba · Apache 2.0
Tool use on 8–12 GB. 9.65B Q4_K_M · 6.3 GB 256K 56 Runs great
11 Qwen3.5 4B
Alibaba · Apache 2.0
Function calling in 8 GB. 4.66B Q4_K_M · 3.5 GB 256K 116 Runs great
12 Ling 3.0 Tiny 7.9B-A1.3B
inclusionAI · MIT
Tool use and reasoning at 1.3B active, sized for edge boxes. 7.9B MoE Q4_K_M · 5.1 GB 128K 195 Runs great
13 Granite 4.1 3B
IBM · Apache 2.0
Small, dense, follows tool schemas. 3.4B Q4_K_M · 3.1 GB 128K 160 Runs great

The whole agents shortlist, and what each one needs

Every model on the list, best first, with the smallest discrete card and the smallest Mac that run it with 15% headroom at the recommended quantisation and 8K context. Every class has its own page, Macs from 8 to 512 GB included — the full list.

#ModelParamsWeightsNeeds (GPU)Needs (Mac)Ollama tag
1 DeepSeek V4.1 Flash 552B-A16B New
Twice V4 Flash's backbone (552B, 16B active per generated token) plus 196B of Engram lookup tables (189 GiB at FP8) that DwarfStar streams from SSD, so they are not counted here. A 512 GB Mac model, and as of October 2026 no mainline llama.cpp or Ollama build: DwarfStar on a Mac, vLLM across four GPUs. DeepSeek puts the cache at 890 bytes/token; modelled conservatively as 4 latent layers plus a 128-token window.
552B MoE 310.4 GB @ Q4_K_M more than one card 512 GB Mac —
2 Kimi K2.6 1T-A32B
The open coding-agent benchmark leader of spring 2026 (80.2 SWE-bench). A 512 GB Mac Studio pair, or a ceiling.
1027B MoE 577.5 GB @ Q4_K_M more than one card no single Mac —
3 GLM-5.3 744B-A40B
Same base as GLM-5.2, new post-training: Z.ai's most capable open-weights coder (Terminal-Bench 3.0 28.3, DeepSWE 66.9). Not MIT any more — its own GLM-5.3 licence. Same ceiling as 5.2: 512 GB of unified memory at Q4.
753B MoE 423.4 GB @ Q4_K_M more than one card no single Mac —
4 GLM-5.3-Flash 320B-A18B
The first natively multimodal GLM-5 and the first hybrid: 34 linear-attention blocks and 11 sparse-attention blocks with a 512-wide latent cache, so a 1M window stays affordable. Z.ai says it beats GLM-5.2 at 18B active; MIT.
321B MoE 180.5 GB @ Q4_K_M more than one card 512 GB Mac —
5 DeepSeek V4 Flash 284B-A13B
The V4 for 192–256 GB machines: Q3 squeezes into 192 GB, Q4 wants 256 GB, and 128 GB falls short even at Q2. Cache is modelled as a 576-wide latent; V4 compresses it further at long context, so this is conservative.
284B MoE 159.7 GB @ Q4_K_M 192 GB card 256 GB Mac —
6 Qwen3.8-Flash-Next 180B-A6B
The open preview of the Qwen4 architecture: a 125B-A6B hybrid (Gated DeltaNet + sparse attention, KV cache on 12 of 48 blocks) plus a 51B n-gram embedding and a 4B draft head — 180B on disk, 6B active. The hosted "Qwen3.8-Flash" is this model with a 1M window.
180B MoE 101.2 GB @ Q4_K_M 141 GB card 192 GB Mac qwen3.8-flash-next:125b-a6b-nvfp4
7 Ling 3.0 Flash 124B-A5B
A 124B hybrid (5 linear-attention layers per MLA layer) with 5.1B active: SWE-bench Pro 56.6 and AIME 93 claimed. Built for 96–128 GB machines.
124B MoE 69.7 GB @ Q4_K_M 141 GB card 128 GB Mac —
8 Qwen3.8 27B
The current default local Qwen: dense 27B, text + image + video, 262K context. Only 16 of its 64 blocks keep a KV cache, so long context is cheap.
27.8B 15.6 GB @ Q4_K_M 24 GB card 32 GB Mac qwen3.8:27b
9 Muse Glimmer 30B
Meta's first open weights since Llama 4: a dense 30B distilled from Muse Spark for always-on local agents. Two KV heads keep the cache small.
29.8B 16.8 GB @ Q4_K_M 24 GB card 32 GB Mac muse-glimmer:30b
10 Qwen3.6 35B-A3B
Mixture of experts with ~3B active: the fastest serious model a 24 GB card runs, and the best MoE under 40B on agentic coding.
35.9B MoE 20.2 GB @ Q4_K_M 32 GB card 36 GB Mac qwen3.6:35b-a3b
11 Nemotron 3.5 Lightning 30B-A3B
Mamba-2 + MoE hybrid built for the execution layer of agents: only 6 attention blocks, so the KV cache is almost free. Weights, data and recipe all open.
31.6B MoE 17.8 GB @ Q4_K_M 24 GB card 32 GB Mac nemotron-3.5-lightning:30b-a3b
12 GLM-4.7-Flash 30B-A3B
MIT-licensed 30B-A3B tuned for agentic coding, with a DeepSeek-style latent KV cache. 60–80 tok/s reported on a 4090.
31.2B MoE 17.5 GB @ Q4_K_M 24 GB card 32 GB Mac glm-4.7-flash:latest
13 gpt-oss 20B
Ships natively in MXFP4, so the 4-bit weights are the reference weights, not a lossy copy. Fits 16 GB.
20.9B MoE 10.8 GB @ MXFP4 16 GB card 24 GB Mac gpt-oss:20b
14 Granite 4.1 8B
Matches the old Granite 4.0 32B MoE at a quarter of the size. Fast, Apache 2.0, no reasoning traces.
8.79B 4.9 GB @ Q4_K_M 10 GB card 16 GB Mac granite4.1:8b
15 Fara 7B
A web computer-use agent on a Qwen2.5-VL base — it clicks, fills forms and stops for permission. Not a chat model; size it like an 8B with vision.
8.29B 4.7 GB @ Q4_K_M 8 GB card 16 GB Mac —
16 Qwen3.5 9B
The default for 8–12 GB cards in 2026: beats every older 8B on every published benchmark, with vision.
9.65B 5.4 GB @ Q4_K_M 10 GB card 16 GB Mac qwen3.5:9b
17 Qwen3.5 4B
The 8 GB coding agent. Q4 lands near 3.4 GB, leaving room for a real context window.
4.66B 2.6 GB @ Q4_K_M 6 GB card 8 GB Mac qwen3.5:4b
18 Ling 3.0 Tiny 7.9B-A1.3B
An 8B MoE with 1.3B active and a latent KV cache on only 6 of 24 layers — reasoning and tool use sized for Apple Silicon and edge boxes.
7.9B MoE 4.4 GB @ Q4_K_M 8 GB card 16 GB Mac —
19 Granite 4.1 3B
Dense, small, enterprise-flavoured: tool calling and instruction following, no thinking mode.
3.4B 1.9 GB @ Q4_K_M 6 GB card 8 GB Mac granite4.1:3b

Tool calling depends on the chat template as much as on the weights. Start llama-server with --jinja so the model's own template formats the calls; Ollama and LM Studio do this for you. A model that fits with headroom but fumbles its JSON usually has a template problem, not a capability one.

Other jobs