Yes — with 19.8 GB to spare

GLM-4.7-Flash 30B-A3B at Q4_K_M fits your A100 40 GB entirely on the GPU at 8K context, at an estimated 215 tokens per second. There is room for its full 198K window.

Fully on GPU 8K context Q4_K_M · 17.5 GB MIT Released 20 Jan 2026

MIT-licensed 30B-A3B tuned for agentic coding, with a DeepSeek-style latent KV cache. 60–80 tok/s reported on a 4090.

What hardware do I need for GLM-4.7-Flash 30B-A3B? →

The VRAM budget

weights 17.5 GB

Weights 17.5 GB KV cache @ 8K 0.41 GB Runtime overhead 0.6 GB Free 19.8 GB of 38.4 GB

Quantisation ladder

Quant	Weights	Total @ 8K	Max context	Tok/s	Quality	Fit
Q8_0	30.9 GB	31.9 GB	133K	122	−0.1% ppl	Long context
Q6_K	23.8 GB	24.8 GB	198K	158	−0.4% ppl	Long context
Q5_K_M	20.6 GB	21.6 GB	198K	183	−0.8% ppl	Long context
Q4_K_M	17.5 GB	18.6 GB	198K	215	−1.9% ppl	Recommended
Q3_K_M	14.2 GB	15.2 GB	198K	265	−5.4% ppl	Long context
Q2_K	12.2 GB	13.2 GB	198K	309	−15% ppl	Long context

Quality is the published perplexity delta against f16 weights. Max context assumes an f16 KV cache; q8_0 roughly doubles it. This model uses multi-head latent attention, so its cache is a compressed latent rather than full K and V.

How to run it

Ollama llama.cpp LM Studio vLLM

terminal

$ ollama pull glm-4.7-flash:latest
$ OLLAMA_CONTEXT_LENGTH=8192 \
    ollama run glm-4.7-flash:latest

The default. One binary, a model registry, an OpenAI-compatible port. More on Ollama.

01Download is 17.5 GB. Keep it on an SSD — a first load off a spinning disk takes minutes.

02Close anything else holding VRAM. A browser with hardware acceleration can sit on 1–2 GB.

03There is room to go to the model's full 198K context on this card.

See all models for this rig Compare with another model