Mixtral 8x7B & 8x22B VRAM Requirements

📚 More on this topic: VRAM Requirements Guide · Mistral & Mixtral Guide · Quantization Explained · What Can You Run on 24GB VRAM

Mixtral models have some of the most confusing VRAM requirements in local AI. “8x7B” sounds like it should need 7B worth of memory. It doesn’t. It needs closer to 47B worth. And “8x22B” isn’t a 22B model — it’s 141B parameters that all needed to live in VRAM at once.

That “all of it in VRAM” rule is the thing worth understanding, because it’s exactly what Mixtral taught a generation of local-AI users in 2024 — and it’s exactly what stopped being true in 2026. Mixtral is retired now (Mistral officially deprecated 8x7B and 8x22B), so this page does two jobs: it keeps the exact Mixtral numbers as a reference for anyone still running these models, and it uses Mixtral as the “before” that makes the current MoE-VRAM story land. Because a larger MoE than Mixtral now runs on a 12GB card — and understanding why is the whole point.

This guide gives you the exact numbers at every quantization level, then the rule that replaced them.

Why MoE VRAM Is Confusing

How Mixture of Experts Works

Mixtral uses a Mixture of Experts (MoE) architecture. Instead of one monolithic feedforward network, the model has 8 separate “expert” networks. For each token, a router selects 2 experts to process it. The other 6 sit idle.

This means Mixtral 8x7B has 46.7 billion total parameters but only activates 12.9 billion per token. Inference speed scales with active parameters — so it runs roughly as fast as a 13B dense model.

Here’s the catch that tripped everyone up in 2024: all 46.7B parameters had to be loaded into VRAM. The router can’t predict which experts it’ll need for the next token, so every expert stayed resident in GPU memory. You paid 47B worth of VRAM for 13B worth of compute.

That was the accepted wisdom — and it’s no longer how the best tooling works. See The A3B-Era Correction below: the weights still have to live somewhere, but “somewhere” can now be system RAM, with only the active experts streamed to the GPU per token. Keep reading for the exact Mixtral numbers first — they’re still accurate for Mixtral — then the rule that supersedes them.

The Numbers That Matter

This is the fundamental tradeoff of MoE: you get the speed of a smaller model and the quality of a larger one, but the VRAM cost of the larger one.

Mixtral 8x7B VRAM Requirements

Every quantization level, with real file sizes from GGUF builds:

*VRAM needed = model size + ~2-3GB for KV cache (4K context) + framework overhead. Longer context adds more.

What GPU Can Run 8x7B?

The honest assessment: Mixtral 8x7B is awkward on consumer hardware. It’s too big for 24GB at good quality, and if you’re buying 32GB+ hardware, a dense model like Qwen 3.6-27B gives you better quality at Q4_K_M (~17GB) with VRAM to spare.

One important caveat on those “won’t fit” rows: they’re true for Mixtral, which predates modern expert-offload. They are not a general ceiling for MoE anymore — a newer, larger MoE runs fine on that same 12GB card by keeping idle experts in system RAM. That’s the A3B-era correction below.

→ Use our Planning Tool to check exact VRAM for your setup.

Mixtral 8x22B VRAM Requirements

The big one. Here’s what you’re looking at:

*Assumes 4K context. 8x22B’s 64K context window at full length adds substantially more.

What GPU Can Run 8x22B?

Mixtral 8x22B is a datacenter model that people try to run on consumer hardware. Unless you have 96GB+ of GPU memory across multiple cards, look at Llama 3.1 70B instead — it fits on 2x 24GB GPUs at Q4_K_M (~40GB) and delivers comparable quality.

How Context Length Eats Your VRAM

The tables above assume short context (~4K tokens). But Mixtral 8x7B supports 32K and 8x22B supports 64K. The KV cache grows linearly with context length and eats into your available VRAM.

Here’s the key insight: KV cache scales with active parameters, not total parameters. Since only 2 experts run per token, the KV cache for 8x7B scales like a ~13B model, and for 8x22B like a ~39B model. This is one area where MoE actually helps.

KV Cache VRAM by Context Length

For a deeper dive into how context length affects VRAM, see our context length explainer.

At 32K context on Mixtral 8x7B with Q4_K_M, you need ~26GB (model) + ~4GB (KV cache) + ~2GB (overhead) = ~32GB total. That’s exactly one RTX 5090. On a 24GB card, you’re limited to roughly 4K-8K tokens before running out of memory.

The A3B-Era Correction: Why a Bigger MoE Now Fits a 12GB Card

Everything above is accurate for Mixtral. But if you take the headline lesson — “an MoE costs you the VRAM of its total parameter count” — and apply it to a 2026 model, you’ll get the wrong answer. Here’s what changed.

The trick is expert offload. In llama.cpp you pass --n-cpu-moe N (it’s -ncmoe in llama-bench), which keeps the attention layers and shared weights on the GPU but leaves the expert weights in system RAM. Because only a couple of experts fire per token, the engine streams just those over PCIe when they’re needed. The weights still have to live somewhere — VRAM or RAM — but they no longer all have to live in VRAM. Total size stops being the ceiling; active parameters plus an offload path decide whether a model is reachable and how fast it runs.

This is why the generation that replaced Mixtral flipped the math. The current sparse MoE models are built around tiny active-parameter counts:

Take Qwen 3.6-35B-A3B: it’s larger than Mixtral 8x7B in total, but with only ~3B active it runs on a 12GB RTX 3060 by offloading experts to RAM — the exact card the table above says “won’t fit at any useful quantization” for Mixtral. A dense model of the same total size collapses under the same offload (it has to stream every parameter each token); the A3B barely notices, because only 3B move per token. We measured this end to end on a 3060: running Qwen 3.6-35B-A3B on a 12GB card has the -ncmoe sweep and the token/sec numbers.

So the corrected rule of thumb for MoE VRAM in 2026:

• VRAM the model needs to be fast: roughly the active-parameter footprint plus KV cache and shared weights — small.
• Memory the model needs to exist: the full total-parameter footprint, but that can be VRAM or system RAM.
• What sets your token/sec: active params and PCIe bandwidth, not total size.

The full per-size numbers and the canonical --cpu-moe rule live in our VRAM Requirements guide; this is the short version, told through the model that created the confusion in the first place.

Mixtral vs Dense Models: The Honest Comparison

This is where people should pay close attention. MoE models made sense when there weren’t good dense alternatives at the same quality tier. In 2026, the landscape has changed — and the live alternatives are no longer other Mixtrals but the A3B-class models above.

Mixtral 8x7B vs Dense Alternatives

Mixtral 8x7B performs roughly like a dense 30B model on benchmarks. Here’s how the VRAM compares:

Qwen 3.6-27B fits on a single 24GB GPU at Q4_K_M with plenty left for context, and beats Mixtral 8x7B on most benchmarks. If you specifically want the MoE speed profile, its own generation now offers it directly: Qwen 3.6-35B-A3B is the live successor — sparser, faster, and (via offload) reachable on far less VRAM than Mixtral ever was. Unless you have legacy Mixtral infrastructure, the current-gen models are the better choice on the same hardware.

For more on these alternatives, see our Qwen Models Guide and DeepSeek Models Guide.

Mixtral 8x22B vs Dense Alternatives

Mixtral 8x22B competes with Llama 3.1 70B. The 64K context window is its main advantage.

Llama 3.1 70B needs less than half the VRAM and has double the context window. On raw benchmarks, it’s competitive or better. The only scenario where 8x22B wins is if you’ve invested in the infrastructure and need its specific MoE speed characteristics — inference speed roughly matches a 39B dense model, which is faster than a 70B dense model on the same hardware.

And if what you actually want is a big MoE done right, that’s now a different model entirely: DeepSeek V4 Flash (284B total / 13B active) and V4 Pro (1.6T / 49B) carry the large-MoE torch in 2026, with far better active-parameter efficiency than 8x22B’s 39B. See our DeepSeek Models Guide for where those land.

Best Quantization Sweet Spots

For Mixtral 8x7B

For Mixtral 8x22B

When Mixtral Still Makes Sense

After all that, you might wonder why anyone runs Mixtral in 2026. There are a few valid reasons:

Inference speed. MoE’s core advantage is speed at quality. Mixtral 8x7B gives you 30B-class quality at 13B-class speed. If you have the VRAM and need fast responses, that’s a real benefit. On a 48GB card, 8x7B at Q6_K generates tokens noticeably faster than a dense 30B at the same quantization.

Fine-tuned variants. There’s a large ecosystem of Mixtral fine-tunes for specific tasks — roleplay, coding, instruction following. If a fine-tune exists for your exact use case and nothing comparable exists for dense models, that’s a reason.

8x22B’s 64K native context. If you need long context on self-hosted infrastructure and have the hardware, 8x22B at Q4 handles it well. But Llama 3.1 70B at 128K context on less VRAM is usually the better choice.

Existing deployments. If your setup already runs Mixtral and works, there’s no urgent reason to migrate. The model hasn’t gotten worse — the alternatives have just gotten better. That said, even this case now has A3B-class successors that do the same MoE job on less hardware, so a migration usually pays for itself.

The Bottom Line

Mixtral 8x7B needs ~26-29GB at Q4_K_M. It won’t fit on a 24GB GPU with usable context. It’s also retired — if you have 24GB and want that quality tier, run Qwen 3.6-27B; if you want the MoE speed profile, run Qwen 3.6-35B-A3B, which reaches down to a 12GB card via expert offload.

Mixtral 8x22B needs ~86-88GB at Q4_K_M. It’s a datacenter model, and the modern large-MoE crown has passed to DeepSeek V4. If you’re keeping a dense 70B on hand, Llama 3.3 70B fits 2x 24GB at Q4_K_M.

Here’s the real takeaway, and it’s bigger than Mixtral: the thing Mixtral taught everyone — “an MoE costs you the VRAM of its full parameter count” — is no longer true. With expert offload, total size sets what memory the model occupies (VRAM or RAM), while active parameters set how fast it runs. That’s why a 35B MoE now fits a card that couldn’t hold Mixtral. Mixtral was the model that created the confusion; the A3B generation is the one that resolved it.

For GPU buying recommendations at every price point, see our GPU Buying Guide and Used RTX 3090 Buying Guide.