Best Way to Run Qwen 3.6 35B MoE Locally: VRAM, Speed, Setup

📚 More on this topic: Complete Qwen Models Guide · Qwen 3.6 Complete Guide · MoE Models Explained · Best Local Coding Models · VRAM Requirements · llama.cpp vs Ollama vs vLLM

If you have 24GB VRAM and you’ve been running Qwen 3.6-27B dense, here’s the question. Would you trade for the MoE 35B-A3B?

The honest answer is “it depends, and the dependencies are not what you’d guess.” More total parameters. Fewer active. Different speed profile. Different tool-use behavior. And the DFlash 2x speedup for the 27B dense doesn’t work on the MoE — though, as of mid-July, llama.cpp’s built-in MTP finally gives the MoE a modest speedup of its own (more on that below).

This piece is the hardware-reality view of the 35B MoE. What runs where, how fast, and the setup commands that actually work.

Measured: the 35B-A3B on an RTX 3090 (July 2026)

This page used to carry ~101 tok/s for a 3090, sourced from the Unsloth model card and Amine Raji’s writeup. We’ve now run it on Miu, our own 3090, and it does 157.66 tok/s. That’s 56% above the figure we’d been repeating.

The finding nobody seems to state plainly, though, is that the whole model fits on the card. No --cpu-moe, no -ncmoe, no offload of any kind. Peak VRAM was 21,652 MiB against the 24,123 MiB a headless 3090 reports free, leaving 2,471 MiB (just under 2.4 GiB) still spare with 8K of context already in the cache. A 35B model, entirely resident on a consumer card, with room left over.

Before you compare that 157.66 against anything else on this page: it’s a llama-bench tg128 figure at up to 8K depth with default KV cache. The community numbers in the hardware table below come from llama-server sessions at 65K context with q8_0 KV quantization, which is heavier and more realistic work. Some of the 56% is our rig. Some of it is that a synthetic decode benchmark flatters everyone. Both numbers are honest and they answer different questions.

The sweep: 11 offload settings, 5 repetitions each

Same method as the 3060 sweep further down — -ngl 99 pins attention and shared weights on the GPU, --n-cpu-moe N pushes the routed experts of N layers out to system RAM. On a 24GB card you want N as low as it will go, and on this card it goes all the way to zero.

Read the top two rows together and one layer of offload looks brutal: 11.2 tok/s gone. An earlier version of this section called that the price of the offload path existing at all, a fixed cost you pay before any question of how much data moves. That was wrong, and the table’s own numbers say so.

Convert tok/s to milliseconds per token and the sweep is a straight line. Every offloaded layer adds about 0.53 ms, and it keeps adding the same 0.53 ms the whole way down: 0.49 ms for the first layer, 0.61 for each of the last eight, 6.34 ms at -ncmoe 0 against 27.69 at -ncmoe 40. A line fitted through all eleven rungs accounts for 99.9% of the variance. There is no knee and no threshold. The first layer is slightly cheaper than the average one, not more expensive.

tok/s is a reciprocal scale, which is the whole illusion. The same 0.53 ms costs 11.2 tok/s when you are sitting at 157 and 0.7 tok/s when you are down at 36. Nothing changes about the hardware between those two points.

The practical advice survives, for a different reason: two layers of “just a bit of headroom” costs about 1 ms per token, 14% of your throughput on a card that never needed to give it up.

And the fully resident config holds up as context grows:

Down 4.4 tok/s across 8K, and the 21,652 MiB peak above already includes that 8K of KV cache. There’s no -ncmoe 20-style trap here the way there is on the 3060: the config that’s fastest is also the config with the most headroom left.

The desktop-session tax: two configs that don’t exist on your daily driver

We ran this sweep twice. The first pass ran with a normal Xubuntu desktop up, which is how most people actually use their machine. A desktop session holds about 4.4 GiB of VRAM, and that changes the answer at the top of the table:

So the headline config is not available to you if you’re running this on the machine you’re looking at. On a 3090 driving a monitor, -ncmoe 4 is the practical top of the table, and it costs about 10% against headless. Free that 4.4 GiB, whether by adding a cheap second GPU for display or just SSHing in with the session stopped, and you get the fully resident config back.

Two details worth separating. The -ncmoe 0 failure and the -ncmoe 2 failure are not the same event: the first never loaded the weights, the second loaded them and then had nothing left for the KV cache. And -ncmoe 2 is not a hardware boundary. It runs fine at 136.03 headless, and it looked like a boundary only until we re-ran without the desktop.

The other thing a desktop session costs is consistency. Error bars on generation went from ±0.02-0.97 tok/s headless to ±0.64-13.47 with the session up. Your compositor and your browser are contending for the same card.

Where a $1,200 card loses to a $300 card

Look at the bottom of the sweep table. At -ncmoe 40, every expert in system RAM, the 3090 does 36.11 tok/s. The 12GB RTX 3060 in the section below, running its recommended -ncmoe 24, does 38.9.

A used 3090 runs about $1,200 and a used 3060 12GB about $300 (our eBay sold-listing tracking, July 2026). Configured badly, the expensive card loses to the cheap one. Not by much, but it loses, and it loses while sitting on 24GB of VRAM with 3GB of it in use.

That’s the active-parameter rule with a number attached. Once every expert lives in system RAM, the thing being measured is RAM bandwidth and PCIe, not the GPU. The 3090’s memory subsystem waits on DDR4 either way. Buy the bigger card to keep more of the model on it. The value is in -ncmoe 0, not in a faster card doing the same streaming.

The confound, measured rather than assumed: these two boxes don’t have matched memory, so I benchmarked both instead of quoting spec sheets. STREAM-style kernels, identical source compiled on each box, 320 MB arrays (well past either L3), best of 8 runs. Sustained read is the figure that matters here, because streaming experts out of RAM is pure load with no write traffic.

Miu peaks at 33.32 GB/s, Rushuna at 27.32 GB/s, which is 78% and 80% of their theoretical dual-channel numbers, normal for DDR4. That gives a measured ratio of 1.220, not the 1.25 the spec sheets imply.

Now put the throughput next to it. At full offload Miu does 36.11 and Rushuna 28.1, a ratio of 1.285, slightly wider than the bandwidth ratio rather than narrower. So the tidy story I told earlier, that the two curves converge toward the RAM ratio, was wrong in its details. They don’t converge on it; they overshoot it a little.

The stronger evidence is what each box does against its own ceiling. Qwen 3.6-35B-A3B has 40 layers, 256 experts, top-8 routing; summing the actual routed-expert tensor bytes in the GGUF gives 19.57 GB, and at 8 of 256 experts per layer that’s 611.5 MB streamed per token. Multiply out:

Two different CPUs, different core counts, different memory speeds, landing within three points of each other as a fraction of their own measured bandwidth. That’s a much better argument that the wall is memory than any ratio comparison, and it’s why “at max offload you’re measuring RAM, not the GPU” holds. The remaining third isn’t slack. STREAM measures sequential streaming; expert routing gathers 8 scattered tensors out of 256 per layer, and 63-66% of sequential peak is about what random access gets you.

Reproducing the 3090 numbers:

GPU:     RTX 3090 24GB, driver 580.159.03, PCIe 3.0 x16 (B365 chipset caps the card at Gen3)
CPU/RAM: i7-8086K (6c/12t), 64GB DDR4-2667 dual channel (4x16GB Samsung, ~43 GB/s)
Backend: llama.cpp b10088 (commit 67b9b0e), CUDA sm_86, flash attention on
Model:   Qwen3.6-35B-A3B, unsloth UD-Q4_K_M (20.6 GiB weights)
State:   headless (lightdm stopped), MemAvailable ~60 GiB, swap 100% free,
         no thermal or clock throttling observed (59-73C, 1800-1920 MHz)

  llama-bench -m Qwen3.6-35B-A3B-UD-Q4_K_M.gguf \
    -ngl 99 -fa 1 -p 512 -n 128 -d 0,4096,8192 -r 5

Note the -r 5. The 3060 sweep below ran -r 3. More repetitions is strictly better and it’s why the error bars on this run are so tight, but it is a difference between the two rigs and not one worth hiding. Every row from both sweeps is in the open benchmark dataset with its repetition count attached.

Low-VRAM, measured: the 35B on a 12GB RTX 3060 (July 2026)

The budget-card question deserved a firsthand answer, so we ran Qwen3.6-35B-A3B on Rushuna, our RTX 3060 12GB — a $200 used Ampere card with 32GB of ordinary DDR4 behind it. It holds ~38 tok/s, flat through 8K of context, on a model whose weights never fully fit the card. That’s well above the ~12-18 tok/s the community had been estimating for this card, and the gap is all in the config.

The mechanism is the A3B math: only ~3B of the 35B params fire per token, so you pin attention and the shared weights on the GPU with -ngl 99, then use --n-cpu-moe N to push the routed experts of N layers out to system RAM. Only the handful of experts a token actually selects gets streamed per step. You want the lowest N that still fits 12GB, because every layer of experts kept on the card is faster. The sweep on Rushuna (40 layers total):

-ncmoe 20 is the fastest that loads — 42.6 tok/s — but treat it as a mirage. At 11.7 GB it has no room for the KV cache to grow, and it threw an out-of-memory error the moment we fed it real context. Run -ncmoe 24, which holds steady under load:

Flat — 0.7 tok/s lost across 8K, because the expert-streaming cost per token is constant and flash attention keeps the KV growth cheap. One caveat that moves the number: Rushuna is headless, so all 12GB is free. A 3060 also driving your desktop loses ~0.5-1 GB to the display, enough to push -ncmoe 24 into an OOM — bump it to 26 or 28 and give up a couple tok/s for the headroom.

Why the offload barely costs anything (and why dense would die)

This works because a dense model has no equivalent escape hatch, and the same card proves it. A dense 14B (Qwen3-14B Q4_K_M, 8.4 GB) fits VRAM fine — then forced to offload its layers to RAM:

Move half a dense model’s layers off the card and it falls from 35.9 to 5.7 tok/s, a 6.3x drop, because a dense model runs every parameter on every token and each offloaded layer streams its full weights across PCIe each step. Push all of them to RAM and it’s 3.05, down 11.8x. The MoE doesn’t work this way: it streams only the selected experts, a few hundred MB per token instead of gigabytes. Same card, same DDR4-2133 (~34 GB/s), same PCIe 3.0. The only difference is how much moves per token, and that’s set by active parameters, not total size.

Line those two up and the MoE looks better than a straight speed comparison suggests. The 35B-A3B holds 38.9 tok/s with the experts of 24 of its 40 layers already sitting in system RAM. The dense 14B manages 35.9 with every layer on the card and nothing left over. Call that a tie on speed, except one of them is carrying 2.5x the parameters to get there, from a position the other can’t survive. Take VRAM away from the dense 14B and it drops 6.3x, then 11.8x once the last layers go. Push every one of the MoE’s experts out to RAM and it still holds 28.1 tok/s, a dip of 1.4x. What the MoE buys you is 35B-class capability at 14B-class speed, plus a graceful failure mode when the card runs out of room.

Scale the dense case up and it’s the honest answer for big dense models on this card: a dense 32B at Q4 lands around 2-3 tok/s, a dense 70B around 1. That’s the cliff the MoE steps around. (The 14B baseline is our own measurement, and it lands about 59% above Hardware Corner’s ~22.6 tok/s for a 14B on the same card. That gap is real but it isn’t like-for-like: different 14B, different settings. Treat theirs as an independent check that a 3060 belongs in this range, and ours as the number with every flag documented.)

Squeezing further, and the floor below the 3060

A few flags buy more context or headroom on top of the base config, all mainline llama.cpp:

• --no-mmap forces the model fully into RAM upfront and avoids page-fault stalls during inference.
• TurboQuant KV cache quantization (K=4 bits, V=3 bits, asymmetric for the 8:1 grouped-query attention) stretches context hard — community reports take a constrained rig from 64K to 256K with negligible quality drop and no speed loss.
• mlock + Docker IPC_LOCK stops the kernel paging expert weights back to disk, which matters for week-long uptime.

The floor goes lower than a 3060. YouTube creator Codacus demonstrated the 35B-A3B on an 8-year-old GTX 1060 6GB with 24GB DDR4 at 17 tok/s and 256K context, production-stable — --n-cpu-moe alone moved that box from 3 to 10 tok/s before the other flags.

What doesn’t help: a separate speculative-decoding drafter. Qwen 3.5 0.8B as drafter dropped throughput to 11 tok/s despite a 65% accept rate — MoE batched verification pulls from up to 64 experts per layer (memory thrash), and the state-space layers can’t parallelize across a draft window. The model’s own MTP head is a different story (see Honest limits). If you’re on the Qwen 3.6-27B dense instead, that’s where DFlash applies — see DFlash on RTX 3090 and the DFlash vs MTP head-to-head.

Reproducing the 3060 numbers:

GPU:     RTX 3060 12GB, driver 580.159.03, PCIe 3.0 x16
CPU/RAM: i7-7700 (4c/8t), 32GB DDR4-2133 (~34 GB/s)
Backend: llama.cpp b10088 (commit 67b9b0e), CUDA sm_86, flash attention on
Model:   Qwen3.6-35B-A3B, unsloth UD-Q4_K_M (20.6 GiB weights)

  llama-bench -m Qwen3.6-35B-A3B-UD-Q4_K_M.gguf \
    -ngl 99 -ncmoe 24 -fa 1 -p 512 -n 128 -d 0,4096,8192 -r 3

RAM speed matters as much as the card on an offload build — DDR4-2133 here is on the slow end, so a DDR5 box will beat these numbers. It’s also why you shouldn’t line these figures up directly against the 3090 sweep above: that box runs DDR4-2667. Measured rather than assumed, that’s 33.32 GB/s of sustained read against this box’s 27.32, a 1.22x advantage on top of the card. At full offload the two rigs land 1.285x apart, so the gap is a shade wider than memory alone explains — but both boxes reach 63-66% of their own measured ceiling, which is the number that actually says the wall is RAM.

What “35B-A3B” actually means

A3B is the part that matters. The model holds 35 billion parameters in memory. For each token it predicts, it routes through 8 of 256 experts plus 1 shared expert. That’s roughly 3 billion parameters worth of compute per forward pass. You pay memory like a 35B and compute like a 3B. That asymmetry is the whole MoE pitch.

Two consequences that change your hardware decision:

• More VRAM than 27B dense. All 35B of weights have to live somewhere. UD-Q4_K_M is 22.1 GB versus ~17 GB for the 27B Q4, the same file llama.cpp reports as 20.6 GiB in the bench block above. Tighter on a 24GB card once you add KV cache — but as our 3090 sweep shows, it does fit, with 2,471 MiB left at 8K context.
• Faster generation than a 27B dense at the same quant. 3B-active means the GPU does roughly 3B’s worth of math per token. We measure the 35B-A3B at 157.66 tok/s on an RTX 3090; the 27B dense on the same card lives in the 33-42 tok/s range depending on harness. Several times faster, despite being a bigger model on disk.

If MoE architecture is new to you, the MoE models explained primer covers the routing logic and why the 256-expert design works for local inference. The short version: experts are small (512-dim intermediate per expert in 3.6), routing is cheap, and the math works out in your favor whenever you’re bottlenecked on memory bandwidth. Most consumer GPUs are.

Hardware reality

Two rows here are firsthand, measured on our own cards: the RTX 3090 (full sweep in the 3090 section above) and the RTX 3060 12GB (low-VRAM section). The rest come from the Unsloth GGUF model card, Amine Raji’s RTX 3090 benchmark, and r/LocalLLaMA week-one reports, on llama.cpp builds b8954 or newer with --flash-attn on, --cache-type-k q8_0 --cache-type-v q8_0, and 65K context unless noted. Tok/s is generation speed, batch=1, greedy.

Caveats, and this table needs a bigger one than it used to. The measured rows and the community rows are not measured the same way. Ours are llama-bench decode rates at up to 8K depth with default KV cache; the community rows are llama-server sessions at 65K context with q8_0 KV quantization, which is heavier work. That’s why the 3090 row went from ~101 to 158 and now sits above the 4090 estimate below it. Don’t read that as a 3090 beating a 4090. Read it as two different tests. The 4090 and 5090 rows were extrapolated from the old ~101 3090 figure plus the usual generational uplift, which makes them the weakest numbers on this page now; we’d want a firsthand 4090 before treating either as more than rough community consensus. Mac numbers are MLX, not GGUF. Metal MoE handling improved a lot in MLX 0.21+.

The 5070 Ti 16GB + 32GB RAM thread on r/LocalLLaMA is the one that surprised people. With --cpu-moe parking the 8 routed experts in system RAM and keeping the shared expert plus attention on GPU, a 16GB card runs the full Q4 model. Slow, but real, and the quality is the full Q4 model rather than a more aggressive quant. For a chat workload that’s a meaningful trade against running UD-IQ2 on-GPU for speed.

–cpu-moe explained, briefly

--cpu-moe is a llama.cpp flag (landed in b8954-era builds) that tells the engine to keep MoE expert weights in CPU RAM and stream them across PCIe per token. The shared expert and attention layers stay on GPU. For a 3B-active MoE the per-token data movement is small enough that the speed hit isn’t fatal. You’re still doing 3B-active math, just with a hop across the bus to fetch which 8 experts to use.

It only makes sense when:

• The model doesn’t fit fully in VRAM (16GB cards, 12GB cards)
• You have enough system RAM to hold the expert weights (32GB+ for Q4, 64GB+ for Q6)
• You can tolerate ~3x slower generation than fully on-GPU

For a 24GB+ card the answer is don’t use it, and our sweep puts a number on how much it costs: 157.66 tok/s fully resident against 36.11 with every expert in RAM, on the same card, same model, same run. Even one layer of offload costs 11 tok/s.

35B MoE vs 27B dense on the same hardware

Here’s the trade table on a single RTX 3090 with 24GB VRAM, Q4 class quants, 64K context:

The MoE column is the awkward one, so treat it as a range rather than a number. At 64K context on UD-Q4_K_XL, which is what this table is scoped to, the community figures around 80-101 are the ones to plan against. Our 158 is the same model at a slightly smaller quant (UD-Q4_K_M) and 8K depth, where the KV cache is small and nothing has to move. Both are real; the gap between them is mostly context.

So on raw tok/s the MoE wins by 3x or better without DFlash. With DFlash on the dense, the gap shrinks. On agentic-coding scores the dense wins. On general chat or RAG workloads the MoE is the better experience.

The interesting wrinkle is a context question, not a capacity one. At UD-Q4_K_M with 8K of context the MoE has 2,471 MiB spare on a headless 3090 — it fits, comfortably. Go up to UD-Q4_K_XL at 65K and that headroom is gone. Push to 128K and you’ll need UD-Q3_K_M or KV cache offload. The 27B dense gives you more context for the same card; the MoE gives you more speed at the context most people actually use.

The Mac path: MLX, not GGUF

Mac users running this on llama.cpp Metal will get it working but slower than they should. The Metal kernels for the Gated DeltaNet layers are still catching up to CUDA. MLX is the right path on Apple Silicon. The Qwen MLX vs Ollama guide walks through the install, and the MLX team shipped 35B-A3B quants in the first week.

For RAM-poor Macs, look for the 3-bit MLX quants in the baa-ai and mlx-community repos. A 3-bit 35B-A3B lands around 14-16 GB and runs on a 16GB M1 Pro. Quality is noticeably below Q4, but it’s the only way to get 35B onto a base Mac. The 27B dense at 4-bit MLX is a better quality-per-byte trade if your Mac has 24GB+ unified memory. See the best local LLMs for Mac writeup for the broader picture.

Setup walkthrough

The fastest path that works today is llama.cpp built from source plus the Unsloth UD quants. Three steps.

1. Build llama.cpp from a recent commit

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release -j

You need a build at b8954 or later for --cpu-moe and the latest Gated DeltaNet kernels — and something recent (mainline crossed b10000 in July) if you want the merged MTP path covered below. On the toolkit: pin CUDA 13.1 or earlier, or 13.3, which shipped the fix. Avoid 13.2 specifically — multiple reports of low-bit 3.6 quants producing gibberish there, and as of July it’s still the one bad version. The issue is in the cuBLAS path, not llama.cpp itself. Amine Raji flagged it explicitly. Unsloth’s docs flag it. If you see garbage tokens, this is the first thing to check.

2. Pull the GGUF + mmproj

The 35B-A3B is multimodal. There are two files: the main GGUF and an mmproj vision sidecar. llama.cpp won’t load the model without mmproj on recent builds, even for text-only inference.

huggingface-cli download unsloth/Qwen3.6-35B-A3B-GGUF \
  Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf mmproj-F16.gguf \
  --local-dir ./qwen36-35b

UD-Q4_K_XL is the Unsloth Dynamic 4-bit “extra large” quant. It’s the recommended balance per their model card. UD-Q3_K_M is the 16GB-card alternative. The full quant table is on the Unsloth GGUF page.

3. Run llama-server

./build/bin/llama-server \
  --model ./qwen36-35b/Qwen3.6-35B-A3B-UD-Q4_K_XL.gguf \
  --mmproj ./qwen36-35b/mmproj-F16.gguf \
  --alias "qwen36-35b-a3b" \
  --ctx-size 65536 \
  --n-gpu-layers 99 \
  --flash-attn on \
  --cache-type-k q8_0 \
  --cache-type-v q8_0 \
  --temp 1.0 \
  --top-p 0.95 \
  --top-k 20 \
  --presence-penalty 1.5 \
  --min-p 0.00 \
  --port 8001

For a 16GB card add --cpu-moe and drop --n-gpu-layers to fit attention + shared expert in VRAM. For instruct (no thinking) mode, swap to --temp 0.7 --top-p 0.8 and pass --chat-template-kwargs '{"enable_thinking":false}'.

LM Studio works too. Point it at the same GGUF and set the mmproj field in the model config. Ollama has caught up for text: Qwen 3.6 is in the official library now (ollama pull qwen3.6:35b-a3b, plus -mlx tags), and chat, code, and tool calls all run fine. The one gap that remains is vision — Ollama still doesn’t wire up the separate mmproj sidecar, so image input fails while text generation works. If you need the multimodal side, use llama.cpp or MLX-VLM; if you’re text-only, Ollama is now a legitimate option. The depth comparison is in llama.cpp vs Ollama vs vLLM.

When to pick the MoE over the 27B dense

Pick 35B-A3B if:

• You want fast everyday chat and RAG workloads on a 24GB card. 158 tok/s feels different from 30 tok/s, and on a 3090 the whole model fits without offload.
• You only have 16GB VRAM and 32GB+ system RAM. CPU-MoE offload makes the full Q4 reachable. The 27B dense with KV offload is slower and worse.
• You’re on Apple Silicon with 64GB+ unified memory. MLX handles MoE well now.
• You want broader world knowledge in the weights. 35B of stored capacity matters for trivia and niche-domain work.

Pick 27B dense if:

• You code primarily and tool-use reliability matters. The 35B-A3B has community reports of dropping tool calls on long agent loops.
• You need the DFlash 2x speedup. It’s NVIDIA-only, sm_86+, and only supports the dense.
• You want strict instruction-following under heavy system prompts. MoE routing drift bites here.
• You want more KV headroom on a 24GB card at long context.

The full Qwen 3.6 guide walks through the dense side in detail and covers the Max-Preview cloud-only variant for completeness.

Honest limits

A few rough edges, with the mid-July 2026 status on each:

• Tool calling regressions. r/LocalLLaMA threads include reports of the 35B-A3B repeating failed tool calls without reading back context, and skipping tool calls entirely on multi-turn loops. The 27B dense is steadier here. If you’re wiring this into Claude Code, OpenCode, or an MCP harness, test before you commit.
• Web search broken in some llama.cpp builds. The web-search tool path through llama-server was reported broken on builds between b8954 and b8967 specifically for the 35B-A3B. Long fixed on mainline (well past b10000 now) — just don’t run an old pinned build for this.
• No DFlash support for MoE. The block-diffusion speculative decoding speedup that doubles 27B dense throughput on a 3090 does not extend to the 35B-A3B. The DFlash team hasn’t said when or whether it will. The MoE routing changes the draft-verify math.
• CUDA 13.2 gibberish. Already covered. Pin 13.1 or 13.3; 13.2 is still the one bad version as of July.
• Speculative decoding: the drafter path doesn’t help, but MTP does now. The classic external-drafter path (PR #19493) still gets eaten by routing overhead on the 35B-A3B. What changed: llama.cpp merged the model’s own multi-token-prediction head (PR #22673, May 16), and paired with MTP-capable GGUF weights (unsloth/Qwen3.6-35B-A3B-MTP-GGUF and others) the MoE picks up a real if modest speedup — community numbers land around 1.17x on an RTX PRO 6000 and 1.47x on an RTX 5060 Ti (98 → 144 tok/s) — note neither is a 3090, so don’t read those as your 3090 figure. It’s smaller than the dense-model gain because a 3B-active model already decodes cheaply, so there’s less target cost for MTP to save.

None of these are fatal, and two of them (Ollama text support, MTP on the MoE) have already been resolved since launch. The rest are current as of mid-July and worth knowing.

Bottom line

The 35B MoE is the right pick for fast general-purpose local AI on a 24GB card or with smart RAM offload on a 16GB card. The 27B dense is the right pick for coding and agentic work on the same hardware, especially with DFlash. Both are Apache 2.0, both are 262K context, both are natively multimodal.

If you only download one, ask yourself which workload dominates. Code all day? 27B dense. Chat, RAG, summarization, multilingual? 35B-A3B. If you have the disk for both, keep both. They cover different jobs and switching takes a llama-server restart.

Related guides

• Qwen 3.6 Complete Guide: 27B Dense, 35B-A3B MoE, and Which to Use
• MoE Models Explained: How Mixture of Experts Cuts Inference Cost
• Best Way to Get 2x Token Output on RTX 3090: Qwen 3.6 + DFlash
• Best Local Coding Models 2026
• Best Local LLMs for Mac 2026
• llama.cpp vs Ollama vs vLLM: When to Use Each
• VRAM Requirements for Local LLMs