Wicked Fast Qwen 3.6 27B: 60 tok/s with MTP on RTX 3090 (2026)

📚 More on this topic: Complete Qwen Models Guide · DFlash vs MTP on RTX 3090 (May 6) · Qwen 3.6 Complete Guide · DFlash 2x Token Output · Speculative Decoding Explained

On my RTX 3090, Qwen 3.6 27B Q4_K_M just hit 60 tok/s with MTP — roughly 1.6x faster than the same setup without MTP on mean per-prompt throughput, and 1.86x faster on wall-clock time across nine mixed prompts. This bench ran on am17an’s mtp-clean branch (commit 2dff7ff8f); 185 commits of polish since May 6 moved the needle from 1.50x mean to that number. PR #22673 has since merged to mainline llama.cpp — May 16 — so MTP now ships in stock builds, no fork required.

Two headline numbers, both real. n=3 gives the better wall-clock total. n=2 gives the better per-prompt throughput on 8 of 9 prompts. Picking between them depends on workload, not on Unsloth’s blanket recommendation. Below: the numbers, what changed, and what to actually run today.

Refresh note (July 2026): The numbers here are the branch measurement from May 19. The first few merged builds (May 16–19) regressed on some configs from a --spec-draft-p-min default of 0.75; ggerganov’s PR #23269 restored the 0.0 default on May 19, so current builds are fine and need no flag. The MTP subsystem still changes commit to commit — separate CPU, Metal, and SYCL slowdowns remain open — so treat 1.86x as the branch-at-May-19 figure and verify on your own build.

The bench

Hardware. Miu — single RTX 3090 24GB on Linux, CUDA 12, sm_86. No second card. MTP carries its draft head inside the target checkpoint, so there is nothing to offload to one. Same workstation as the May 6 head-to-head against DFlash.

Model. RDson/Qwen3.6-27B-MTP-Q4_K_M-GGUF — 15.35 GiB target with the MTP layer baked into the same file. No separate draft weights to manage.

Bench harness. am17an’s gist — nine mixed prompts at n_predict=192, temp=0.0, seed=42. Same harness as cturan’s 3090+3060 run, the Strix Halo reproduction, the M1 Ultra run, and my own May 6 bench. Shared harness, different boxes: the dual-GPU one is cturan’s, not mine. Numbers compose with the community thread.

llama.cpp build. am17an/llama.cpp at the mtp-clean branch, commit 2dff7ff8f (May 19, 2026). Built with CUDA: cmake -B build -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release.

Server flags (common). -ngl 99 -c 10000 --no-mmap --no-cache-prompt -fa on -np 1. Baseline runs no spec flags. MTP runs add --spec-type draft-mtp --spec-draft-n-max 3 (or 2).

Three benches, three numbers

Same hardware, same model, same prompts. Three configurations.

Two speedup metrics because they measure different things. Mean tok/s averages each prompt’s individual generation rate — useful for comparing how fast tokens stream once generation starts. Wall-clock total time is what users actually feel: how long from prompt submitted to all responses complete across the full bench. For most readers building an agent or chat workflow, wall-clock is the metric that matters. Mean tok/s is the one Unsloth and DataCamp quote.

n=3’s 1.86x wall-clock speedup is the practical headline. Same RTX 3090, same Qwen 3.6 27B Q4_K_M, same prompts. Add --spec-type draft-mtp --spec-draft-n-max 3 and the bench finishes in 27.2s instead of 50.64s. That’s a real, reproducible, today-available number — provided you build from the mtp-clean branch.

The surprise: n=2 isn’t always the sweet spot

Unsloth’s official Qwen 3.6 docs say to set --spec-draft-n-max 2 for most setups. On my RTX 3090, per-prompt throughput agrees with them — n=2 wins 8 of 9 prompts on tok/s. But total wall time tells a different story.

n=3 wins exactly one prompt on per-prompt throughput: stepwise_math, at 68.6 vs 58.1 tok/s. That prompt’s acceptance rate on n=3 was 80.8% — when the MTP layer’s predictions match the target reliably, larger draft batches generate more accepted tokens per cycle, and the per-cycle gain dominates. n=2 is the safer default at 80% mean acceptance, but its smaller batch size means each verified cycle produces fewer tokens.

The wall-clock story is more dramatic. n=3 totals 27.2s; n=2 totals 34.44s. That entire 7.24s gap traces to one prompt: long_code_review. n=3 finished it in 5.77s; n=2 in 13.26s — a 7.49s delta on a single prompt. Every other prompt n=2 wins by a small margin.

The mechanism: prefill. PR #22673 documents that MTP slows prompt processing. On this run, n=2 spent ~10s on prefill for long_code_review’s ~700-token system prompt before generation even started. n=3 spent ~2.5s on the same prompt’s prefill. n=2’s larger prefill overhead is the wall-clock killer on long inputs.

The pattern: n=2 generates faster but starts slower. On short prompts that’s a win. On long contexts the prefill penalty erases the per-token advantage and then some.

Practical takeaway:

• Coding agents and short-prompt workflows → --spec-draft-n-max 2. Higher acceptance, more consistent per-prompt throughput, what Unsloth recommends.
• Long-form generation (analysis, creative writing, long code reviews) → --spec-draft-n-max 3. Larger batches pay off when the model has runway to commit to a direction.
• Pick based on what you’re running, not based on a blanket “n=2 is best” line. Both work. Workload determines which wins.

What changed since May 6

On May 6 I benched the same hardware against the same model on PR #22673 at commit 267f8afe8 and hit 1.50x mean MTP speedup. Today the same setup hits 1.86x wall-clock speedup. What moved the needle in 13 days:

• 185 commits on the mtp-clean branch. am17an force-pushed continuously from May 4 onward. The branch is structurally rewritten in places — handle_mtp_for_ubatch lifted out of llama_context per ggerganov’s review feedback.
• GDN partial rollback shipped across all backends. CUDA, Vulkan, and Metal all support it now. This is the core acceptance-rate improvement — when a draft batch contains a partial rejection, the rest of the batch’s accepted tokens get kept instead of the whole batch being discarded.
• Multi-backend support stabilized. Metal’s keep_intermediates=true path landed for unified-memory devices. Vulkan partial rollback works. CUDA was already working but got polish.
• n-gram + MTP compatibility fixes. The commit message spec : fix compatibility with n-gram and add TODOs lands a fix for the prior interaction issue.
• The flag renamed. --spec-type mtp is now --spec-type draft-mtp. The change unifies MTP under the draft-* taxonomy alongside draft-eagle3 and draft-simple. Documentation hasn’t caught up: Unsloth’s docs and DataCamp’s complete tutorial both still reference the old mtp name. Neither works on a current mainline build without substituting draft-mtp.

Worth flagging: ggerganov, the llama.cpp maintainer, drove this to the finish. The handle_mtp_for_ubatch rework he asked for went in, and PR #22673 merged to mainline on May 16 — three days after this bench. MTP is no longer a branch-only feature.

For the May 6 baseline numbers and the head-to-head against DFlash on identical hardware, see the May 6 piece. DFlash still leads on mean speedup (2.56x vs MTP’s 1.60x mean today), but the gap is narrower than 13 days ago and MTP’s mainline path is more accessible.

The honest framing

When this piece first ran (May 19), MTP was still a branch-only feature and the caution here was “PR #22673 hasn’t merged — master llama.cpp doesn’t have MTP.” That caution has expired: PR #22673 merged to mainline on May 16. A recent stock llama.cpp build has MTP; you no longer clone a fork or checkout a branch.

Two things still worth saying plainly. First, the docs lag: Unsloth’s Qwen 3.6 pages and DataCamp’s tutorial still reference the old --spec-type mtp flag, which the merge renamed to --spec-type draft-mtp — substitute it or the flag silently does nothing. Second, “merged” doesn’t mean “settled”: the earliest merged builds regressed on some configs from a --spec-draft-p-min default of 0.75 (issue #23230) — fixed May 19 by #23269, which put the default back to 0.0, so a current build needs no flag. Other backend-specific slowdowns (CPU, Metal, SYCL) are still open with separate causes. Build recent, use the new flag name, and verify your own throughput.

How to actually run MTP today

Five steps. The bench ran on am17an’s branch in May; since the May 16 merge, mainline llama.cpp is the path — no fork, no branch checkout.

1. Get mainline llama.cpp (stock master has MTP as of the PR #22673 merge):

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp

2. Build with CUDA:

cmake -B build -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release --target llama-server -j

3. Download an MTP GGUF. Three options I’d consider:

• RDson/Qwen3.6-27B-MTP-Q4_K_M-GGUF — what I benched, 15.35 GiB, single Q4_K_M quant.
• unsloth/Qwen3.6-27B-MTP-GGUF — Dynamic 2.0 variants across multiple quants. Pick what fits your VRAM.
• havenoammo/Qwen3.6-27B-MTP-UD-GGUF — UD XL quants for users who want maximum quality fit.

All three include the MTP layer baked into the GGUF. No separate draft model to manage.

4. Run llama-server with MTP enabled:

./build/bin/llama-server \
  -m /path/to/Qwen3.6-27B-MTP-Q4_K_M.gguf \
  --host 127.0.0.1 --port 8080 \
  -ngl 99 -c 10000 -fa on -np 1 \
  --spec-type draft-mtp --spec-draft-n-max 3

The flag to get right is --spec-type draft-mtp, not --spec-type mtp — the merge renamed it, and the Unsloth docs and DataCamp tutorial both still use the old name, which silently does nothing on a current build. (Historical note: the first merged builds, May 16–19, defaulted --spec-draft-p-min to 0.75 and ran slow — issue #23230; #23269 restored the 0.0 default on May 19, so you only need to add --spec-draft-p-min 0 if you’re pinned to a build in that window.)

5. For coding agents or short-prompt workflows:

--spec-type draft-mtp --spec-draft-n-max 2

Use 2 instead of 3. Per the table above, n=2 wins on per-prompt throughput for code generation and short responses. Pick by workload.

What this means for your local AI setup

For 24GB cards (RTX 3090, RTX 3090 Ti, RTX 4090): MTP is a clear win. Qwen 3.6 27B Q4_K_M at 60 tok/s is faster than most people can read in real time. Local coding agents that previously felt sluggish at ~40 tok/s become smooth. Long-context workflows benefit even more on n=3. See the Qwen 3.6 setup guide for the model overview and the coding-models roundup for context.

For 12-16GB cards: MTP costs roughly 2.5 GiB extra VRAM beyond the base model — the MTP layer plus its KV cache. On a 16GB card running 14B Q4 with a generous context, that overhead may not fit. Test before committing. If you’re hitting VRAM pressure, the 35B-A3B MoE path trades model quality for headroom in a way MTP doesn’t.

For Mac (M4 Max, M5 Max): This article is CUDA-only. Apple Silicon has its own MTP path through MLX — separate work, separate bench. Treat the numbers above as not portable.

Honest trade-offs

Three things MTP costs you alongside the speedup.

Prefill is slower. PR #22673’s own benchmarks show ~0.51x prefill at long prompts on the same hardware. Time-to-first-token goes up. Generation speed goes from low-40s to low-60s, but you wait longer to see the first token, especially with large input contexts. For chat with short prompts the cost is invisible. For RAG with 8K+ context the cost is real.

VRAM overhead. ~2.5 GiB for the MTP layer plus its KV cache. At 27B Q4_K_M on a 24GB card, that’s comfortable. On a 16GB card running 14B at any reasonable context, it may not fit. Run with --no-mmap and watch nvidia-smi during model load to see your actual footprint.

Build complexity. You’re building llama-server from source — mainline now, since the May 16 merge, not a fork. Prebuilt release binaries may lag on MTP depending on when they were cut, so a source build is still the reliable path. CUDA toolchain required. If you’ve never built llama.cpp from source, expect a couple of hours on first try — see the Qwen 3.6 troubleshooting piece for the common errors. After that, rebuilds are fast.

What’s next

PR #22673 merged into llama.cpp master on May 16. What that changed, and what’s still moving:

• MTP is in mainline — the clone-a-fork step is gone; a recent stock build has it.
• The first merged builds had a --spec-draft-p-min regression (issue #23230); ggerganov’s PR #23269 fixed it on May 19 (default back to 0.0). Other backend-specific slowdowns — CPU (#23698), Apple Metal (#23011), SYCL (#23203) — have separate causes and some remain open.
• Numbers move commit to commit. Whether the 1.86x wall-clock holds on your build depends on the exact commit and flags — verify locally rather than trusting a figure pinned to one date.

A merged-mainline re-bench is on the list. The newsletter catches the follow-up when it lands.

Bench data, build details, and the per-prompt acceptance rates from this run are reproducible via am17an’s gist. If you’ve got numbers from your own hardware, post them on the PR #22673 thread — that’s where the maintainers are watching.