Why Qwen 3.8 27B Feels Slow: Reasoning Tokens Measured

📚 More on this topic: Qwen 3.8 vs 3.6 on RTX 3090 · Why Is My Local LLM So Slow? · Context Length Explained · KV Cache Optimization · Best Qwen Models Ranked

HumanEval problem 108 asks you to count how many numbers in a list have a digit sum greater than zero, with the wrinkle that a negative number’s first digit counts as negative. It is a middling problem. Not trivial, not hard.

Qwen 3.8-27B spent 32,000 tokens on it and never answered.

Not “answered badly” — the response field came back empty. It burned 14.2 minutes of an RTX 3090, hit the token ceiling, and returned no code at all.

What it did with those tokens is not what I assumed before reading the trace. It reasoned coherently for 6,400 characters, worked through the negative-digit rule correctly, and got as far as considering what a hidden test case might contain. Then it emitted a single unbroken run of 30,037 zero characters and nothing else. That run is 82.4% of its output and it consumed every remaining token.

That is a decode collapse, not a reasoning loop. The model did not think itself into a corner; its generation degenerated and the ceiling merely ended it. It is also the only problem in all 164 with a degenerate run of even 100 characters, so 108 is the extreme of the distribution and a different phenomenon from the rest of it. One problem, 7.2% of everything the model generated across the whole benchmark.

Four days earlier I had run problem 108 through the same model file, on the same card, with the same seed, as a plain completion with the chat template switched off. That run answered it in 24 tokens:

return sum(1 for x in arr if sum(map(int, str(abs(x)))) > 0)

That answer is wrong — it fails on negatives, the one thing the problem asks you to get right, and the official scorer marks it failed. So the model gets this problem wrong either way. What changed was the price: 24 tokens against 32,000 for the same wrong result, a factor of roughly 1,300, and in the expensive case you do not even get the wrong answer to look at.

The second-worst case is stranger. Problem 2 asks for the decimal part of a float: given 3.5, return 0.5. The model spent 14,953 tokens, got there, and wrote one line.

return number - int(number)

126 tokens of that were the answer. Two different failures — one reaching a trivial answer expensively, the other never reaching one — and neither is predictable from looking at the task.

I want to be careful about what this proves. It is not evidence the model is bad: it is a strong model, and further down I score it. It is not evidence the model is slow either, because it generated at 37.2 tok/s throughout, which is roughly what this file does on this card. What it shows is where the time goes.

What I ran

Qwen3.8-27B at Unsloth’s UD-Q4_K_XL quant, one RTX 3090, llama-server from llama.cpp b10088. The important part is that this run went through /v1/chat/completions with --jinja, which means the model’s own chat template was applied and the reasoning path was live.

That matters because it is the opposite of how I benchmarked this model the first time.

reasoning_effort   model default (xhigh, resolved from the GGUF template)
temperature        1.0
top_p              0.95
top_k              20
context            -c 32768
max_tokens         32000
stop sequences     none
seed               42

Sampling follows the model card’s own recommendation for thinking mode. I did not pick these numbers; this is what you get if you follow the instructions. The model file was checksummed against upstream immediately before load and again after the run, and both digests matched. That is footing rather than story, after this rig turned up a single-bit page-cache fault last week.

All 164 problems, 3 hours 19 minutes, 444,556 tokens generated. That run is what the tables below describe. Three more have finished since — seed 43 on this model, and both seeds of Qwen 3.6 — and where they move a number I say so at the number rather than saving it for the end.

A note on the one truncated problem. HumanEval/108 hit the ceiling with an empty response, so it is reported as a truncation and excluded from the accuracy question — scoring an empty string as a wrong answer would turn a budget artifact into a capability claim, which is the exact confusion this page exists to undo. It stays in the token distribution, where its true cost is unknown: all we know is that it is at least 32,000.

Where the tokens go

Across all 164 problems: 444,556 tokens generated, 412,441 of them thinking. Thinking is 92.8% of output. The answer is 7.2%. Percentiles are nearest-rank over all 164, including the truncated problem at its censored value of 32,000.

The number to look at is that the mean is 2.7 times the median — the signature of a heavy tail, where a handful of problems set the average while typical behaviour sits far below it. Half the set finishes under 993 tokens. The ten most expensive problems produced 44.9% of all output between them; the cheapest hundred produced 14.4%.

Now compare against the raw completions path, from the benchmark I published on the 14th. Same model file, same card, same seed, no chat template, same 164 problems:

24x at the median. 48x at the mean. Same weights, same hardware, same seed. The only thing that changed is whether the chat template was applied.

Problem 108 from the opening is the extreme of that distribution, not an exception to it. Problem 2 is the same shape at smaller scale: the raw path emitted nine tokens, return number - int(number), and the chat path took 14,953 to reach the identical line.

The part that actually matters

Look at the answer row again. 0 to 602 tokens, median 173: bounded and boring, which is what a benchmark of small functions should look like. Now the thinking row. 100 to 32,000, a 320x spread. Almost all the variance in what this model costs you lives there, and that would be manageable if it tracked how hard the problem was.

320x is the conservative figure. Seed 43 ran the same model wider, 59 to 32,000, 542x. Both ends of both ranges are single observations, so the honest version is a range that is not narrowing: somewhere between 320x and 542x.

Qwen 3.6 on the same protocol does not behave this way at all. Its thinking spans 257 to 7,413 tokens on one seed and 504 to 6,136 on the other, 12x to 29x. Same benchmark, same card, same scorer, two orders of magnitude less spread. What this page describes is a property of this model rather than of reasoning models generally.

Answer length is the obvious thing to check and it is only a loose guide. The two correlate at Spearman r = 0.53, about a quarter of the variance. Among the 39 problems whose answers land between 150 and 200 tokens, thinking still ranges from 131 to 17,636.

Does the thinking track difficulty?

So I checked four better proxies. None of them is difficulty itself, and I would not lean on any single one, but they disagree usefully.

Every one is positive and every one is weak. The best predictor is not a property of the problem at all. It is how many tokens the same model spent on the same problem with the template off, which is closer to how much this model wanted to say than to how hard the task was. Assert count is the outlier at the bottom, and HumanEval/32 shows why: a single assert, 26,850 thinking tokens.

Quartiles by reference-solution length separate what these proxies do and do not capture:

The medians rise about fivefold, monotonically. Difficulty sets the floor, and the cheap end confirms it: the ten cheapest problems have one-line, 32-character reference solutions, three asserts each, a median of 12 tokens on the raw path, and the raw path got all ten right.

What difficulty does not set is the ceiling. The most expensive problem in the run sits in Q3, and the worst blow-up in the easiest quartile ran to 14,827. Every quartile has a tail. Problem 2 belongs to that easy group by every measure I have — one line, 24 characters, three asserts, answered correctly in nine tokens with the template off — and on the chat path it cost 14,827.

It is also why the usual local-LLM speed playbook does nothing here. If you came from the slow-model diagnostic route you have already checked offload, quant size, and num_ctx overflow, and all three came back clean, because this is not a tokens-per-second problem. Your card is fine. You are being billed for tokens you did not know you ordered.

This is what reviewers have been running into

The behaviour has been visible since launch, mostly as frustration without a mechanism attached.

Simon Willison covered it on August 16 in a post about the model’s default reasoning behaviour: one SVG drawing, around 21 minutes and roughly 22,000 reasoning tokens for about 3,200 tokens of output. His advice is to skip the default and start at low or no reasoning.

Two YouTube reviewers hit it independently in agent workflows. One kept 3.6’s settings, ran into session compaction three times and failed all three tasks he set. Another reports burning 87% of a 128K context on a single Kanban app. Those are their observations, not mine, and I repeat them because the numbers above explain what they were seeing: none of them were hitting a broken model, they were hitting a context budget sized for a model that no longer exists.

Why my own benchmark showed none of this

Here is the awkward part, and I would rather raise it than have someone else raise it for me.

The head-to-head I published on August 14 found 3.8 and 3.6 statistically indistinguishable on HumanEval, 80.49% against 82.32%, McNemar p=0.70. That run used /v1/completions with --no-jinja and raw prompts. In llama.cpp’s source the two endpoints have entirely separate parsers, and the one behind /v1/completions contains no reference to templates, reasoning, or thinking at all. Everything on this page lives in the Jinja template, which only /v1/chat/completions reaches. I checked all 164 completions from that run: not one contains a <think> tag, and the median was 41 tokens. The reasoning path never fired.

That null still stands and this page does not contradict it. It compared two models as raw code completers, treated identically, which is how HumanEval is normally run. What it measured was not the reasoning path. This page measures that path’s cost.

Whether the cost buys accuracy is a third question, and since I had the completions I scored them. The chat path solved 155 of the 163 problems it answered, 95.09%, against the raw path’s 80.49%, fixing 25 the raw path got wrong and breaking 2 it got right. HumanEval/108 is out of that denominator, because scoring an empty string as a wrong answer is the exact mistake this page exists to warn about.

Three more runs have since finished and they answer that question properly: two seeds per model, 12.87 hours of GPU time on the one card, everything else held.

The denominators differ because truncations are excluded rather than scored as failures, which is the distinction this entire page exists to make. Score them as wrong answers instead and 3.8 drops to 94.51% and 93.90% while 3.6 does not move. That is a sensitivity check, not a result.

The reasoning buys about 14 points, and it buys them for both models. Against the raw path’s 82.32% and 80.49%, turning the template on is worth roughly 13 points to 3.6 and 15 to 3.8. Same weights, same quant, same card, same scorer. The protocol is worth more than the model is.

Between the two models there is nothing, and the nothing is the result rather than a failure to find one. The gap is 0.35 points. Two seeds of 3.6 differ by 0.61. The models sit closer to each other than one model sits to itself, paired McNemar returns p = 1.0000 on both seeds, and the problems they disagree on number 4 and 7, split almost evenly in each direction.

That is a limit of the instrument and not proof the models code equally well. At 95% HumanEval has very little headroom left: four problems fail in all four runs, nine more flip between them, and what is left to separate two models is a handful of coin flips. A benchmark this saturated cannot rank them. Saying so is a different claim from saying they are equal.

One thing did separate them, and it belongs to this page’s subject rather than to the accuracy question. The two models spend about the same and spend it in opposite shapes. 3.6 generated more total tokens across its two seeds, 911k against 840k, and put roughly twice as much of it into the answer you actually read: median 320 tokens against 177. Its thinking is uniform, mean and median almost coinciding, p95 around 5,100, and across 328 problems it never once failed to terminate. 3.8 is bimodal, mean about three times its median, cheap on most problems and occasionally catastrophic. All three truncations in the whole set are 3.8’s.

Provenance, briefly, because this rig has an unresolved page-cache fault. Every run was bracketed the same way: file evicted to zero resident pages and confirmed with mincore, hashed against upstream immediately before launch and again after the run. All eight hashes matched. 653 of the 656 responses extracted cleanly, and the three that did not are the genuine truncations.

On what a HumanEval number is worth either way, we’ve written the long version.

What to actually do about it

Qwen 3.8 ships a reasoning_effort parameter with three levels. Qwen 3.6 has no such parameter. This is new, and it is the single biggest reason 3.8 does not behave like a drop-in swap even though the model card never says so. It defaults to xhigh.

Set it per request:

{
  "model": "qwen3.8-27b",
  "messages": [{"role": "user", "content": "..."}],
  "chat_template_kwargs": {"reasoning_effort": "low"}
}

Server-wide, Unsloth documents the same keys as --chat-template-kwargs '{"reasoning_effort":"low"}'.

Two traps in that knob, both found by dumping the template out of the GGUF and neither in any documentation I can find. high is silently rewritten to xhigh, so if you assumed the ladder ran xhigh, high, medium, low and picked “high” to dial back, you set the default and nothing warns you. And none is not valid on this file despite Unsloth listing it — the template raises on anything outside xhigh, medium, low. medium injects no instruction at all, making it the absence of steering rather than moderate steering.

Because of that I would reach for llama.cpp’s own flags, which sit outside the template and behave predictably. --reasoning takes on, off, or auto and defaults to auto, which does whatever the template says. --reasoning-budget defaults to -1 for no limit; 0 ends thinking immediately, any positive number caps it there. That is a real ceiling, not a suggestion the model can talk itself out of, and the next section is about where to set it. A companion --reasoning-budget-message injects text before the closing tag when the budget runs out.

For multi-turn agent sessions, --no-reasoning-preserve stops the template carrying every earlier thinking block forward, so your prompt does not grow while your conversation stands still. I have not tested whether that is what the reviewers hit — it is a hypothesis about their sessions, not a measurement of them.

None of this applies to raw completions. Drive /v1/completions with --no-jinja, as a code-completion backend does, and the template never runs and the thinking never fires.

Cap the thinking first, then size the context

On the first twenty problems the obvious conclusion was that you should give the model 32K of context and get on with it. The full run says that would have been wrong advice, and it is worth being precise about why, because the obvious correction is also wrong.

The obvious correction is “32K was too small, use more.” It is not. HumanEval/108 truncates at 16,384 and at 32,000, and nothing about a generation that collapsed into repeating one character suggests 64,000 would have rescued it. Seven problems went past 16,384 here and four past 20,000. The tail does not taper politely.

So the control is the cap rather than the context. And the reason is not that the tail is expensive; it is that the tail is where the evidence says the thinking stops paying.

The 25 problems the reasoning genuinely fixed have a median thinking length of 1,697 tokens. The set median is 790; the eight problems over 10,000 have a median of 20,420. The work that changes outcomes happens in the middle of the distribution, an order of magnitude below the tail.

The tail bears that out. Of the seven expensive problems that answered at all, the chat path got 4 right against the raw path’s 2. Better, and nowhere near the 95% it manages across the set. Two of those four were ones the raw path had already solved for nine and 116 tokens. Genuine gain from the roughly 148,000 thinking tokens spent above 10,000: two problems.

Set --reasoning-budget first and treat context as whatever you need once the cap is in place. From this distribution I would start at 8192:

llama-server -m Qwen3.8-27B-UD-Q4_K_XL.gguf --jinja \
  -c 16384 --reasoning-budget 8192

8,192 sits above the productive region and below the tail. 154 of 164 problems finish their reasoning without ever reaching it, and so do 23 of the 25 the reasoning fixed. Seed 43 lands on the same 154 of 164, which is more agreement than a heavy-tailed statistic usually offers.

On Qwen 3.6 the cap is a no-op. Its longest reasoning across both seeds is 7,413 tokens, so 8,192 never fires on any of its 328 problems. If both models sit behind one server, setting it costs the older one nothing.

To be exact about what 8,192 is: a number fitted to Qwen 3.8’s distribution, which held on both of its seeds, and which happens to be free on 3.6. It is not a recommendation for reasoning models in general, and I have no evidence it is the right number for any model I did not measure. The method transfers — find where your model’s productive region ends and cap above it. The figure does not.

The two it costs you are the honest price and I would rather name them than wave at them: HumanEval/132 and /137 were genuine gains at 17,636 and 30,073 thinking tokens, and the cap gives them up. In exchange you bound every worst case in the run. Two problems out of 164 against that is a trade I would take, and it is a trade rather than a free win.

Tighten it further if your work is routine: 4096 touches 23 problems, 2048 touches 42, which is defensible for bulk work where a retry is cheap and a fourteen-minute stall is not.

Both of those are 3.8’s numbers and they do not carry to 3.6, whose thinking centres near 2,250-2,365 tokens against 3.8’s 703-790. A 2048 cap fires on 100 and 95 of its 164 problems across the two seeds, cutting into the productive middle rather than the tail. Size the cap to the distribution of the model you are running, not to a number from a page about a different one.

Context then follows from the cap instead of trying to contain the model. With reasoning bounded at 8,192, the longest prompt here at 457 tokens and answers topping out at 602, 16K is comfortable. Uncapped, no context size is, which is the whole point. On what the cache costs in VRAM at each size, we’ve measured it. For multi-turn agent work the reviewers say 128K uncapped is still not enough; cap the reasoning there too.

What this does not tell you

164 problems of self-contained Python function synthesis, two models, two seeds each, one quant, one card. Specifically not established:

• That the tail is pinned down. Seed 43 reproduced the shape and hit the same 154 of 164 at an 8,192 cap, so the structure replicates. It also widened the spread from 320x to 542x. Both ends of both ranges are single observations at temperature 1.0, so the tail is measured now rather than assumed, which is not the same as being bounded.
• That HumanEval can separate these two models. At around 95% it cannot, and more seeds will not fix a saturated benchmark. A harder suite might rank them. This one returned p = 1.0000 twice and there is no winner to read out of that.
• Anything about a different quant or a different card. One UD-Q4_K_XL file per model, one RTX 3090, one llama.cpp build.
• Anything about other models. Every current reasoning model has some version of this. I measured one. Do not read these ratios onto GPT-OSS, DeepSeek, or anything else.
• Multi-turn or agentic behaviour, which is where reviewers hit the wall and where I have no measurements at all.

One loose end: generation averaged 39.2 tok/s over the first twenty problems and 37.2 tok/s over all 164. Decode slowing as the KV cache fills would produce that, and so would thermal drift over three hours. I did not isolate them.

The bottom line

Qwen 3.8-27B is not slow. It runs at 37.2 tok/s on a 3090, which is where 3.6 sits too. It generates 24 to 48 times more tokens than the same file does with the template switched off, 92.8% of them are thinking, and the amount varies by 320x to 542x depending on the seed, in a way that answer length only weakly predicts.

The fix is a cap, not a bigger room. Set --reasoning-budget 8192 and give it 16K of context; that leaves 154 of 164 problems untouched and bounds the ten that would otherwise cost you minutes each. Drop to --reasoning off when you already know the task is easy, and check you are not accidentally setting high and getting xhigh. Then judge the model on what it produces rather than on how long you waited. They are separate questions with different answers: the cost is measured and it is large, and the accuracy is a tie that HumanEval is too saturated to break.