Best Local LLMs for Math & Reasoning: What Actually Works

📚 More on this topic: DeepSeek Models Guide · Qwen Models Guide · Llama 3 Guide · VRAM Requirements

Standard LLMs are bad at math. Ask Llama 3.1 8B to solve a competition-level problem and it’ll confidently produce wrong answers. The model doesn’t reason — it pattern-matches, and math requires actual step-by-step logic.

That changed in 2025 with reasoning models. DeepSeek R1 proved that chain-of-thought training could make open-source models competitive with OpenAI’s o1 on math benchmarks. Now there’s a whole class of models — R1 distills, Qwen 3’s thinking mode, Phi-4-reasoning — that genuinely think through problems before answering.

The difference is dramatic. Qwen 3 32B scores 95.2% on MATH-500 with thinking enabled. Turn thinking off: 43.6%. Same model, same weights, same hardware. The reasoning process is the entire game.

This guide covers which reasoning model to run on your hardware, what the benchmarks actually mean, and when local models fall short.

Why Math Is Different

Regular LLMs generate text left-to-right, picking the most likely next token. This works for chat, summarization, and creative writing. It fails for math because math requires working through intermediate steps — and a wrong step early means everything after it is wrong too.

Reasoning models fix this by generating a chain of thought before the final answer. The model literally writes out its reasoning — “let me try substituting x = 3… that gives 27 + 9 = 36, not 42, so let me try x = 4…” — then produces the answer based on that work.

This costs more tokens (2-5x more per response) and takes longer, but the accuracy improvement is massive. It’s the difference between a calculator and a student who shows their work.

The benchmarks that matter

GSM8K (grade school math) is saturated — top models score 95%+. It no longer separates good from great. The benchmarks that actually differentiate reasoning models in 2026:

One benchmark-year note before the tables: through 2026 the field moved from AIME 2024 to AIME 2026 (each year’s exam is a fresh, unseen set), and MATH-500 has saturated — most current model cards have dropped it. Scores across benchmark years aren’t directly comparable, so where this guide shows an old model’s AIME 2024 figure next to a new model’s AIME 2026 figure, treat the generational gap as directional, not a same-ruler comparison. GPQA Diamond is the cleanest cross-generation reasoning yardstick.

The Contenders at a Glance

Every model worth considering for local math and reasoning, with the numbers that matter:

The current generation first, then the models this guide used to recommend (now legacy). Note the AIME year per row — current cards report AIME 2026, older ones AIME 2024; the two aren’t the same exam.

The standout in 2026: Qwen 3.6-35B-A3B posts 92.7 on AIME 2026 with only ~3B active parameters per token — so via --n-cpu-moe expert offload it runs on a 16GB card, not the 24GB its 35B size implies. The efficiency story moved from “a 14B beats a 32B” (Phi-4’s old claim) to “a 35B-that-runs-like-a-3B beats everything you could fit before.” (GPQA-Diamond is the cleanest cross-year yardstick here; a reliable AIME 2026 figure for the 27B dense wasn’t published at writing, but its 87.8 GPQA-D leads the open dense field.)

Best Model by VRAM Tier

Skip the analysis if you just want a recommendation for your GPU:

One structural note before the tiers: Qwen 3.6 ships only a 27B dense and a 35B-A3B MoE — no small dense sizes. So the 4-12GB tiers point at Qwen 3.5 (its 4B/9B thinking models), and Qwen 3.6 takes over from 16GB up.

4-8 GB VRAM

Pick: Qwen 3.5 9B (thinking) — the best small-card reasoner. On 8GB it runs at Q4-Q5; the 4B fits below 4GB.

ollama run qwen3.5:9b          # 8GB, /think for math
ollama run qwen3.5:4b          # ~4GB, still genuinely useful

Qwen 3.5 9B with thinking scores GPQA-Diamond 81.7 — it beats models several times its size on reasoning, and the /think toggle means it doubles as a general chat model when you don’t need the chain of thought. This is the small-card default now; the old DeepSeek-R1-Distill-7B still runs but trails it.

12-16 GB VRAM

Pick: Qwen 3.5 9B at Q6/Q8 fully on GPU, or Qwen 3.6-35B-A3B via offload

ollama run qwen3.5:9b          # fully on-GPU, high quant, long context
ollama run qwen3.6:35b-a3b     # the ceiling-breaker, --n-cpu-moe offload

At 12GB, Qwen 3.5 9B at Q6/Q8 gives near-lossless quality with room to think. At 16GB the bigger move is Qwen 3.6-35B-A3B (AIME 2026 92.7) via --n-cpu-moe expert offload — 35B-level reasoning at ~38 tok/s, experts streamed from RAM. See the firsthand offload guide. (The old Phi-4-reasoning-plus math specialist still runs here, but current thinking models have passed it — keep it only if you want a math-only model in ~8GB.)

24 GB VRAM

Pick: Qwen 3.6-27B dense for the best all-round reasoner, Qwen 3.6-35B-A3B for the MoE alternative

ollama run qwen3.6:27b         # dense, GPQA-Diamond 87.8, ~17GB
ollama run qwen3.6:35b-a3b     # MoE, AIME 2026 92.7, ~101 tok/s on a 3090

At 24 GB the pick is Qwen 3.6. The 27B dense (GPQA-Diamond 87.8, the strongest open dense reasoner near its size) fits fully with context room and toggles thinking on/off. The 35B-A3B MoE posts AIME 2026 92.7 and runs at ~101 tok/s on a 3090 because only ~3B params fire per token. Both are a clear generation ahead of the DeepSeek-R1-Distill-32B this tier used to recommend. For the dense-vs-MoE tradeoff and VRAM specifics, see the Qwen 3.6 guide; for the full family, the Qwen models guide.

→ Check what fits your hardware with our Planning Tool.

DeepSeek R1 Distills: The Math Specialists

These were the 2025-era picks — they still run, but they’ve been overtaken. The DeepSeek R1 distills are purpose-built for reasoning, trained by distilling the full 671B R1 model’s reasoning behavior into smaller models. They always generate chain-of-thought tokens — no way to turn it off. Worth knowing where the DeepSeek line went since: DeepSeek R2 never shipped, and DeepSeek V4 (April 2026) folded the R-series reasoning into the main V-series as one adaptive-thinking model — but V4 is a 284B+ MoE, not a consumer-local pick. So the distills’ local successor isn’t a newer distill; it’s Qwen 3.5/3.6 thinking.

Key observations:

The 14B is the value king. Scoring 69.7% on AIME at only ~6.5 GB VRAM, it’s the most reasoning per gigabyte in the R1 lineup. Fits comfortably on a 12 GB card.

The 32B beats OpenAI’s o1-mini on AIME and MATH-500. On a single consumer GPU.

The 70B isn’t worth it. Going from 32B to 70B (Llama base) only gains 0.2% on MATH-500 and drops slightly on AIME. Double the VRAM for negligible improvement. The Qwen-based 32B is the sweet spot.

The always-think tradeoff. Every response includes reasoning tokens — even for “what’s 2+2.” This makes them slower for simple tasks and annoying for general chat. They’re specialists, not daily drivers.

# Pull the one that fits your GPU
ollama run deepseek-r1:7b
ollama run deepseek-r1:14b
ollama run deepseek-r1:32b

Phi-4-Reasoning: The Dark Horse

Once the per-parameter champion, now a generation behind — but still a capable math-only pick in ~8-10GB. Microsoft’s Phi-4-reasoning-plus, at 14.7B, scores (on AIME 2024, the exam of its era):

• 81.3% on AIME 2024 — higher than DS R1-Distill-32B (72.6%) and Qwen 3 32B (70.0%)
• ~97.7% on MATH-500 — higher than full DeepSeek R1 671B (97.3%)
• 69.3% on GPQA Diamond — competitive with models 4x its size

This is a 14B model outperforming 32B reasoning specialists. At Q4 quantization, it needs ~8-10 GB VRAM — meaning it runs on an RTX 3060 12GB or RTX 4060 Ti 16GB.

The catch

Phi-4-reasoning-plus is a math-only model. Microsoft’s model card explicitly states it’s “designed and tested for math reasoning only.” Don’t expect good creative writing, chat, or code generation. It does one thing and does it extraordinarily well.

The base Phi-4 (non-reasoning) is a general-purpose model, but it scores dramatically lower on math benchmarks. The reasoning version was fine-tuned on o3-mini demonstrations plus 90 steps of reinforcement learning on ~6,000 math problems.

There’s also Phi-4-mini-reasoning at 3.8B — a tiny reasoning model for edge devices. Useful if you’re truly VRAM-constrained but still need reasoning capabilities.

Setup

ollama run phi4-reasoning-plus   # 14.7B, math specialist
ollama run phi4-mini-reasoning   # 3.8B, lightweight

When to pick Phi-4 over R1

• You have 12-16 GB VRAM and want the absolute best math performance
• Your use case is specifically math, physics, or formal logic
• You don’t need the model for chat, code, or creative tasks

When to pick R1 instead

• You want stronger performance on code (LiveCodeBench, Codeforces)
• You want a model that handles reasoning across domains, not just math
• You have 24 GB and the R1 32B distill fits

Qwen Thinking Mode: The All-Rounder (Now the Default)

Qwen’s hybrid thinking mode is what makes it the current local math pick: competitive on reasoning, useful for everything else, and toggleable per message. The numbers just below are from Qwen 3 (the generation this section was written around); the mechanism is identical in Qwen 3.5 and 3.6, which post higher scores (35B-A3B: AIME 2026 92.7). The thinking-on-vs-off gap is the whole point, and it holds across the family.

The numbers with thinking enabled:

That last column is the key stat. Qwen 3 32B drops from 95.2% to 43.6% on MATH-500 when thinking is disabled. The chain of thought more than doubles the model’s math accuracy. Same weights, same quantization, same hardware — the only difference is whether it reasons through the problem first.

The toggle advantage

Unlike R1 distills that always think, Qwen 3 lets you switch per-message:

>>> /think Prove that the square root of 2 is irrational
[detailed chain-of-thought reasoning...]

>>> /no_think What's the capital of France?
Paris.

This means one model handles your math homework and your quick questions without wasting tokens on reasoning when it’s not needed.

Qwen 3 4B: The surprise

Qwen 3 4B with thinking scores 73.8% on AIME 2024. That’s higher than the DeepSeek R1-Distill-Qwen-32B (72.6%). A model that fits in 2.5 GB of VRAM outperforming a model that needs 18 GB on competition math. The 4B won’t match the 32B on complex multi-step reasoning or non-math tasks, but for the AIME benchmark specifically, the result speaks for itself.

Gemma 4: The Multimodal Option

Google’s Gemma 4 is the current Gemma generation, and unlike Gemma 3 it’s a genuine contender on math: the 31B dense scores 89.2% on AIME 2026, and the 26B-A4B MoE (4B active) brings that reasoning to a lighter, offload-friendly footprint. Gemma 4 is natively multimodal, so its niche for math is unchanged and still unique among strong reasoners: it can read equations from images — photos of textbooks, handwritten problems — and reason through them. If your workflow is image-in math, Gemma 4 is the pick no text-only reasoner matches.

At Q4 the 26B-A4B lands ~16-18 GB (offloadable), the 31B dense ~18 GB. See the Gemma 4 guide for setup.

Setup Guide

Quick start: Best model for your GPU

# 4 GB VRAM — tiny reasoning model
ollama run qwen3.5:4b

# 8 GB VRAM — best small-card reasoner
ollama run qwen3.5:9b

# 12 GB VRAM — high-quant 9B, or math-only Phi-4 if you prefer
ollama run qwen3.5:9b

# 16 GB VRAM — the ceiling-breaker via expert offload
ollama run qwen3.6:35b-a3b

# 24 GB VRAM — best open dense reasoner
ollama run qwen3.6:27b

System prompts for better math output

Reasoning models benefit from system prompts that set expectations:

# Modelfile for math-focused Qwen 3
FROM qwen3:14b

PARAMETER num_ctx 8192
PARAMETER temperature 0.1

SYSTEM """You are a precise math and reasoning assistant. When solving problems:
1. Show your complete work step by step
2. Verify your answer by checking it against the original problem
3. If you're uncertain, state your confidence level"""

ollama create math-assistant -f Modelfile
ollama run math-assistant

Low temperature (0.1-0.3) helps for math — you want deterministic reasoning, not creative variation.

Thinking mode with Qwen 3

Force thinking on for all math sessions:

# Interactive toggle
>>> /set think

# Or per-message
>>> /think Solve: find all real solutions to x^4 - 5x^2 + 4 = 0

For a permanently thinking version:

FROM qwen3:14b
PARAMETER think true
PARAMETER temperature 0.1

Managing multiple reasoning models

Keep both a specialist and an all-rounder pulled:

# Pull both
ollama pull deepseek-r1:14b
ollama pull qwen3:14b

# Use R1 for heavy math
ollama run deepseek-r1:14b "Prove that there are infinitely many primes"

# Use Qwen for math + everything else
ollama run qwen3:14b

Ollama keeps models on disk when not in use and loads them into VRAM on demand. Having multiple models pulled doesn’t cost VRAM — only the running model uses GPU memory.

When Local Falls Short

Honesty time. Local reasoning models are impressive, but they have limits.

Frontier math. On the hardest problems — FrontierMath, ARC-AGI, novel research-level math — cloud models (o1-pro, o3, Claude with extended thinking) still lead by a wide margin. The full DeepSeek R1 at 671B is competitive, but you can’t run that locally.

Multi-step proofs. Long chain-of-thought reasoning (20+ steps) degrades in smaller models. The 7B and 14B models sometimes lose track of earlier steps or contradict themselves mid-proof. The 32B models are significantly more reliable here.

Context length under load. Reasoning tokens eat into your context window. A problem that needs 2,000 tokens of thinking plus 500 tokens of answer needs 2,500 tokens of KV cache space. On VRAM-constrained setups, this limits how complex your problems can be.

Verification. Reasoning models can produce confident, well-structured proofs that are wrong. The chain of thought looks convincing even when it contains logical errors. Always verify important results independently.

For day-to-day math help — homework, data analysis, engineering calculations, statistics — local reasoning models are more than capable. For publishing mathematical proofs or tackling unsolved problems, cloud APIs are still the safer bet.

Bottom Line

The local math-reasoning landscape keeps moving, and the picks that led this guide a few months ago are a generation behind now. Where it stands in mid-2026:

• Qwen 3.6-35B-A3B posts AIME 2026 92.7 and runs on a 16GB card via --n-cpu-moe offload — 35B reasoning at ~3B compute cost per token.
• Qwen 3.6-27B dense (GPQA-Diamond 87.8) is the strongest open dense reasoner near its size, fully on 24GB with thinking on/off.
• Qwen 3.5 9B with thinking is the small-card default, beating models several times its size.
• The old R1-distills and Phi-4-reasoning-plus still run, but their AIME 2024 numbers no longer lead.

The key insight hasn’t changed: thinking is everything. A model that shows its chain of thought crushes one that just predicts — the same Qwen jumps dramatically when /think is on. If you care about math accuracy, run a model that reasons, and in 2026 that means Qwen 3.6, or Qwen 3.5 on a small card. For image-in math (photos of problems), Gemma 4 is the multimodal pick.

# Just try it — the difference is immediately obvious
ollama run qwen3.5:9b
# Then type: /think Solve: x^3 - 6x^2 + 11x - 6 = 0