Quantization
Inkling 975B vs Your 3090: The Real Memory Math (2026)
Inkling's smallest 1-bit quant is 270GB. A maxed consumer desktop holds 256GB. The open frontier left consumer hardware behind. Here's the honest math.
How to Run GLM 5.2 Locally: GPU, VRAM & Quant Guide
GLM 5.2 is 753B params and 1.51TB at full precision. Run it locally: the live Unsloth quant ladder, every GPU and RAM path, and the quant to actually target.
Qwen 3.6: Why Q4 Quant Breaks Local Coding Agents (And the Fix)
A viral thread says Q4-to-Q6 fixes Qwen 3.6 coding, but the test was confounded. What four independent reports show about the quant tax on coding agents.
FP4 Just Landed in llama.cpp: NVFP4 vs MXFP4 Explained (2026)
NVFP4 in llama.cpp, MXFP4 in ik_llama.cpp. The first practical FP4 quantization for the GGUF ecosystem — what works, what doesn't, and what to test.
Best 8GB GPU Model: How to Set Up Qwen 3.5 9B (Step by Step)
Qwen 3.5 9B fits in 6.6GB and beats Qwen 3-class models 3x its size. Setup on Ollama/llama.cpp, quant table, where 9B still fits in the May 2026 lineup.
Best Qwen 3.5 Models Ranked: Every Size, Every GPU, Every Quant
Complete ranking of all Qwen 3.5 models from 0.8B to 397B. VRAM requirements, speed benchmarks, and which model to pick for your hardware.
Running 70B Models Locally — Exact VRAM by Quantization
Llama 3.3 70B needs 43GB at Q4, 75GB at Q8, 141GB at FP16. Every quant level, which GPUs fit, real speeds, and when a 27B or MoE model is the smarter buy.
Mixtral 8x7B & 8x22B VRAM Requirements
Mixtral 8x7B and 8x22B VRAM requirements at every quantization level — plus why the 'every expert must sit in VRAM' rule Mixtral taught no longer holds in the A3B era.
Model Formats Explained: GGUF vs GPTQ vs AWQ vs EXL2
GGUF vs GPTQ vs AWQ vs EXL2/EXL3 model formats explained — plus the new FP4 (MXFP4/NVFP4) and Apple's MLX. What each does, which tools run it, and how to choose for your GPU.
Best VRAM Cheat Sheet for Local LLMs: Every Model, Every Quant
Exact VRAM for Qwen 3.6, Qwen 3.5, Llama, Mistral, and DeepSeek at Q3 through FP16. Lookup tables for 7B, 9B, 13B, 27B, 32B, 70B, and 120B models with real measurements and GPU recommendations. Updated July 2026.
Quantization Explained: What It Means for Local AI
Q4_K_M shrinks a 7B model from 14GB to ~4GB while keeping 90-95% quality. What every quantization format means, how much VRAM each saves, and which to pick for your GPU.