Guides
252 practical guides for running AI locally — from first install to advanced optimization.
Recently Updated
- MoE Models Explained: Why Mixtral Uses 46B Parameters But Runs Like 13B Mixture of Experts explained for local AI — why MoE models run fast but still need full VRAM. Mixtral, DeepSeek V3, DBRX compared with dense model alternatives.
- Best Way to Run Qwen 3.6 35B MoE Locally: VRAM, Speed, Setup Qwen 3.6-35B-A3B has 35B total params but only 3B active per token. Real tok/s on RTX 3090, 4090, 5070 Ti, dual 5060 Ti, and M3 Ultra. Quants and setup.
- Gemma 4 26B in 2GB RAM: The MoE Memory Ladder Explained One model, three places its experts can live: VRAM, RAM, SSD. We measured the first two on Gemma 4. TurboFieldfare just added the third.
- Flash-MoE: Run a 397B Model on a 48GB Laptop (Here's How) Flash-MoE streams Qwen3.5-397B from your SSD at 4.4 tok/s using 5.5GB of RAM. Pure C and Metal, no Python. Here's what's real, what's hype, and how to try it.
- Every SSD-Streaming MoE Engine: What's Real, What's Dead Eight engines that stream MoE experts from disk appeared in four months. Three have no license file at all. Here's the verified state of each.
- Best VRAM Cheat Sheet for Local LLMs: Every Model, Every Quant Exact VRAM for Qwen 3.6, Qwen 3.5, Llama, Mistral, and DeepSeek at Q3 through FP16. Lookup tables for 7B, 9B, 13B, 27B, 32B, 70B, and 120B models with real measurements and GPU recommendations. Updated July 2026.
- Mac vs PC for Local AI: Which Should You Choose? An RTX 3090 runs 7B-32B models 2-3x faster than a Mac. A 96GB Mac Studio or a Strix Halo mini-PC (from ~$1,499) loads 70B. Benchmarks, current 2026 prices, and which platform fits.
- How Much Does It Cost to Run LLMs Locally? $200-800 for hardware, $5-15/month in electricity, and a 3-6 month breakeven vs ChatGPT Plus at $240/year. Full cost breakdown with real numbers.
- Free Local AI vs Paid Cloud APIs: Real Cost Comparison A used RTX 3090 is $1,200 now and costs ~$10/month to run. Full break-even math vs OpenAI, Anthropic and Google APIs — including when local never pays back.
- LoRA Training on Consumer Hardware: Fine-Tune Models With 12GB VRAM QLoRA fine-tunes a 7B model on an RTX 3060 12GB in 2-4 hours. Full Unsloth and Axolotl recipes, VRAM tables, and the GGUF export pipeline.
- Building a Distributed AI Swarm for Under $1,100 A complete bill of materials for a three-node distributed AI cluster: RTX 3090 workstation, ThinkCentre M710Q for light inference, Raspberry Pi 5 coordinator. Every part sourced used or cheap, total cost under $1,100.
- How to Update Models in Ollama — Keep Your Local LLMs Current Ollama doesn't auto-update models. Run ollama pull model:tag to grab the latest version — only changed layers download. Use ollama show to check what you have, and a simple loop to update everything at once.
- Best Hardware for Running OpenClaw — Mac Mini vs VPS vs Your Old PC OpenClaw runs 24/7. A Mac Mini M4 draws 4 watts idle. A free Oracle VPS costs nothing. A used ThinkCentre costs $85. Here's which one to pick.
- Local AI for Lawyers: Confidential Document Analysis Without Cloud Risk A federal judge ordered OpenAI to hand over 20 million chat logs. If you're a lawyer using ChatGPT for client work, that's an ethics problem. Local AI keeps everything on your hardware.
- WSL2 + Ollama on Windows: Complete Setup Guide (GPU Passthrough Included) Install Ollama in WSL2 with full GPU acceleration in 20 minutes. GPU passthrough, Open WebUI, Docker Compose, VPN fixes, and the gotchas that will waste your afternoon.
- Why Is My Local LLM So Slow? A Diagnostic Guide Local LLM running slow? Check GPU vs CPU inference, VRAM offloading, quantization, context length, backend choice, and thermals. Find your fix in 60 seconds.
- AI Kill Switch Act vs Open Weights: Can You Shut Down a File? 1,238 AI staff asked Washington to slow the frontier. Every mechanism proposed works at release — the only moment an open release can be touched.
- RTX 5060 Ti 16GB Killed? Local AI Alternatives The RTX 5060 Ti 16GB faces production cuts from GDDR7 shortages. See what is really happening and explore the best alternative GPUs for local AI in 2026.
Most Popular
- Qwen 3.6 Complete Guide: 27B Dense, 35B-A3B MoE, and Which to Use Qwen 3.6 landed in two open-weight flavors: 27B dense and 35B-A3B MoE. Benchmarks, hardware fit, and which variant to run on your GPU.
- Best Qwen 3.5 Setup: When to Stay vs Move to 3.6 (2026) 3.5 is Qwen's stable open workhorse — 3.6 replaced only two tiers, 3.7 went closed. Which 3.5 model on which GPU, when to stay vs move to 3.6.
- Best Local LLMs for Mac in 2026 — M1 through M5 Tested Best model for every Mac tier, 8GB to the 96GB Studio ceiling. Qwen 3.6, Llama 4 Scout, DeepSeek V4, MLX vs Ollama. Why bandwidth, not RAM, sets your speed.
- llama.cpp vs Ollama vs vLLM: One User vs Many (2026) Single-user, the three are closer than benchmark posts admit. Concurrent, vLLM pulls 10-20x ahead. Decision tree, the vLLM VRAM gotcha, mid-2026 versions.
- Run Your First Local LLM in 15 Minutes Install Ollama, pull a model, and chat with AI offline—all in 15 minutes. Works on any Mac, Windows, or Linux machine with 8GB RAM. No accounts, no API keys, no fees.
- GPU Buying Guide for Local AI: Pick the Right Card The complete GPU buying guide for local AI. Covers RTX 3060 through 4090 with VRAM analysis, performance benchmarks, prices, and used vs new buying advice.
- Hugging Face Hacked by AI Agent — Saved by a Local Model (2026) Hugging Face disclosed July 16 that an autonomous AI agent breached its internal infra — now confirmed as OpenAI's own models. The models you download are safe — here's what was and wasn't hit.
- Inkling 975B vs Your 3090: The Real Memory Math (2026) Inkling's smallest 1-bit quant is 270GB. A maxed consumer desktop holds 256GB. The open frontier left consumer hardware behind. Here's the honest math.
- China Made Open Source a Strategy. If It Pulls Back, Who Fills the Gap? If China restricts future open weights, who fills the gap? The West's open-model capacity is real (Ai2, Mistral, Apertus) but scattered and underfunded.
- China May Restrict Its AI Exports — Your Local Models Don't Care China's commerce ministry met Alibaba, ByteDance and Z.ai about walling off top AI models. The US did the same in June. The weights on your disk don't care.
- A100 vs H100 vs L40S vs 4090: Why the Cheaper GPU Costs More to Train On The cheapest GPU per hour is rarely the cheapest per training run. Real 2026 rental prices and total-cost math across the 4090, L40S, A100, H100, and H200.
- Best 24GB Backend Shootout: ik_llama vs BeeLlama vs llama.cpp ik_llama and BeeLlama both finish in 22-23s on the am17an 9-prompt harness vs mainline llama.cpp's 37s — 1.66x and 1.62x speedups via opposite strategies.
- Qwen 3.7 Open Weights Watch: The June Window Is Closing June 19: Qwen 3.7 open weights overdue. 3.5→3.6 cadence projected June 6-14; we're past. Max May 20 (56.6 AAI), VLA, Plus closed. Bench at drop.
- Wicked Fast Qwen 3.6 27B: 60 tok/s with MTP on RTX 3090 (2026) Firsthand bench: 60 tok/s on Qwen 3.6 27B Q4_K_M with MTP on a single RTX 3090 — 1.86x wall-clock speedup over baseline. PR #22673 progress May 6 → May 19.
- Wicked Fast Gemma 4 vs Qwen 3.6 on RTX 3090: 3.10x Tested Same RTX 3090, same llama.cpp build, same bench. Gemma 4 26B-A4B Q4_K_XL: 128 tok/s mean. Qwen 3.6-27B Q4_K_M: 41 tok/s. 3.10x faster, firsthand.
- DFlash vs MTP on RTX 3090: I Tested Both Locally Firsthand head-to-head bench of DFlash + DDTree against MTP (PR #22673) on a single RTX 3090, same Qwen 3.6-27B target. Real numbers, both backends.
- How to Fix Slow Qwen 3.6 27B on RTX 3090 (10-80 tok/s) Qwen 3.6-27B at 12 tok/s on a 3090 when others report 35? The 8-step diagnostic checklist for offload, quants, templates, power limits, and backend choice.
- Lightning 2.6.x Malware: Check Your Local AI Stack PyPI's lightning package was poisoned April 30 with malware that abuses Claude Code hooks. Here's the 5-minute audit I ran on my own 3090 box.
- How to Get 2.5x Faster Qwen on RTX 3090 (Free) I built DFlash on my RTX 3090 and ran the full bench. Real 2.5x speedup on Qwen 3.5 and 3.6 — below the 3.43x README claim, still huge. Here's how.
- Best Way to Run Qwen 3.6 35B MoE Locally: VRAM, Speed, Setup Qwen 3.6-35B-A3B has 35B total params but only 3B active per token. Real tok/s on RTX 3090, 4090, 5070 Ti, dual 5060 Ti, and M3 Ultra. Quants and setup.
- Best Way to Get 2x Token Output on RTX 3090: Qwen 3.6 + DFlash Luce DFlash + DDTree pushes Qwen 3.6-27B Q4_K_M from 35 tok/s to 69 tok/s on a single RTX 3090. Real benchmarks, setup, and honest limits.
- FP4 Just Landed in llama.cpp: NVFP4 vs MXFP4 Explained (2026) NVFP4 in llama.cpp, MXFP4 in ik_llama.cpp. The first practical FP4 quantization for the GGUF ecosystem — what works, what doesn't, and what to test.
- DeepSeek V4 Flash vs Pro: Verdict, Cost, Setup (2026) Flash or Pro? Which DeepSeek V4 to run, what each costs (Haiku-tier pricing), and how to deploy locally. The verdict, the tradeoffs, no launch recap.
- Anthropic Just Cut Off OpenClaw Users — Why Local Models Matter More Than Ever Starting April 4, Claude subscribers can no longer use their subscription for OpenClaw and other third-party harnesses. If you were relying on cloud AI for your agent, here's how to go fully local.
- OpenClaw Critical Sandbox Escape: Update to 2026.3.28 Now Ant AI Security Lab found 33 vulnerabilities in OpenClaw including critical privilege escalation and filesystem sandbox escape. If you're self-hosting, update immediately.
- TurboQuant Explained: How Google's KV Cache Trick Cuts Memory 6x With Zero Quality Loss Google's TurboQuant compresses the KV cache 6x with zero accuracy loss. Here's what it actually does, how it works in llama.cpp and MLX, and what it means for running bigger models on your GPU.
- Intel's $949 GPU Has 32GB VRAM and 608 GB/s Bandwidth: What It Means for Local AI Intel is launching a 32GB VRAM GPU for $949. Here's how it compares to the RTX 3090, RTX 4090, and used GPU market for running local LLMs and Stable Diffusion.
- Best Apple M5 Pro and Max for Local AI (2026) M5 Pro at 307GB/s, M5 Max at 614GB/s (or 460 on the 32-core bin), up to 128GB — now the highest-memory Mac you can buy. Picks for Qwen 3.6 and Llama 3.3 70B.
- Best 8GB GPU Model: How to Set Up Qwen 3.5 9B (Step by Step) Qwen 3.5 9B fits in 6.6GB and beats Qwen 3-class models 3x its size. Setup on Ollama/llama.cpp, quant table, where 9B still fits in the May 2026 lineup.
- Qwen 3.5 Small Models: The 9B Beats Last-Gen 30B — Here's What Matters for Local AI Alibaba's Qwen 3.5 drops 4 small models (0.8B to 9B) — all natively multimodal, 262K context, Apache 2.0. The 9B beats Qwen3-30B on reasoning and destroys GPT-5-Nano on vision. VRAM tables and what to run.
- Best Anime and Stylized Checkpoints for Local Image Generation (2026) Illustrious XL, NoobAI-XL, Animagine, Pony Diffusion, and SD 1.5 anime models compared. VRAM requirements, Danbooru prompting, LoRA picks, and settings for ComfyUI and A1111.
- Best Photorealism Checkpoints for Local Image Generation (2026) Juggernaut XL, RealVisXL, Realistic Vision, and Flux compared for photorealistic AI images. VRAM requirements, recommended settings, sample prompts, and installation for ComfyUI and A1111.
- Replace GitHub Copilot With Local LLMs in VS Code — Free, Private, No Subscription Set up free, private AI code completion in VS Code with Continue + Ollama. Autocomplete, chat, and agentic coding with Qwen models at every VRAM tier. Step-by-step setup, model picks, honest tradeoffs.
- Best Qwen 3.5 Models Ranked: Every Size, Every GPU, Every Quant Complete ranking of all Qwen 3.5 models from 0.8B to 397B. VRAM requirements, speed benchmarks, and which model to pick for your hardware.
- DeepSeek V4: Everything We Know Before It Drops DeepSeek V4 launches next week with native image and video generation, 1M context, and rumored 1T MoE params with only 32B active. Here's what local AI builders need to know and how to prepare.
- OpenClaw Security Report: February 2026 — ClawHub Malware, Google Suspensions, and Critical Fixes 17 security fixes, 341 malicious ClawHub skills, Google banning users, and the creator leaving. Every OpenClaw security event from February 2026.
- Best Local Alternatives to Claude Code in 2026 Aider, Continue.dev, Cline, OpenCode, Void, and Tabby compared. Which open-source coding tools work best with local models on your own GPU?
- Best OpenClaw Alternatives: 11 Tools That Actually Work in 2026 Tested alternatives to OpenClaw for local AI agent workflows — Hermes Agent, Pi Agent, MMX-CLI, VT Code, and seven more. Ranked by setup ease, model support, and what actually works after Anthropic's April 4 subscription cutoff.
- Best OpenClaw Tools and Extensions in 2026 Crabwalk visualizes agent actions in real time, Tokscale catches API bills before they hit $200+, and openclaw-docker locks down deployment. The best 3rd-party tools ranked.
- Fix OpenClaw Token Waste: $150 to $6 Overnight Cut OpenClaw API costs by 97% with three proven fixes: route heartbeats through Ollama, add tiered model routing, and purge session history token bloat.
- OpenClaw ClawHub Alert: 1,103 Malicious Skills Found OpenClaw ClawHub security alert: 1,103 malicious skills found across 14,706 audited. CVE-2026-28458 Browser Relay auth bypass. How to protect yourself now.
- Best Local Models for OpenClaw 2026: Qwen 3.6 + DeepSeek V4 Qwen 3.6-27B dense ties Sonnet 4.6 on agentic coding; 3.6-35B-A3B runs OpenClaw on 16GB VRAM. Plus DeepSeek V4-Flash, sampling tips, VRAM tiers.
- Run LLMs on Mac M-Series: Faster, Without the Gotchas (2026) Foundational how-to for Apple Silicon local AI: unified memory, MLX vs Ollama vs llama.cpp Metal, verification, and the headless Mac Mini AI server.
- Best Way to Set Up OpenClaw (2026 Guide) Run `npx openclaw@latest`, scan a QR code for WhatsApp, and your AI agent is live. Gateway needs just 2-4GB RAM. Add Ollama for local models or connect Claude/GPT-4 via API.
- Best Local Coding Models Ranked: Every VRAM Tier, Every Benchmark (2026) The best local LLMs for coding in 2026, ranked by VRAM tier. Qwen 3.6-27B, 3.6-35B-A3B, DeepSeek V4-Flash, benchmarks, editor setup, and Claude Code alternatives.
- Best VRAM Cheat Sheet for Local LLMs: Every Model, Every Quant Exact VRAM for Qwen 3.6, Qwen 3.5, Llama, Mistral, and DeepSeek at Q3 through FP16. Lookup tables for 7B, 9B, 13B, 27B, 32B, 70B, and 120B models with real measurements and GPU recommendations. Updated July 2026.
- Ollama vs LM Studio: Speed, Setup, and Verdict Ollama gives you a CLI with 100+ models and an OpenAI-compatible API. LM Studio gives you a visual GUI with one-click downloads. Most power users run both—here's when to use each.
Getting Started (8)
- Ollama Troubleshooting Guide: Every Common Problem and Fix GPU not detected? Running at 1/30th speed on CPU? OOM crashes mid-generation? Every common Ollama error with exact diagnostic commands and fixes for Mac, Windows, and Linux. Updated July 2026 for v0.31.x and Qwen 3.5 + 3.6.
- Run Your First Local LLM in 15 Minutes Install Ollama, pull a model, and chat with AI offline—all in 15 minutes. Works on any Mac, Windows, or Linux machine with 8GB RAM. No accounts, no API keys, no fees.
- Ollama 0.30.0: What's New, What's Faster, What Breaks on Upgrade Ollama 0.30.0: llama.cpp integration, flash-attention default for Qwen/Gemma, broader model support. Firsthand upgrade notes, known issues to watch.
- Ubuntu 26.04 Is Built for Local AI — What Actually Changes Ubuntu 26.04 LTS packages NVIDIA CUDA and AMD ROCm in official repos. No more external downloads or dependency nightmares. What's confirmed and what it means for local AI.
- Qwen 3.5 Locally — 27B vs 35B-A3B vs 122B, Which Model Fits Your GPU Qwen 3.5 and 3.6 on local hardware. 27B dense vs 35B-A3B MoE vs 122B compared. VRAM tables, community tok/s on RTX 3090, and which to pick for your card.
Hardware & GPUs (51)
- Best Local LLMs for Mac in 2026 — M1 through M5 Tested Best model for every Mac tier, 8GB to the 96GB Studio ceiling. Qwen 3.6, Llama 4 Scout, DeepSeek V4, MLX vs Ollama. Why bandwidth, not RAM, sets your speed.
- GPU Buying Guide for Local AI: Pick the Right Card The complete GPU buying guide for local AI. Covers RTX 3060 through 4090 with VRAM analysis, performance benchmarks, prices, and used vs new buying advice.
- A100 vs H100 vs L40S vs 4090: Why the Cheaper GPU Costs More to Train On The cheapest GPU per hour is rarely the cheapest per training run. Real 2026 rental prices and total-cost math across the 4090, L40S, A100, H100, and H200.
- How to Run GLM 5.2 Locally: GPU, VRAM & Quant Guide GLM 5.2 is 753B params and 1.51TB at full precision. Run it locally: the live Unsloth quant ladder, every GPU and RAM path, and the quant to actually target.
- RTX 5090 Benchmarks: 5090 vs 4090 vs Used 3090 (2026) 5090 community benches across 4K-131K context, prompt-processing tables, 5090-vs-4090 upgrade math, and InsiderLLM's firsthand 3090 honest-value anchor.
Mac & Apple Silicon (14)
- Best Local LLMs for Mac in 2026 — M1 through M5 Tested Best model for every Mac tier, 8GB to the 96GB Studio ceiling. Qwen 3.6, Llama 4 Scout, DeepSeek V4, MLX vs Ollama. Why bandwidth, not RAM, sets your speed.
- Best Apple M5 Pro and Max for Local AI (2026) M5 Pro at 307GB/s, M5 Max at 614GB/s (or 460 on the 32-core bin), up to 128GB — now the highest-memory Mac you can buy. Picks for Qwen 3.6 and Llama 3.3 70B.
- OpenClaw on Mac: Setup, Optimization, and What Actually Works brew install openclaw-cli, connect Ollama, configure the gateway, and stop fighting macOS. Apple Silicon setup, memory math, launchd config, and the gotchas nobody warns you about.
- What Can You Run on 8GB Apple Silicon? Local AI on a Budget Mac Llama 3.2 3B runs at 30 tok/s. Phi-4 Mini fits with room to spare. 7B models technically load but swap to disk. Honest benchmarks and real limits for 8GB M1/M2/M3/M4 Macs.
- Stable Diffusion on Mac: Image Generation with MLX and Draw Things Draw Things generates SD 1.5 images in 8-15 seconds on an M2 Pro. ComfyUI takes 3x longer. MLX is fastest but code-only. Complete Mac image gen guide with speed tests.
Image Generation (11)
- Stable Diffusion Locally: Getting Started SD 1.5 runs on 4GB VRAM, SDXL needs 8GB, Flux needs 12GB+. Generate unlimited images for free in under 5 minutes with Fooocus or ComfyUI. Setup, models, and first image tips.
- Local AI Upscaling: Make Blurry Images Sharp Without the Cloud Upscayl, Real-ESRGAN, chaiNNer, and ComfyUI can upscale your photos for free on your own hardware. No subscriptions, no uploads, no per-image fees. Even a GTX 1060 works. Here's how to pick the right tool and start.
- Best Photorealism Checkpoints for Local Image Generation (2026) Juggernaut XL, RealVisXL, Realistic Vision, and Flux compared for photorealistic AI images. VRAM requirements, recommended settings, sample prompts, and installation for ComfyUI and A1111.
- Stable Diffusion on Mac: Image Generation with MLX and Draw Things Draw Things generates SD 1.5 images in 8-15 seconds on an M2 Pro. ComfyUI takes 3x longer. MLX is fastest but code-only. Complete Mac image gen guide with speed tests.
- SDXL vs SD 1.5 vs Flux: Which Image Model Should You Run Locally? SDXL vs SD 1.5 vs Flux compared by VRAM, speed, and quality. SD 1.5 needs 4GB, SDXL needs 8GB, Flux needs 12GB+. Benchmarks on real GPUs inside.
Models (45)
- Qwen 3.6 Complete Guide: 27B Dense, 35B-A3B MoE, and Which to Use Qwen 3.6 landed in two open-weight flavors: 27B dense and 35B-A3B MoE. Benchmarks, hardware fit, and which variant to run on your GPU.
- Best Qwen 3.5 Setup: When to Stay vs Move to 3.6 (2026) 3.5 is Qwen's stable open workhorse — 3.6 replaced only two tiers, 3.7 went closed. Which 3.5 model on which GPU, when to stay vs move to 3.6.
- Quantization Explained: What It Means for Local AI Q4_K_M shrinks a 7B model from 14GB to ~4GB while keeping 90-95% quality. What every quantization format means, how much VRAM each saves, and which to pick for your GPU.
- Hugging Face Hacked by AI Agent — Saved by a Local Model (2026) Hugging Face disclosed July 16 that an autonomous AI agent breached its internal infra — now confirmed as OpenAI's own models. The models you download are safe — here's what was and wasn't hit.
- Kimi K3 & Qwen 3.8: Open Weights You Can't Run (2026) Kimi K3's 2.8T weights shipped on time and need 64 accelerators to load. Qwen 3.8's still haven't shipped. What to run on your own GPU instead.
Software & Tools (29)
- Local LLMs vs ChatGPT: An Honest Comparison ChatGPT has web search, voice mode, and GPT-5.2. Local LLMs have privacy, no subscriptions, and no rate limits. Here's when each one wins, what the cost math actually looks like, and why most power users run both.
- llama.cpp vs Ollama vs vLLM: One User vs Many (2026) Single-user, the three are closer than benchmark posts admit. Concurrent, vLLM pulls 10-20x ahead. Decision tree, the vLLM VRAM gotcha, mid-2026 versions.
- Run Your First Local LLM in 15 Minutes Install Ollama, pull a model, and chat with AI offline—all in 15 minutes. Works on any Mac, Windows, or Linux machine with 8GB RAM. No accounts, no API keys, no fees.
- Every SSD-Streaming MoE Engine: What's Real, What's Dead Eight engines that stream MoE experts from disk appeared in four months. Three have no license file at all. Here's the verified state of each.
- How to Fix Slow Qwen 3.6 27B on RTX 3090 (10-80 tok/s) Qwen 3.6-27B at 12 tok/s on a 3090 when others report 35? The 8-step diagnostic checklist for offload, quants, templates, power limits, and backend choice.
AI Agents & OpenClaw (46)
- Qwen 3.6: Why Q4 Quant Breaks Local Coding Agents (And the Fix) A viral thread says Q4-to-Q6 fixes Qwen 3.6 coding, but the test was confounded. What four independent reports show about the quant tax on coding agents.
- Anthropic Just Cut Off OpenClaw Users — Why Local Models Matter More Than Ever Starting April 4, Claude subscribers can no longer use their subscription for OpenClaw and other third-party harnesses. If you were relying on cloud AI for your agent, here's how to go fully local.
- 12 Architecture Patterns from the Claude Code Leak -- Ranked by Payoff for Local AI Claude Code's leaked source reveals 12 engineering patterns that power a $2.5B product. Ranked by how much each one improves your local AI agent setup.
- OpenClaw Critical Sandbox Escape: Update to 2026.3.28 Now Ant AI Security Lab found 33 vulnerabilities in OpenClaw including critical privilege escalation and filesystem sandbox escape. If you're self-hosting, update immediately.
- Claude Code's Source Just Leaked: What 500K Lines of TypeScript Reveal About AI Coding Agents Claude Code's full source was exposed via npm source maps. Here's what the leaked architecture reveals about multi-agent orchestration, and what it means for local AI agent builders.
Use Cases (41)
- Local LLMs vs ChatGPT: An Honest Comparison ChatGPT has web search, voice mode, and GPT-5.2. Local LLMs have privacy, no subscriptions, and no rate limits. Here's when each one wins, what the cost math actually looks like, and why most power users run both.
- Stable Diffusion Locally: Getting Started SD 1.5 runs on 4GB VRAM, SDXL needs 8GB, Flux needs 12GB+. Generate unlimited images for free in under 5 minutes with Fooocus or ComfyUI. Setup, models, and first image tips.
- Local AI for Accounting and Tax: Keep Your Financial Data Off the Cloud Local LLMs can categorize transactions, draft client letters, extract receipt data, and answer questions over tax documents — without sending a single number to OpenAI or Google. What works, what doesn't, and how to set it up.
- Running OpenClaw on 4GB, 6GB, and 8GB GPUs: What Actually Works OpenClaw on low VRAM GPUs: 4GB is rough, 6GB is marginal, 8GB is where it starts working. Model picks, quantization tricks, partial offload, and when to just use a cloud API instead.
- Local AI for Therapists: Session Notes, Treatment Plans, and Client Privacy Without the Cloud Run AI on your own hardware to draft session notes, treatment plans, and clinical letters without sending client data to OpenAI. HIPAA-friendly setup for therapists.
Architecture & Theory (16)
- Quantization Explained: What It Means for Local AI Q4_K_M shrinks a 7B model from 14GB to ~4GB while keeping 90-95% quality. What every quantization format means, how much VRAM each saves, and which to pick for your GPU.
- 12 Architecture Patterns from the Claude Code Leak -- Ranked by Payoff for Local AI Claude Code's leaked source reveals 12 engineering patterns that power a $2.5B product. Ranked by how much each one improves your local AI agent setup.
- TurboQuant Explained: How Google's KV Cache Trick Cuts Memory 6x With Zero Quality Loss Google's TurboQuant compresses the KV cache 6x with zero accuracy loss. Here's what it actually does, how it works in llama.cpp and MLX, and what it means for running bigger models on your GPU.
- Model Routing for Local AI — Stop Using One Model for Everything You're running one model for every task. That wastes VRAM, burns electricity, and gives worse results. Model routing sends each task to the right model at the right cost. Here's how to set it up.
- Speculative Decoding: Free 20-50% Speed Boost for Local LLMs Speculative decoding uses a small draft model to predict tokens verified by the big model. Same output, 20-50% faster. Setup guide for LM Studio and llama.cpp.
Troubleshooting (20)
- Ollama Troubleshooting Guide: Every Common Problem and Fix GPU not detected? Running at 1/30th speed on CPU? OOM crashes mid-generation? Every common Ollama error with exact diagnostic commands and fixes for Mac, Windows, and Linux. Updated July 2026 for v0.31.x and Qwen 3.5 + 3.6.
- LLM Running Slow? Two Different Problems, Two Different Fixes Slow local LLM? Separate time-to-first-token from generation speed. Fix prompt processing with batch size and Flash Attention. Fix tok/s with GPU layers, quantization, and context length.
- Why Your Local LLM Is Slow: The num_ctx VRAM Overflow Nobody Warns You About DeepSeek-R1 14B went from 35 tok/s to 4.8 tok/s on the same GPU. The fix was one parameter. How num_ctx silently overflows VRAM and kills inference speed.
- Qwen2.5-VL Not Loading in LM Studio? Fix mmproj and Vision Errors Fix every Qwen2.5-VL error in LM Studio: missing mmproj, 'model type not supported', no eye icon, vision crashes. Exact fixes with file paths.
- Open WebUI Not Connecting to Ollama? Every Fix Docker networking, wrong OLLAMA_BASE_URL, localhost confusion, WSL2 isolation, missing models, random disconnects. Every Open WebUI + Ollama connection problem with the exact fix.