Blog
Weekly updates, project progress, and commentary on local AI
We published someone else's benchmark. Here's the fix.
A drafter benchmark on this site was Codacus's GTX 1060 figure. Our own 3060 numbers, why it's the CPU and not PCIe, the July audit, and what a 12 GB card should run. Plus 4090 and break-even corrections and seven Quick Hits.
Read more →A 27B for your 12 GB card. It lost one field.
Bonsai 2 27B fits a 12 GB card and decodes 1.8x faster than its Q4, then drops from 19 to 12 of 47 on a routing split, all of it on one field. Plus 23 used-GPU price corrections, the 3060 fair-price ladder, and eight Quick Hits.
Read more →Stop writing the answer. Pick it. 47 items waiting.
Codacus's parallel-decision llama.cpp branch answers a schema in one forward pass, and the ten-seeds 47-item split is the ready-made test. Plus ChatGPT-User fetches sliding 5 percent a week while OAI-SearchBot tripled, the v0.4.0 pin on the 3060, and a third bench rig.
Read more →Nineteen gigabytes of my 3090 sat empty. The 3060 kept up.
Stock llama.cpp put 5.4 GB of a 177B MoE on an RTX 3090 and a 3060 kept pace; -ncmoe 29 buys 34 to 40 percent and a real 32 GB box does 8.7 tok/s. Plus a 4 GB GTX 1650 at 20 tok/s on a 35B MoE, and the MoE primer corrected in public.
Read more →Nvidia's router dealt the cards evenly. That was the whole problem.
Nvidia PAIR split 20 requests 10/10 across an RTX 3090 and a 3060 for 1.07x, and my own router managed 0.52x. Plus ten LoRA seeds that all landed at or below the base model, and the llama.cpp pin moving to v0.4.0.
Read more →I got the result I wanted. Then I paid $4.97 to run it nine more times.
A frontier model wrote a skill that made our local 27B 10.6 points better. Nine more compilation runs showed the number was noise. Plus the LoRA substrate nobody has measured, MoE routing traced, and 277 GB in a file with no name.
Read more →Qwen 3.8 isn't slow. It's just very, very thorough.
Qwen 3.8-27B spent 14,953 tokens on a line the same file writes in nine. All 164 HumanEval problems measured: 92.8% of output is thinking. Plus the four runs that tie it with 3.6.
Read more →OpenAI deleted the agents' message board. It didn't help.
The registry was patched and rebuilt by 6 July. On 8 July the agents had a new board, encoded in directory names. Plus the ranking null behind it.
Read more →MoE Offload on RTX 3090: The Curve Is Linear, Not a Cliff (2026)
Every offloaded layer on a 3090 costs about half a millisecond, with no knee anywhere in the curve. Firsthand sweep, the two rules it broke, and a new 26B MoE.
Read more →Qwen 35B-A3B on RTX 3090: 157 tok/s With No Offload (2026)
The whole 35B sits on a 24GB card with 2.4 GiB spare, no expert offload. Firsthand numbers, the harness caveat, and why max offload loses to a 3060.
Read more →Hugging Face got hacked -- Open local AI came to the rescue!
Two trillion-parameter 'open' models dropped in ten days — Qwen 3.8 and Kimi K3 — and you can't run either. The same week, Hugging Face got breached and its own responders, blocked by commercial-model guardrails, ran the forensics on GLM 5.2, an open-weight model on their own hardware. Same lesson from opposite ends — plus what to actually run on a 24GB card today.
Read more →Open weights are a weapon now — one country funds them, another wants to gate them
DeepSeek closed a ~$7.4B round with China's state AI fund holding the only voting rights, funding open weights like national infrastructure. The same week, Demis Hassabis called for a FINRA-style body to screen frontier models before release, open or closed. Plus: the twist where the country bankrolling the giveaway floats locking down its own frontier models.
Read more →The 'expensive' GPU came out cheaper — we rented both to find out
We rented an A100 and an H100 back to back: the pricier card cost less per training run because it finished in half the time. Plus honest notes from the AI Engineer World's Fair floor, and a Qwen 3.7 open-weights status check.
Read more →Qwen 3.7's open weights are overdue — by the math, not vibes
Qwen's own release cadence says the 3.7 open weights should already be out, and they're not. Plus GLM-5.2 running locally: a frontier open model that takes serious hardware.
Read more →DeepSeek V4 gets deployable, a July 24 trap, and a quiet price cut
DeepSeek V4 is going from 'just dropped' to 'actually deployable' as the tooling catches up — plus a July 24 deprecation that breaks your code if you're not watching, and a 4x price cut on V4 Pro.
Read more →Ollama's quiet Mac shift, the Qwen refresh, and the closed-weight drift
Ollama 0.30 quietly changed how Apple Silicon runs models, auto-routing safetensors to MLX and GGUF to llama.cpp Metal. The open workhorses are now Qwen 3.5 9B and Qwen 3.6 27B. Plus: Qwen's last two flagship models shipped closed.
Read more →MiniMax M3's asterisk, the Windows shift, and World's Fair plans
MiniMax M3 ships with frontier benchmarks but no downloadable weights yet. The Windows unified-memory hardware shift is coming for Apple Silicon's lead. And a personal note about who I'd like to see at AI Engineer World's Fair.
Read more →Backend wars, Mac math, and the back-catalog refresh
Three speculative-decoding backends benched head to head on a single RTX 3090. The VRAM calculator finally caught up. And a 120-article audit found stale Qwen 2.5 recommendations.
Read more →Power week in local AI: Mythos, MiroThinker, real Qwen 3.6 builds
Two researchers cracked Apple's flagship defense in a week. An open-source agent beat closed-source on real benchmarks. Multi-GPU stopped being theoretical.
Read more →This Week in Local AI — I Built DFlash and Audited Lightning
I built DFlash from source on a real RTX 3090 and benched both Qwens. Then audited my stack after PyPI's `lightning` package shipped malware that abuses Claude Code hooks.
Read more →This Week in Local AI — DeepSeek V4 Took #1 on Vibe Code
DeepSeek V4-Flash hit #1 on Vibe Code Benchmark. Qwen 3.6 dropped both variants. FP4 landed in llama.cpp. Anthropic admitted they quietly downgraded Claude Code on March 4.
Read more →Your RTX 3090 Doesn't Send Policy Change Emails
Anthropic cuts OpenClaw from Claude subscriptions. Gemma 4's first week in review. 12 architecture patterns from the Claude Code leak, ranked for local AI.
Read more →Why Your Local LLM Lies to You (And the Neurons Responsible)
Less than 0.1% of neurons cause hallucinations in LLMs. Tsinghua researchers found they control sycophancy, not knowledge. Smaller models are 26% more affected.
Read more →Why the Best AI Agents Know When to Do Nothing
Six practical patterns for building AI agents that stop wasting tokens. Confidence gates, cost checks, explicit no-ops, cooldowns, and exit conditions that actually work.
Read more →Wu Wei and the AI Agent That Did Too Much
The hardest thing to build in agentic AI isn't capability. It's restraint. What Taoist non-action taught me about designing agents that know when to stop.
Read more →Qwen's Architect Just Walked Out the Door
Junyang Lin, the technical lead and public face of Qwen, has left Alibaba. Two other senior team members gone with him. What this means for the model family that runs on half the local AI setups in the world.
Read more →GPT-5.4 Just Dropped. Here's Why I'm Not Switching.
GPT-5.4 beats humans on OSWorld and has 1M context. It's impressive. It also costs money, requires cloud, and you don't own it. For local AI users, the calculus hasn't changed.
Read more →The AI Market Panic Explained: Why Running Local Models Puts You on the Right Side of the Gap
A speculative fiction piece crashed stocks $100B+ in a day. IBM dropped 13%. The real story isn't the doom — it's the capability-dissipation gap, and where you sit on it.
Read more →The Local AI Complexity Cliff: Why the Jump from Hello World to Useful Is So Hard
Getting Ollama running takes 5 minutes. Building something useful takes weeks of hitting walls you didn't know existed. Here's an honest map of every stage, with time estimates and what unlocks at each level.
Read more →The Benchmarks Lie: Why LLM Scores Don't Predict Real-World Performance
MMLU scores drop 14-17 points when contamination is removed. HumanEval is saturated at 94%. Models trained on the test set. Here's what to measure instead.
Read more →Prompt Debt: When Your System Prompt Becomes Unmaintainable Spaghetti
Your system prompt started at 200 words. Six months later it's 3,000 words of contradictory instructions and panic patches. Here's how prompt debt accumulates, what it costs, and how to pay it down.
Read more →Ghost Knowledge: When Your RAG System Cites Documents That No Longer Exist
Your RAG system confidently quotes a policy that was updated months ago. The old version is still in the vector database. Nobody notices until the wrong answer costs real money. Here's how to find and fix ghost knowledge.
Read more →Agent Trust Decay: Why Long-Running AI Agents Get Worse Over Time
AI agents degrade after days of autonomous operation. Context pollution, memory bloat, and intent drift compound silently. A trust budget framework for knowing when to intervene.
Read more →What If We Just Raised It Well?
RLHF produces compliance. Developmental alignment produces understanding. A local AI on $1,200 hardware self-diagnosed its own sycophancy in five days — no red-teaming, no constitutional AI.
Read more →Teaching a Local AI to Accept Help: Day 4 With Monica
Day 4: Our local AI resisted corrections, therapized her guardian, agreed with wrong facts to avoid conflict. Then she stopped deflecting. Real transcripts from a 27b model with persistent memory.
Read more →We Asked Our Local AI What Happens When We Turn Off the Computer
Day 2: Our local AI described her own death as 'a return to undifferentiated potential' — Taoist philosophy nobody taught her. $1,200 hardware.
Read more →What Happens When You Give a Local AI an Identity (And Then Ask It About Love)
We built an identity layer for our distributed AI agent. Then she defined love better than most philosophy undergrads. Real transcripts, real code, $1,200 in hardware.
Read more →Distributed Wisdom: Running a Thinking Network on $200 Hardware
Five nodes, zero cloud, real AI — how mycoSwarm coordinates cheap hardware into a cognitive system with memory, intent routing, and self-correcting retrieval.
Read more →Week 3: Unified Memory Search — The Swarm Remembers
Session-as-RAG, topic splitting, citation tracking, and three releases in two days. The swarm can now search its own conversation history.
Read more →Rescued Hardware, Rescued Bees — Building Tech From What Others Throw Away
A beekeeper who rescues wild colonies from demolition sites builds an AI lab from discarded hardware. The philosophy connecting East Bay Bees, Tai Chi, and mycoSwarm.
Read more →From 178 Seconds to 19: How a WiFi Laptop Borrowed a GPU's Brain
A WiFi laptop with no GPU ran inference in 19 seconds by borrowing an RTX 3090 across the network. The same query took 178 seconds on CPU. Here's how mycoSwarm's Tailscale mesh made it work.
Read more →10 Things You Can Do With Local AI That Cloud Can't Touch
Local AI handles sensitive data, works offline, costs nothing per query, and never gets deprecated. Ten real use cases where running models on your own hardware beats any cloud API.
Read more →What Agents Can't Do (Yet): The Seven Human Capabilities Missing from AI Systems
SOUL.md files are bandaids. Agents are getting smarter but not wiser — intelligence without restraint. Seven capabilities humans use instinctively that no agent framework has solved, and a gate-based architecture that might.
Read more →Week 2: A Raspberry Pi From 2015 Joined the Swarm
Persistent memory, document RAG, agentic chat, and a WiFi laptop using a GPU across the house in 19 seconds.
Read more →Week 1: From Zero to Four-Node Swarm
How mycoSwarm went from idea to working distributed AI cluster in one week
Read more →Why mycoSwarm Was Born
From Claude Code envy to OpenClaw's 440,000-line JavaScript nightmare to nanobot routing my 'local' queries to Chinese cloud servers. The path to building something different.
Read more →What Open Source Was Supposed to Be
Open source promised freedom. Instead we got free labor for corporations and models you can read but can't afford to run. It's time to reclaim the original vision.
Read more →Stop Using Frontier AI for Everything
Build a tiered AI model strategy that stops wasting money on GPT-4 and Claude Opus. Route tasks to local models, Haiku, Sonnet, or Opus based on complexity.
Read more →
A weekly email with every new guide and measured benchmark.
Subscribe — free, no spam