Every SSD-Streaming MoE Engine: What's Real, What's Dead

📚 Related: The MoE Memory Ladder · Flash-MoE: 397B on a Laptop · MoE Models Explained · Qwen 3.6 35B MoE Locally · Our benchmark dataset

In March, one person published an engine that ran a 397-billion-parameter model off a laptop SSD. By the end of July there were eight of them.

That’s the story. Not any single project, and definitely not which one is fastest, because as you’ll see, the speeds these projects publish cannot be laid against each other without lying. The story is that a technique went from one demo to a small field with its own lineage in about four months, and that a third of that field is unusable for reasons nobody writing about it has mentioned.

One thing to be clear about up front: we have not run any of these. Everything below is verified repo state (existence, language, license file, commit history, contributor count) plus figures the projects publish themselves. Where we have our own measurements, they’re of llama.cpp, which is not one of the eight, and they’re labelled.

The eight, verified

Checked against the GitHub API on 1 August 2026.

The oldest is hypura at 13 March. The newest is TurboFieldfare at 17 July. Four months, start to finish.

Star counts are a poor proxy for anything, and this table is the clearest illustration I’ve seen. Pulsar has 76 stars and runs eleven model architectures. flash-moe has 4,000 and has not been touched since March.

One caveat on that table: hypura is in it on a technicality. It bills itself as a storage-tier-aware scheduler for Apple Silicon rather than an MoE engine, placing tensors across GPU, RAM and NVMe by access pattern. Expert streaming is one thing it does, for Mixtral. Its other headline number, Llama 70B at 0.3 tok/s, is a dense model with no experts to stream at all. It belongs in the survey because it solves the same problem, but it isn’t the same kind of thing as the other seven, and it stopped in April.

Three more exist below the threshold I’d take seriously: tonbistudio/moe-ssd-streaming-windows (25 stars, one day of commits), ssd-moe/deepseek-v4-flash-mlx (2 stars), GOBA-AI-Labs/moe-stream (3 stars). Each is a single-day drop with no follow-up. I mention them so the count is honest rather than tidy.

Finding one: this field has a lineage

The eight are not eight independent inventions, which is what the coverage implies by treating each launch as a standalone story.

antirez, the author of Redis, published ds4 on 6 May. It grew into DwarfStar: DeepSeek V4 Flash and PRO plus GLM-5.2, across Metal, CUDA and ROCm, consuming GGUF. In SSD mode the non-routed weights stay resident and routed experts load from the GGUF on cache miss. It carries 40 contributors and describes itself, in its own README, as “beta quality” and “very fast changing.”

On 4 July, neutronstar appeared: explicitly “CUDA fork of antirez/ds4.” Nine days later the same author started pulsar, a ground-up Rust and CUDA rewrite rather than a C fork, and neutronstar stopped. Pulsar now claims eleven architectures including GLM-5.2 743B, Kimi K2.7, DeepSeek-V4-Flash, Gemma 4 and Qwen3.6-35B-A3B, with routed experts on NVMe and everything that makes decisions resident in VRAM.

So one branch of this field reads ds4 → neutronstar → pulsar, in under three months, ending in a rewrite in a different language.

Colibri took the opposite structural bet. Rather than one engine that dispatches across architectures, it ships a separate C amalgam per model: colibri.c for GLM-5.2, plus kimi_k3.c, inkling.c and olmoe.c. There’s no model registry and no config-driven loading. Adding a model means writing a new engine against shared safetensors, quantization and tokenizer infrastructure. That sounds like a weakness and it partly is, but it’s also why the project has 87 contributors: the work parallelises cleanly, because two people porting two models barely touch the same lines.

You can watch that happening right now. A contributor has an open pull request adding DeepSeek V4-Flash as its own engine, roughly 21,000 lines across 56 files, which the maintainer has confirmed works against the current checkpoint but has not merged. It’s being split into two stacked PRs and is blocked on a scale format the shared code doesn’t handle yet.

rabbit is the odd one. It’s Rust, it runs Kimi K3 and GLM-5.2, and its description (“experts streamed from disk. Tiny engine, immense model”) is near-verbatim Colibri’s tagline. It also runs a checkpoint published in Colibri’s own int4 format. Whether that’s a port, a tribute, or convergent phrasing, I could not establish, and I’d rather flag it than guess.

Finding two: none of these numbers can be compared

Every project publishes tokens per second. Here they are, and then here’s why you should not rank them.

Read that top to bottom and pulsar looks roughly 60 times faster than Colibri. It is not. Pulsar’s 51.8 is a 35B model with 3B active; Colibri’s 0.868 is a 284B model. Those are different jobs. TurboFieldfare’s own two rows differ by six times on identical code and an identical model, purely because one machine has 8GB and the other 24GB.

Different models, different active-parameter counts, different quantizations, different storage speeds, different context depths, different definitions of what counts as decode. A league table built from this would be fiction, and building one is the single easiest way to write this article badly.

Two entries deserve individual notes.

rabbit’s 50-70 seconds per token is the most honest number in the survey. The project publishes it under a heading called “Honest numbers,” alongside a 610-second model load and a 412-second prefill, and immediately says it’s “the correctness-first floor, not a tuned number” — the point being that it validated bit-exact against Moonshot’s PyTorch reference before optimising anything. Its other architecture, GLM-5.2, has since gone 3.5× faster across eight measured versions. Publishing your slow, correct floor is rarer than it should be.

flash-moe is the cautionary case. Its widely-quoted speed is 5.74 tok/s. That’s the 2-bit configuration, which the author’s own results file flags as emitting \name\ instead of "name" in JSON, breaking tool calling. The production 4-bit config is 4.36. We wrote that up in March, in detail, including a section on the trap. The caveat has been public since the day the number was, sitting in the same repo. Bare numbers travel. The conditions attached to them do not, and that’s a property of how this stuff gets shared rather than any failure by the author, who documented it properly.

Finding three: three of the eight have no license

This is the part with a practical consequence, and I have not seen it mentioned anywhere.

flash-moe, hypura and rabbit ship no license file. Confirmed directly: LICENSE returns 404 on each default branch, and there’s no license block in the README. That isn’t an unusual license or a permissive default. Under copyright, code published without a license grant is all rights reserved. Being public on GitHub gives you the right to view and fork within the platform’s terms. It does not give you permission to use, modify, or ship it.

The split is clean:

For a weekend experiment on your own machine this is mostly academic. For anything that touches work, it’s the first thing to check and the easiest to miss, because a 4,000-star repo with a polished README reads as more established than a 76-star one that happens to have an actual MIT file in it.

The fix is usually trivial, since maintainers often just forget. In fairness, we haven’t asked: we have not opened an issue on any of the three, so this is a gap we’re pointing at rather than one we’ve tried to close. If you want to use one of them, the trackers are open — flash-moe (19 issues, none answered since March), hypura, rabbit — and asking beats assuming.

The upgrade question, which inverts

Here’s a finding that only becomes visible once you put SSD streaming next to the older technique, and it’s the one that changes what you should buy.

There are two ways to run an MoE that won’t fit in VRAM. llama.cpp’s --n-cpu-moe parks routed experts in system RAM. The eight engines above put them on disk. They respond to a memory upgrade in opposite directions.

With experts in RAM, capacity is irrelevant. This is ours, measured. Our two bench boxes run the same Qwen 3.6-35B-A3B at full offload:

Miu has exactly twice the RAM. The throughput ratio is 1.285×, which tracks the measured bandwidth ratio of 1.220×, nowhere near the 2× capacity ratio. Both boxes land at 63-66% of their own memory ceiling despite different CPUs, core counts and DDR4 grades. At full offload this model moves 611.5 MB per token, computed from the GGUF’s actual tensor bytes, and that figure doesn’t change if you add sticks. The experts already fit. They just have to be read, every token, forever.

Worth knowing before you spend anything: on Miu, running -t 12 instead of -t 8 on this workload drops throughput from 32.62 to 16.46 tok/s. Using every SMT sibling on a bandwidth-bound job halves it. That’s free to fix.

With experts on disk, capacity is the biggest lever there is. This is theirs, cited. On Colibri’s V4-Flash pull request, contributor DrewZt reported throughput against RAM budget on one machine: 16 GiB gave 0.087 tok/s, 64 GiB gave 0.240, and 108 GiB gave 0.868. Roughly a 10× spread with the disk unchanged.

Those are three reported points from two dates on a moving branch, not a controlled sweep, so treat the shape rather than the values. But the shape is clear and the mechanism explains it: RAM here is the cache in front of the disk. Buy more of it and more of the working set stops being read at all.

So the advice reverses depending on which technique you’re running. If you’re on --n-cpu-moe, more RAM buys you nothing and you’re shopping for memory channels. If you’re on any engine in the table above, RAM capacity is the first thing to spend on, ahead of a faster drive.

What we haven’t done

We have not run a single one of these eight engines. This is a map of a field, assembled from repository state and self-published figures, and it should be read as that rather than as a review.

The measurement that would make it a review is the one nobody has published: one machine, one model, several engines. Every number above comes from a different rig, and until someone holds the hardware fixed there is no way to say whether pulsar’s approach genuinely beats Colibri’s or whether it just had a better GPU that day.

Our own ladder piece covers TurboFieldfare in depth, and our March writeup covers flash-moe. Both are single-project articles. This one deliberately isn’t.

The bottom line

If you want to try this today on a Mac, ds4 and TurboFieldfare are the two with real licenses and recent commits, and ds4 calls itself beta in its own README. On Linux with an NVIDIA card, pulsar is the most capable thing here by breadth and almost nobody has noticed it. Colibri is the most populated project and the safest bet if you want something that will still exist in six months.

Do not start from flash-moe or hypura. They stopped in March and April, and neither one gives you permission to use it.

And whatever you pick: the number in the headline was measured on someone else’s machine, on a different model, at a quantization the author may have flagged as broken. That’s not a knock on any of these projects. Most of them documented their caveats carefully. It’s just that the caveats stay in the repo, and the numbers go everywhere.