Strata HotRod, Tested: Too RAM-Hungry for a 32 GB 3060, Slower on a 3090
📚 More on this topic: Strata vs llama.cpp on a 3060 · Strata vs llama.cpp on an RTX 3090 · Every SSD-Streaming MoE Engine · A 177B Model on a 3060: The 32 GB Number
Strata is an engine that runs one model family, Qwen3.8-Flash-Next, and streams the experts it can’t fit from RAM and SSD. Strata HotRod is a fork of it. Its README promises 50-70 tok/s decode on a 12 GB card with 30 GB of RAM, the low-memory, SSD-streaming case. Then a YouTube reviewer, No Place Like Localhost, posted “Doubling Qwen Flash Next speed with Strata HotRod”. On their RTX 4070 12 GB, HotRod read a 4K prompt 5.3x faster than upstream and decoded short chats 1.4x faster.
The reviewer’s machines both have 125 GB of RAM, so the whole model sits in memory. Neither tested the case the README is about. I have that box: an RTX 3060 12 GB, an i7-7700 and 32 GB of DDR4-2133 on PCIe 3.0. I also have a 3090 with 64 GB, where the model fits in RAM, for the reviewer’s other result.
Here’s the bottom line first: on my hardware, HotRod didn’t beat upstream Strata in any result my rules let me count. On the 3060 its default setup used so much RAM that my memory guard ended the run during load. On the 3090 it was slower than upstream, and decoded slower than the upstream release it forked from. Stick with upstream Strata.
I pre-registered it before running anything: three arms, the predictions, the void rules and a memory floor, all published before the first number.
The three arms
| arm | what it is | engine |
|---|---|---|
| U | upstream Strata v0.2.16, the current release | 0.1.62 |
| H | Strata HotRod at commit 251dec43 | 0.1.45 + 471 fork commits |
| B | upstream Strata v0.1.45, the release HotRod forked from | 0.1.45 |
B is there because the reviewer’s baseline was a newer upstream than HotRod’s base. H vs B is the fork’s own effect, and U vs B is what upstream changed since.
Same model file for all three (ISTA’s Qwen3.8-Flash-Next GSQ-RCO IQ3_S, 83 GB), 131,072 context, identical setup flags, each config used exactly as its setup wrote it. All three engines were compiled from source with the network cut. Each cell is three priming plus three measured requests at 4K, 16K and 32K tokens, with a fresh nonce on every prompt so no cache helps. A cell is void if the prompt token counts spread by more than 3, if anything swaps out, or if the PCIe link drops a generation. Void cells get one re-run, and void twice stays void.
RTX 3060, 32 GB: HotRod’s run ended during load
The memory floor was simple: if available RAM drops under 2 GiB, the run ends. Upstream and B never came close.
HotRod’s setup on this box turns on a RAM tier for experts. At load, the engine copies about 23 GiB of experts into RAM as a resident cache, sized automatically from free memory, plus 1.55 GiB of pinned memory for KV streaming. On a 32 GB machine that leaves about 1.9 GiB. Thirty-six seconds into loading, available memory hit 1,682 MiB, and the run ended. Under the rules that ends the arm. HotRod has no scored numbers on the 3060, and both of its 3060 predictions are unscorable. (My guard killed the wrapper script but not the server under it, so Strata’s benchmark tool kept going unguarded for another 75 seconds before I stopped it by hand. Those numbers are in the record, marked post-trip, and I’m not using them. The harness now kills the whole tree.)
I wanted to know whether that was a leak or a fixed allocation, so I ran two more loads, both unscored, sampling every process once a second:
- Default config: only the engine grows. Its memory climbs about 1.1 GB per second, 0.4 to 25.3 GB in 30 seconds, then flattens. It’s a deliberate allocation that stops where it was told to, not a leak. The budget is just sized so tight that a 32 GB machine is left with under 2 GiB.
- Same install, one change,
--resident-budget-gib 20: it loaded, ran Strata’s own benchmark suite to the end and never dipped below 3.2 GiB available.
That second run is the result HotRod’s author will like:
on the 3060, Strata’s own strata_bench.py (median) | upstream v0.2.16 | HotRod, budget 20 GiB (unscored) |
|---|---|---|
| short-chat decode | 17.1 tok/s | 26.0 |
| 4K prompt | 172.8 tok/s | 649.6 |
| 16K prompt | 326.4 tok/s | 633.3 |
| agent decode | 14.5 tok/s | 25.0 |
That’s 3.8x on 4K prefill and 1.5x on decode. I’m not counting it. It’s a config I changed after seeing the failure, which is exactly the kind of fix a pre-registration exists to stop me grading myself on. It deserves its own pre-registered test. And even capped, 26 tok/s is half the README’s 50-70.
How the two upstream releases did on the 3060, in my own cells (mean of three, tok/s):
| prompt | U prefill | B prefill | U decode | B decode |
|---|---|---|---|---|
| 4,096 | 183.5 | 192.2 | 18.8 | 17.2 |
| 16,384 | 329.7 | 221.7 | 18.8 | 17.0 |
| 32,512 | 341.0 | 251.9 | 18.5 | 17.2 |
Upstream got faster between 0.1.45 and 0.2.16: about 50% at 16K prefill and 10% on decode. At 4K there’s no difference outside the noise. These are slower than my October 3060 numbers because that test used the 1-bit Coder file. This is IQ3_S, a 50 GB working set that can’t fit in 32 GB, so it streams from the NVMe every token.
RTX 3090, 64 GB: HotRod is slower than upstream
Here everything fits and nothing tripped. Every run was inside a sandbox with no network.
| prompt | U prefill | H prefill | B prefill | U decode | H decode | B decode |
|---|---|---|---|---|---|---|
| 4,096 | void | 2,385 | 2,223 | void | 94.6 | 115.8 |
| 16,384 | 2,951 | 2,405 | 2,668 | 135.4 | 92.0 | 112.8 |
| 32,512 | 3,102 | 1,968 | 2,806 | 141.0 | 91.5 | 119.7 |
At 16K and 32K, HotRod is slower than upstream at both prefill and decode, and none of the ranges overlap. At 32K it reads prompts at 63% of upstream’s speed and decodes at 65%.
My predictions for this box, copied from the reviewer’s 24 GB result:
- HotRod decodes slower than upstream: true at 16K and 32K. I can’t call it a hit, because upstream’s 4K cell came back void twice (prompt token counts spread by 5, then 4). The prediction has to hold at every length.
- HotRod prefills 0-30% faster than upstream: missed. At 16K it’s 18% slower.
There’s a reason not to read the decode gap as all HotRod. On the 3090, upstream’s setup picked its newer DFlash2 drafter on its own, while HotRod and B use the older MTP draft layer. Part of upstream’s lead is the drafter, as shipped. The cleaner comparison is HotRod against B, same base engine and same drafter: HotRod still decodes 18-24% slower than the release it forked from. It prefills 7% faster than B at 4K, then falls behind at 16K and 32K.
Strata’s own benchmark tool agrees with the reviewer at 4K. It puts HotRod’s 4K prompt speed 8% ahead of upstream’s (2,575 vs 2,393 tok/s), close to the reviewer’s +14%. It doesn’t agree at 16K, where HotRod is 4% behind. Short prompts look good, and long ones are where it loses.
What I’d do with this
If you have a 24 GB card and 64 GB of RAM, run upstream Strata. It was faster at every length I could score, and it ships the better drafter.
If you have 32 GB of RAM, don’t run HotRod’s default setup and walk away. It sizes its RAM tier to leave you under 2 GiB, and anything else you open comes out of that. If you want to try it anyway, cap the tier with --resident-budget-gib (20 left me 3.2 GiB on 32 GB) and keep free -m open through the load.
The capped run on the 3060 is the interesting loose end. If I test it, it gets a fresh pre-registration, not a patch to this one.
Every log, config, hash and the scoring script are in the public record, including the runs I threw out and why. The reviewer’s raw files are on GitHub too, and worth comparing.
A weekly email with every new guide and measured benchmark.
Subscribe — free, no spam