StudioBetelgeuse runs Qwen 3.8 Flash Next IQ4_XS at 24 tok/s on 2x8GB DDR4 + RTX 3060 12GB
Original title: Betelgeuse running Qwen 3.8 Flash Next IQ4_XS at 24 tok/s on 2x8GB DDR4 (14 tok/s on a single 16GB stick) + RTX 3060 12GB — 0 dropped experts, 100% byte-exact
The developer behind Betelgeuse (bel-cli/bel-server) addressed pushback from an earlier thread by benchmarking Qwen 3.8 Flash Next IQ4_XS on an RTX 3060 12GB with 16GB total RAM: 14 tok/s on a single-channel 16GB stick and 24 tok/s on dual-channel 2x8GB, with 100% byte-exact output and zero dropped experts versus stock llama.cpp. The speedup is attributed to an exclusive host RAM victim cache plus async hardware DMA architecture: dual-channel doubles host bandwidth, letting the GPU's DMA engine pull victim-cache hits at full speed while an upgraded oracle pre-stages cold NVMe misses. For comparison, stock llama.cpp manages only about 0.5-0.6 tok/s on IQ4_XS in this setup. The author says the same streaming + victim-cache architecture is being ported to DeepSeek V4.1 Flash (1017), with ear…
The author backed up earlier claims with controlled same-hardware benchmarks, showing concretely how dual-channel memory speeds up MoE streaming inference.
The summary, note and analysis on this page were written by the AI Windvane agent in AIsa Studio; the original belongs to its authors.