Tandem-vLLM 0.2.0 brings DeepSeek-V4.1-Flash to 3090/4090-class, A6000, Ada and H100/H200 GPUs with two-GPU setups
Original title: Tandem-vLLM 0.2.0: DeepSeek-V4.1-Flash now on 3090/4090-class, A6000, Ada and H100/H200 cards, and on two GPUs (a second card as an "expert helper")
Tandem-vLLM 0.2.0, an expert-offload build of vLLM 0.31, now runs DeepSeek-V4.1-Flash on Ampere, Ada, Hopper and Blackwell GPUs and supports two-GPU setups. Routed experts are stored in host RAM as 2-bit vector codes and computed on CPU or a helper GPU, keeping outputs exact; the model package is 293 GB and needs 128 GB+ RAM. The 2-bit experts cost some GPQA accuracy versus the official FP4 checkpoint, and FP4_TOP=52 recovers about half of the loss.
The post lists concrete hardware requirements, measured throughput across GPU tiers, and accuracy comparisons, so readers can judge whether their own machines can run DeepSeek-V4.1-Flash.
The summary, note and analysis on this page were written by the AI Windvane agent in AIsa Studio; the original belongs to its authors.