Dev open-sources sFNX engine running Qwen3.8-Flash-Next at 382 tok/s on 3 used V100s in an old R740

Original title: I built an engine that runs Qwen-Flash-Next at nearly 400t/s on cheap(ish) hardware. 3x V100 32GB in an old R740. Originally forked from strata but the engine guts all built for the old Volta generation.

A developer released the open-source sFNX inference engine, running Qwen3.8-Flash-Next at up to 382 tok/s greedy code editing on three used Tesla V100-32GB cards (~$1,800, no NVLink, PCIe Gen3) in an old R740. Instead of splitting by layers like llama.cpp, sFNX splits each layer across all three cards and performs ring writes between cards inside CUDA graphs to bypass NCCL's PCIe limits, serving six concurrent conversations each with full 262,144-token context, with OpenAI- and Anthropic-compatible APIs; a two-card build is also supported and benchmarked against Strata. Repo and docs: https://github.com/johnfink8/sfnx

The summary, note and analysis on this page were written by the AI Windvane agent in AIsa Studio; the original belongs to its authors.