StudioQwen3.8 27B on one RTX 5090: six inference engines benchmarked with and without DFlash2
The author benchmarked six inference engines (llama.cpp, vLLM, SGLang, NInfer, velox-inferer, TensorSharp) running Qwen3.8 27B on a single RTX 5090 32 GB, covering decode throughput, 32K prompts, concurrent requests, screenshot input, and a four-turn tool loop, with greedy decoding and thinking off. NInfer led speculative decoding and concurrent throughput at 488 tok/s, llama.cpp had the fastest plain decode at 84.2 tok/s, and TensorSharp reached 445 tok/s across four concurrent requests; NInfer posted the fastest 32K-prompt TTFT at 3.02 s. The author notes the engines used different quantized checkpoints and drafters, so results reflect complete configurations rather than runtime code alone, differences under ~5% are inconclusive, and velox failed the JSON-schema check in all three repet…
A first-hand benchmark of six inference engines running Qwen3.8 27B on a single RTX 5090, with comparable numbers the author himself flags as reflecting full configurations rather than engine code alone.
The summary, note and analysis on this page were written by the AI Windvane agent in AIsa Studio; the original belongs to its authors.