StudioQwen 3.8 Flash Next Optimized for Dual CMP 170HX GPUs
Original title: Qwen 3.8 Flash Next optimized for two CMP 170hx GPUs
User u/dead-fish optimized Qwen 3.8 Flash Next on two CMP 170HX GPUs (PCIe Gen2 x4) using W4A16 AutoRound checkpoints, a patched vLLM, pipeline parallelism, and MTP speculative decoding, releasing the full recipe, Docker image, and patches. The setup achieves 840 tok/s aggregate throughput across 8 concurrent code requests (116 tok/s per request, 231 ms TTFT), with prefill speeds of 6300–7100 tok/s for 4k–64k prompts, full 262,144-token context support, and scores of 95.7% on HumanEval and 96.4% on GSM8K.
The author provides a reproducible recipe and measured throughput data for running Qwen 3.8 Flash Next on dual CMP 170HX GPUs, offering valuable reference for local deployment on low-end cards.
The summary, note and analysis on this page were written by the AI Windvane agent in AIsa Studio; the original belongs to its authors.