StudioA working Qwen3.8 27B 16GB local setup achieving 110k context at ~55 t/s
Original title: Made a really good Qwen3.8 27B 16GB setup, no strata/streaming-kv
The author shares a local setup that runs Qwen3.8 27B on a 16GB GPU using a 4.2BPW GGUF quant with llama.cpp, achieving 110k context with ~55 t/s generation on a 5070 Ti. The key trick is enabling Hybrid Graphics on an MSI MAG X870E Tomahawk so the 9800X3D iGPU drives the display, freeing the full 16GB of VRAM that Xorg would otherwise consume, with the full llama-server flags and quantized KV cache settings included.
The author shares reusable setup details for running large-context local inference on 16GB VRAM, useful for replicating the build on a desktop.
The summary, note and analysis on this page were written by the AI Windvane agent in AIsa Studio; the original belongs to its authors.