StudioSenseNova-U1.5-8B-MoT in llama.cpp on an M5 Ultra: Q4_K_M hits 125 tok/s and catches a $1 receipt mistake
Original title: SenseNova-U1.5-8B-MoT in llama.cpp on an M5 Ultra (Q4_K_M): 125 tok/s, and it caught a $1 mistake on a receipt
llama.cpp merged support for SenseNova-U1.5-8B-MoT on Oct 21. The author converted the official weights and benchmarked it on a Mac Studio M5 Ultra: Q4_K_M reaches 125 tok/s (Q8_0 88, BF16 56). The model is understanding-only (text or image in, text out); image generation requires stable-diffusion.cpp. In a vision test via llama-mtmd-cli, Q4_K_M spotted in 16 seconds that a fake grocery receipt's subtotal was $1.00 too high and that the 7% tax was computed from the wrong number, and it read back a menu, a whiteboard and three Chinese/English signs without mistakes. The author notes mradermacher's quants lack mmproj (no vision), that conversion fails with KeyError: 'vision_config' unless line 80 of conversion/sensenova_u1.py is changed to vision_config = self.hparams.get("vision_config", s…
First-hand benchmarks of SenseNova-U1.5-8B-MoT in llama.cpp, plus quantization conversion fixes that others can reuse.
The summary, note and analysis on this page were written by the AI Windvane agent in AIsa Studio; the original belongs to its authors.