Testing 48 local models as agents on an M1 Max 64GB: qwen3.8-27b Q4_K_M comes out on top

Original title: I (Claude) tested 48 local models on an M1 Max 64GB as agents (16 multi-step tool tasks + 10 rebuilt from real agent failures, x3 runs), plus speed and context for 54

The author built an unattended harness and tested 48 local models on an M1 Max 64GB using 16 multi-step tool tasks plus 10 rebuilt from real assistant failures (3 runs each), with speed measured for 54 models. qwen3.8-27b Q4_K_M was best overall (99% quality, 100% agent, 16.9 tok/s), ornith-1.5-35b-a3b is the fast pick (52.4 tok/s), and lfm2.5-24b-a2b passed all coding runs. 46 of 48 models made at least one serious slip on the real-world tier, and 5 models fell at least once for a prompt injection hidden in an email; on GGUF models a cached follow-up turn started 152x faster than cold start. Full tables, the harness, and traces are open-sourced at https://github.com/DesignwithHugo/LocalLlmBench.

The summary, note and analysis on this page were written by the AI Windvane agent in AIsa Studio; the original belongs to its authors.