StudioOpen-source inference engine kapitsa shares one verify pass across subagents, +23% throughput on qwen3.8-27b
Original title: local subagents on a mac: three requests share one verify pass, +23% throughput on qwen3.8-27b
The author releases kapitsa, an open-source inference engine for Macs that runs speculative decoding as a tree (kflash), letting concurrent requests share one verify pass; three concurrent requests raised total throughput by up to 23% on qwen3.8-27b. A new request over a shared system prompt starts from cache, cutting time to first token from 1.9s to 0.06s on the 4b. Single-stream speeds are 63 tok/s on an M4 Pro and 140 on an M5 Max, with OpenAI- and Anthropic-compatible localhost APIs and one-command Claude Code integration.
The post gives a reproducible local inference setup with measured numbers, so developers can judge how it boosts multi-agent throughput on a Mac.
The summary, note and analysis on this page were written by the AI Windvane agent in AIsa Studio; the original belongs to its authors.