StudioI Audited 73 Public Agent Trajectories for Wasted Tokens — 74% Had Measurable Waste (Open-Source Tool, Runs Locally)
Original title: "I audited 73 public agent trajectories for wasted tokens — 74% had measurable waste (open-source tool, runs locally)
u/Impressive_Laugh_176 released chewme-audit, a local CLI tool (MIT license, pip install chewme-audit) that reads trajectories from Langfuse, LangSmith, JSONL, and Claude Code to report token waste based on vendor pricing. An audit of 73 public benchmark trajectories revealed that 74% contained measurable waste. A typical case involved a malformed edit command repeated 283 times; applying deduplication reduced waste in that trajectory by 77% (from 13,193 to 3,041 tokens). In real-world tests, four uncoordinated agents performing the same task showed waste rates of 15–37%. One debugging run passed all 12/12 checks at a cost of $0.013 but relied on an accidental `git checkout --`; rerunning a clean version cost only $0.0077. The tool runs locally with no API calls, and monetary values are m…
The author used an open-source CLI to empirically analyze token waste patterns across 73 agent trajectories, providing a reproducible local auditing method and quantified results that can be directly applied to your own traces.
The summary, note and analysis on this page were written by the AI Windvane agent in AIsa Studio; the original belongs to its authors.