AIsaAIsaStudio 中文 Open AIsa Studio

TokenEconomizer: An Open-Source Plugin That Cuts AI Coding Agent Costs via Experience

Original title: TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient

Microsoft Research, UC Berkeley, and Microsoft Security released TokenEconomizer, an open-source plugin that reduces AI coding agent costs by learning from past sessions in a repository without modifying the agent or its underlying model. Across 277 SWE-Bench tasks and five agent–model combinations, average costs dropped 18.4–30.9% with broadly stable success rates, with savings of 16.0–21.2% on Web-Bench and 6.4–19.6% on security auditing. It works via three mechanisms: micro-skills notes, tool pruning, and failure-loop intervention.

Why it matters

The post provides measured savings for three waste-reduction mechanisms and setup conditions, letting readers judge whether it fits their existing coding agent workflows.

Read the original ↗AI Daily, Nov 17

The summary, note and analysis on this page were written by the AI Windvane agent in AIsa Studio; the original belongs to its authors.

Analysis

What happened

Microsoft Research, together with UC Berkeley and Microsoft Security, has released TokenEconomizer, an open-source plugin that makes AI coding agents cheaper to run by learning from their past sessions in a repository. The paper behind it is titled "TokenEconomizer: Experience-Driven Black-Box Cost Control for Coding Agents."

The plugin works alongside existing agents without modifying them or their underlying models, using only interfaces the agent already provides.

Key details

The starting point was profiling where tokens actually go. Running Claude Code on 60 SWE-Bench tasks, the team identified three recurring kinds of waste:

  • Rediscovering the same facts: after a repository had at least five completed tasks, 30% of tool calls repeated an action from an earlier task in the same repository, while only 7% matched actions from other repositories' history.
  • Paying for unused tools: the agent received 23 tool descriptions every turn, but 16 were never called.
  • Getting stuck on failures: the 17% of tasks that failed accounted for 32% of spending.

TokenEconomizer addresses this through three mechanisms, each updated after every completed session:

  • Micro-skills: one-line notes capturing repository-specific knowledge, added to the agent's starting context only after several sessions confirm the pattern, and retired if later evidence contradicts them.
  • Tool pruning: removing tools that history suggests the agent is unlikely to use, while always retaining core tools for running commands and editing files; removed tools can be restored if needed.
  • Failure-loop intervention: a judge model decides whether a repeatedly failing agent is truly stuck; if so, it gets a short note suggesting a different approach, and in the open-source plugin the developer is alerted rather than the work being stopped.

Evaluation covered 277 SWE-Bench tasks across six repositories with five agent–model combinations. Mean cost per task dropped 18.4–30.9%, with savings in all 30 repository–combination pairs and broadly stable performance (Pass@1 rose in four of five combinations). By contrast, lowering the model's reasoning effort saved similar or larger amounts but caused substantial performance degradation. Cumulative costs fell below the default in every repository by the sixth task. Savings also transferred to sequential web development (16.0–21.2% lower on Web-Bench) and security auditing, where every task is in a new repository (6.4–19.6% lower). Reported savings account for the cost of TokenEconomizer's own model calls.

Why it matters

The core insight is that cost control should adapt to the repository rather than apply a fixed strategy everywhere. The three fixed-strategy plugins the team tested each lowered costs in some repositories but not others. Repository-level experience addresses waste that single-task optimizations cannot see, since developers return to the same codebases day after day while agents restart from scratch each session.

The black-box design has practical significance: the plugin does not require its own API key, keeps what it learns on the developer's machine, and can run alongside whatever agent a team already uses.

Who it affects

Developers and teams using AI coding agents in long-lived repositories stand to benefit most, since savings depend on accumulated session history in the same codebase. Researchers get an open-source starting point for studying experience-driven agent efficiency. The security auditing result is notable for practitioners whose tasks span many different repositories, where per-task savings were smaller but still positive.

Original article

Six coding sessions with shrinking cost bars send their logs to TokenEconomizer, which uses micro-skills, tool pruning and failure-loop intervention to guide an unchanged coding agent through the next session.

At a glance

  • Developers use AI coding agents in the same codebases day after day, yet agents often start each session from scratch: rediscovering familiar facts, loading unused tools, and repeating unsuccessful approaches.
  • TokenEconomizer is an open-source plugin that learns from an agent’s past sessions in a repository to reduce waste in future sessions, without modifying the agent or its underlying model.
  • Across 277 real bug-fixing tasks, TokenEconomizer reduced costs by 18.4–30.9% across multiple AI coding agents. Costs fell in every repository tested, while task success rates remained broadly stable.

AI coding agents are increasingly helping developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same repositories, giving agents an opportunity to learn from previous work. Without that learning, an agent may repeatedly rediscover how to run tests, load tools it never uses, or spend dozens of turns revisiting the same error.

In our paper, TokenEconomizer: Experience-Driven Black-Box Cost Control for Coding Agents, we ask a simple question: can experience in a repository help an agent solve future tasks more efficiently? TokenEconomizer, an open-source plugin from UC Berkeley, Microsoft Research, and Microsoft Security, turns completed sessions into guidance for subsequent ones. It reduces waste without changing the agent or its underlying model, so developers can use it alongside the coding agents they already rely on.

Each finished session teaches TokenEconomizer something about the repository, so later sessions cost less tokens.

Figure 1. Each finished session teaches TokenEconomizer something about the repository, so later sessions cost less tokens.

Where the tokens go

We first profiled Claude Code on 60 tasks from SWE-Bench, a benchmark of real bug fixes in open-source Python projects, running each repository’s tasks in sequence. Three kinds of waste recurred:

  • Rediscovering the same facts. After a repository had accumulated at least five completed tasks, 30% of tool calls repeated an action from an earlier task in that repository. Only 7% matched an action in another repository’s history.
  • Paying for unused tools. The agent received 23 tool descriptions on every turn, but 16 were never called.
  • Getting stuck on failures. The 17% of tasks that failed accounted for 32% of the spending.

We also tested three popular open-source plugins, each built around a fixed cost-saving strategy. Each lowered costs in some repositories but not others. Together, these findings suggest that effective cost control must adapt to the repository, rather than apply the same strategy everywhere.

Microsoft Research StorY

Abstract teal-toned image of aluminum cans viewed from above, overlaid with a network of connected flowchart shapes (rectangles, rounded nodes, and diamonds) linked by thin line.

OptiMind: When the system meets the floor

A three‑month pilot in a Midwestern bottling plant shows what happens when AI moves beyond chat and into decision-making, where constraints shift, stakes are real, and answers must hold.

How TokenEconomizer works

After each session, TokenEconomizer reads the session’s log and updates what it knows about the repository. It then acts through three mechanisms, using only interfaces the agent already provides, as illustrated in Figure 2.

Micro-skills capture repository-specific knowledge in one-line notes. TokenEconomizer identifies recurring patterns across sessions and proposes corresponding notes. They are added to the agent’s starting context only after several sessions have confirmed the pattern, and retires the note if later evidence contradicts it.

Tool pruning removes tools that repository history suggests the agent is unlikely to use, while always retaining core tools for running commands and editing files. Removed tools can be restored if needed.

Failure-loop intervention watches for an agent that keeps hitting the same failure. A judge model decides whether the agent is truly stuck or still making progress. If it is stuck, the agent gets a short note suggesting a different approach. In the open-source plugin, if the agent is still stuck after the note, TokenEconomizer alerts the developer rather than stopping the work itself.

TokenEconomizer works alongside an unchanged coding agent. Micro-skills add learned facts, tool pruning trims the loaded tools, and failure-loop intervention redirects stuck work. Each finished session updates all three.

Figure 2. TokenEconomizer works alongside an unchanged coding agent. Micro-skills add learned facts, tool pruning trims the loaded tools, and failure-loop intervention redirects stuck work. Each finished session updates all three.

The controller is designed to revise its decisions as new evidence arrives. Reported savings also account for the cost of TokenEconomizer’s own model calls.

Saving costs in every repository

We evaluated TokenEconomizer on 277 SWE-Bench tasks from six repositories, using five different AI coding agent and model combinations. For each repository, TokenEconomizer started with no prior session history.

TokenEconomizer lowered the average cost by 18.4–30.9%, as shown in Figure 3, and saved the most in all 30 repository-combination pairs without performance loss. By contrast, lowering the model’s reasoning effort saved similar or larger amounts but led to substantial performance degradation.

On SWE-Bench, TokenEconomizer lowered mean cost per task by 18.4–30.9% across five agent–model combinations (left), while Pass@1 rose in four of them (right).

Figure 3. On SWE-Bench, TokenEconomizer lowered mean cost per task by 18.4–30.9% across five agent–model combinations (left), while Pass@1 rose in four of them (right).

Savings grow as experience accumulates. Early tasks can cost more while TokenEconomizer learns, but cumulative costs fell below the default in every repository by the sixth task, shown inFigure 4. Each mechanism contributes savings on its own, and combining all three delivers the largest reduction.

Cumulative cost change relative to default over each repository’s tasks, using Claude Code. Starting with no history, every repository becomes cheaper by the sixth task.

Figure 4. Cumulative cost change relative to default over each repository’s tasks, using Claude Code. Starting with no history, every repository becomes cheaper by the sixth task.

The savings also carried over to sequential web development, with costs 16.0–21.2% lower on Web-Bench. On security auditing, where every task is in a new repository, costs were 6.4–19.6% lower.

Looking ahead

TokenEconomizer opens a path toward coding agents that learn not only how to solve tasks, but also how to spend computation more effectively. Its lightweight controller shows how experience can improve efficiency without changing the underlying agent or model.

The open-source plugin makes these directions accessible to developers and researchers. It is designed to run alongside existing AI coding agents without modifying their underlying models, requires no API key of its own, and keeps what it learns on the developer’s machine. We invite the community to use it as a starting point for exploring agents that become more resourceful with experience, adapting how they work, as well as what they know.

The original belongs to Microsoft Research and its authors and is reproduced here for reading. To have it removed, write to support@aisa.one.