Accenture 在电信、保险、采矿和大型企业的四个客户项目中应用了这些杠杆。正如本文所述,Amazon Bedrock 原生提供了相同的杠杆。在有衡量节省效果的地方,token 支出下降了约 40–85%。结果因每个客户的工作负载而异,可能会有所不同。为了保护客户机密性,Accenture 隐去了客户名称。
在您优化任何内容之前,您需要查看消耗发生在哪里、谁拥有它以及它产生了什么业务价值。在团队、工作负载和代理级别使用成本分配标签是将消耗与负责的负责人联系起来并最终与业务成果联系起来的先决条件。您可以通过 Amazon Bedrock 应用程序推理配置文件 应用这些标签,每个应用程序或工作负载一个配置文件,并按团队、成本中心和环境进行标记。在 AWS 账单和成本管理控制台 中将其激活为成本分配标签后,您可以使用这些标签来追踪哪些团队、应用程序和代理推动了支出。它们还使得能够衡量您引入的每项更改的影响。
仅有遥测数据是不够的。工程师可以在仪表板上读取每次调用的成本,但产品负责人和项目领导者需要以业务术语表达的指标。Accenture 的“每笔业务交易成本”(CpBT)框架弥补了这一差距。CpBT 将所有导致一次成功业务输出的模型调用、重试、工具调用和代理交接聚合到一个单一的完全加载成本数字中。它揭示了产生结果的真正成本,包括原始 token 定价所隐藏的失败尝试。通过与每个工作流提供的业务价值进行比较,CpBT 区分了反映扩展成功的不断增长的费用和反映浪费的费用。
这六个杠杆的目标不是不惜一切代价地最小化 token。而是使用适当数量和水平的智能,以适当的成本、质量和风险交付每项业务成果。这些杠杆根据更改的复杂性和所需投资分为三个阶段。AWS 和 Accenture 建议按顺序处理这些阶段,并且您可以独立地将每个阶段应用于最能产生影响的负载。
这两个杠杆都为您的现有设置添加了配置。提示缓存需要在您的提示中添加缓存检查点标记,而批量推理需要将符合条件的请求作为批处理作业提交,而不是调用按需 API。将缓存应用于交互式、按需流量,将批量推理应用于异步工作。两者都不需要新的基础设施或架构更改。
两种互补的 Accenture 模式与这些 AWS 原生杠杆协同工作。上下文范围限制每个代理接收的数据量,仅限于与其工作流程步骤相关的文档切片,而不是在每个交接处传递完整的案例上下文。上下文压缩在文档到达模型之前对其进行总结或过滤。这两者都减少了输入 token 的数量,尤其是在多代理链中。
语义缓存存储响应,并在新请求与已回答的请求在语义上相似时重用它们,即使措辞不同。这避免了冗余的模型调用,并降低了成本和延迟。在 AWS 上,您可以在模型前面使用向量存储(如 Amazon ElastiCache)来实现它。在针对 63,796 个真实聊天机器人查询的 AWS 基准测试中,0.90 的相似度阈值产生了 74.5% 的缓存命中率、92.3% 的准确性和 72.5% 的成本节省(请参阅 ElastiCache 语义缓存基准测试)。当您应用路由和语义缓存时,请在每条路由上保持 Amazon Bedrock Guardrails 开启,并设置语义缓存过期时间,以免用户收到过时的答案。
Accenture 在四个客户参与项目中应用了这些杠杆。每个项目都针对不同的成本模式。在其中两个项目中,Accenture 同时应用了几项更改,因此这些结果反映了综合影响,而不是任何单个杠杆的效果。
第一个参与项目是一个大型企业部署,其中每个请求都发送到前沿模型,没有缓存、模型分层或批处理。Accenture 同时应用了三项更改:模型分层以将每项任务匹配到能很好地处理它的最大模型,提示缓存用于重复上下文,以及批处理用于夜间和非紧急作业。他们共同将每月 token 成本削减了 85%,且节省额未按更改分开统计。大约 15% 的任务仍然需要顶级模型来进行困难、开放式的推理,其余任务则在较轻的模型上运行。您的拆分比例将取决于您的工作负载组成。
第二个参与项目是一家电信运营商,年 token 支出为 1200 万美元。大部分支出来自没有限制地扇出请求的代理,以及无论复杂性如何都将每个请求发送到前沿模型。Accenture 同时应用了分诊路由、语义缓存和上下文压缩。输入 token 减少了 70%,年 token 支出从 1200 万美元降至 380 万美元,减少了 68%。Accenture 根据运营商变更前的性能基准验证了输出质量。
第三和第四个参与项目各自应用了一个缓存层。一家保险公司使用了模型层前缀缓存,在多次调用中重用其提示的静态部分。一家矿业运营商使用了应用层语义缓存,在查询到达模型之前返回存储的响应以应对重复查询。在这两个参与项目中,token 支出总共下降了约 40%,没有进行模型路由更改或训练投资。
结果范围从约 40% 到约 85%,反映了每个客户的起点。大型企业部署最初没有任何优化措施,达到了 85%。电信运营商在 1200 万美元的年 token 支出上达到了 68%,保险公司和矿业运营商各自使用一个缓存杠杆,总共达到了约 40%。结果因每个客户的工作负载而异,可能会有所不同。
这些节省并不是最终目标。它们创造了扩展 AI 工作负载的能力,重新投资于更高价值的用例,并提高了每美元 AI 消费的回报。其中两个参与项目同时应用了几项更改。当您一次应用一个杠杆时,您可以将每个结果归因于特定的更改。AWS 和 Accenture 建议您对自己的工作负载采用此序列:将消耗与负责人和成果联系起来,一次应用一个杠杆,衡量成本、质量和业务价值,并在有效的基础上构建。
查看原文
As enterprises scale AI on Amazon Bedrock, token spend often grows even as per-token prices fall. Without cost allocation tags in place, your cloud invoice shows AI spend as a single line item rather than by team, workload, agent, or model. Cheaper tokens expand adoption, and agentic workloads multiply invocations. As successful AI use cases scale, consumption can rise faster than unit prices drop. The economic question is whether the business value generated grows faster than the cost of the intelligence required to produce it. The Linux Foundation has launched the Tokenomics Foundation to develop open standards for AI cost economics, a sign that managing the economics of AI is now an industry discipline.
In this post, you will learn about six levers for optimizing token spend, how Accenture applied them in production, and how to apply them using Amazon Bedrock.
Accenture applied these levers in four client engagements across telecom, insurance, mining, and a large enterprise deployment. The same levers are available natively on Amazon Bedrock as described in this post. Where savings were measured, token spend fell by about 40–85 percent. Results are specific to each client’s workload and may vary. Client names are withheld by Accenture to protect client confidentiality.
Attribution first: Measuring cost at the business level
Before you optimize anything, you need to see where consumption is occurring, who owns it, and what business value it produces. Cost allocation tags at the team, workload, and agent level are the prerequisite for connecting consumption to accountable owners and, ultimately, to business outcomes. You apply them through Amazon Bedrock application inference profiles, one profile per application or workload, tagged by team, cost center, and environment. After you activate them as cost allocation tags in the AWS Billing and Cost Management console, you can use these tags to trace which teams, applications, and agents are driving spend. They also make it possible to measure the impact of each change you introduce.
Three AWS services provide the underlying telemetry. AWS Cost and Usage Reports (CUR 2.0), delivered through AWS Data Exports, handle financial attribution, breaking down billed inference cost by workload and team after your cost allocation tags are active. Amazon CloudWatch captures operational signals such as invocation counts and latency. Amazon Bedrock model invocation logging records request-level detail, including the model used and the input and output token counts for each call.
Telemetry alone isn’t enough. Engineers can read cost per invocation on a dashboard, but product owners and program leaders need a metric expressed in business terms. Accenture’s Cost per Business Transaction (CpBT) framework bridges this gap. CpBT aggregates all the model invocations, retries, tool calls, and agent handoffs that contribute to one successful business output into a single fully loaded cost figure. It surfaces the true cost of producing a result, including the failed attempts that raw token pricing hides. Tracked against the business value each workflow delivers, CpBT distinguishes a growing bill that reflects scaling success from one that reflects waste.
The six optimization levers
The goal of these six levers isn’t to minimize tokens at all costs. It’s to use the right amount and level of intelligence to deliver each business outcome at the appropriate cost, quality, and risk. The levers are organized into three phases based on the complexity of the change and the investment it requires. AWS and Accenture recommend working through the phases in order, and you can apply each phase independently to the workloads where it will have the most impact.
Phase 1: Configuration only, no infrastructure change
With Amazon Bedrock prompt caching, the model reuses the parts of a prompt that repeat across requests, such as system instructions, tool definitions, and reference documents, so you stop paying full price to process the same content again. Amazon Bedrock stores the key-value state of previously computed prompt prefixes and is designed to skip recomputation on later requests. This is designed to reduce cached input token costs by up to 90 percent for supported models (see Amazon Bedrock Prompt Caching).
Amazon Bedrock batch inference processes non-urgent workloads asynchronously as a group at 50 percent lower cost than on-demand pricing for select models (see Amazon Bedrock Pricing). You submit inference requests as a batch job, and results are written to Amazon Simple Storage Service (Amazon S3) when processing completes. Batch inference suits nightly reports, document processing pipelines, and evaluation runs that don’t need an immediate response.
Both levers add configuration to your existing setup. Prompt caching requires cache checkpoint markers in your prompts, and batch inference requires submitting eligible requests as a batch job instead of calling the on-demand API. Apply caching to interactive, on-demand traffic and batch inference to asynchronous work. Neither requires new infrastructure or architecture changes.
Phase 2: Architectural changes
Amazon Bedrock intelligent prompt routing directs requests across models within the same family, matching each task to the level of intelligence its business value, complexity, and risk require, rather than defaulting every request to the most powerful model. Intelligent prompt routing can reduce costs by up to 30 percent without compromising accuracy (see Amazon Bedrock intelligent prompt routing). Check Amazon Bedrock Pricing for any routing charges. In the following engagements, Accenture’s model tiering and triage routing apply this same routing lever.
Retrieval Augmented Generation (RAG) optimization tunes the retrieval step in the RAG process so that fewer unnecessary context tokens reach the model. Techniques such as hierarchical chunking, metadata-based filtering, and reranking narrow the retrieved context to the most relevant chunks, so you pay primarily for content that improves the answer.
Two complementary Accenture patterns work alongside these AWS native levers. Context scoping limits the data each agent receives to the document slice relevant to its step of the workflow, rather than passing the full case context at every handoff. Context compression summarizes or filters documents before they reach the model. Both reduce input token volume, especially across multi-agent chains.
Phase 3: Infrastructure and training investment
Semantic caching stores responses and reuses them when a new request is semantically similar to one already answered, even if the wording differs. This avoids redundant model invocations and reduces both cost and latency. On AWS, you can implement it with a vector store such as Amazon ElastiCache in front of the model. In an AWS benchmark on 63,796 real chatbot queries, a 0.90 similarity threshold produced a 74.5 percent cache hit rate, 92.3 percent accuracy, and 72.5 percent cost savings (see ElastiCache semantic caching benchmarks). As you apply routing and semantic caching, keep Amazon Bedrock Guardrails on every route, and set semantic cache expiry so that users don’t receive outdated answers.
Model distillation transfers knowledge from a large teacher model to a smaller, faster student model. Distilled models in Amazon Bedrock are up to 500 percent faster and up to 75 percent less expensive than original models, with less than 2 percent accuracy loss for use cases such as RAG. Distillation is best suited to stable, high-volume workloads where the transaction volume justifies the training investment.
In production: Four client engagements
Accenture applied these levers in four client engagements. Each engagement targeted a different cost pattern. In two of them, Accenture applied several changes together, so those results reflect the combined effect rather than any single lever.
The first engagement was a large enterprise deployment where every request went to a frontier model, with no caching, model tiering, or batching. Accenture applied three changes together: model tiering to match each task to the smallest model that could handle it well, prompt caching for repeated context, and batch processing for overnight and non-urgent jobs. Together they cut monthly token cost by 85 percent, and the saving was not separated by change. About 15 percent of tasks still needed the top model for hard, open-ended reasoning, and the rest ran on lighter models. Your split will depend on your workload composition.
The second engagement was a telecom operator with $12 million in annual token spend. Most of that spend came from agents fanning out requests without limits and from every request going to a frontier model regardless of complexity. Accenture applied triage routing, semantic caching, and context compression together. Input tokens fell by 70 percent, and annual token spend fell from $12 million to $3.8 million, a 68 percent reduction. Accenture validated output quality against the operator’s pre-change performance benchmarks.
The third and fourth engagements each applied a single cache layer. An insurer used model-layer prefix caching, reusing the static portions of its prompts across invocations. A mining operator used an application-layer semantic cache, returning stored responses for repeated queries before they reached the model. Across the two engagements, token spend fell by about 40 percent combined, without model routing changes or training investment.
Results
Outcomes ranged from about 40 percent to about 85 percent, reflecting each client’s starting point. The large enterprise deployment began with no optimization in place and reached 85 percent. The telecom operator reached 68 percent on $12 million in annual token spend, and the insurer and mining operator reached about 40 percent combined with one caching lever each. Results are specific to each client’s workload and may vary.
These savings aren’t the end goal. They create capacity to scale AI workloads, reinvest in higher-value use cases, and improve the return generated by each dollar of AI consumption. Two of these engagements applied several changes at once. When you apply one lever at a time, you can attribute each result to a specific change. AWS and Accenture recommend this sequence for your own workloads: connect consumption to owners and outcomes, apply one lever at a time, measure cost, quality, and business value, and build on what works.
Production governance: Four operating tenets
Optimizing AI economics in production requires an ongoing governance cycle spanning cost, quality, risk, and business value, not a one-time technical change. Before you apply any lever, establish a quantitative baseline for both cost and quality so that you can measure the impact of each change against a known starting point. When you validate output quality, use held-out evaluation sets drawn from your actual production traffic rather than synthetic benchmarks. Apply one lever at a time per workload so that you can attribute results to a specific change and roll back cleanly if quality drops below your acceptable thresholds. If you operate in a regulated industry such as financial services, insurance, or healthcare, document your quality test results and rollback procedures as part of your standard change management record.
These four tenets apply regardless of which levers you are using or what phase of the sequencing you are in. Amazon Bedrock AgentCore is an environment where you can build, connect, and optimize agents at scale with any framework or model. It includes AgentCore Evaluations, a capability of Amazon Bedrock AgentCore that provides batch evaluation to support your quality validation cycle as you progress through more advanced levers.
Getting started
You can begin with three steps. First, turn on cost attribution for Amazon Bedrock by applying cost allocation tags at the team and workload level so that your spend is traceable from day one. Second, activate those tags in the AWS Billing and Cost Management console. Then use CUR 2.0 and model invocation logging to identify high-consumption workloads, and assess each one based on spend, business value, quality requirements, risk, volume, and optimization potential. Third, apply Amazon Bedrock prompt caching to that workload’s interactive traffic and Amazon Bedrock batch inference to its asynchronous jobs as configuration-only changes, and measure the result before you move to architectural changes in Phase 2.
The AWS Well-Architected Agentic AI Lens includes model invocation and token cost optimization (AGENTCOST02), a focus area within the cost optimization pillar for agentic workloads. This focus area provides a structured framework you can use to apply these levers at scale across your organization.
About the authors