AIsaAIsaStudio English 打开 AIsa Studio

Amazon Bedrock AgentCore runtime 快速会话启动的架构模式

原标题:Architecture patterns for fast session startup on Amazon Bedrock AgentCore runtime

AWS 介绍 Amazon Bedrock AgentCore runtime 在 platformVersion 设为 V2 时的会话快速启动架构模式。V2 在创建或更新 runtime 时启动容器并保存快照,每个新会话恢复快照,启动时间约 2 秒且更一致。文章讲解如何用 CloudWatch 指标区分平台开销与代码耗时,建议把导入、客户端创建和静态数据加载放在 init 阶段,为大文件和 CLI 提供 build hook,并通过对用户打开应用等行为预热会话实现亚秒级响应。

为什么值得看

文章讲清了 V2 快照机制下初始化与运行时工作的分配原则,并给出可迁移的测量和预热方法。

阅读原文 ↗11月14日 AI 日报

本页的摘要、推荐理由与解读出自 AIsa Studio 的 AI 风向智能体,原文版权归原作者所有。

深度解读

发生了什么

AWS Machine Learning 发布了一篇关于 Amazon Bedrock AgentCore runtime 快速会话启动架构模式的指南,核心是 platformVersion 设为 "V2" 后的会话启动方式变化:不再为每个会话初始化容器,而是只准备一次环境、生成快照,之后每个新会话都从快照恢复。

文章讲解了 V2 的会话启动流程、如何测量会话启动时间、如何把工作移出首个请求(包括 AgentCore runtime 新支持的 build hooks),并给出四种模式,从代码层面的改动到在用户需要之前提前启动会话。

关键细节

  • V1 上冷启动主要由从 Amazon ECR 拉取容器镜像决定。V2 上容器只在创建或更新时启动一次,等待首个健康 GET /ping(须在 120 秒内到达),清空文件缓存后将 microVM 冻结为快照。
  • 新会话到达时,快照被恢复进新的 microVM。快照时进程持有的导入、对象、客户端以内存速度恢复;文件读取不会延续(缓存已清空),镜像文件在首次读取时才拉取。
  • 关键结论:导入、客户端(如 boto3)和保存在对象中的静态数据随会话恢复;网络连接恢复后已失效,应在快照前关闭;推迟到 handler 的任何工作都要由每个新会话付出代价。
  • 文章给出三种优化模式:(1) 切换到 V2 并在 app.run() 之前完成静态初始化;(2) 为无法保存在进程内的大文件、CLI 或浏览器添加 build hook;(3) 在用户参与时预热——用户打开应用或聊天窗口时在后台启动会话,实现亚秒级首响应。
  • 测量方面:AWS/Bedrock-AgentCore 命名空间的 CloudWatch 指标提供 Latency(首字节时间)和 OverheadLatency(平台时间)。Init 时间记录在构建日志中,因为在 V2 上它只在构建时运行一次。文章建议使用 P90/P99 等百分位或修剪均值,并按生产级别的会话频率测试。
  • Build hook 会在每次构建时向容器内暴露的 HTTP 端点发送两次 POST——一次在快照前("memory" 预热),一次在测试恢复后("disk" 预热)——两者共享 120 秒,由 X-Amzn-Bedrock-AgentCore-Runtime-Warmup-Type 头区分。失败会导致创建或更新失败,且不会重试。
  • 陷阱提示:动态数据(工具目录、feature flags)和每会话值(随机值、密钥、当前时间、计时器)必须在 handler 中创建,因为 init 中的所有内容会被原样冻结进每个会话。会话在空闲超时(默认 15 分钟,可设 60 秒到 8 小时)或最大生命周期(8 小时)时结束。

为什么重要

V2 把大部分会话启动工作移到了部署时,使会话启动更快更一致——新会话约 2 秒——同时内存回收意味着账单跟踪代理实际使用的资源。一条反直觉的建议是:在 V2 上应与通常的冷启动建议相反,把初始化前置而非推迟,因为推迟的工作每个会话都要付出代价。

静态与动态工作的区分很关键:快照意味着过时的 init 时间戳、完全相同的随机状态或过期的凭证可能变成 bug 或安全问题。build hook 机制将这一模型扩展到需要以独立进程运行的 CLI 或浏览器的场景。

对于连 2 秒都嫌慢的交互式聊天和语音体验,预热模式利用用户参与信号完全隐藏会话启动,代价是为每个打开但从不发送消息的用户多付一个会话。

对谁有影响

  • 在 Amazon Bedrock AgentCore runtime 上构建代理的团队,尤其是有交互式(用户等待首个响应)体验或需长时间无人值守运行的代理。
  • 代理需要在请求时读取大文件的开发者——本地模型权重、嵌入、数据集、浏览器二进制、CLI 工具链——或每个请求启动子进程的团队。
  • 关注延迟测量的运维者,因为 V2 改变了 init 时间出现的位置(构建日志而非请求指标),也改变了 OverheadLatency 等指标的解读方式。
全文翻译

以下翻译只覆盖原文的前一部分,其余内容请看下方原文。

如今的 AI Agent 会审查代码、处理理赔请求,并无人值守地运行数小时。许多 Agent 还位于交互式体验之后,用户在此等待首次响应。Amazon Bedrock AgentCore 运行时 为这些 Agent 提供了完全托管的无服务器环境:每个会话都在其自己的隔离微虚拟机(microVM)中运行,当没有工作时平台会自动缩容至零,且您只需为会话消耗的资源付费。

将 platformVersion 设置为 “V2” 的 AgentCore 运行时 改变了会话启动的方式。它不再为每个会话初始化您的容器,而是预先准备一次环境,将其制作成快照,并为每个新会话恢复该快照。这使得会话启动速度更快且更加一致。V2 还会回收会话未使用的内存,因此您的账单仅反映 Agent 实际使用的资源。

对于大多数 Agent 而言,切换到 V2 就足够了。但有些 Agent 需要重构才能充分利用其优势,而对于聊天和语音体验来说,即使是 2 秒也太慢了。本文解释了 V2 如何启动会话、如何测量会话启动时间,以及如何避免让工作堆积在第一个请求上,包括 AgentCore 运行时现在支持的构建钩子(build hooks)。我们涵盖了四种模式,从修改 Agent 的代码到在用户需要之前预启动会话。

您需要哪种模式?

您的情况

操作建议

章节

约 2 秒内启动新会话即可接受

切换到 V2,并在 app.run() 之前完成所有初始化

将工作移出首个请求

您的 Agent 早期就需要大文件、命令行工具(CLI)或浏览器,且无法将它们保留在进程内存中

添加构建钩子,以便在构建快照时加载并预热这些数据

使用钩子

您需要亚秒级延迟,且用户在发送第一个请求前就会进行交互(例如打开应用或聊天窗口)

在用户交互时在后台启动会话

基于用户交互预加热

V2 如何启动会话

AgentCore 运行时在每个新微虚拟机中运行一个会话。在 V1 版本中,AgentCore 会从 Amazon Elastic Container Registry (Amazon ECR) 拉取您的容器镜像,启动容器,并在处理第一个请求之前运行您的初始化代码。镜像拉取是主要耗时环节,因此冷启动时间随镜像大小增长。

V2 只执行一次这项工作。当您创建或更新运行时配置时,AgentCore 运行时会在一个新的微虚拟机中启动您的容器,并等待第一次健康检查 ping(GET /ping),该请求必须在 120 秒内到达。随后,它会清除操作系统的文件缓存以保持快照体积较小,冻结微虚拟机,并将其内存和磁盘状态存储为快照。

这与 V1 不同:每次创建或更新现在都需要几分钟来构建快照。

在本文中,init 指的是 /ping 报告健康之前 Agent 所做的所有工作。使用 AgentCore Python SDK 时,即指 app.run() 之前的所有内容。如果您运行自己的 HTTP 服务器(使用 Node.js 或其他语言),请在初始化完成后才报告健康状态。

构建流程:容器仅启动一次,报告健康后,AgentCore 清除文件缓存,冻结微虚拟机并存储快照

图 1:每次创建或更新时仅构建一次

当新会话到来时,AgentCore 将快照恢复到新的微虚拟机中。您的进程从被冻结的位置继续运行,其导入模块、对象和客户端均已就位。来自您镜像的文件采用懒加载方式:会话首次读取某个块时才会从存储中获取该块。

调用流程:每个新会话将快照恢复到新的微虚拟机中,进程恢复运行,镜像文件在首次读取时获取

图 2:调用,每个新会话

从上述流程可以得出三个事实,本文后续内容将基于这些事实展开:

  • 内存整体返回。 快照时刻进程持有的任何内容,以及 RAM 支持的文件系统(如 /dev/shm)中的任何内容,在首次访问时都以内存速度可用。
  • 文件读取不会延续。 冻结前会清除文件缓存,因此在初始化期间读取文件并不会使恢复后的会话中读取变快。
  • 镜像文件在首次读取时获取。 块的初次读取来自快照存储,随后缓存在靠近主机的位置。

下表展示了这对典型 Agent 所做工作的影响,具体取决于工作是在何处执行的:

工作

在 init 期间执行

在处理器(handler)中执行

导入库

会话启动时已在内存中

每个新会话都从镜像中读取

创建客户端(例如 boto3)

随端点和凭据解析一起恢复。套接字在首次使用时重新连接

每个新会话从头构建

打开网络连接

恢复后失效。在快照前关闭它们

全新打开

从镜像加载静态数据并将其保留在对象中

会话启动时已在内存中

每个新会话都从镜像中读取

从镜像读取文件但不保留数据

无影响,因为文件缓存已被清除

每个新会话都从镜像中读取

如果您熟悉 AWS Lambda SnapStart,这看起来很熟悉。Marc Brooker 关于 SnapStart 如何加载快照的文章以及 SnapStart 预热策略文章是很好的背景资料。

表中的所有假设都基于您的 Agent 能够将其所需内容保留在自己的进程中。但这并不总是可行的。作为单独进程运行并退出的 CLI,或者浏览器二进制文件,在 init 期间被读取然后消失,因此恢复后的会话会再次从镜像中获取它们。对于这些情况,您可以添加构建钩子,详见“使用钩子”部分。图 3 显示了它的运行位置。

带有钩子的构建和调用流程:内存预热调用在快照前运行,磁盘预热调用在测试恢复后运行

图 3:带钩子的构建和调用

优化前先测量

智能体应用程序中的延迟不是一个单一数值。请求花费时间在 AgentCore 运行时中将会话准备好以运行您的代码,然后在您的代码中花费时间:模型调用、工具调用、子 Agent 以及流式传输响应回传。解决这些问题需要不同的方法,因此首先要找出哪个环节占主导。

AgentCore 运行时通过 Amazon CloudWatch 指标 发布这两项指标,位于 AWS/Bedrock-AgentCore 命名空间中。Latency(延迟)是指首字节时间(Time to First Byte, TTFB)。OverheadLatency(开销延迟)是属于平台的部分,例如在新会话中启动微虚拟机和恢复快照的时间。

您想要的数据

获取方式

TTFB(首字节时间)

Latency

运行时时间

OverheadLatency

您的代码时间

Latency − OverheadLatency

初始化时间

在您的代码启动时测量(见以下示例)。它在 V2 上不属于任何请求的一部分

由于在 V2 上 init 仅在构建快照时运行一次,而不是在每个会话中运行,因此 init 不会出现在这些指标中。它增加了达到 READY 状态的创建或更新时间,而不增加请求延迟。要查看其耗时,请在 app.run() 之前记录一次日志。该行会出现在您运行时的构建日志中:

import time
INIT_START = time.time()  # 模块的第一行
# 导入、客户端、数据加载 ...
print(f"init_ms={round((time.time() - INIT_START) * 1000)}")
app.run()

如何解读拆分结果:

  • 新会话中 OverheadLatency 较高: 时间花在会话启动上。使用本文后面的模式,在用户需要之前启动会话。
  • 同一会话中第一个请求的代码时间远高于第二个请求: 属于 init 的工作正在您的处理器中运行。在 AgentCore 可观测性(Amazon Bedrock AgentCore 的一项功能)中添加自定义跨度,围绕工具设置、文件加载和延迟导入,以找到它们,然后将它们移至 init。

报告 P90 和 P99 等百分位数,或 TM90 等修剪均值,后者计算最快 90% 请求的平均值并丢弃最慢的尾部。简单的平均值会掩盖慢速请求。OverheadLatency 涵盖运行中会话和新会话的请求,因此其分布取决于您的请求中有多少比例启动了新会话。比较具有相似混合比例的运行结果。

以您在生产中预期的新会话速率进行测试,而不是在部署后仅用少量会话进行测试。

若要包含来自客户端的网络时间,请在客户端也进行测量。计时 invoke 调用,让处理器返回其运行时长,然后相减。两个时间戳来自同一台机器,因此客户端与微虚拟机之间的时钟差异无关紧要。如果您使用流式传输,非常小的事件可能会被推迟,直到积累了几百字节的输出,因此首个事件的时间可能高于会话启动时间。

将工作移出首个请求

通常的冷启动建议是延迟导入并保持初始化小巧。在 V2 上,请反其道而行之:init 仅在构建快照时运行一次,而推迟到处理器中的工作将由每个新会话承担。这适用于静态工作:来自您镜像的导入、客户端和数据。动态数据无法被快照。下一节将介绍它。

受影响最大的 Agent 是在请求时从其镜像读取大文件的 Agent:小型本地模型的权重、嵌入向量、数据集、大型库、用于自动化的浏览器二进制文件,或由编码 Agent 作为子进程运行的 CLI 工具链。读取许多小文件(如编译 TypeScript 项目)也会产生相同的效果。仅导入几个库并调用托管模型的 Agent 主要需要正确配置其导入和客户端。

如何重构您的 Agent

在 init 期间创建对象,而不仅仅是导入。 创建对象会加载其依赖的模块并将它们保留在活动内存中。也在 init 期间初始化使用外部库的本地工具,而不是在 Agent 首次调用它们时才初始化。

在 init 期间创建您的客户端。 端点解析、凭据解析以及客户端的模块都会随快照一起返回。套接字则不会,但 boto3 客户端在第一次调用时会干净地重新连接。其他一些网络客户端会先尝试过期的套接字,这会增加数百毫秒,因此在快照前关闭它们的连接。

将静态数据保留在内存中,例如提示词、技能前置元数据、配置和小索引。将数据保存在对象中,而不仅仅是读取它:

INDEX = pathlib.Path("/opt/data/index.bin").read_bytes()  # 在 init 期间:保留字节数据

保持工具运行,而不是每个请求启动一个新实例。 在处理器中启动的子进程在每个新会话中都从镜像中读取其二进制文件和库。在 init 期间启动的长期运行的辅助程序(如语言服务器或已完成导入的工作进程)会作为进程内存的一部分返回。通过管道或套接字与其通信。

将无法保留在内存中的文件复制到 RAM 支持的磁盘中。 像 tsc 这样的工具链作为单独的程序运行,并读取自身的文件。在 init 期间将它们复制到 /dev/shm 或 ramfs 挂载点,以便它们的读取来自内存。

分发编译后的字节码。 uv pip install 默认不编译,因此延迟导入也会从源代码编译。设置 UV_COMPILE_BYTECODE=1 或使用 pip install --compile。

下面的示例在 init 期间构建 Agent、其模型及其客户端,因此每个会话都继承它们:

import
查看原文

Agents today review code, process claims, and run unattended for hours. Many also sit behind interactive experiences where a person waits for the first response. Amazon Bedrock AgentCore runtime gives these agents a fully managed, serverless environment: each session runs in its own isolated microVM, the platform scales to zero when there’s no work, and you only pay for the resources your sessions consume.

AgentCore runtime with platformVersion set to “V2” changes how sessions start. Instead of initializing your container for every session, it prepares your environment once, snapshots it, and restores that snapshot for each new session. Session start times are faster and more consistent. V2 also reclaims memory your sessions don’t use, so your bill tracks what your agent uses.

For most agents, switching to V2 is all it takes. Some agents need restructuring to get the full benefit, and for chat and voice experiences even 2 seconds is too slow. This post explains how V2 starts a session, how to measure session start time, and how to keep work off the first request, including build hooks, which AgentCore runtime now supports. We cover four patterns, from changes to your agent’s code to starting sessions before users need them.

Which pattern do you need?

Your situation

What to do

Section

A new session in about 2 seconds is fine

Switch to V2 and do all initialization before app.run()

Keep work off the first request

Your agent needs large files, a CLI, or a browser early, and can’t hold them in its process

Add a build hook to load and prime them while the snapshot is built

Working with hooks

You need sub-second, and users engage before they send their first request (opening the app or a chat window)

Start the session in the background when the user engages

Pre-warm on user engagement

How V2 starts a session

AgentCore runtime runs each session in a new microVM. On V1, AgentCore pulls your container image from Amazon Elastic Container Registry (Amazon ECR), starts the container, and runs your initialization code before it can process the first request. The image pull dominates, so cold start grows with image size.

V2 does that work once. When you create or update a runtime, AgentCore runtime starts your container in a fresh microVM and waits for the first healthy ping (GET /ping), which must arrive within 120 seconds. It then clears the operating system’s file cache, to keep the snapshot small, freezes the microVM, and stores its memory and disk as a snapshot.

This is a change from V1: Every create or update now takes a few minutes while the snapshot is built.

In this post, init means everything your agent does before /ping reports healthy. With the AgentCore Python SDK, that’s everything before app.run(). If you run your own HTTP server, in Node.js or another language, report healthy only after your initialization finishes.

Build flow where the container starts once, reports healthy, then AgentCore clears the file cache, freezes the microVM, and stores a snapshot

Figure 1: Build, once per create or update

When a new session arrives, AgentCore restores the snapshot into a new microVM. Your process resumes exactly where it was frozen, with its imports, objects, and clients in place. Files from your image are loaded lazily: each block is fetched the first time a session reads it.

Invoke flow where each new session restores the snapshot into a fresh microVM, the process resumes, and image files are fetched on first read

Figure 2: Invoke, every new session

Three facts follow from these flows, and the rest of this post builds on them:

  • Memory comes back whole. Anything your process held at snapshot time, and anything in a RAM-backed file system such as /dev/shm, is available at memory speed from the first access.
  • File reads don’t carry over. The file cache is cleared before the freeze, so reading a file during init doesn’t make it fast in a restored session.
  • Image files are fetched on first read. The first read of a block comes from snapshot storage and is then cached closer to the host.

The following table shows what that means for the work a typical agent does, depending on where it does it:

Work

Done during init

Done in the handler

Import libraries

In memory when the session starts

Read from the image in every new session

Create clients (for example, boto3)

Restored with endpoint and credential resolution done. Sockets reconnect on first use

Built from scratch in every new session

Open network connections

Stale after the restore. Close them before the snapshot

Opened fresh

Load static data from the image and keep it in an object

In memory when the session starts

Read from the image in every new session

Read a file from the image without keeping the data

No effect, because the file cache is cleared

Read from the image in every new session

If you know AWS Lambda SnapStart, this will feel familiar. Marc Brooker’s post on how SnapStart loads snapshots and the SnapStart priming strategies post are useful background.

Everything in the table assumes your agent can keep what it needs in its own process. That isn’t always possible. A CLI that runs as a separate process and exits, or a browser binary, is read during init and then gone, so a restored session fetches it from the image again. For these cases, you can add a build hook, covered in “Working with hooks.” Figure 3 shows where it runs.

Build and invoke flow with hooks, where a memory warmup call runs before the snapshot and a disk warmup call runs after a test restore

Figure 3: Build and invoke with hooks

Measure before you optimize

Latency in an agentic application isn’t one number. A request spends time in AgentCore runtime getting a session ready to run your code, then in your code: model calls, tool calls, sub-agents, and streaming the response back. You fix these in different ways, so start by finding out which one dominates.

AgentCore runtime publishes both as Amazon CloudWatch metrics in the AWS/Bedrock-AgentCore namespace. Latency is the time to first byte. OverheadLatency is the part that belongs to the platform, such as starting the microVM and restoring the snapshot on a new session.

What you want

How to get it

TTFB (Time to first byte)

Latency

Runtime time

OverheadLatency

Your code’s time

Latency − OverheadLatency

Init time

Measured in your code at startup (see the following example). It isn’t part of any request on V2

Init doesn’t appear in these metrics because on V2 it runs once, while the snapshot is built, and not in each session. It adds to how long a create or update takes to reach READY, not to request latency. To see how long it takes, log it once before app.run(). The line appears in your runtime’s build logs:

import time
INIT_START = time.time()  # first line of your module
# imports, clients, data loading ...
print(f"init_ms={round((time.time() - INIT_START) * 1000)}")
app.run()

How to read the split:

  • High OverheadLatency on new sessions: the time is in session start. Use the patterns later in this post to start sessions before users need them.
  • Your code’s time is much higher on the first request than on the second in the same session: work that belongs in init is running in your handler. Add custom spans in AgentCore Observability (a capability of Amazon Bedrock AgentCore) around tool setup, file loads, and late imports to find it, then move it into init.

Report percentiles such as P90 and P99, or trimmed means such as TM90, which averages the fastest 90% of requests and drops the slowest tail. A plain average hides the slow requests. OverheadLatency covers requests to running sessions as well as new ones, so its distribution depends on how many of your requests start a new session. Compare runs with a similar mix.

Test at the rate of new sessions you expect in production, not with a handful of sessions after a deploy.

To include network time from your client, measure on the client as well. Time the invoke call, have your handler return how long it ran, and subtract. Both timestamps come from the same machine, so clock differences between the client and the microVM don’t matter. If you stream, very small events can be held back until a few hundred bytes of output build up, so time to first event can read higher than session start.

Keep work off the first request

The usual cold start advice is to import lazily and keep initialization small. On V2, do the opposite: init runs once, when the snapshot is built, and work deferred to the handler is paid by every new session. This applies to static work: imports, clients, and data from your image. Dynamic data can’t be snapshotted. The next section covers it.

The agents that notice most are the ones that read large files from their image at request time: weights for a small local model, embeddings, datasets, large libraries, a browser binary for automation, or a CLI toolchain that a coding agent runs as subprocesses. Reading many small files, as compiling a TypeScript project does, has the same effect. An agent that imports a few libraries and calls a hosted model mostly needs to get its imports and clients right.

How to restructure your agent

Create objects during init, not only imports. Creating an object loads the modules it depends on and keeps them in active memory. Initialize local tools that use external libraries during init too, not when the agent first calls them.

Create your clients during init. Endpoint resolution, credential resolution, and the client’s modules all come back with the snapshot. Sockets don’t, but boto3 clients reconnect cleanly on the first call. Some other network clients first try the stale socket, which adds hundreds of milliseconds, so close their connections before the snapshot.

Keep static data in memory, such as prompts, skill front matter, configuration, and small indexes. Hold the data in an object, not only a read of it:

INDEX = pathlib.Path("/opt/data/index.bin").read_bytes()  # during init: keep the bytes

Keep tools running instead of starting them per request. A subprocess launched in the handler reads its binary and libraries from the image in every new session. A long-running helper started during init, such as a language server or a worker process with its imports done, comes back as part of process memory. Talk to it over a pipe or socket.

Copy files you can’t hold in memory to a RAM-backed disk. Toolchains like tsc run as separate programs that read their own files. Copy them to /dev/shm or a ramfs mount during init so their reads come from memory.

Ship compiled bytecode. uv pip install doesn’t compile by default, so late imports also compile from source. Set UV_COMPILE_BYTECODE=1 or use pip install --compile.

The example below builds the agent, its model, and its clients during init, so every session inherits them:

import json, pathlib
import boto3
import pandas as pd  # import during init, including rarely used libraries
from strands import Agent
from strands.models import BedrockModel
from bedrock_agentcore.runtime import BedrockAgentCoreApp

MODEL_ID = "us.anthropic.claude-haiku-5-5"

# Init: runs once, before the snapshot. Every new session inherits these objects.
PROMPTS = json.loads(pathlib.Path("prompts.json").read_text())
ddb = boto3.client("dynamodb")
agent = Agent(model=BedrockModel(model_id=MODEL_ID), system_prompt=PROMPTS["system"])

app = BedrockAgentCoreApp()

@app.entrypoint
def invoke(payload):
    return agent(payload["prompt"])

app.run()  # /ping answers only after this line

The trade-off: time to first byte on a new session is the time to start the microVM and restore the snapshot, plus the time your handler takes. Moving work into init shortens the handler, but anything you hold in memory makes the snapshot larger. You also pay for that held memory in every session. Measure end-to-end session start before and after the change, so you know you’ve reduced latency and not only moved it from the handler to the restore.

Reference card

Data

Approach

Large files the first request needs, such as a model, a CLI, or a browser

Hold them in memory during init, or prime them with a build hook

Large reference data read selectively

Mount Amazon S3 Files or Amazon Elastic File System (Amazon EFS) during init, through an access point (requires virtual private cloud (VPC) mode)

Files the agent rarely reads

Leave them on the image, or move them to Amazon Simple Storage Service (Amazon S3), S3 Files, or Amazon EFS

Files the session creates

No change needed

Working with hooks

A build hook builds on init and doesn’t replace it. Do what you can in init and use the hook for what init can’t cover. You expose an HTTP endpoint in your container, and AgentCore runtime sends it a POST request twice while it builds your runtime:

  1. Before the snapshot. On the build microVM, after your agent reports healthy and before the freeze. Anything your agent loads here and keeps in memory is part of the snapshot, so every session starts with it.
  2. After a test restore. On a separate microVM that restores the snapshot to check it. Nothing from this call is snapshotted. Use it to read the files your sessions will need, which shapes what restored sessions fetch first.

The X-Amzn-Bedrock-AgentCore-Runtime-Warmup-Type header tells your agent which call it’s handling: memory for the first, disk for the second. The two calls share 120 seconds, so the disk call gets whatever time the memory call leaves. Return a 2xx response only after the work is done. Any other response, a failed connection, or running out of time fails the create or update, and AgentCore runtime doesn’t retry. The build-logs- streams in your runtime’s log group show which call failed and why.

Hooks are available on V2 runtimes. Configure them with hooksConfiguration when you create or update the runtime:

client = boto3.client("bedrock-agentcore-control")
client.create_agent_runtime(
    agentRuntimeName="my_agent",
    agentRuntimeArtifact={"containerConfiguration": {"containerUri": IMAGE_URI}},
    roleArn=ROLE_ARN,
    networkConfiguration={"networkMode": "PUBLIC"},
    platformVersion="V2",
    hooksConfiguration={
        "build": {
            "warmup": {
                "action": {"httpPost": {"path": "/warmup", "port": 8080}}
            }
        }
    },
)

The port is optional and defaults to your agent’s serving port. If you set it, use a port from 8000 to 12000 that isn’t reserved by the platform. The path can’t match a route AgentCore runtime already uses on that port, such as POST /invocations or GET /ping. UpdateAgentRuntime replaces the whole hook configuration, so include hooksConfiguration in every update where you want to keep the hook.

In the agent, handle both call types on the same endpoint:

import asyncio
from fastapi import FastAPI, Request, Response

app = FastAPI()

# Files that each session reads when it starts.
STARTUP_FILES = ["/app/models/tokenizer.json", "/app/config/agent.json"]

def load_shared_state():
    # Load what every session needs into memory, such as heavy imports,
    # model weights, and caches.
    ...

def read_startup_files():
    # Read the files that a session reads when it starts, so that new
    # sessions can use them faster.
    for path in STARTUP_FILES:
        with open(path, "rb") as f:
            while f.read(1024 * 1024):
                pass

@app.get("/ping")
def ping():
    return {"status": "Healthy"}

@app.post("/warmup")
async def warmup(request: Request):
    # Header lookup in Starlette and FastAPI is case-insensitive.
    warmup_type = request.headers.get(
        "X-Amzn-Bedrock-AgentCore-Runtime-Warmup-Type", ""
    ).lower()
    # Run the work in a worker thread so that /ping keeps answering.
    if warmup_type == "memory":
        await asyncio.to_thread(load_shared_state)
    elif warmup_type == "disk":
        await asyncio.to_thread(read_startup_files)
    # Respond only after the work is done.
    return Response(status_code=200)

@app.post("/invocations")
async def invoke(request: Request):
    payload = await request.json()
    # Per-request work runs here on every session.
    return {"response": payload.get("prompt", "")}

For the full configuration reference, see Configure lifecycle hooks for AgentCore runtime. For a complete code example, see the sample code.

Snapshot pitfalls: What must stay in the handler

Everything created during init is frozen into the snapshot until your next deploy, and every session starts with the same copy. For most objects that’s the point. For the values below, it’s a bug, and for random values, a security issue.

Load dynamic data in the handler, once per session. Tool catalogs from AgentCore Gateway (a capability of Amazon Bedrock AgentCore), feature flags, and price lists change after you deploy. Load them in the handler and cache them at module level, so each session fetches them once. Sessions can last up to 8 hours, so give the cache an expiry:

_tools, _loaded_at = None, 0.0

def get_tools():
    global _tools, _loaded_at
    if _tools is None or time.time() - _loaded_at > 300:  # wall clock: refresh after 5 minutes
        _tools, _loaded_at = load_gateway_tools(), time.time()
    return _tools

Call get_tools() from the handler and pass the result to the agent you created during init.

Create per-session values in the handler.

Value

Where to create it

Why

Random values, nonces, secrets, keys

In the handler, with secrets, os.urandom(), or uuid.uuid4()

Library random state (Python’s random, NumPy) created during init is identical in every session. Values from the operating system are unique

Current and elapsed time

In the handler, with time.time()

An init timestamp is fixed at snapshot time, and time.monotonic() doesn’t count the time the snapshot spent stored

Timers, rate limiters, TTL caches

Use wall-clock time, or reset them on the first request

Anything measuring elapsed time with a monotonic clock sees almost no gap across the restore

Credentials and tokens you fetch yourself

Refresh them when they expire

Values loaded during init can expire before a session starts

Know the limits. A session ends when its idle timeout expires (idleRuntimeSessionTimeout: 15 minutes by default, from 60 seconds up to 8 hours), when it reaches its maximum lifetime (maxLifetime: 8 hours by default and at most), or when you stop it.

Sub-second: Start the session before the user needs it

Suppose your new session takes about 2 seconds to start, including all the timing breakdown covered in the preceding sections. A session that’s already running answers in about 200 ms. For a chat or voice experience, the way to sub-second is to make sure the session has started before the user’s first request arrives. There are two ways to do that.

Pre-warm on user engagement

Most interactive applications get a signal before the first prompt: the user opens the app, a chat window, or a support page. Use it. Generate a session ID, send the agent a warm-up request in the background, and keep the session ID with the conversation. By the time the user has typed their first message, the session is running.

# Backend, when the user opens the chat (don't block the UI on it)
session_id = f"chat-{uuid.uuid4().hex}"  # session IDs need at least 33 characters
agentcore.invoke_agent_runtime(agentRuntimeArn=ARN, runtimeSessionId=session_id,
                               payload=json.dumps({"action": "prime"}).encode())
save_conversation(user_id, session_id)  # every later turn reuses this session ID

# Agent handler: answer the warm-up before any model call
@app.entrypoint
def invoke(payload):
    if payload.get("action") == "prime":
        return {"primed": True}
    return agent(payload["prompt"])

This pattern needs no extra infrastructure and fully hides session start whenever users take more than about 2 seconds to send their first message. It costs a session for each user who engages but never sends a prompt. A short idle timeout keeps that small.

Conclusion

On V2, AgentCore runtime moves most of the work of starting a session to deployment time. Your container starts once, and every new session restores the result: memory comes back whole, and files are fetched on first read.

To keep session startup time near 2 seconds, do your static work during init, hold the data the first request needs in memory, and keep dynamic and per-session values in the handler. For files your process can’t hold, add a build hook. To get under a second, start the session before the user needs it: on user engagement when you get a signal.

To get started, set platformVersion to V2 when you create or update a runtime, then review Optimize your agent for AgentCore runtime V2. For questions about your workload, reach out to your AWS account team.


About the author

原文版权归 AWS Machine Learning 及原作者所有,本站转载仅供阅读。如需删除,请联系 support@aisa.one。

事件综述

AWS 发布新一代 Bedrock AgentCore runtime 并详解 V2 快速启动架构

2026 年 11 月 12 日,AWS 发布下一代 Amazon Bedrock AgentCore runtime:会话启动最快提速 15 倍(低于 2 秒),可扩展至 8 vCPU 和 32 GB 内存并支持最大 10 GB 容器镜像,内存消耗最多降低 68%,同时每个会话仍在硬件隔离的 Firecracker MicroVM 中运行。空闲 Agent 可缩容至零,内存释放后数分钟内回收,支持按基线计费并可突发至 4 倍;Agent 可通过 FUSE 挂载 Amazon S3 等文件系统,使数据保持在信任边界内。按会话绑定存储的凭据机制计划在 re:Invent 前推出。该 runtime 已在北美、南美、欧洲和亚太的 AWS 区域可用。次日(11 月 13 日),AWS 发文介绍 runtime 在 platformVersion 设为 V2 时的会话快速启动架构模式:V2 在创建或更新 runtime 时启动容器并保存快照,每个新会话恢复快照,启动时间约 2 秒且更一致;文章讲解如何用 CloudWatch 指标区分平台开销与代码耗时,建议把导入、客户端创建和静态数据加载放在 init 阶段,为大文件和 CLI 提供 build hook,并通过对用户打开应用等行为预热会话实现亚秒级响应。报道 1 称启动最快提速 15 倍(低于 2 秒),报道 2 称 V2 模式下启动时间约 2 秒且更一致,两者表述口径不同但均指快速启动能力,未构成实质性矛盾。

其他来源的报道 1