AIsaAIsaStudio English 打开 AIsa Studio

BMW Group 如何在数千名开发者规模下治理数百个 AI 智能体

原标题:How the BMW Group governs hundreds of agents across thousands of developers

BMW Group 研发团队与 AWS 解决方案架构师在 AWS 官方博客发文,详解在超过 12,500 名开发者、约 5 亿行车载软件代码规模下治理数百个 Agent 的实践。方案以 agent harness 运行时为核心,将认证、限流等治理规则外置到 MCP gateway、中央注册表等集成层,按步骤设置三级自治信任(人工审批、异步审查、护栏内自主),并在 CI/CD 管线中以确定性代码强制执行合规检查。文章还提出 MCP tools、skills、flows 三层组合架构,通过 A2A 协议将复杂 Agent 拆分为专职 Agent,用 Terraform 等 IaC 框架落地治理,并给出四阶段部署管线与运行时漂移监控流程。

为什么值得看

BMW 一线团队复盘了在大规模研发组织中治理数百个 Agent 的结构化做法,给出了可直接借鉴的分级自治与 CI/CD 门禁模式。

阅读原文 ↗11月20日 AI 日报

本页的摘要、推荐理由与解读出自 AIsa Studio 的 AI 风向智能体,原文版权归原作者所有。

深度解读

发生了什么

AWS Machine Learning 博客发布文章,介绍宝马集团研发部门如何在数千名开发者规模下治理数百个 AI 智能体。该部门有超过 12,500 名开发者维护约 5 亿行车软件代码,并由 1,300 多个微服务组成的 Connected Vehicle 后端为 2,400 万辆在途车辆提供数字服务,属于监管要求极高的行业。

关键细节

  • 治理规则外置且与框架无关:认证、授权、限流和内容过滤在集成层(MCP 网关、中央智能体注册表、编排边界)执行,因此能「在框架迁移、模型更换和团队重组中得以保留」。
  • 按步骤而非按智能体分级授权:Level 0 为人工审批每一步,Level 1 为自主执行加异步人工审查,Level 2 为在确定性策略边界内自主运行。边界通过 CI/CD 流水线以确定性代码强制执行,智能体及其提示词都无法绕过。
  • 三层组合体系:MCP 工具作为原子集成契约,技能为面向领域的工具组合,流程为端到端编排。团队在实现前需产出范围定义、交互模型、工具定义、真值数据集和评估标准五项产物。
  • 四阶段部署流水线依次执行快速确定性检查、安全与治理检查、评估门禁和去重检查。治理智能体会标记范围蔓延、提示词膨胀、工具误用和性能退化,供人工确认,保持四眼原则。注册表元数据包含责任人、SLA 目标、数据治理标记,以及对高影响或不可逆操作要求人工介入的标记。
  • 运行时监控数据漂移与模型漂移,告警触发分级响应并转发人工审查。采纳模式是「自下而上加中央策展」:团队自由构建技能和工具并通过注册表共享,中央团队提供治理即代码模块。

为什么重要

文章的核心论点是:大规模构建与运营智能体是一门工程学科,而非实验性练习;治理必须「从一开始就是结构性的」——外置、领域敏感、确定性执行。据其经验,可复用模板和共享基础设施降低了开发成本,标准化的交互模型和自动化部署让智能体更快上线,流水线在生命周期各阶段强制执行成熟模式也提升了质量。这一模式展示了大型组织如何在团队创新速度与企业安全、安全合规、可组合性和成本效率标准之间取得平衡,避免形成割裂的「AI 孤岛」和重复建设。

对谁有影响

这些实践源自宝马自 2023 年以来的企业级智能体构建经验,直接面向需要在大型、受监管组织中工业化智能体的团队。文章还列出了后续方向:发布架构权衡与反模式指南、扩展基础设施即代码模板、深化合规测试、扩展智能体 FinOps,并将这些模式集成到宝马的架构指导智能体 Archimedes 中。

全文翻译

以下翻译只覆盖原文的前一部分,其余内容请看下方原文。

当数百个智能体(agents)服务于数千名开发人员时,企业治理绝不能被视为事后补充。

在宝马集团(BMW Group)的研发部门,超过 12,500 名开发人员正在处理约 5 亿行车辆软件代码。这些软件由 BMW Connected Vehicle 后端增强,这是一个基于云的系统,包含 1,300 多个微服务,为现场 2400 万辆汽车提供数字服务。在受监管的行业里,对(软件)质量、稳定性和治理合规性的要求极高。

虽然目前的研究、框架和指南有助于构建智能体,但以高效、可扩展的方式工业化运营高质量智能体仍然是一个重大挑战。

要在大型组织中扩展智能体 AI,仅靠孤立的团队是无法实现的。这需要集体努力、本地驱动的实验以及跨企业的积极知识共享。这就产生了一个根本性的治理两难困境:开发团队需要快速创新的自由,而企业必须执行安全、安全性、可组合性和成本效率的标准。如果没有这种平衡,组织将面临人工智能孤岛碎片化和昂贵重复工作的风险。鉴于宝马集团内部有数百个全球分布的团队正在构建智能体,这确实是一个现实挑战。

在本文中,我们详细介绍了支持宝马研发团队在不牺牲速度、质量或合规性的情况下,大规模构建、治理和运营智能体的实践和模式。

解决方案

我们的前提很简单:从一开始就构建能够协作的智能体。这需要一种架构,使多样性具备可组合性、可治理性和可自动化性,而无需依赖单一框架或工具链。

该架构的核心是智能体运行环境(agent harness),它是包裹在大语言模型(LLM)周围的一层运行时层。它为自主操作提供了执行循环、工具调度、内存管理和可观测性钩子。

正如 AWS 架构最佳实践框架 标准化了云基础设施设计一样,智能体系统也需要等效的模式、可重用的蓝图,从开始就编码治理、可组合性和运营最佳实践。

我们将解决方案围绕五个相互关联的实践进行组织,这些实践源自自 2023 年以来在企业规模上构建和运营智能体系统的实践经验。每项实践都固化为基础设施即代码(IaC),使得采用过程在各团队之间可重复、可审计且可强制执行,同时不会阻碍 DevOps 的速度。

主动治理,而非事后补救

治理必须从开始就在结构上确立,并贯穿整个生命周期。我们提倡有效智能体治理的三个属性:

与框架无关且外部化: 身份验证、授权、速率限制和内容过滤等治理规则在集成层强制执行。这包括模型上下文协议(MCP)网关、中央智能体注册表和编排边界,而不是在单个智能体代码库中。这意味着治理可以 survive 框架迁移、模型替换和团队重组。

领域敏感且分级: 并非每个智能体动作都承担相同的风险:起草文档、更新客户记录和写入生产数据库需要不同级别的监督。我们通过每步自主权级别实施分级信任:

  • 第 0 级(人在回路中): 每个动作都需要明确的人类批准。这是新智能体和高危操作的起点。
  • 第 1 级(人在环路上): 智能体自主执行,但将决策呈现给异步人类审查。适用于具有定义输入、输出和回滚选项的中风险任务。
  • 第 2 级(在护栏内自主): 智能体在严格定义的政策边界内独立运行,通过确定性方式强制执行,而不是通过提示指令(见下文)。保留用于低风险、已充分理解且具有验证记录的操作。

按步骤分配信任级别,而不是按智能体分配。单个工作流可以包含所有三个级别的步骤,因此团队可以从保守开始,随着信心的建立逐步增加自主权,这基于可衡量的评估数据,而不是假设。

确定性强制执行: 治理作为确定性代码在智能体的推理循环之外运行。在实践中,这主要发生在 CI/CD 管道中。权限检查、策略即代码规则和合规性验证在每个部署之前都在那里进行门控,直到智能体到达生产环境。因为这些检查与智能体分开运行,所以智能体及其提示都无法覆盖它们。出口控制(收件人白名单、出站密钥扫描)有助于防止持有机密数据的智能体通过看似合法的调用泄露数据。

持续仪器化、验证和评估

智能体系统中的可观测性超越了传统的应用程序监控。真正的问题不再是服务是否返回了 HTTP 200 状态码,而是智能体是否产生了忠实且符合政策的输出,以及如何通过可执行的执行轨迹来证明这一点。回答这个问题需要一个涵盖常见查询和边缘情况的真实数据集(ground-truth dataset)。这个数据集从第一天起就用于评估,因此质量是针对代表性且适当治理的评估数据集进行衡量的,而不是针对合成提示。

这种基础设施无法后期添加,必须是初始设计的一部分。持续集成和持续交付(CI/CD)管道强制执行这一点,因此缺乏仪器化的智能体在评估运行之前就会失败验证。

我们建议仪器化三个维度:

  • 意图追踪: 捕获每次执行的可见智能体轨迹(推理步骤、工具调用和上下文检索)。OpenTelemetry span 创建从用户请求到最终输出的可查询谱系。
  • 验证和评估作为 CI: 对智能体的任何更改(提示编辑、工具添加、模型替换)都会触发管道。验证首先运行,涵盖智能体组件及其交互的自动回归测试。由于大多数运行环境组件是确定性的,这些测试速度快且完全可自动化。评估其次运行,根据真实数据集对非确定性输出进行评分,并在质量阈值上进行门控。这种排序是故意的:验证充当闸门,因此评估仅在通过的构建上运行。
  • 模型分层和路由遥测: 当智能体在模型之间路由时,捕获路由决策及其理由,而不仅仅是结果。分层分配是根据每种任务类型的评估结果得出的,不良结果会作为新的评估案例反馈回来。

专注起步,设计为可组合

对于分布式云系统,行业通过采用微服务或服务网格等架构模式解决了端到端工作流的复杂性。智能体系统的演变需要遵循类似的轨迹,从单体、通用型智能体转向多个协作的专业化智能体。

团队应在考虑智能体功能之前定义问题。在编写任何编排逻辑之前,他们应生成五个具体工件:

  1. 清晰的范围定义: 智能体应该做什么和不应该做什么。
  2. 交互模型: 用户和其他智能体如何调用它。
  3. 明确定义的工具: 每个工具和参数都精确定义。
  4. 真实数据集: 上述评估参考,在任何实现开始之前建立。
  5. 评估标准: 智能体将被衡量的质量阈值和基准。

这种纪律防止了我们观察到的最常见的故障模式:承担过多责任的智能体会积累臃肿的提示,降低准确性并增加延迟。

一旦单个智能体范围清晰,就必须将其设计为可组合的构建块。因此,我们定义了三层组合层次结构:

  • MCP 工具 是原子集成契约。每个工具封装单一能力(例如 API 调用或数据库查询),而不编码工作流假设。这是稳定、可重用的层。
  • 技能(Skills) 是 MCP 工具的捆绑、特定领域的组合。例如,代码审查技能可能会链接静态分析工具、风格检查工具和基于 LLM 的推理步骤。技能编码领域逻辑,但在工作流之间保持便携。
  • 流程(Flows) 是技能的端到端编排,代表完整的业务流程(例如,整个 PR 审查流水线或需求到部署周期)。

每一层都在 CI/CD 管道中独立验证。组件测试涵盖工具和技能,交互测试涵盖它们组合成流程的情况。

这些层通过开放标准连接:MCP 和 Agent Skills 用于工具集成,A2A 协议用于智能体间通信,OpenTelemetry 用于跨由此产生的交互的可观测性。

图 1 显示了这三层如何相互构建,从底部的单个 MCP 工具到顶部的完整流程。

图 1:构建可组合智能体的三层系统

图 1:构建可组合智能体的三层系统

边学习边专业化

在生产环境中,即使范围清晰的智能体也会随着实际使用与设计时的预期偏离而崩溃:查询模式发生变化,工具选择逻辑变得困难,性能不可预测地下降。

正确的响应是将复杂的智能体分解为多个专业化的智能体,每个智能体都有更清晰的指令、更简单的工具集和更集中的推理。好处是具体的:

  • 独立演进: 更新需求工程智能体不需要触碰编码智能体。
  • 单独扩展: 当范围和接口定义良好时,单个智能体可以独立扩展,而无需重新部署或调整更大系统的规模。
  • 最小化范围: 一个智能体的故障或幻觉不会在整个工作流中级联。
  • 成本节约: 范围狭窄的智能体通常可以在较小的模型上运行,从而降低推理和托管成本。

复杂性可以沿两个轴减少:

  • 在一个智能体内: 将能力分解为技能,因为工具和少样本示例在每次调用时都会消耗上下文,同时降低准确性并增加成本。通过经验确定粒度:合并智能体很少选择的技能,并根据输入拆分产生不同行为的技能。
  • 跨智能体: 将超出智能体领域的工作委托给 A2A 协议上的专业化同行,而不是扩展当前智能体。智能体应在能力上增长,而不是在上下文上增长。

关键在于,由于每个专业化的智能体只访问其角色所需的数据和工具,这种架构有助于在设计上强制执行最小权限原则,这是受监管行业的合规要求。

然而,要实现这些好处,需要智能体能够找到彼此并互操作。为了防止出现 AI 孤岛,我们维护一个 集中式 的智能体、工具和技能注册表。此注册表用作:

  • 发现机制: 根据构建期间存储的元数据编目可用的智能体、工具和技能。
  • 治理检查点: 确认资产类型获得生产使用的批准并保持审计跟踪。
  • 安全检查点: 在批准之前应用自动安全检查,例如提示注入
查看原文

When hundreds of agents serve thousands of developers, enterprise governance can’t be an afterthought.

At the BMW Group’s R&D, more than 12,500 developers work on approximately 500 million lines of vehicle software code. This software is augmented by the BMW Connected Vehicle backend, a cloud-based system of more than 1,300 microservices providing digital services for 24 million vehicles in the field. In regulated industry, the bar for (software) quality, stability, and governance compliance is exceptionally high.

While research, frameworks, and guidelines now facilitate building agents, industrializing and operating high-quality agents in an efficient, scalable manner remains a significant challenge.

Scaling agentic AI across a large organization cannot be achieved by isolated teams. It requires collective effort, locally driven experimentation, and active knowledge sharing across the enterprise. This creates a fundamental governance dichotomy: Development teams need the freedom to innovate rapidly, while the enterprise must enforce standards for safety, security, composability, and cost-efficiency. Without that balance, organizations risk a fragmented landscape of AI islands and costly duplicate work. This presents a real challenge given hundreds of globally distributed teams building agents across the BMW Group.

In this post, we detail the practices and patterns that support BMW’s R&D teams in building, governing and operating agents at scale without sacrificing speed, quality or compliance.

Solution

Our premise is straightforward: build agents that can cooperate from the start. This requires an architecture that makes diversity composable, governable, and automatable, without requiring a single framework or toolchain.

Central to this architecture is the agent harness, a runtime layer wrapped around a large language model (LLM). It provides the execution loop, tool dispatch, memory management, and observability hooks for autonomous operation.

Just as the AWS Well-Architected Framework standardized cloud infrastructure design, agentic systems need equivalent patterns, reusable blueprints encoding governance, composability, and operational best practices from the start.

We organize our solution around five interlocking practices, drawn from hands-on experience building and operating agentic systems at enterprise scale since 2023. Each practice is codified as infrastructure as code (IaC), making adoption repeatable, auditable, and enforceable across teams without blocking DevOps velocity.

Govern proactively, not retroactively

Governance must be structural from the start and maintained across the full lifecycle. We advocate for three properties of effective agentic governance:

Framework-agnostic and externalized: Governance rules such as authentication, authorization, rate limiting and content filtering are enforced at the integration layer. This includes a Model Context Protocol (MCP) gateway, a central agent registry and orchestration boundary, rather than within individual agent codebases. This means governance survives framework migrations, model swaps, and team reorganizations.

Domain-sensitive and graduated: Not every agent action carries the same risk: drafting documentation, updating customer records, and production database writes demand very different levels of oversight. We implement graduated trust through per-step autonomy levels:

  • Level 0 (Human-in-the-loop): Every action requires explicit human approval. This is the starting point for new agents and high-risk operations.
  • Level 1 (Human-on-the-loop): The agent executes autonomously but surfaces decisions for asynchronous human review. Suitable for medium-risk tasks with defined inputs, outputs, and rollback options.
  • Level 2 (Autonomous within guardrails): The agent operates independently within strictly defined policy boundaries, enforced deterministically rather than through prompt instructions (see below). Reserved for low-risk, well-understood operations with verified track records.

Assign trust levels per step, not per agent. A single workflow can include steps at all three levels, so teams can start conservatively and incrementally increase autonomy as confidence builds, grounded in measurable evaluation data, not assumptions.

Deterministically enforced: Governance runs as deterministic code outside the agent’s reasoning loop. In practice, this happens mainly in the CI/CD pipeline. Permission checks, policy-as-code rules, and compliance validations gate every deployment there, before an agent reaches production. Because these checks run separately from the agent, neither the agent nor its prompts are in a position to override them. Egress controls (recipient allowlisting, outbound secret scanning) help prevent an agent holding confidential data from disclosing it through an otherwise legitimate call.

Instrument, verify, and evaluate continuously

Observability in agentic systems goes beyond traditional application monitoring. The real question is no longer whether a service returned an HTTP 200 status, but whether an agent produced a faithful, policy-compliant output, and how this can be demonstrated through observable execution traces. Answering that requires a ground-truth dataset covering common queries and edge cases. This dataset is used for evaluation from day one, so quality is measured against representative and appropriately governed evaluation datasets for evaluation, not synthetic prompts.

This infrastructure cannot be retrofitted and must be part of the initial design. The continuous integration and continuous delivery (CI/CD) pipeline enforces this, so agents lacking instrumentation fail validation before evaluation runs.

We recommend instrumenting three dimensions:

  • Intent tracing: Capture the observable agent trajectory (reasoning steps, tool invocations, and context retrievals) for each execution. OpenTelemetry spans create a queryable lineage from user request to final output.
  • Verification and evaluation as CI: Any change to the agent (prompt edits, tool additions, model swaps) triggers the pipeline. Verification runs first with automated regression tests covering agent components and then their interactions. As most harness components are deterministic, these tests are fast and fully automatable. Evaluation runs second, scoring non-deterministic output against the ground-truth dataset, gated on quality thresholds. This sequencing is deliberate: Verification acts as a gate, so evaluations only run on a passing build.
  • Model tiering and routing telemetry: When agents route between models, capture the routing decision and its justification, not just the outcome. Tier assignments are derived from evaluation results per task type and poor outcomes flow back as new evaluation cases.

Start focused, design for composition

For distributed cloud systems, the industry solved the complexity of end-to-end workflows by adopting architectural patterns such as microservices or service meshes. The evolution of agentic systems needs to follow a similar trajectory, moving from monolithic, generalist agents toward multiple collaborating specialized agents.

Teams should define the problem before considering agents’ functionality. Before writing any orchestration logic, they should produce five concrete artifacts:

  1. Clear scope definition: what the agent should and shouldn’t do.
  2. Interaction model: how users and other agents will invoke it.
  3. Unambiguous tool definitions: each tool and parameter precisely defined.
  4. Ground-truth dataset: the evaluation reference described above, established before any implementation begins.
  5. Evaluation criteria: quality thresholds and benchmarks the agent will be measured against.

This discipline prevents the most common failure modes we observe: agents that take on too many responsibilities accumulate bloated prompts, reducing accuracy and increasing latency.

Once a single agent is well-scoped, it must be designed as a composable building block. Therefore, we define a three-tier composition hierarchy:

  • MCP tools are atomic integration contracts. Each tool wraps a single capability (for example, an API call or a database query) without encoding workflow assumptions. This is the stable, reusable layer.
  • Skills are bundled, domain-specific compositions of MCP tools. A code review skill, for example, might chain a static analysis tool, a style-check tool, and an LLM-based reasoning step. Skills encode domain logic but remain portable across workflows.
  • Flows are end-to-end orchestrations of skills, representing complete business processes (for example, an entire PR review pipeline or a requirements-to-deployment cycle).

Each tier is validated independently in the CI/CD pipeline. Component tests cover tools and skills, interaction tests cover their composition into flows.

These tiers are connected through open standards: MCP and Agent Skills for tool integration, the protocol for agent-to-agent communication, and OpenTelemetry for observability across the resulting interactions.

Figure 1 shows how the three tiers build on each other, from individual MCP tools at the base to complete flows at the top.

Figure 1: Three-tier system to build composable agents

Figure 1: Three-tier system to build composable agents

Specialize as you learn

In production, even well-scoped agents can break down as real-world usage diverges from what was anticipated at design time: query patterns shift, tool-selection logic struggles, and performance degrades unpredictably.

The right response is to decompose complex agents into multiple specialized ones, each with clearer instructions, simpler tool sets, and more focused reasoning. The benefits are concrete:

  • Evolve independently: updating a requirements engineering agent does not require touching the coding agent.
  • Scale individually: when scope and interfaces are well-defined, individual agents can be scaled independently without redeploying or resizing the broader system.
  • Minimized scope: a failure or hallucination of one agent does not cascade across the entire workflow.
  • Cost savings: A narrowly scoped agent can often run on a smaller model, which cuts inference and hosting costs.

Complexity can be reduced along two axes:

  • Within an agent: decompose capabilities into skills since tools and few-shot examples consume context on every invocation while degrading accuracy and increasing cost. Determine granularity empirically: merge skills the agent rarely selects and split skills that produce different behavior depending on the input.
  • Across agents: delegate work outside an agent’s domain to a specialized peer over the A2A protocol rather than extending the current agent. An agent should grow in capability, not in context.

Critically, since each specialized agent only accesses the data and tools required for its role, this architecture helps to enforce least-privilege by design, a compliance requirement in regulated industries.

Realizing these benefits, however, requires that agents can find and interoperate with each other. To prevent the emergence of AI islands, we maintain a centralized registry of agents, tools, and skills. This registry serves as:

  • Discovery mechanism: cataloging available agents, tools, and skills based on metadata stored during build.
  • Governance checkpoint: confirming approval of the asset types for production use and maintaining an audit trail.
  • Security checkpoint: automated security checks are applied before approval, such as prompt injection and similar issues.
  • Deduplication safeguard: identifying equivalent capabilities already maintained by other teams to avoid duplicate work. This check runs automatically at registration, flagging duplicates before approval.

Scale through codification

The shift from managing individual agents to governing an agent fleet requires agent architectures to be codified. Using Infrastructure-as-code frameworks such as Terraform or AWS CloudFormation, teams provision and manage agent infrastructure with the same rigor applied to any production system:

  • Fine-grained least-privilege roles: scoped per agent or tool.
  • CI/CD pipelines: consistent, validated deployments across accounts and regions, with automated deployment stops on failed tests or metric deterioration.
  • Policy-as-code: rules (for example, AWS Config, Open Policy Agent) scanning deployed agents for compliance with organizational standards.

All three are enforced as pipeline gates, validated before registration, not audited after the fact.

The adoption model is deliberately bottom-up with central curation. Individual teams build skills and MCP tools to solve their immediate problems. All teams can contribute by sharing their assets through the registry. A central team builds automatic governance checks as part of the CI pipeline to uphold standards. To support development teams and foster shift-left, the central team additionally provides building blocks as governance-as-code modules.

This balance avoids both centrally mandated architectures and unmanaged experimentation. As a result, when new operational concerns arise, teams can extend existing patterns rather than redesign their architectures.

Operationalizing best practices across the agent lifecycle

Architectural patterns and governance standards are only effective if they are embedded into the workflows teams use to build and operate agents before and at runtime.

Before deployment

Governance begins before a single line of orchestration logic is written. Teams are provided with reference architecture patterns and examples drawn from the registry. These pre-validated blueprints cover common agent configurations, tool integration patterns, and composition structures, which accelerate development while encoding proven defaults. Before anything reaches production, standardized CI/CD controls enforce these practices. Figure 2 shows the four phases of a reference pipeline, which run in order from left to right.

Figure 2: Reference pattern for a four-phase agent deployment pipeline.

Figure 2: Reference pattern for a four-phase agent deployment pipeline.

Phase 1 from Figure 2 runs fast, deterministic checks: supply chain security (approved registries only), component and interaction testing, evaluation library references, and observability instrumentation.

Phase 2 runs security and governance checks in parallel. Static analysis scans the code for issues, and security tests probe the agent in an isolated staging environment. The pipeline also validates egress controls and least-privilege permissions. Two more checks confirm that the agent’s declared autonomy level matches its actual permissions and that its configuration passes the policy-as-code rules. A governance agent accelerates audits by checking the agent against established standards and flagging findings for human review, such as:

  • Scope creep: operating beyond originally defined purpose or permissions.
  • Prompt bloat: unnecessary complexity in instructions.
  • Tool misuse: calling tools outside their intended use case.
  • Performance degradation: declining accuracy, latency, or cost-efficiency.

Figure 3: Example governance agent workflow

This assessment surfaces concrete, actionable measures that teams can apply immediately, rather than abstract recommendations. Figure 3 shows the governance agent’s dashboard for an agent maturity assessment. The governance agent provides evidence and actionable recommendations, while accountable humans own governance decisions. Recommendations require human confirmation before being applied, maintaining a four-eyes principle for accountability. Findings update the agent’s governance status in the registry, while full details are logged separately in a central knowledge base for audit and traceability.

Phase 3 evaluates the agent as a running system: the ground-truth evaluation gate (scores outputs against benchmarks established at design time) and agent performance (latency, token consumption). Figure 4 details the metrics, scoring criteria and the relationships between them.

Figure 4: Overview of evaluation metrics

Figure 4: Overview of evaluation metrics

Phase 4 checks for duplicates before registering the agent, its sub-agents, tools, and skills. The following table shows the metadata fields stored per agent in the registry.

Agent metadata

Core data:

  • unique identifier
  • name
  • description
  • version
  • business owner
  • IT owner
  • Capabilities: names, descriptions
  • Functions list: name, description, direction: provides/ uses
  • Status: plan / active / deprecated
  • Link to specification, agent card or documentation
  • Hosting mode: on-prem, cloud, cloud-region

Service level agreement data:

  • Target availability in percent
  • Target P95 response time in ms
  • Support hours / operational support model

Data governance flags:

  • information classification level
  • personal data processed flag
  • data retention period/ policy
  • request and response payloads logged flag
  • PII data redacted flag
  • Human in the loop required flag (required for high-impact / irreversible actions)

Table 1: Example agent metadata stored in the registry

If any step fails or metrics deteriorate, the deployment stops with logs for root cause analysis. If all checks pass, the agent is packaged and deployed as a containerized workload, producing consistent, reproducible deployments. This prevents ungoverned agents from reaching production, automates compliance checks, and keeps the registry accurate.

At runtime

At runtime, observability has to catch two kinds of drift. Data drift shows up as new or shifting user query patterns, or changes in upstream APIs. Model drift shows as rising error rates or falling tool-selection accuracy. When a customer flags a response, that also triggers an investigation. Alerts for drift, distribution shifts, declining evaluation scores, or direct customer flags prompt a staged response. These events are logged and forwarded to human review. Additionally, teams can run the governance agent continuously to flag drift and operational issues for the responsible team to investigate.

Summary

Building and operating agents at scale is an engineering discipline, not an experimentation exercise.

Effective governance must be structural from the start, externalized, domain-sensitive, and deterministically enforced. We introduced a three-tier composition hierarchy (MCP tools, skills, and flows) as the architectural foundation for reusable, composable, and interoperable agents, decomposed into specialized agents to contain complexity and scope. We showed how verification and evaluation in every CI/CD run make quality assurance part of everyday development, and how defining agent infrastructure as code lets teams apply the same governance rules to every agent in the fleet. Finally, we operationalized these patterns across the agent lifecycle, from pre-validated reference architectures and structured pipelines to a runtime governance agent that continuously evaluates deployed agents.

Our experience has shown these practices pay off in a few ways. Reusable templates and shared infrastructure reduced development costs. Teams also got agents to production faster, thanks to standard interaction models and automated deployment. Quality improved too, because the pipeline enforces proven patterns at every stage of the lifecycle.

Scaling agentic governance is an industry-wide imperative. The shift toward AI and generative-AI-supported workflow automation is already reshaping how engineering organizations operate. The winners will be those who move beyond experimentation and establish the discipline to govern autonomous agent networks at scale. With these practices in place, BMW engineers can focus on solving the problems that matter: Building the ultimate driving machine.

To apply these ideas to your own agents, start with the AWS Well-Architected Framework. This post used it as the model for reusable agent blueprints. To see how BMW codifies governance across its AWS environment, read How BMW Group uses automation to achieve end-to-end compliance at scale on AWS.

Outlook and next steps

The patterns and practices described in this post are a foundation, not a finish line. We are actively extending them across several fronts:

  • Improve architectural guidance: publishing best practice guides on architectural trade-offs, composition patterns, and anti-patterns for teams building agents.
  • Expand infrastructure-as-code templates: providing agent blueprints as usable IaC modules with built-in security and governance standards by default.
  • Deepen governance compliance testing: adding CI/CD test cases verifying governance compliance.
  • Automate standards enforcement: expanding automated checks and managed compliance-scanning rules across pre-deployment and runtime stages.
  • Extend agentic FinOps: Govern the right-sizing of models and tool calls.
  • Agentic user guidance: integrating described patterns into Archimedes, BMW’s architectural guidance agent, to provide contextual architectural recommendations.

Acknowledgements

The authors would like to thank (in alphabetical order) Alexander Stark, Alexander Zimmer, Dr. Céline Laurent-Winter, Dominik Segl, Florian Seidel, Hugo Lerias, Dr. Lars Lewerenz, Mark Neumann, Ralf Bottesch, Ramona Weber, Satish Vasireddy, Samuel Voss, Xagarey-Louis Meyer.


About the authors

原文版权归 AWS Machine Learning 及原作者所有,本站转载仅供阅读。如需删除,请联系 support@aisa.one。