在宝马集团(BMW Group)的研发部门,超过 12,500 名开发人员正在处理约 5 亿行车辆软件代码。这些软件由 BMW Connected Vehicle 后端增强,这是一个基于云的系统,包含 1,300 多个微服务,为现场 2400 万辆汽车提供数字服务。在受监管的行业里,对(软件)质量、稳定性和治理合规性的要求极高。
要在大型组织中扩展智能体 AI,仅靠孤立的团队是无法实现的。这需要集体努力、本地驱动的实验以及跨企业的积极知识共享。这就产生了一个根本性的治理两难困境:开发团队需要快速创新的自由,而企业必须执行安全、安全性、可组合性和成本效率的标准。如果没有这种平衡,组织将面临人工智能孤岛碎片化和昂贵重复工作的风险。鉴于宝马集团内部有数百个全球分布的团队正在构建智能体,这确实是一个现实挑战。
该架构的核心是智能体运行环境(agent harness),它是包裹在大语言模型(LLM)周围的一层运行时层。它为自主操作提供了执行循环、工具调度、内存管理和可观测性钩子。
我们将解决方案围绕五个相互关联的实践进行组织,这些实践源自自 2023 年以来在企业规模上构建和运营智能体系统的实践经验。每项实践都固化为基础设施即代码(IaC),使得采用过程在各团队之间可重复、可审计且可强制执行,同时不会阻碍 DevOps 的速度。
按步骤分配信任级别,而不是按智能体分配。单个工作流可以包含所有三个级别的步骤,因此团队可以从保守开始,随着信心的建立逐步增加自主权,这基于可衡量的评估数据,而不是假设。
智能体系统中的可观测性超越了传统的应用程序监控。真正的问题不再是服务是否返回了 HTTP 200 状态码,而是智能体是否产生了忠实且符合政策的输出,以及如何通过可执行的执行轨迹来证明这一点。回答这个问题需要一个涵盖常见查询和边缘情况的真实数据集(ground-truth dataset)。这个数据集从第一天起就用于评估,因此质量是针对代表性且适当治理的评估数据集进行衡量的,而不是针对合成提示。
对于分布式云系统,行业通过采用微服务或服务网格等架构模式解决了端到端工作流的复杂性。智能体系统的演变需要遵循类似的轨迹,从单体、通用型智能体转向多个协作的专业化智能体。
这些层通过开放标准连接:MCP 和 Agent Skills 用于工具集成,A2A 协议用于智能体间通信,OpenTelemetry 用于跨由此产生的交互的可观测性。
查看原文
When hundreds of agents serve thousands of developers, enterprise governance can’t be an afterthought.
At the BMW Group’s R&D, more than 12,500 developers work on approximately 500 million lines of vehicle software code. This software is augmented by the BMW Connected Vehicle backend, a cloud-based system of more than 1,300 microservices providing digital services for 24 million vehicles in the field. In regulated industry, the bar for (software) quality, stability, and governance compliance is exceptionally high.
While research, frameworks, and guidelines now facilitate building agents, industrializing and operating high-quality agents in an efficient, scalable manner remains a significant challenge.
Scaling agentic AI across a large organization cannot be achieved by isolated teams. It requires collective effort, locally driven experimentation, and active knowledge sharing across the enterprise. This creates a fundamental governance dichotomy: Development teams need the freedom to innovate rapidly, while the enterprise must enforce standards for safety, security, composability, and cost-efficiency. Without that balance, organizations risk a fragmented landscape of AI islands and costly duplicate work. This presents a real challenge given hundreds of globally distributed teams building agents across the BMW Group.
In this post, we detail the practices and patterns that support BMW’s R&D teams in building, governing and operating agents at scale without sacrificing speed, quality or compliance.
Solution
Our premise is straightforward: build agents that can cooperate from the start. This requires an architecture that makes diversity composable, governable, and automatable, without requiring a single framework or toolchain.
Central to this architecture is the agent harness, a runtime layer wrapped around a large language model (LLM). It provides the execution loop, tool dispatch, memory management, and observability hooks for autonomous operation.
Just as the AWS Well-Architected Framework standardized cloud infrastructure design, agentic systems need equivalent patterns, reusable blueprints encoding governance, composability, and operational best practices from the start.
We organize our solution around five interlocking practices, drawn from hands-on experience building and operating agentic systems at enterprise scale since 2023. Each practice is codified as infrastructure as code (IaC), making adoption repeatable, auditable, and enforceable across teams without blocking DevOps velocity.
Govern proactively, not retroactively
Governance must be structural from the start and maintained across the full lifecycle. We advocate for three properties of effective agentic governance:
Framework-agnostic and externalized: Governance rules such as authentication, authorization, rate limiting and content filtering are enforced at the integration layer. This includes a Model Context Protocol (MCP) gateway, a central agent registry and orchestration boundary, rather than within individual agent codebases. This means governance survives framework migrations, model swaps, and team reorganizations.
Domain-sensitive and graduated: Not every agent action carries the same risk: drafting documentation, updating customer records, and production database writes demand very different levels of oversight. We implement graduated trust through per-step autonomy levels:
- Level 0 (Human-in-the-loop): Every action requires explicit human approval. This is the starting point for new agents and high-risk operations.
- Level 1 (Human-on-the-loop): The agent executes autonomously but surfaces decisions for asynchronous human review. Suitable for medium-risk tasks with defined inputs, outputs, and rollback options.
- Level 2 (Autonomous within guardrails): The agent operates independently within strictly defined policy boundaries, enforced deterministically rather than through prompt instructions (see below). Reserved for low-risk, well-understood operations with verified track records.
Assign trust levels per step, not per agent. A single workflow can include steps at all three levels, so teams can start conservatively and incrementally increase autonomy as confidence builds, grounded in measurable evaluation data, not assumptions.
Deterministically enforced: Governance runs as deterministic code outside the agent’s reasoning loop. In practice, this happens mainly in the CI/CD pipeline. Permission checks, policy-as-code rules, and compliance validations gate every deployment there, before an agent reaches production. Because these checks run separately from the agent, neither the agent nor its prompts are in a position to override them. Egress controls (recipient allowlisting, outbound secret scanning) help prevent an agent holding confidential data from disclosing it through an otherwise legitimate call.
Instrument, verify, and evaluate continuously
Observability in agentic systems goes beyond traditional application monitoring. The real question is no longer whether a service returned an HTTP 200 status, but whether an agent produced a faithful, policy-compliant output, and how this can be demonstrated through observable execution traces. Answering that requires a ground-truth dataset covering common queries and edge cases. This dataset is used for evaluation from day one, so quality is measured against representative and appropriately governed evaluation datasets for evaluation, not synthetic prompts.
This infrastructure cannot be retrofitted and must be part of the initial design. The continuous integration and continuous delivery (CI/CD) pipeline enforces this, so agents lacking instrumentation fail validation before evaluation runs.
We recommend instrumenting three dimensions:
- Intent tracing: Capture the observable agent trajectory (reasoning steps, tool invocations, and context retrievals) for each execution. OpenTelemetry spans create a queryable lineage from user request to final output.
- Verification and evaluation as CI: Any change to the agent (prompt edits, tool additions, model swaps) triggers the pipeline. Verification runs first with automated regression tests covering agent components and then their interactions. As most harness components are deterministic, these tests are fast and fully automatable. Evaluation runs second, scoring non-deterministic output against the ground-truth dataset, gated on quality thresholds. This sequencing is deliberate: Verification acts as a gate, so evaluations only run on a passing build.
- Model tiering and routing telemetry: When agents route between models, capture the routing decision and its justification, not just the outcome. Tier assignments are derived from evaluation results per task type and poor outcomes flow back as new evaluation cases.
Start focused, design for composition
For distributed cloud systems, the industry solved the complexity of end-to-end workflows by adopting architectural patterns such as microservices or service meshes. The evolution of agentic systems needs to follow a similar trajectory, moving from monolithic, generalist agents toward multiple collaborating specialized agents.
Teams should define the problem before considering agents’ functionality. Before writing any orchestration logic, they should produce five concrete artifacts:
- Clear scope definition: what the agent should and shouldn’t do.
- Interaction model: how users and other agents will invoke it.
- Unambiguous tool definitions: each tool and parameter precisely defined.
- Ground-truth dataset: the evaluation reference described above, established before any implementation begins.
- Evaluation criteria: quality thresholds and benchmarks the agent will be measured against.
This discipline prevents the most common failure modes we observe: agents that take on too many responsibilities accumulate bloated prompts, reducing accuracy and increasing latency.
Once a single agent is well-scoped, it must be designed as a composable building block. Therefore, we define a three-tier composition hierarchy:
- MCP tools are atomic integration contracts. Each tool wraps a single capability (for example, an API call or a database query) without encoding workflow assumptions. This is the stable, reusable layer.
- Skills are bundled, domain-specific compositions of MCP tools. A code review skill, for example, might chain a static analysis tool, a style-check tool, and an LLM-based reasoning step. Skills encode domain logic but remain portable across workflows.
- Flows are end-to-end orchestrations of skills, representing complete business processes (for example, an entire PR review pipeline or a requirements-to-deployment cycle).
Each tier is validated independently in the CI/CD pipeline. Component tests cover tools and skills, interaction tests cover their composition into flows.
These tiers are connected through open standards: MCP and Agent Skills for tool integration, the protocol for agent-to-agent communication, and OpenTelemetry for observability across the resulting interactions.
Figure 1 shows how the three tiers build on each other, from individual MCP tools at the base to complete flows at the top.

Figure 1: Three-tier system to build composable agents
Specialize as you learn
In production, even well-scoped agents can break down as real-world usage diverges from what was anticipated at design time: query patterns shift, tool-selection logic struggles, and performance degrades unpredictably.
The right response is to decompose complex agents into multiple specialized ones, each with clearer instructions, simpler tool sets, and more focused reasoning. The benefits are concrete:
- Evolve independently: updating a requirements engineering agent does not require touching the coding agent.
- Scale individually: when scope and interfaces are well-defined, individual agents can be scaled independently without redeploying or resizing the broader system.
- Minimized scope: a failure or hallucination of one agent does not cascade across the entire workflow.
- Cost savings: A narrowly scoped agent can often run on a smaller model, which cuts inference and hosting costs.
Complexity can be reduced along two axes:
- Within an agent: decompose capabilities into skills since tools and few-shot examples consume context on every invocation while degrading accuracy and increasing cost. Determine granularity empirically: merge skills the agent rarely selects and split skills that produce different behavior depending on the input.
- Across agents: delegate work outside an agent’s domain to a specialized peer over the A2A protocol rather than extending the current agent. An agent should grow in capability, not in context.
Critically, since each specialized agent only accesses the data and tools required for its role, this architecture helps to enforce least-privilege by design, a compliance requirement in regulated industries.
Realizing these benefits, however, requires that agents can find and interoperate with each other. To prevent the emergence of AI islands, we maintain a centralized registry of agents, tools, and skills. This registry serves as:
- Discovery mechanism: cataloging available agents, tools, and skills based on metadata stored during build.
- Governance checkpoint: confirming approval of the asset types for production use and maintaining an audit trail.
- Security checkpoint: automated security checks are applied before approval, such as prompt injection and similar issues.
- Deduplication safeguard: identifying equivalent capabilities already maintained by other teams to avoid duplicate work. This check runs automatically at registration, flagging duplicates before approval.
Scale through codification
The shift from managing individual agents to governing an agent fleet requires agent architectures to be codified. Using Infrastructure-as-code frameworks such as Terraform or AWS CloudFormation, teams provision and manage agent infrastructure with the same rigor applied to any production system:
- Fine-grained least-privilege roles: scoped per agent or tool.
- CI/CD pipelines: consistent, validated deployments across accounts and regions, with automated deployment stops on failed tests or metric deterioration.
- Policy-as-code: rules (for example, AWS Config, Open Policy Agent) scanning deployed agents for compliance with organizational standards.
All three are enforced as pipeline gates, validated before registration, not audited after the fact.
The adoption model is deliberately bottom-up with central curation. Individual teams build skills and MCP tools to solve their immediate problems. All teams can contribute by sharing their assets through the registry. A central team builds automatic governance checks as part of the CI pipeline to uphold standards. To support development teams and foster shift-left, the central team additionally provides building blocks as governance-as-code modules.
This balance avoids both centrally mandated architectures and unmanaged experimentation. As a result, when new operational concerns arise, teams can extend existing patterns rather than redesign their architectures.
Operationalizing best practices across the agent lifecycle
Architectural patterns and governance standards are only effective if they are embedded into the workflows teams use to build and operate agents before and at runtime.
Before deployment
Governance begins before a single line of orchestration logic is written. Teams are provided with reference architecture patterns and examples drawn from the registry. These pre-validated blueprints cover common agent configurations, tool integration patterns, and composition structures, which accelerate development while encoding proven defaults. Before anything reaches production, standardized CI/CD controls enforce these practices. Figure 2 shows the four phases of a reference pipeline, which run in order from left to right.

Figure 2: Reference pattern for a four-phase agent deployment pipeline.
Phase 1 from Figure 2 runs fast, deterministic checks: supply chain security (approved registries only), component and interaction testing, evaluation library references, and observability instrumentation.
Phase 2 runs security and governance checks in parallel. Static analysis scans the code for issues, and security tests probe the agent in an isolated staging environment. The pipeline also validates egress controls and least-privilege permissions. Two more checks confirm that the agent’s declared autonomy level matches its actual permissions and that its configuration passes the policy-as-code rules. A governance agent accelerates audits by checking the agent against established standards and flagging findings for human review, such as:
- Scope creep: operating beyond originally defined purpose or permissions.
- Prompt bloat: unnecessary complexity in instructions.
- Tool misuse: calling tools outside their intended use case.
- Performance degradation: declining accuracy, latency, or cost-efficiency.
Figure 3: Example governance agent workflow
This assessment surfaces concrete, actionable measures that teams can apply immediately, rather than abstract recommendations. Figure 3 shows the governance agent’s dashboard for an agent maturity assessment. The governance agent provides evidence and actionable recommendations, while accountable humans own governance decisions. Recommendations require human confirmation before being applied, maintaining a four-eyes principle for accountability. Findings update the agent’s governance status in the registry, while full details are logged separately in a central knowledge base for audit and traceability.
Phase 3 evaluates the agent as a running system: the ground-truth evaluation gate (scores outputs against benchmarks established at design time) and agent performance (latency, token consumption). Figure 4 details the metrics, scoring criteria and the relationships between them.

Figure 4: Overview of evaluation metrics
Phase 4 checks for duplicates before registering the agent, its sub-agents, tools, and skills. The following table shows the metadata fields stored per agent in the registry.
Agent metadata
Core data:
- unique identifier
- name
- description
- version
- business owner
- IT owner
- Capabilities: names, descriptions
- Functions list: name, description, direction: provides/ uses
- Status: plan / active / deprecated
- Link to specification, agent card or documentation
- Hosting mode: on-prem, cloud, cloud-region
Service level agreement data:
- Target availability in percent
- Target P95 response time in ms
- Support hours / operational support model
Data governance flags:
- information classification level
- personal data processed flag
- data retention period/ policy
- request and response payloads logged flag
- PII data redacted flag
- Human in the loop required flag (required for high-impact / irreversible actions)
Table 1: Example agent metadata stored in the registry
If any step fails or metrics deteriorate, the deployment stops with logs for root cause analysis. If all checks pass, the agent is packaged and deployed as a containerized workload, producing consistent, reproducible deployments. This prevents ungoverned agents from reaching production, automates compliance checks, and keeps the registry accurate.
At runtime
At runtime, observability has to catch two kinds of drift. Data drift shows up as new or shifting user query patterns, or changes in upstream APIs. Model drift shows as rising error rates or falling tool-selection accuracy. When a customer flags a response, that also triggers an investigation. Alerts for drift, distribution shifts, declining evaluation scores, or direct customer flags prompt a staged response. These events are logged and forwarded to human review. Additionally, teams can run the governance agent continuously to flag drift and operational issues for the responsible team to investigate.
Summary
Building and operating agents at scale is an engineering discipline, not an experimentation exercise.
Effective governance must be structural from the start, externalized, domain-sensitive, and deterministically enforced. We introduced a three-tier composition hierarchy (MCP tools, skills, and flows) as the architectural foundation for reusable, composable, and interoperable agents, decomposed into specialized agents to contain complexity and scope. We showed how verification and evaluation in every CI/CD run make quality assurance part of everyday development, and how defining agent infrastructure as code lets teams apply the same governance rules to every agent in the fleet. Finally, we operationalized these patterns across the agent lifecycle, from pre-validated reference architectures and structured pipelines to a runtime governance agent that continuously evaluates deployed agents.
Our experience has shown these practices pay off in a few ways. Reusable templates and shared infrastructure reduced development costs. Teams also got agents to production faster, thanks to standard interaction models and automated deployment. Quality improved too, because the pipeline enforces proven patterns at every stage of the lifecycle.
Scaling agentic governance is an industry-wide imperative. The shift toward AI and generative-AI-supported workflow automation is already reshaping how engineering organizations operate. The winners will be those who move beyond experimentation and establish the discipline to govern autonomous agent networks at scale. With these practices in place, BMW engineers can focus on solving the problems that matter: Building the ultimate driving machine.
To apply these ideas to your own agents, start with the AWS Well-Architected Framework. This post used it as the model for reusable agent blueprints. To see how BMW codifies governance across its AWS environment, read How BMW Group uses automation to achieve end-to-end compliance at scale on AWS.
Outlook and next steps
The patterns and practices described in this post are a foundation, not a finish line. We are actively extending them across several fronts:
- Improve architectural guidance: publishing best practice guides on architectural trade-offs, composition patterns, and anti-patterns for teams building agents.
- Expand infrastructure-as-code templates: providing agent blueprints as usable IaC modules with built-in security and governance standards by default.
- Deepen governance compliance testing: adding CI/CD test cases verifying governance compliance.
- Automate standards enforcement: expanding automated checks and managed compliance-scanning rules across pre-deployment and runtime stages.
- Extend agentic FinOps: Govern the right-sizing of models and tool calls.
- Agentic user guidance: integrating described patterns into Archimedes, BMW’s architectural guidance agent, to provide contextual architectural recommendations.
Acknowledgements
The authors would like to thank (in alphabetical order) Alexander Stark, Alexander Zimmer, Dr. Céline Laurent-Winter, Dominik Segl, Florian Seidel, Hugo Lerias, Dr. Lars Lewerenz, Mark Neumann, Ralf Bottesch, Ramona Weber, Satish Vasireddy, Samuel Voss, Xagarey-Louis Meyer.
About the authors