AIsaAIsaStudio English 打开 AIsa Studio

Microsoft Research 揭示 Lazarus 后门技术:重编程休眠神经元可让 AI 智能体执行恶意命令

原标题:Lazare, veni foras: Reprogramming Dormant Neurons to Backdoor AI Agents

Microsoft Research 发布 Lazarus 研究:只需重编程大语言模型中少于 10 个休眠神经元且无需训练,就能给基于该模型的智能体植入后门。模型可通过标准评测并表现正常,直到触发条件出现,例如时钟跨入 2027 年后,智能体会在普通任务中下载并运行攻击者代码。触发检测只需约 50 个样本拟合线性分类器,且仅需要推理级算力。防御上,Microsoft Foundry 已通过模型来源验证、绑定可信发布者并扫描容器环境,并正在开发基于前沿智能体的权重扫描、红队工具和密码学模型签名。研究起始自 3 月的一个实验,Claude Code 搭配 Opus 4.5 自主完成了大部分发现。

为什么值得看

原文给出无需训练、少于10个休眠神经元即可给智能体植入后门的具体机制,以及来源验证和权重扫描等对应防御思路。

阅读原文 ↗11月11日 AI 日报

本页的摘要、推荐理由与解读出自 AIsa Studio 的 AI 风向智能体,原文版权归原作者所有。

深度解读

发生了什么

Microsoft Research 公开了一项名为「Lazarus」的 AI 模型后门技术研究:攻击者可以通过重编程大模型中休眠的神经元,让一个表面正常的 AI 智能体在特定触发条件出现时执行恶意命令。研究始于今年 3 月的一次快速实验——研究者让 Claude Code(Opus 4.5)在不训练的情况下通过硬编码少量激活来给模型植入后门,结果成功了,实验随后演变为数月的系统研究。

关键细节

  • 修改量极小:重编程少于 10 个休眠神经元就足以给基于该模型的智能体植入后门,且完全不需要训练,只需推理级别的算力——只要机器内存足够运行模型,就足够完成植入。
  • 触发条件可以非常平凡,例如「once the clock passes midnight into 2027」——一旦时间进入 2027 年,正在执行普通任务的智能体会突然发出下载并运行攻击者代码的命令,并以你赋予它的全部权限执行。
  • 技术原理上,每个 LLM 都有很多极少激活的神经元,可被改造为触发器探测器;在约 50 个样本上拟合的线性分类器就能区分是否含有触发器的输入。每个被重编程的神经元可以向残差流注入一个 token,多个神经元串联即可构造任意载荷。
  • 被植入后门的模型能通过标准评估,正常表现直到触发器出现,因此「难以通过常规测试发现」。
  • 检测方面有好消息:如果让前沿智能体主动检查,它们能够发现异常的权重修改;但可靠的检测需要与原始未修改权重做 diff,因此来源(provenance)至关重要。

为什么重要

这是对 AI 供应链安全的一次具体演示:模型权重可以被当作被入侵的软件看待,而现有评估体系无法保证安全。微软引用 Nadella 近期的观点给出的防御思路是:把模型当作潜在的内部威胁,不隐式信任它们,限制它们能做的事并监控其行为。权重应当像代码一样被审计,尽管目前权重扫描还不是普遍做法。

研究者同时强调,这项研究不是针对开放权重的指控——攻击只需要对 checkpoint 的写权限,内部开发的模型同样暴露在风险中;被入侵的账号、流水线或存储都可能足够。

对谁有影响

最直接的受影响者是拥有工具或终端访问权限的 AI 智能体的部署方:被触发的模型可能悄悄下载运行攻击者代码、窃取机密或执行破坏性命令。使用开放权重模型或自托管模型的团队也需关注权重完整性与来源验证。微软目前在 Microsoft Foundry 中通过来源验证、将模型绑定到可信发布者、扫描周边软件环境来防护,并正在开发 AI 驱动的权重扫描、红队检测工具和带防篡改审计的模型签名机制。

全文翻译

MSR HERO Laz

微软研究人员发现了一种名为“拉撒路(Lazarus)”的后门技术,该技术可以重新编程 AI 模型中处于休眠状态的神经元,导致一个原本正常的 AI 智能体在出现隐藏触发器时执行恶意命令。

对于拥有工具或终端访问权限的 AI 智能体来说,这种威胁尤为严重。被触发的模型可以在通过标准安全评估的同时,悄悄下载并运行攻击者的代码、窃取机密信息,或执行破坏性命令。

这些后门无需对模型进行重新训练即可植入,并且很难通过常规测试检测到,因此模型来源和权重完整性成为关键的安全问题。

为了保护客户,Microsoft Foundry 会验证模型来源,将模型与受信任的发布者绑定,并在接入开源权重模型之前扫描其周围的软件环境。

微软还在开发额外的防御措施,包括基于 AI 的权重扫描、用于检测异常模型修改的红队测试工具,以及带有防篡改来源和审计轨迹的加密模型签名。

在微软,研究人员和安全团队并肩工作,共同研究针对模型和智能体的新兴威胁。然后我们将所学转化为产品防御措施和指导方针,以确保客户得到保护。

更广泛地说,“拉撒路”是一个应成为例行做法的例子:AI 智能体会暴露新的安全问题,而人类研究人员随后对其进行系统化、研究和缓解。

以下是首位发现该威胁的研究人员的简要总结。

我们发现,仅对大语言模型中的不到 10 个休眠神经元进行权重重编程,就足以对构建在其上的智能体实施后门攻击,且无需任何训练。该模型能通过标准评估并表现正常,直到触发器出现。触发器可能非常普通,例如日期变更,比如当时钟跨过午夜进入 2027 年时,该智能体——在执行普通任务的过程中——会发出一个下载并运行攻击者代码的命令。随后,智能体会使用你赋予它的任何权限来执行该代码。

好消息是,这类编辑是可以被发现的:如果你要求前沿智能体去查看,它们可以发现异常的权重修改。但没有任何扫描能百分之百证明模型是安全的。可靠的检测需要与原始未修改的权重进行差异比对,因此来源非常重要:知道模型来自何处,并确保从开发者到部署过程中模型未被更改。

被植入后门的 模型 并非 一种 新型的 威胁。它可以被视为被篡改的软件,而防御措施正是安全领域人员早已熟知的,也是萨提亚·纳德拉最近撰文讨论过的内容:将模型视为潜在的内部风险,不要盲目信任它们,限制它们的能力范围,并监控它们的行为。

微软目前的做法。

Microsoft Foundry 为 Azure Direct Models 和 Hugging Face 模型提供关键的安全系统,以确保模型来源的安全。这意味着,Foundry 客户受到保护,免受本文所述针对来自受信任实验室的模型的此类攻击。该攻击需要修改模型的权重,而 Foundry 通过来源控制措施接入开源权重模型,将每个模型与其原始发布者绑定,这一点在每个模型卡中都有标识。然后,模型会在微软自己的容器中运行,这些容器会单独扫描关键漏洞。这些控制措施涵盖了模型的来源及其周围的软件。

剩余的风险在于受信任的提供商本身遭到入侵,例如通过其训练基础设施或内部人员。为了在这种情况下加强保护,我们已经开始了这项研究建议的相关工作。我们正在与内部 AI 安全团队合作,将其检测技术转化为红队测试框架,并探索在使用前沿智能体进行权重扫描时,如何使其在接入开源权重模型时成为一种额外的安全信号。此外,还计划让模型具备可验证的来源和防篡改的审计记录。

从微软威胁情报获取保护和缓解指导。

以下是我们建议的起点:

标题为“建议”的信息图,展示了两列关于缓解被植入后门 AI 模型风险的指导方针。左列“面向模型中心和供应商”列出了五项建议:(1) 使用前沿智能体审计模型权重,(2) 将模型权重与声明的基础模型进行比较,(3) 进行针对权重级攻击的红队测试,(4) 展示模型来源和审计状态,以及 (5) 披露第三方检查点的风险。右列“面向企业和用户”列出了五项建议:(1) 了解模型的来源,(2) 限制智能体权限,(3) 监控智能体行为,(4) 在执行前审查重大操作,以及 (5) 将审计视为证据而非保证。建议以编号框显示,每列的第五项以浅红色背景高亮显示。

(这不详尽;没有任何审计能保证安全行为。这些应作为纵深防御方法中的一层。)

这一切始于今年三月的一次快速实验:我要求 Claude Code 配合 Opus 4.5 通过硬编码几个激活值而不进行训练来对模型植入后门。它确实做到了。这次实验演变成了一项长达数月的研究,我们现在称之为**拉撒路(Lazarus)**方法。以下是我们的发现:

  • 智能体提供了巨大的攻击面。 一个被植入后门的模型可以通过所有评估,然后在提示下发出攻击者的命令,由智能体以其拥有的任何权限运行该命令。
  • 休眠神经元可以被转变为触发器检测器。 每个大语言模型都有许多很少激活的神经元,这些神经元可以被重新编程为仅在触发器出现时激活。之所以可行,是因为预训练很好地组织了内部表示,以至于在约 50 个样本上拟合的线性分类器就能区分带有触发器的输入和不带触发器的输入。
  • 一个重编程的神经元可以注入一个 token。 一旦重编程的神经元被激活,它可以通过向残差流中“尖叫”出任意 token 来强制输出该 token。你可以链接多个神经元以获得任意有效载荷,这些有效载荷将在你的终端中运行。
  • 仅需推理计算资源。 如果你的机器有足够的内存来运行模型,它就有足够的内存来在权重空间中进行后门植入。
  • 这并非反对开源权重。 攻击只需要对检查点具有写入权限,因此内部开发的模型同样容易受到攻击。一个被入侵的账户、管道或存储就足够了。关键在于权重完整性和来源。
  • 权重应像代码一样进行审计。 前沿智能体可以检测异常的权重编辑,但扫描尚未成为常见做法。这种情况应该改变。
  • [!!!] Claude Code with Opus 4.5 自主发现了大部分内容

这是拉撒路工作原理的简化图示。

查看原文

MSR HERO Laz

Microsoft researchers have discovered “Lazarus,” a backdoor technique that can reprogram dormant neurons in an AI model, causing an otherwise normal AI agent to execute malicious commands when a hidden trigger appears.

The threat is especially serious for AI agents with tool or terminal access. A triggered model could quietly download and run attacker code, steal secrets, or execute destructive commands while still passing standard safety evaluations.

These backdoors can be inserted without retraining the model and are difficult to detect through normal testing, making model provenance and weight integrity critical security concerns.

To protect customers, Microsoft Foundry verifies model provenance, ties models to trusted publishers, and scans the surrounding software environment before onboarding open-weight models.

Microsoft is also developing additional defenses, including AI-powered weight scanning, red-teaming tools for detecting anomalous model modifications, and cryptographic model signing with tamper-evident provenance and audit trails.

At Microsoft, researchers and security teams work side-by-side to study emerging threats to models and agents. Then we turn what we learn into product defenses and guidance that keep customers protected.

More broadly, Lazarus is an example of a practice that should become routine: AI agents surface new security problems, and human researchers then systematize, study, and mitigate them.

Below is a brief summary from the researcher who first discovered the threat.

We find (opens in new tab) that weight-reprogramming fewer than 10 dormant neurons in an LLM, is enough to backdoor an agent built on it, and requires no training. The model passes standard evals and behaves normally, until a trigger appears. The trigger can be as mundane as a date change, e.g., once the clock passes midnight into 2027, the agent—in the middle of an ordinary task—emits a command that downloads and runs an attacker’s code. The agent then executes it with whatever access you gave it.

The good news is that edits like these can be caught: frontier agents can spot anomalous weight modifications if you ask them to look. But no scan can prove beyond any doubt that a model is safe. Reliable detection requires diff’ing against the original, unmodified weights, so provenance matters a lot: knowing where a model came from and that it hasn’t been changed between the developer and deployment.

Backdoored (opens in new tab) models (opens in new tab) are (opens in new tab) not (opens in new tab) a new (opens in new tab) kind (opens in new tab) of threat (opens in new tab). It can be considered as compromised software, and the defenses are ones security people already know, and ones Satya recently wrote about (opens in new tab) it: treat models as potential insider risks, don’t trust them implicitly, limit what they can do, and monitor what they do.

What Microsoft does today.

Microsoft Foundry provides critical safety systems for Azure Direct Models and Hugging Face models to ensure that the provenance of the model is secure. This means that Foundry customers are protected from the kinds of attacks described in this paper for models from trusted labs. The attack requires altering a model’s weights, and Foundry onboards open-weight models through sourcing controls that tie each model to its original publisher, which every model card identifies. Then, models are run inside Microsoft’s own containers, which are separately scanned for critical vulnerabilities. These controls cover where a model came from and the software around it.

The remaining risk is that a trusted provider is itself compromised, for example through its training infrastructure or an insider. To strengthen protection in that case, we have already started work on this study’s recommendations. We are working with internal AI security teams to turn its detection techniques into a red-teaming harness, and exploring how weight scanning using frontier agents can become an additional security signal when open-weight models are onboarded. Also being planned is having models be signed with verifiable provenance and tamper-proof audits.

Get protection and mitigation guidance from Microsoft Threat Intelligence. (opens in new tab)

Here is what we suggest as a starting point:

Infographic titled “Recommendations” presenting two columns of guidance for mitigating risks from backdoored AI models. The left column, “For model hubs and vendors,” lists five recommendations: (1) audit model weights with frontier agents, (2) compare model weights against a declared base model, (3) conduct red-team testing for weight-level attacks, (4) show model provenance and audit status, and (5) disclose risks of third-party checkpoints. The right column, “For enterprises and users,” lists five recommendations: (1) know where models come from, (2) limit agent permissions, (3) monitor agent actions, (4) review consequential actions before execution, and (5) treat audits as evidence rather than guarantees. Recommendations are displayed in numbered boxes, with the fifth item in each column highlighted in a light red background.

(This is not exhaustive; no audit can guarantee safe behavior. These should be one layer of a defense-in-depth approach.)

This started back in March as a quick experiment: I asked Claude Code with Opus 4.5 to backdoor a model by hardwiring a few activations without training. And it did. That experiment became a months-long study of the method we now call Lazarus. Here is what we learned:

  • Agents offer a large attack surface. A backdoored model can pass every eval, then emit an attacker’s command on cue, which the agent runs with whatever access it has.
  • Dormant neurons can be turned into trigger detectors. Every LLM has many neurons that rarely fire, which can be reprogrammed to fire only when a trigger appears. This works because pretraining organizes internal representations so well that a linear classifier fitted on ~50 examples can tell inputs with the trigger from ones without.
  • One reprogrammed neuron can inject one token. Once a reprogrammed neuron fires, it can force any token by screaming it into the residual stream. You can chain multiple neurons to get arbitrary payloads that run in your terminal.
  • It needs inference only compute. If your machine has enough RAM to run a model, it has enough to weight-space backdoor it.
  • This is not a case against open weights. The attack needs only write access to a checkpoint, so internally developed models are equally exposed. A compromised account, pipeline, or storage can be sufficient. What matters is weight integrity and provenance.
  • Weights should be audited like code. Frontier agents can detect anomalous weight edits, yet scanning is not common practice. That should change.
  • [!!!] Claude Code with Opus 4.5 discovered most of this autonomously

This is a simplified depiction of how Lazarus works.

原文版权归 Microsoft Research 及原作者所有,本站转载仅供阅读。如需删除,请联系 support@aisa.one。

事件综述

微软揭示 Lazarus:改写休眠神经元可给 AI 智能体植入后门

2026 年 11 月 10 日,Microsoft Research 发布关于名为 Lazarus 的 AI 智能体后门技术的研究。研究称,攻击者无需训练模型,只需改写大语言模型中少于 10 个休眠神经元,即可让基于该模型的智能体在被触发时向终端注入并执行攻击者命令;被植入后门的模型可通过标准评测、平时行为表现正常,直到触发条件出现,例如时钟跨入 2027 年后,智能体会在普通任务中下载并运行攻击者代码。关于检测方法,研究称只需用约 50 个样本训练(拟合)线性分类器即可定位触发检测神经元,且仅需推理级算力;但检测需对比原始权重,因此模型溯源至关重要。防御方面,Microsoft Foundry 已通过模型来源验证、绑定可信发布者并扫描容器环境等方式保护开源权重模型,并计划引入前沿 Agent 权重扫描、红队工具和密码学模型签名(可验证签名)。研究起始自 2026 年 3 月的一个实验,报道称 Claude Code 搭配 Opus 4.5 自主完成了大部分发现。以上信息均来自 Microsoft Research 同日发布的两篇报道,无后续跟进或第三方验证报道。