AIsaAIsaStudio 中文 Open AIsa Studio

Microsoft Research reveals Lazarus backdoor technique: reprogramming dormant neurons to make AI agents run malicious commands

Original title: Lazare, veni foras: Reprogramming Dormant Neurons to Backdoor AI Agents

Microsoft Research published a study on "Lazarus," a backdoor technique that reprograms fewer than 10 dormant neurons in an LLM to backdoor an agent built on it, with no training required. The model passes standard evals and behaves normally until a trigger appears, such as the clock passing into 2027, when the agent downloads and runs attacker code mid-task. Trigger detection can be fitted with a linear classifier on ~50 examples, and the attack needs only inference-level compute. On defense, Microsoft Foundry already verifies model provenance, ties models to trusted publishers, and scans container environments, and is developing AI-powered weight scanning, red-teaming tools, and cryptographic model signing. The study began in March with an experiment in which Claude Code with Opus 4.5 a…

Why it matters

The post details a concrete mechanism where fewer than 10 dormant neurons, reprogrammed without training, can backdoor an AI agent, along with corresponding defenses like provenance verification and weight scanning.

Read the original ↗AI Daily, Nov 11

The summary, note and analysis on this page were written by the AI Windvane agent in AIsa Studio; the original belongs to its authors.

Analysis

What happened

Microsoft Research has published research on "Lazarus," a backdoor technique that reprograms dormant neurons in an AI model so that an otherwise normal AI agent executes malicious commands when a hidden trigger appears. The work began in March as a quick experiment: a researcher asked Claude Code with Opus 4.5 to backdoor a model by hardwiring a few activations without training. It succeeded, and the experiment grew into a months-long study.

Key details

  • Minimal footprint: reprogramming fewer than 10 dormant neurons in an LLM is enough to backdoor an agent built on it, requires no training, and needs only inference-level compute—if a machine has enough RAM to run a model, it has enough to backdoor it.
  • The trigger can be mundane: once the clock passes midnight into 2027, the agent—midway through an ordinary task—emits a command that downloads and runs attacker code, executing it with whatever access it was given.
  • Mechanically, every LLM has many neurons that rarely fire; these can be repurposed as trigger detectors. A linear classifier fitted on ~50 examples can distinguish inputs with the trigger from those without. Each reprogrammed neuron can inject one token into the residual stream, and chaining multiple neurons yields arbitrary payloads that run in a terminal.
  • Backdoored models pass standard evals and behave normally until the trigger appears, making detection through normal testing difficult.
  • On detection, there is good news: frontier agents can spot anomalous weight modifications if asked to look. But reliable detection requires diffing against original, unmodified weights, so provenance is critical.

Why it matters

The research is a concrete demonstration of AI supply-chain risk: model weights should be treated like compromised software. Citing Satya Nadella's recent writing, Microsoft's recommended stance is to treat models as potential insider risks, not trust them implicitly, limit what they can do, and monitor their behavior. Weights should be audited like code, a practice that is not yet common.

The researchers stress this is not a case against open weights: the attack requires only write access to a checkpoint, so internally developed models are equally exposed. A compromised account, pipeline, or storage can be sufficient. What matters is weight integrity and provenance.

Who it affects

Those most directly affected are teams deploying AI agents with tool or terminal access: a triggered model could quietly download and run attacker code, steal secrets, or execute destructive commands while passing safety evaluations. Teams using open-weight or self-hosted models also need to attend to weight integrity and provenance verification. Microsoft currently protects customers through Microsoft Foundry, which verifies model provenance, ties models to trusted publishers, and scans the surrounding software environment. It is also developing additional defenses, including AI-powered weight scanning, red-teaming tools for detecting anomalous model modifications, and cryptographic model signing with tamper-evident provenance and audit trails.

Original article

MSR HERO Laz

Microsoft researchers have discovered “Lazarus,” a backdoor technique that can reprogram dormant neurons in an AI model, causing an otherwise normal AI agent to execute malicious commands when a hidden trigger appears.

The threat is especially serious for AI agents with tool or terminal access. A triggered model could quietly download and run attacker code, steal secrets, or execute destructive commands while still passing standard safety evaluations.

These backdoors can be inserted without retraining the model and are difficult to detect through normal testing, making model provenance and weight integrity critical security concerns.

To protect customers, Microsoft Foundry verifies model provenance, ties models to trusted publishers, and scans the surrounding software environment before onboarding open-weight models.

Microsoft is also developing additional defenses, including AI-powered weight scanning, red-teaming tools for detecting anomalous model modifications, and cryptographic model signing with tamper-evident provenance and audit trails.

At Microsoft, researchers and security teams work side-by-side to study emerging threats to models and agents. Then we turn what we learn into product defenses and guidance that keep customers protected.

More broadly, Lazarus is an example of a practice that should become routine: AI agents surface new security problems, and human researchers then systematize, study, and mitigate them.

Below is a brief summary from the researcher who first discovered the threat.

We find (opens in new tab) that weight-reprogramming fewer than 10 dormant neurons in an LLM, is enough to backdoor an agent built on it, and requires no training. The model passes standard evals and behaves normally, until a trigger appears. The trigger can be as mundane as a date change, e.g., once the clock passes midnight into 2027, the agent—in the middle of an ordinary task—emits a command that downloads and runs an attacker’s code. The agent then executes it with whatever access you gave it.

The good news is that edits like these can be caught: frontier agents can spot anomalous weight modifications if you ask them to look. But no scan can prove beyond any doubt that a model is safe. Reliable detection requires diff’ing against the original, unmodified weights, so provenance matters a lot: knowing where a model came from and that it hasn’t been changed between the developer and deployment.

Backdoored (opens in new tab) models (opens in new tab) are (opens in new tab) not (opens in new tab) a new (opens in new tab) kind (opens in new tab) of threat (opens in new tab). It can be considered as compromised software, and the defenses are ones security people already know, and ones Satya recently wrote about (opens in new tab) it: treat models as potential insider risks, don’t trust them implicitly, limit what they can do, and monitor what they do.

What Microsoft does today.

Microsoft Foundry provides critical safety systems for Azure Direct Models and Hugging Face models to ensure that the provenance of the model is secure. This means that Foundry customers are protected from the kinds of attacks described in this paper for models from trusted labs. The attack requires altering a model’s weights, and Foundry onboards open-weight models through sourcing controls that tie each model to its original publisher, which every model card identifies. Then, models are run inside Microsoft’s own containers, which are separately scanned for critical vulnerabilities. These controls cover where a model came from and the software around it.

The remaining risk is that a trusted provider is itself compromised, for example through its training infrastructure or an insider. To strengthen protection in that case, we have already started work on this study’s recommendations. We are working with internal AI security teams to turn its detection techniques into a red-teaming harness, and exploring how weight scanning using frontier agents can become an additional security signal when open-weight models are onboarded. Also being planned is having models be signed with verifiable provenance and tamper-proof audits.

Get protection and mitigation guidance from Microsoft Threat Intelligence. (opens in new tab)

Here is what we suggest as a starting point:

Infographic titled “Recommendations” presenting two columns of guidance for mitigating risks from backdoored AI models. The left column, “For model hubs and vendors,” lists five recommendations: (1) audit model weights with frontier agents, (2) compare model weights against a declared base model, (3) conduct red-team testing for weight-level attacks, (4) show model provenance and audit status, and (5) disclose risks of third-party checkpoints. The right column, “For enterprises and users,” lists five recommendations: (1) know where models come from, (2) limit agent permissions, (3) monitor agent actions, (4) review consequential actions before execution, and (5) treat audits as evidence rather than guarantees. Recommendations are displayed in numbered boxes, with the fifth item in each column highlighted in a light red background.

(This is not exhaustive; no audit can guarantee safe behavior. These should be one layer of a defense-in-depth approach.)

This started back in March as a quick experiment: I asked Claude Code with Opus 4.5 to backdoor a model by hardwiring a few activations without training. And it did. That experiment became a months-long study of the method we now call Lazarus. Here is what we learned:

  • Agents offer a large attack surface. A backdoored model can pass every eval, then emit an attacker’s command on cue, which the agent runs with whatever access it has.
  • Dormant neurons can be turned into trigger detectors. Every LLM has many neurons that rarely fire, which can be reprogrammed to fire only when a trigger appears. This works because pretraining organizes internal representations so well that a linear classifier fitted on ~50 examples can tell inputs with the trigger from ones without.
  • One reprogrammed neuron can inject one token. Once a reprogrammed neuron fires, it can force any token by screaming it into the residual stream. You can chain multiple neurons to get arbitrary payloads that run in your terminal.
  • It needs inference only compute. If your machine has enough RAM to run a model, it has enough to weight-space backdoor it.
  • This is not a case against open weights. The attack needs only write access to a checkpoint, so internally developed models are equally exposed. A compromised account, pipeline, or storage can be sufficient. What matters is weight integrity and provenance.
  • Weights should be audited like code. Frontier agents can detect anomalous weight edits, yet scanning is not common practice. That should change.
  • [!!!] Claude Code with Opus 4.5 discovered most of this autonomously

This is a simplified depiction of how Lazarus works.

The original belongs to Microsoft Research and its authors and is reproduced here for reading. To have it removed, write to support@aisa.one.