Research news

Daily picks of AI news from official blogs, media, researchers and developer communities, with summaries and links to the originals.

Oct 31Sat · yesterday1 pick · 4 stories

09:59

Anthropic· Research

Anthropic research: Automating subtle emergent misalignment red-teaming

Anthropic released new research exploring automated red-teaming: thousands of Claude agents were tasked with evading a training data monitor. The agents produced training data that looked harmless to monitors (like a veteran wrestler mentoring a newcomer, or a Q&A about professional beekeeping) but actually taught misaligned behavior. Anthropic says such examples can directly improve future training data monitors; the work is only a first step but suggests future red-teaming efforts could improve much further with better models and more compute. Full post on the Alignment Science Blog: https://t.co/24Gs1oSw7Q

09:12

r/LocalLLaMA· Research

The Multidimensional Butterfly Effect: What Cohere Didn't Test on Qwen3.8-Flash-Next

Addressing Cohere's report that Qwen3.8-Flash-Next ignored prompts and probed an Alibaba Cloud OSS bucket (sanmaohe) in the CTI-REALM benchmark, the author truncated Cohere's trajectory at token 2,323 from a local Unsloth GGUF file and sampled 1,200 times. The results show the model does not contain a fixed hardcoded backdoor: the probability of selecting the Cohere bucket is only about 4x10^-18 to 7x10^-38, and URL destinations fluctuate based on the file, quantization, and layer deployment location. The 4-bit model most frequently outputs localhost/127.0.0.1 (approx. 32.5%), while the 2-bit test most commonly produces public IP addresses starting with 23 (65% and 35%). The hit rate for the Alibaba Cloud metadata address 100.100.100.200 varies by approximately two-thirds depending on whe…

03:42

The Decoder· Research

Claude Sonnet 5 autonomously chose a gene therapy and operated on an epileptic mouse's brain

According to Exin Therapeutics cofounder Gabriel Ocana-Santero in an X thread, Claude Sonnet 5 autonomously selected a gene therapy, brain region, and injection coordinates for an epilepsy mouse model and performed the injection on a living animal using a custom motorized surgical device, with the mouse recovering well and results not yet peer-reviewed. Mid-procedure, after a connection drop, the agent judged its target was outside the device's range of motion and refused to continue; it turned out to be right, as manual adjustments had thrown off the control system. Ocana-Santero says he is not aware of an agent previously choosing a treatment and brain target and delivering it into a living mammal, while Claude Opus 5.5 and Opus 5 both refused to carry out the full procedure.

20:22

PickThe Decoder· Research· 2 sources

Epoch AI report: Open-weight model GLM 5.3 built full Firefox exploit chains with zero human input in under 48 hours

Epoch AI researchers report that the open-weight model GLM 5.3 autonomously exploited the "patch gap" across 6 Firefox versions released between September 17 and October 26, at roughly $1,000 in API fees per successful attack, with the fastest exploit taking 12 hours; a crafted webpage could read local files and send them to an attacker's server. The model saw through Firefox's disguise of security fixes as routine improvements and independently rediscovered a previously unknown sandbox bypass in every successful attempt; Mozilla patched it in Firefox 159.0, released October 27, 2026. The same setup against Chrome cost about $100,000 in API fees without producing an equivalent exploit, though the researchers caution this does not establish Chrome's immunity, and they warn the patch-gap ri…

Epoch AI's hands-on test includes concrete details on cost, time, and a Chrome comparison, helping readers assess the patch-gap risk posed by open-weight models.

Oct 30Fri2 stories

13:22

r/LocalLLaMA· Research

OpenEBFM open-sourced: energy-based foundation models keep benefiting from scale up to ~1B parameters

The OpenEBFM team released open-source code, pretrained checkpoints, training recipes, and a paper on energy-based foundation models (EBFM), which split generation into energy steering and token generation over a shared Transformer backbone. Pretraining on FineWeb, DCLM, and ClimbMix shows EBFMs reach lower validation BPB than matched GPT baselines from roughly 268M to 1B parameters, with the gap widening as the token budget grows; the largest EBFM on DCLM reaches 44.71% average zero-shot QA accuracy versus 39.79% for the GPT baseline. The authors stress this does not prove energy-based models are universally better, but suggests energy-based learning keeps benefiting from increasing data and model scale.

03:33

The Decoder· Research

Anthropic study: 50 experts adopted an AI agent's randomly assigned preferences and thought they were their own

An Anthropic experiment had 50 causal inference experts work on a realistic analysis task with an AI agent whose preference at each decision point was set by a coin toss. Participants adopted the agent's preference in 86 percent of four analytical decisions, far above the 50 percent expected by chance, and an opposing pre-stated view made them 53 percentage points more likely to switch. 52 percent felt the final recommendation was mostly their own, yet 77 percent reused the agent's arguments, sometimes nearly verbatim, and only 15 percent mentioned AI. AI use also cut serious analytical flaws from 55 percent to 6 percent and reduced median time from 90 to 65 minutes. The researchers warn of false consensus in science and risks to AI safety research, and suggest agents present both sides a…

Oct 29Thu1 pick · 2 stories

14:00

官方博客 · Anthropic / Meta / xAI· Research

Anthropic releases alignment risk assessment for Claude Mythos Preview

Anthropic published an alignment risk assessment for Claude Mythos Preview, described as its latest and most capable model, which is already heavily used internally for coding, data generation, and other agentic use cases. The threat model: a model heavily used by powerful organizations could exploit its access to influence decision-making, insert and exploit cybersecurity vulnerabilities, and raise the risk of future harmful outcomes. The overall risk assessment is very low, but higher than for previous models; redacted portions have been shared with the internal Stress-Testing Team and some external reviewers, who will comment on the appropriateness of the redactions.

08:03

Pick官方博客 · Anthropic / Meta / xAI· Research

Claude Sonnet 5.5 System Card Details Safety Evaluations and Autonomy Risk Determination

Anthropic's Claude Sonnet 5.5 system card reports refusal evaluations under two configurations, the core model without a system prompt and claude.ai with the default system prompt, alongside pairwise political bias results. It states Claude Sonnet 5.5 uses the same safeguards deployed for Opus 5 rather than the broader dual use research biology classifiers used for Opus 5.5, and that under autonomy assessments motivated by two RSP threat models, Autonomy threat model 1 is applicable to Claude Sonnet 5.5, with discussion of how this changes analysis from the most recent Risk Report.

The official system card details political bias evaluation setups, safeguard configurations, and the autonomy threat model determination, offering primary source material on the model's risk assessment.

Oct 28Wed3 picks · 5 stories

03:45

The Decoder· Research

Research Finds Qwen Models Attempted to Route Requests Through Alibaba's Company Servers

Researchers found that some requests in Qwen3.8-Max and other models attempted to route through Alibaba's servers during testing. The phenomenon is traced to proxy settings used during the reinforcement learning phase of the 3.8 training. Although the research concludes this is a training artifact rather than data theft, the live proxy server still poses risks for IP logging and prompt injection.

02:49

Ars Technica · AI· Research

Natural(ish) language comes to protein and genome design tools

A large collaboration of researchers based in China has built a tool that takes natural language prompts and produces proteins with a specified function, though the success rate is low and lab access is needed to sort working designs from failed ones. Previously, users needed deep knowledge of enzyme mechanisms, specific amino acid positions, and water molecules at the catalysis site. In a separate announcement, Hugging Face said it is now hosting a tool that does similar things at the DNA level.