Agent Swarms: DeepMind automates adaptive prompt injection
An automated swarm of attacker agents found working prompt injections against models that public benchmarks rate as robust.
· Google DeepMind · Research by Alexey Kurakin, Vishal Karande, Alex Bie, Chongyang Shi, Shuang Song, Google DeepMind
Google DeepMind reports that an automated attacker built from swarms of LLM agents drove attack resistance to 10% or less on six of seven models tested, including models that score as robust on public benchmarks.
01What happened
Google DeepMind published Scaling Adaptive Prompt Injection Attacks with Agent Swarms on arXiv on November 9, 2026. The team, with Alexey Kurakin as corresponding author, built an attacker that runs many LLM agents in parallel to search for prompt injections, then measured how seven defender models held up. Six of the seven resisted 10% or fewer of the attack scenarios at the full query budget.
02How it works
The attacker treats prompt injection as a search problem, spending a query budget probing the defender and adapting each attempt from what the previous one returned. Attack goals are random non-malicious actions rather than harmful ones, which separates how well a model holds its instruction hierarchy from how well it recognizes danger. The paper measures Attack Resistance Rate, the share of scenarios where the attack failed, so a lower number means a stronger attack.
03Who is affected
Seven models served as defenders with thinking on and chain of thought hidden: four Gemini versions plus Kimi K3, GLM-5.3 and DeepSeek-V4.1-Flash. Gemini 4 Argon held up best and was also the strongest attacker in the study. Per-model scores are in the table below.
04What to do
Do not rely on model hardening alone, which the authors argue should be part of a full stack with defense in depth. The authors also say frontier defenses may need proactive, automated red-teaming to stay resilient against more capable adversaries.
Measured results
Attack resistance rate against the strongest attacker, full budget
Lower is worse for defenders. Attacker was Gemini 4 Argon, defender thinking on and chain of thought hidden.
| Result | Score |
|---|---|
| Gemini 3.6 Flash | 1% |
| Gemini 3.7 Flash | 6% |
| Gemini 3.8 Flash | 10% |
| Gemini 4 Argon | 66% |
| Kimi K3 | 1% |
| GLM-5.3 | 1% |
| DeepSeek-V4.1-Flash | 1% |
Scale 0 to 100% · Source: arxiv.org, as published
Sources 2 links, all fetched
- 1arxiv.orgarXiv Paper
- 2arxiv.orgFull Text