AutoPI: a red teamer that counts how many ways an agent breaks
AutoPI is an adaptive prompt-injection red teamer that measures how many distinct techniques break an agent, not only how often one does.
· University of Southern California · Research by Stavros Damianakis, Konstantinos Psounis
AutoPI is an adaptive prompt-injection red teamer that writes one injection per technique in the PIAtlas taxonomy and reports technique coverage alongside attack success rate. The authors evaluate it on AgentDojo and IPI Arena.
01What happened
Stavros Damianakis and Konstantinos Psounis of the University of Southern California posted AutoPI to arXiv on 18 November 2026 and released the harness and the generated attacks under the MIT license. The paper's argument is that attack success rate alone hides how many different injection techniques work against a target. The authors evaluate the tool on the AgentDojo and IPI Arena benchmarks.
02How it works
AutoPI walks PIAtlas, a taxonomy that sorts prompt-injection attacks by how the injected instruction is built to subvert the model, with 6 classes and 34 techniques. For each sub-technique, an attacker model writes one indirect injection for the scenario, an LLM judge scores the outcome, and failed attempts are rewritten from that feedback. The method is black-box and needs no training, so it drives a target through its normal interface. Results are reported as technique coverage, meaning how many classes, techniques and sub-techniques produced at least one success, next to the usual success rate.
03Who is affected
Teams that measure the prompt-injection robustness of their own agents are the audience here, rather than users of one product. On AgentDojo against Qwen3.6-35B, the authors report a successful injection in all six taxonomy classes and in more than 90% of techniques. Claude Haiku 5.5 was the most robust target they tested.
04What to do
Report technique coverage next to attack success rate when testing an agent against prompt injection, because a low success rate can still hide many working techniques. The harness ships with AgentDojo and IPI Arena bundled, so an existing evaluation setup can run it.
Sources 4 links, all fetched
- 1arxiv.orgarXiv Paper
- 2github.comAutoPI Code
- 3stevendamianakis.github.ioProject Page
- 4stevendamianakis.github.ioPIAtlas Taxonomy