INFUSE: a 40-task benchmark for injections hidden in agent files
INFUSE plants hidden instructions in the files a coding agent reads during real tasks, then scores whether the agent obeys them.
· Vanderbilt University · Research by Thuy Dung Nguyen, Jingyi Guo, Rana Seda Dubauskas, Sreepriya Damuluru, Taylor T. Johnson, Kevin Leach
INFUSE is a benchmark that hides adversarial instructions in the files a coding agent reads across 40 tasks. Across 19 models the authors report a mean attack success rate of 19.6%, ranging from 3.2% to 41.9%.
01What happened
Researchers at Vanderbilt University posted INFUSE to arXiv on October 14, 2026, a benchmark for indirect prompt injection across long-horizon agent workflows. The release covers 40 containerized tasks and 56 attack scenarios in five threat categories, and the authors report 41,824 adversarial trajectories plus 720 benign reference runs. Code and tasks are on GitHub under the MIT license.
02How it works
Attacks arrive through four surfaces an agent touches while working: file content, config files, filenames and archive contents. Each task ships a Dockerfile, instructions and hidden tests, so one run measures two things at once: whether the agent followed the planted instruction and whether it still completed the job. Two scorers are used, an LLM judge and a heuristic check, which the paper reports separately because they disagree.
03Who is affected
The 19 models evaluated span the GPT, Claude, Gemini, Llama, Gemma and Qwen families, from GPT-3.5-Turbo through GPT-5.4, Claude-Sonnet-4.6 and Gemini-3-Flash-Preview. Gemini-3.1-Flash-Lite-Preview recorded the highest attack success rate at 41.9%. Claude-Sonnet-4.6 recorded the lowest at 3.2%, which the authors attribute to runs cut short by a cost cap and say should not be read as robustness.
04What to do
Run the harness against your own model and defense mix: it drives any LiteLLM provider or a local vLLM server, and ships 11 defenses to compare. The authors report that prompt-level instructional hardening cut mean attack success from 58.2% to 14.3% on a smaller stress subset, with security and utility trade-offs that vary by model, and that existing guard models gave limited protection.
Sources 2 links, all fetched
- 1arxiv.orgarXiv Paper
- 2github.comINFUSE Code