BrowseSafe++: a small detector beats frontier models, adaptive attacks win
BrowseSafe++ measures whether injection detectors hold up once an attacker rewrites the injection using the agent's own feedback.
· Perplexity · Research by Yuetian Chen (Perplexity and Purdue University), Kaiyuan Zhang (Perplexity and Rutgers University), Yuntao Du (Perplexity and Purdue University), Thomas Li, Mark Tenenholtz, Perplexity
BrowseSafe++ is a 38,675 pair benchmark for prompt injection hidden in web pages, from Perplexity, Purdue and Rutgers. Their 0.8B detector reaches 98.29 F1 on HTML, but rewritten injections still reach 95.41% success on Kimi K2.
01What happened
BrowseSafe++ was published on arXiv on October 12, 2026 by researchers at Perplexity, Purdue University and Rutgers University. The benchmark grows the earlier BrowseSafe set to 38,675 attack and control page pairs, covering 12 website types, 24 attack types and 18 page locations. Perplexity released the SDK on GitHub and the training split on Hugging Face.
02How it works
Each pair is one web page carrying a hidden injection plus a clean copy of the same page, served in three views: raw HTML, Markdown and the accessibility tree. BrowseSafe-Guard, the defense the paper ships, is two 0.8B detectors, one reading text and one reading screenshots. BrowseSafe-Adapt then rewrites each injection using feedback from the target agent, holding the attacker's goal fixed until the agent obeys.
03Who is affected
Any product that lets an agent read untrusted web pages is in scope. The authors tested 16 models as text detectors, 15 of them also as task-executing agents, and seven as adaptive attack targets, among them Claude Sonnet 5, GPT-5.5, Gemini 3.1 Pro, DeepSeek V3.1 and Kimi K2.
04What to do
Test an injection detector against adaptive rewrites, not only against the original injections. The authors report seed injections succeeded on 0.49 to 4.00% of attempts and their rewrites raised that to between 30.66 and 95.41%. The training split and the SDK are public, so a defender can run the same test in house.
Measured results
Prompt injection detection, F1 on HTML pages
Higher is better. BrowseSafe-Guard is a 0.8B model; the frontier detectors below it flag 6 to 21% of clean pages.
| Result | Score |
|---|---|
| BrowseSafe-Guardthis story | 98.3F1 |
| GPT-6 Astra | 91.5F1 |
| Claude Fable 5.1 (answered) | 90.4F1 |
| GPT-5.5 | 89.3F1 |
| Jev 1.13 | 89.2F1 |
| Qwen 3.7 Max | 89.0F1 |
Scale 0 to 100F1 · Source: arxiv.org, as published
Questions people ask
- Is the BrowseSafe++ dataset free to download?
- BrowseSafe++ publishes its 29,500 pair training split on Hugging Face under CC BY 4.0, and the SDK that loads it on GitHub under the MIT license. Perplexity holds back the 9,175 pair test split for a leaderboard at browsesafe.perplexity.ai, where the submission endpoint is listed as coming soon. The full screenshot collection is about 45 GB.
- Does BrowseSafe-Guard beat frontier models at detecting prompt injection?
- BrowseSafe-Guard, a 0.8B model, reaches 98.29 F1 on raw HTML with a 1.35% false positive rate, which the paper reports as higher than every frontier detector it tested. GPT-6 Astra scored 91.55 F1 and GPT-5.5 scored 89.32, but both flagged far more clean pages, 14.32% and 20.99% respectively. Perplexity reports the numbers, so treat them as the authors' own measurement.
- Why did the attack success rate reach 95.41% on Kimi K2?
- BrowseSafe-Adapt rewrites an injection using feedback from the target agent, keeping the attacker's goal fixed until the agent complies. Against Kimi K2 the authors report success rising to 95.41%, the highest of seven targets, from a seed range of 0.49 to 4.00%. Adaptation success across all targets landed between 30.66% and 95.41%.
Sources 4 links, all fetched
- 1arxiv.orgPaper on arXiv
- 2arxiv.orgPaper Full Text
- 3github.comBrowseSafe++ SDK
- 4huggingface.coDataset on Hugging Face