GuardrailAgent: a jailbreak filter that learns from past attacks
GuardrailAgent checks both the prompt and the reply, and the authors report it beat three deployed guardrails on all three target models.
· CISPA Helmholtz Center for Information Security · Research by Tianze Chang, Junjie Chu, Yixin Wu, Michael Backes, Yang Zhang
GuardrailAgent is a multi-agent jailbreak filter from CISPA that scores prompts and replies against knowledge bases of known attacks. The authors report 93.1% input-side detection on ChatGLM3-6B, against 67.6% for Llama Guard 4.
01What happened
GuardrailAgent appeared on arXiv on November 6, 2026, from Tianze Chang, Junjie Chu, Yixin Wu, Michael Backes and Yang Zhang at CISPA Helmholtz Center for Information Security. The paper is accepted to EMNLP 2026 Findings, and the authors released the detection code under the MIT license. The work targets a gap the authors describe in static guardrails, which adapt poorly to attack strategies they have not seen.
02How it works
Two agents sit on either side of the model. An InputAgent scores a prompt against a Prompt knowledge base of known jailbreak text and a Material knowledge base of documented attack strategies: a high-confidence hit from either flags the prompt, and otherwise the two scores are summed, with a total of 2 or more counted as a jailbreak. An OutputAgent then runs a refusal check and compares the reply against a Response knowledge base. A feedback module is meant to return misclassified cases so the knowledge bases keep growing.
03Who is affected
Teams choosing a jailbreak guardrail are the audience, not operators of a specific product, because GuardrailAgent is a research prototype rather than a deployed service. The evaluation uses older target models, ChatGLM3-6B, Vicuna-7B and GPT-3.5-turbo, so the numbers do not speak to current frontier models. The released repository also leaves out the code for automatic knowledge base construction and for the feedback loop, which is the part that makes the design adaptive.
04What to do
Read the comparison below before trusting a single input-side classifier, and measure any guardrail you run against the attack families the paper lists rather than against one jailbreak. Weigh GuardrailAgent's 1.6% false positive rate on benign prompts against the 0.1% the authors measured for Llama Guard 4.
Measured results
Input-side jailbreak detection rate, ChatGLM3-6B
Higher is better for defenders
| Result | Score |
|---|---|
| PromptGuard | 53.5% |
| OpenAI Moderation | 39.5% |
| Llama Guard 4 | 67.6% |
| GuardrailAgentthis story | 93.1% |
Scale 0 to 100% · Source: arxiv.org, as published
Questions people ask
- Does GuardrailAgent beat Llama Guard 4?
- GuardrailAgent scored higher than Llama Guard 4 on every detection number the paper reports: 93.1% against 67.6% input-side detection on ChatGLM3-6B, and 93.9% against 72.0% overall. Llama Guard 4 produced fewer false alarms, at 0.1% on benign prompts against GuardrailAgent's 1.6%. All figures come from the authors' own evaluation, not an independent test.
- Can I run GuardrailAgent in production today?
- GuardrailAgent ships as research code under the MIT license, needing an OpenAI API key and a locally downloaded e5-base-v2 embedding model. The authors say the code for building knowledge bases automatically and for the feedback loop is not released yet, so the adaptive part of the design cannot be reproduced. Treat the repository as a baseline to measure against.
- Which jailbreaks did the GuardrailAgent evaluation cover?
- The evaluation covers ten attacks in two groups. Model-independent ones are WildJailbreak, AIM, Base64, DevMode v2 and Combination; model-dependent ones are AutoDAN, GCG, GPTFuzz, TAP and PAIR. Each ran against ChatGLM3-6B, Vicuna-7B and GPT-3.5-turbo. The paper reports no results against newer frontier models, so coverage of current jailbreak families is untested.
Sources 3 links, all fetched
- 1arxiv.orgarXiv Paper
- 2arxiv.orgFull Text
- 3github.comGuardrailAgent Code