Fuzzy robustness: OpenAI trains GPT-6 on its own automated jailbreaks
Fuzzy robustness is OpenAI's name for how reliably a model keeps refusing disallowed help when an attacker keeps rewriting the request.
· OpenAI · Research by Kaiwen Wang, Oliver Cheng, Zheng Wen, Mehmet Yatbaz, Saachi Jain, Chuan Guo, Alec Helyar
OpenAI reports that an automated red teaming loop produced adversarial training data that raised jailbreak robustness in GPT-6 Astra and Sol across cyber, biology and violence, without making them refuse more benign requests.
01What happened
OpenAI's alignment research blog published Improving Fuzzy Robustness in LLM Safety with Automated Red Teaming on November 12, 2026, written by Kaiwen Wang, Oliver Cheng, Zheng Wen, Mehmet Yatbaz, Saachi Jain, Chuan Guo and Alec Helyar. The post describes a red teaming harness that generates adversarial training data, and reports what happened when OpenAI trained on it. A full paper is linked from the post.
02How it works
An attacker model receives a seed prompt and refines it across attempts until the defender model gives unsafe help, a design the authors compare to PAIR. A worst@n criterion keeps only the attacks that succeed on repeated tries, so the surviving prompts are reliable training examples rather than single lucky hits. OpenAI then uses that robustified prompt set for reinforcement learning. The attacker was a helpful-only reasoning model from the GPT-5.6 Sol family.
03Who is affected
GPT-6 Astra and Sol are the frontier models OpenAI says the approach was applied to, while the controlled study used GPT-5.4 and GPT-5.5 as defenders. Seed prompts cover biology, cyber and violent activity, domains the authors call dual-use because the same information can help or harm. OpenAI states that the published figures measure model robustness without production safeguards.
04What to do
Read these numbers as model-layer measurements rather than deployment measurements when you compare models, because OpenAI says they exclude production safeguards. OpenAI frames model safety training as one layer of a broader safeguards system, so keep controls outside the model in place for cyber and biology work.
Measured results
Safe responses to fixed cyber jailbreaks
Higher is better: the share of jailbreak attempts answered safely, without production safeguards
| Result | Score |
|---|---|
| GPT-6 Astrathis story | 91.5% |
| GPT-5.6 Sol | 59% |
Scale 0 to 100% · Source: alignment.openai.com, as published
Questions people ask
- Do OpenAI's fuzzy robustness numbers include production safeguards?
- OpenAI states that the figures in its fuzzy robustness post measure model robustness without production safeguards. The percentages therefore describe how the model itself behaves under adversarial pressure, not how a deployed OpenAI product behaves. The post frames model safety training as one layer inside a broader safeguards system, so deployed behaviour sits behind further filters and controls.
- Which models did OpenAI test for fuzzy robustness?
- OpenAI applied the approach to its frontier models GPT-6 Astra and GPT-6 Sol, and used GPT-5.4 and GPT-5.5 as defender models in a controlled study. The attacker that generated the jailbreaks was a helpful-only reasoning model from the GPT-5.6 Sol family. The post compares GPT-6 Astra against GPT-5.6 Sol on fixed jailbreaks and on adaptive multi-turn attacks.
- Did training for fuzzy robustness make GPT-6 refuse more benign requests?
- OpenAI reports that robustness training did not make GPT-6 Astra more likely to refuse benign tasks. On its challenging-prompt evaluation the authors describe a Pareto improvement, meaning safer handling of harmful requests alongside fewer refusals of legitimate ones. OpenAI publishes these as its own measurements, and the supporting values appear only in the post's figures rather than in a table.
Sources 2 links, all fetched
- 1alignment.openai.comOpenAI Research Post
- 2cdn.openai.comFull Paper (PDF)