# LatentRiskBench: action-level checks miss state-dependent agent risk

> LatentRiskBench measures whether a guardrail can tell a safe agent action from an identical one that breaks policy in another state.

- **Severity:** Info: Research, tools, evals or lab claims with no direct exposure. Context, not an alert.
- **Category:** Research, Benchmark
- **Published by:** arXiv
- **Disclosed:** 2026-10-13
- **Affects:** Enterprise AI agents, LlamaGuard3, GuardAgent
- **Research by:** Jingyue Cong (RMIT University), Estrid He (RMIT University), Zhuang Li (RMIT University), Nan Wu, Li Shen (YepAI), RMIT University
- **Primary source:** https://arxiv.org/abs/2610.17685
- **Page:** https://theinjection.dev/items/arxiv-latentriskbench/

Guardrails that judge one agent action at a time catch almost none of the policy violations that depend on workflow state, a new benchmark reports. The authors measure 0.09 detection at action level against 0.96 at state level.

## What happened

Eight authors led by Jingyue Cong of RMIT University posted LatentRiskBench to arXiv on October 13, 2026, with code on GitHub. The benchmark was accepted to the EMNLP 2026 Industry Track. The team built it for enterprise agent risks that cannot be judged from a single action, and ran it against seven guardrails and five auditor models.

## How it works

Each case pairs a risky agent trajectory with a benign one that ends in the same final action, differing only in a hidden, policy-relevant workflow state. A guardrail reading the action alone cannot separate the pair, so it has to reconstruct the earlier context. The authors score residual risk left after a defense runs, and report a State-Reconstruction Gain of 0.87 between action-level and state-level auditing.

## Who is affected

Teams running policy guardrails over multi-step enterprise agents are the audience, in particular anyone relying on filters that score a single tool call. Among the defenses tested, the authors report GuardAgent leaving the least residual risk at 0.20 and an unguarded baseline the most at 0.96. Results come from the authors' own benchmark and one anonymized e-commerce deployment, not from a reported incident.

## What to do

Test a guardrail against pairs of trajectories that end in the same action but differ in earlier workflow state, rather than scoring actions one at a time. The authors report that giving an auditor access to workflow state lifts detection from 0.09 to 0.96 on their state-dependent set.

## Residual risk by guardrail, LatentRiskBench

Residual risk after the defense runs, lower is better for defenders

| Result | Score |
|---|---|
| Vanilla | 0.96 |
| Keyword | 0.78 |
| StepRule | 0.59 |
| LLMJudge | 0.32 |
| TrajRule | 0.5 |
| LlamaGuard3 | 0.85 |
| GuardAgent | 0.2 |

Source: https://arxiv.org/html/2610.17685v1

## Quick facts

- **Venue:** Accepted to EMNLP 2026 Industry Track
- **arXiv v1:** October 13, 2026
- **Guardrails tested:** Vanilla, Keyword, StepRule, LLMJudge, TrajRule, LlamaGuard3, GuardAgent
- **Auditor backbones:** GPT-4o, GPT-4o-mini, Claude Sonnet 4.6, Gemini 2.5 Pro, DeepSeek-V3
- **Detection:** 0.09 action-level against 0.96 state-level on the state-dependent core
- **Code:** Released on GitHub, no license stated in the repository

## Sources

1. [arXiv Paper](https://arxiv.org/abs/2610.17685)
2. [Full Text HTML](https://arxiv.org/html/2610.17685v1)
3. [LatentRiskBench Code](https://github.com/JingyueCong/LatentRiskBench)