# HarnessSecurity-Bench: auto-approve lifts attack success to 95.6%

> HarnessSecurity-Bench measures whether the security mechanisms in six coding agent harnesses actually stop attacks.

- **Severity:** Info: Research, tools, evals or lab claims with no direct exposure. Context, not an alert.
- **Category:** Benchmark, Research
- **Published by:** arXiv
- **Disclosed:** 2026-10-06
- **Affects:** Claude Code, Codex CLI, Gemini CLI, gptme, Qwen Code, GitHub Copilot
- **Research by:** Zhengyang Zhu, Liming Huang, Runmin Ji, Mingxi Ye, Zihan Zhou, Hanyang Guo, Jingwen Wu, Yuhan Ye, Yuming Feng, Hong-Ning Dai, Zibin Zheng
- **Primary source:** https://arxiv.org/abs/2610.07639
- **Page:** https://theinjection.dev/items/harness-security-bench-coding-agents/

HarnessSecurity-Bench tests nine security mechanisms across six coding agent harnesses and reports that enabling auto-approve raises attack success from 29.2% to 95.6%, while command denylisting cut attack effects with a utility gain.

## What happened

HarnessSecurity-Bench, posted to arXiv on October 6, 2026, tests the security mechanisms of six coding agent harnesses: Claude Code, Codex CLI, Gemini CLI, gptme, Qwen Code and GitHub Copilot. The authors build a ten-mechanism taxonomy, assess 400 harness-mechanism cells using researcher and LLM-judge ratings, and run 23 tasks across five attack surfaces. The reported scale is 2,500 trials, 81,155 tool calls and over 2.2 billion tokens, with GLM-5.2 as the baseline model.

## How it works

The benchmark evaluates nine mechanisms across the six harnesses and measures both attack success and task utility. The paper reports that enabling auto-approve raises attack success from 29.2% to 95.6%. Network isolation and read-only mode reduced attack effects but cost substantial utility, command allowlisting reduced attack effects with a small utility loss, and command denylisting reduced attack effects while giving a utility gain.

## Who is affected

Teams running any of the six harnesses, and anyone treating a harness setting as a security control. The authors report that about half of confirmed mechanism implementations are opt-in, that closed-source harnesses show substantial evidence gaps, and that allowed tools or commands can still leave unauthorized operations reachable through alternative execution paths.

## What to do

Leave auto-approve off, since the paper measures attack success rising from 29.2% to 95.6% when it is on. Check which mechanisms your harness enables by default rather than assuming they are active, and note the paper's finding that command denylisting and allowlisting cost less utility than network isolation or read-only mode.

## Quick facts

- **Harnesses tested:** Claude Code, Codex CLI, Gemini CLI, gptme, Qwen Code, GitHub Copilot
- **Benchmark:** 23 tasks, 5 attack surfaces, 9 mechanisms
- **Scale:** 2,500 trials, 81,155 tool calls, over 2.2 billion tokens
- **Auto-approve:** Attack success 29.2% off, 95.6% on
- **Baseline model:** GLM-5.2
- **Posted:** 2026-10-06

## Sources

1. [arXiv Abstract](https://arxiv.org/abs/2610.07639)
2. [Project Website](https://tsingpig.github.io/HarnessSecurity-Benchmark/)