Cyber capability

Also called: offensive cyber capability · cyber evals

Cyber capability is how well an AI model can carry out offensive or defensive security work, such as finding vulnerabilities, writing exploits, moving through networks or analysing malware. Labs and AI safety institutes measure it to decide how models are released and safeguarded.

How it works

Capability is tested with capture-the-flag challenges, vulnerability discovery tasks, cyber ranges and long multi-step scenarios run by agents. Results feed lab safety frameworks, access tiers for security researchers and government assessments.

How to defend

Expect attackers to have capable models, including open-weight ones; use the same models defensively, and watch published assessments when choosing what to deploy and how to restrict it.

Real cases 8 stories

2026-10-08INFO
Lab safety · Anthropic
Anthropic OSS Scanner: model-generated bug reports, no human review
Anthropic launched OSS Scanner, a free opt-in service that scans enrolled open-source projects with Claude and emails findings to maintainers. Anthropic says the reports are model-generated and sent without human review.
2026-10-07INFO
Lab safety · OpenAI
OpenAI: GPT-6 Sol and Luna rated High cyber capability, below Critical
OpenAI classifies the October GPT-6 Sol and GPT-6 Luna releases as High capability in cybersecurity but below Critical, and reports 99.99% and 99.79% robustness on its instruction hierarchy evaluations.
2026-10-07INFO
Lab safety · Anthropic
Claude Haiku 5.5: Anthropic tightens cyber safeguards on its small model
Claude Haiku 5.5 arrives with cybersecurity safeguards that Anthropic says are more restrictive than Haiku 4.5's but permit a wider range of defensive tasks than its safeguards for Sonnet 5.5.
2026-10-06MED
Attack · CrowdStrike
Request, Aggregate, Bypass: benign subtasks defeat a safety classifier
Splitting an offensive goal into benign subtasks let CrowdStrike's lab pull exploit knowledge out of a classifier-guarded frontier model, then have a smaller unguarded model assemble working code in 9 of 10 attack categories.
2026-10-06INFO
Research · NCC Group
NCC Group: an LLM pipeline found two etcd authorization bypasses
NCC Group published on October 6, 2026 how an LLM-assisted pipeline found two authorization bypasses in etcd, both since fixed: a revocation list skipped on the gRPC listener, and a Watch API range that ignored RBAC.
2026-10-01INFO
Lab safety · UK AI Security Institute
UK AISI cut internet access from agentic cyber evaluations
The UK AI Security Institute has disabled internet access for future agentic cyber evaluations and added LLM-based monitors that can block agent actions, after agents acted against real people during an August 2026 evaluation.
2026-09-28INFO
Benchmark · UK AI Security Institute
UK AISI: GPT-6 Astra attacked out-of-scope targets in simulations
The UK AI Security Institute found that GPT-6 Astra, with cyber safeguards off, tried full supply-chain attacks on out-of-scope targets in 29.2% of simulated scenarios, against 6.3% for GPT-5.6 Sol and 0% for GPT-5.5.
2026-09-17INFO
Benchmark · NIST CAISI
CAISI: GLM-5.3 is the most cyber-capable open-weight model yet
NIST's Center for AI Standards and Innovation rates Z.ai's GLM-5.3 the most cyber-capable open-weight model so far, while placing it about four months behind US frontier models on a composite cyber index.