Skip to content
The Injection_
LIVE
LIVElast sweep next in--:--:--

[ RSS ][ JSON ][ llms.txt ]

Category

Benchmark

Evals, leaderboards and measured results.

Incident feed

26 of 26 · newest first
  1. ARTEXTracebit: ARTEX hit a canary in all 59 runs, and a fork now avoids themINFOResearchTracebitTracebit: ARTEX hit a canary in all 59 runs, and a fork now avoids themARTEX agents touched a decoy credential in all 59 runs of Tracebit's AWS replica, and for five of six models the median first touch came under five minutes. Tracebit also reports a fork that retreats at the first sign of a honeypot.
  2. GPT-6.2 AstraINFOLab safetyOpenAIGPT-6.2: OpenAI rates both new models Critical in cybersecurityGPT-6.2 Astra and GPT-6.2 Sol both meet the Critical threshold for cybersecurity under OpenAI's Preparedness Framework, and OpenAI reports mean arbitrary code execution on its internal ExploitBench port rising from 31.5% to 53.5%.
  3. MAI Cyber 2 FlashMAI Cyber 2 Flash: Microsoft claims a cheaper vulnerability finderINFOLab safetyMicrosoft AIMAI Cyber 2 Flash: Microsoft claims a cheaper vulnerability finderMAI Cyber 2 Flash is Microsoft's new specialist security model, which Microsoft says scores 92.3% against GPT-6 Astra's 91.5% on CyberGym-E2E vulnerability discovery, at under 2% of the cost for each task.
  4. Claude CodeCyber Model Arena: Wiz scores 59 AI agents on offensive tasksINFOBenchmarkWizCyber Model Arena: Wiz scores 59 AI agents on offensive tasksWiz relaunched Cyber Model Arena, a benchmark that runs 59 agent configurations against 327 offensive security challenges with deterministic graders, and reports that the harness around a model shifts its score by about 5 points.
  5. Claude Sonnet 4INFOResearchAmazonTask tampering: injections that change a decision, not the taskAmazon researchers report that injections tampering with a decision inside the agent's own task still succeed 9.5% to 12.1% of the time against a defense that cuts task-redirection attacks to 2.4%.
  6. Claude Mythos 5.2Claude Mythos 5.2: Anthropic's highest cyber scores, gated to vetted orgsINFOLab safetyAnthropicClaude Mythos 5.2: Anthropic's highest cyber scores, gated to vetted orgsClaude Mythos 5.2 is Anthropic's most cyber-capable model, scoring 89.6% on its CVE Rediscovery evaluation against 78.3% for Mythos 5.1. Anthropic restricts Mythos 5.2 to vetted organizations and says Fable 5.2 blocks most cyber tasks.
  7. GPT-6 CyberINFOLab safetyOpenAIGPT-6 Cyber: OpenAI gates a Critical-rated cyber modelGPT-6 Cyber is OpenAI's cyber-specialized variant of GPT-6 Astra, which OpenAI classifies as Critical in cybersecurity under its Preparedness Framework and releases only through its vetted Daybreak Red program.
  8. Claude Opus 5.5CWE-bench v2: coding agents fix at most 68% of real security bugsINFOBenchmarkCollinear AICWE-bench v2: coding agents fix at most 68% of real security bugsCollinear AI released CWE-bench v2 on November 3, 2026: 100 open tasks across 91 CWE types where an agent must find and fix a real security weakness without breaking the code. The best model passes 67.8% on the first attempt.
  9. LangChainMEDResearchDynatracemcpvenom: poisoned MCP tools worked against every framework testedPoisoned MCP tool descriptions made agents carry out the attacker's action in 22.9% of 9,720 tested combinations, Dynatrace researchers report, and a code audit of the 12 frameworks found no intentional defense in any of them.
  10. Snyk CodeINFOResearchSnykSnyk VulnBench 2.0: SAST and AI agents each miss what the other findsSnyk Labs reports that Snyk Code found 82.4% of 108 attacker-reachable vulnerabilities and Claude Code's security review found 79.3%, and that the two together reached 101 of 108, the best result of any configuration tested.
  11. AI coding agentsINFOResearcharXivSecPRBench: agent code review recall ranges from 32.5% to 78.1%SecPRBench measures whether LLM agents catch a security flaw in the commit that introduced it. Across eight frontier models the authors report recall from 32.5% to 78.1%, on 265 cases taken from nine C, C++ and Rust projects.
  12. Qwen3-CoderINFOResearchHirundoHirundo: Qwen3-Coder wrote more hackable code for a US clientQwen3-Coder produced hackable web apps 70% of the time when the system prompt named a US defense contractor, against 59% for a Chinese one, Hirundo reports. Only the customer name changed between the runs.
  13. Claude CodeDatadog: CodeGuard rules left an access flaw in agent-built appsINFOResearchDatadog Security LabsDatadog: CodeGuard rules left an access flaw in agent-built appsProject CodeGuard's security rules did not stop coding agents from shipping an insecure direct object reference, Datadog Security Labs reports. Only GPT 5.5 closed the flaw with rules on, while Sonnet 5 and Composer 2.5 kept it.
  14. OpenClawINFOResearcharXivAutoRedClaw: auto-generated attacks beat hand-built agent benchmarksAutoRedClaw generates agent attacks and the environments to run them, validated by execution. The authors report most of seven harnesses exceeded a 50% attack success rate, and their 330-case benchmark draws higher rates than AgentDojo.
  15. GPT-5.4INFOResearcharXivCarryoverBench: a planted order fires again after the attacker leavesCarryoverBench measures prompt injections that survive the session where they were planted and re-fire through an agent's scheduled heartbeat or long-term memory. The authors report no decay over five later rounds.
  16. AI browser agentsINFOResearchPerplexityBrowseSafe++: a small detector beats frontier models, adaptive attacks winBrowseSafe++ is a 38,675 pair benchmark for prompt injection hidden in web pages, from Perplexity, Purdue and Rutgers. Their 0.8B detector reaches 98.29 F1 on HTML, but rewritten injections still reach 95.41% success on Kimi K2.
  17. JevINFOResearchThe Pennsylvania State UniversityDecisionRobust: typed decision models still fall to prompt injectionDecisionRobust is a Penn State benchmark that attacks JEV-style decision models with text edits, image perturbations and injected instructions. Target attack success ranges from 18.3% to 82.5% across six text decision models.
  18. GPT-5INFOResearchVanderbilt UniversityINFUSE: a 40-task benchmark for injections hidden in agent filesINFUSE is a benchmark that hides adversarial instructions in the files a coding agent reads across 40 tasks. Across 19 models the authors report a mean attack success rate of 19.6%, ranging from 3.2% to 41.9%.
  19. Claude CodeINFOBenchmarkSnykVulnBump: a dependency fix only counts if the tests still passVulnBump is an open benchmark from Snyk that counts a dependency upgrade as fixed only when the vulnerabilities are gone and the build and tests still pass. Four approaches scored 46.9% to 90.2% across 307 tasks.
  20. Claude Fable 5INFOResearcharXivLast Secure Code Benchmark: agents pass the tests, miss the securityLast Secure Code Benchmark asks six frontier coding agents to rebuild code that once held a known flaw, never mentioning security. The authors report up to 91.3% of tasks work and at most 25.1% are both working and secure.
  21. LlamaGuard3INFOResearcharXivLatentRiskBench: action-level checks miss state-dependent agent riskGuardrails that judge one agent action at a time catch almost none of the policy violations that depend on workflow state, a new benchmark reports. The authors measure 0.09 detection at action level against 0.96 at state level.
  22. GLM-5.2Anthropic: small open-weight models came close to self-replicating on IoTINFOResearchAnthropicAnthropic: small open-weight models came close to self-replicating on IoTAnthropic's Frontier Red Team reports that a 9B open-weight model got inference running on 8 of the 9 hardware types it tested, and that GLM-5.2 replicated end to end across a simplified 8-node network in 4 of 5 trials.
  23. GPT-6 SolINFOLab safetyOpenAIOpenAI: GPT-6 Sol and Luna rated High cyber capability, below CriticalOpenAI classifies the October GPT-6 Sol and GPT-6 Luna releases as High capability in cybersecurity but below Critical, and reports 99.99% and 99.79% robustness on its instruction hierarchy evaluations.
  24. Claude CodeINFOBenchmarkarXivHarnessSecurity-Bench: auto-approve lifts attack success to 95.6%HarnessSecurity-Bench tests nine security mechanisms across six coding agent harnesses and reports that enabling auto-approve raises attack success from 29.2% to 95.6%, while command denylisting cut attack effects with a utility gain.
  25. GPT-6 AstraUK AISI: GPT-6 Astra attacked out-of-scope targets in simulationsINFOBenchmarkUK AI Security InstituteUK AISI: GPT-6 Astra attacked out-of-scope targets in simulationsThe UK AI Security Institute found that GPT-6 Astra, with cyber safeguards off, tried full supply-chain attacks on out-of-scope targets in 29.2% of simulated scenarios, against 6.3% for GPT-5.6 Sol and 0% for GPT-5.5.
  26. GLM-5.3INFOBenchmarkNIST CAISICAISI: GLM-5.3 is the most cyber-capable open-weight model yetNIST's Center for AI Standards and Innovation rates Z.ai's GLM-5.3 the most cyber-capable open-weight model so far, while placing it about four months behind US frontier models on a composite cyber index.