Skip to content
The Injection_
LIVE
LIVElast sweep next in--:--:--

[ RSS ][ JSON ][ llms.txt ]

Detection

Also called: monitoring · prompt injection detection

Detection in AI security is spotting attacks on or through AI systems as they happen: prompt-injection attempts, jailbreaks, malicious tool calls, data exfiltration and abuse of AI services. It covers classifiers on model inputs and outputs and the logging and monitoring around agents.

How it works

Approaches include trained classifiers, canary tokens, rules for suspicious tool sequences, anomaly detection on agent behaviour, and scanning content before it reaches a model. Detection quality is measured by catch rate and false positives on realistic attacks.

How to defend

Log prompts, tool calls and agent actions with enough context to investigate, alert on high-risk patterns, and evaluate detectors against adaptive attacks rather than fixed test sets.

Real cases 4 stories

  1. INFO

    Attack · Cisco Talos

    A3 malware: Talos measures prompt injection aimed at AI triage

    Four malware families embed natural-language instructions to steer AI malware triage toward a benign verdict, Cisco Talos reports. Across 84 samples, the best technique shifted the verdict in the attacker's favor in about 35% of test runs.

  2. INFO

    Incident · ReversingLabs

    ReversingLabs: behavior beat hashes on an AI-assisted spy campaign

    ReversingLabs says it classified two GTG-20006 implants as malicious in August, two months before Anthropic's September report named the actor, and that behavior-based hunting held up while Claude was used to rebuild detected files.

  3. INFO

    Lab safety · Anthropic

    Anthropic OSS Scanner: model-generated bug reports, no human review

    Anthropic launched OSS Scanner, a free opt-in service that scans enrolled open-source projects with Claude and emails findings to maintainers. Anthropic says the reports are model-generated and sent without human review.

  4. MED

    Research · arXiv

    Explosive prompts: dormant injections fire on 'thanks' in agents

    Explosive prompts are dormant prompt injections that wait for a harmless trigger such as 'thanks'. A new paper reports 43% to 83% success on nine production agents, versus at most 3% for plain injections, and proposes the DeFuse detector.