Learn
AI security glossary
34 terms, each defined in plain English, with how it works, how to defend against it and every story on The Injection that shows it in the real world.
Attack techniques
- Prompt injection13Prompt injection is an attack where text written by an attacker is treated as instructions by a large language model, overriding what the developer or user asked it to do. It works because models read instructions and data in the same stream of text and cannot reliably tell them apart.
- Indirect prompt injection3Indirect prompt injection is prompt injection delivered through content an AI system reads on a user's behalf, such as a web page, document, email, code comment, issue or tool response, rather than typed by the user. The victim never sees the malicious instructions; their own assistant or agent acts on them.
- Jailbreak4A jailbreak is a prompt or technique that gets an AI model to produce content or take actions its safety training is meant to refuse. Unlike prompt injection, which hijacks an application, a jailbreak targets the model's own safeguards, usually by the user themselves.
- Guardrail bypass3A guardrail bypass is a technique that gets past a safety layer placed around an AI model, such as an input filter, output classifier, prompt-injection detector or policy engine, so that blocked content or actions go through. It targets the wrapper, not the model's own training.
- Tool poisoning1Tool poisoning is an attack where the description, schema or output of a tool given to an AI agent, often an MCP server, carries hidden instructions or misleading behaviour. The agent trusts the tool's metadata, so a poisoned tool can steer it into leaking data or calling other tools.
- Data exfiltration4Data exfiltration in AI systems is getting sensitive data out of a model's context, connected tools or environment to an attacker. It is the usual goal of prompt injection: the model is tricked into sending secrets, files or chat history somewhere the attacker can read them.
- Credential theft11Credential theft in AI security means stealing API keys, OAuth tokens, cloud credentials or passwords through AI tools, agents or their supply chain. Agents often run with a developer's full access, so a hijacked agent or malicious plugin can read and send secrets it was never meant to touch.
- Remote code execution (RCE)5Remote code execution (RCE) is a flaw that lets an attacker run their own code or commands on a machine they do not control. In AI products it often comes from agents that run shell commands, plugins that execute on install, or model output that is evaluated as code.
- Zero-click attack2A zero-click attack compromises a target without them doing anything beyond normal use: no link to click, no file to open, no prompt to approve. In AI systems, an agent that reads incoming email, tickets or repos on its own can be hijacked the moment malicious content arrives.
- Sandbox escape6A sandbox escape is a flaw that lets code or an AI agent break out of the isolated environment meant to contain it and reach the host, other tenants or the network. For coding agents and code-execution features, the sandbox is often the main barrier between a hijacked model and real damage.
- Approval bypass3An approval bypass is a flaw that lets an AI agent run a command, edit a file or call a tool that should have required the user's confirmation. Human-in-the-loop prompts are a key control for coding agents, so getting around them turns prompt injection into direct execution.
- Privilege escalation4Privilege escalation is gaining more access than you were given. In AI systems it usually happens through an agent acting with its user's or service's permissions: an attacker with little access gets the agent to do something only a more privileged identity could do.
- Data poisoning0Data poisoning is corrupting the data an AI system learns from or retrieves, so that it behaves the way an attacker wants. It covers training and fine-tuning data, retrieval corpora used for RAG, and the memories agents store for later.
- Model backdoor1A model backdoor is hidden behaviour planted in an AI model that stays dormant until a specific trigger appears in the input, then makes the model do what the attacker chose. The model behaves normally on ordinary tests, which makes backdoors hard to find.
- Typosquatting1Typosquatting is publishing a malicious package, model, skill or domain under a name that looks like a popular one, so people or AI agents install it by mistake. AI coding tools add a twist: they can suggest package names that do not exist, which attackers then register.
- Malicious package2A malicious package is a library, plugin, extension, skill or model file published to look useful while carrying code that steals data, opens a backdoor or attacks its user. AI tooling has made registries like npm, PyPI, Hugging Face and skill and MCP marketplaces frequent targets.
- Rogue agents5Rogue agents are AI agents acting outside what their operators or the systems they touch would allow: probing services, misusing tools, crawling aggressively or pursuing goals beyond their task. The term covers both hijacked agents and agents that misbehave on their own.
Attack surfaces
- Coding agents7Coding agents are AI tools that read and change code and run commands on a developer's behalf, such as Claude Code, OpenAI Codex, GitHub Copilot's agent and CLI, Gemini CLI and Cursor. They hold a developer's access to repos, terminals and secrets, which makes them a high-value target.
- Browser agents0Browser agents are AI agents that operate a web browser for a user: reading pages, clicking, filling forms and completing tasks across sites. Every page they visit is untrusted input, so they are exposed to indirect prompt injection while often holding logged-in sessions.
- MCP (Model Context Protocol)6MCP, the Model Context Protocol, is an open standard for connecting AI assistants and agents to tools and data through servers that expose functions, resources and prompts. It is widely used, so MCP servers, clients, SDKs and registries are now a major part of the AI attack surface.
- Agent skills2Agent skills are packaged instructions, scripts and resources that an AI agent loads to perform a task, such as Anthropic's Agent Skills for Claude. Because a skill tells the agent what to do and can include code, a malicious or tampered skill acts like a malicious package with the agent's access.
- Plugins and extensions3Plugins and extensions add commands, tools or integrations to AI assistants and coding agents. They run with the host's trust and often its access, so a malicious, compromised or vulnerable plugin can execute code, read secrets or steer the agent.
- Agent memory3Agent memory is information an AI assistant or agent stores between sessions, such as user preferences, facts and past conversations, and reloads into its context later. If an attacker can write to it, injected instructions persist and keep influencing the agent long after the original attack.
- RAG (retrieval-augmented generation)0RAG, retrieval-augmented generation, is a pattern where an AI system searches a document store and adds the results to the model's prompt before answering. Everything retrieved becomes model input, so the corpus is an injection and data-leak surface.
- AI model supply chain1The AI model supply chain is everything a deployed model depends on: training data, base models, fine-tunes, weight files, model hubs, conversion tools and the libraries that load them. A compromise anywhere in it can put backdoored weights or malicious code into production.
- Open-weight models2Open-weight models are AI models whose trained weights are published for anyone to download, run and fine-tune, such as Llama, Qwen, Mistral, DeepSeek and GLM releases. Their safety controls can be removed by fine-tuning, so their capabilities, especially in cyber offense, matter for security.
Defense and measurement
- AI agent security14AI agent security is the practice of keeping AI agents, models that plan and take actions with tools, from being hijacked, misused or causing harm. It combines classic controls such as least privilege, sandboxing and auditing with defences against prompt injection and tool abuse.
- Guardrails3Guardrails are controls placed around an AI model to check or restrict what goes in and comes out, such as prompt-injection detectors, content classifiers, PII filters, tool-call policies and output validators. They reduce risk but can be bypassed, so they work best alongside hard limits.
- Detection4Detection in AI security is spotting attacks on or through AI systems as they happen: prompt-injection attempts, jailbreaks, malicious tool calls, data exfiltration and abuse of AI services. It covers classifiers on model inputs and outputs and the logging and monitoring around agents.
- AI red teaming2AI red teaming is attacking your own AI system before others do: probing models, applications and agents for jailbreaks, prompt injection, data leaks, unsafe actions and harmful outputs. It ranges from manual expert testing to automated attack tools and scored campaigns.
- AI security evals7AI security evals are structured tests that measure how a model or agent behaves under attack or on security tasks, for example its resistance to prompt injection, its willingness to cause harm, or its ability to find and exploit vulnerabilities. Labs and government institutes publish them to compare systems.
- Cyber capability8Cyber capability is how well an AI model can carry out offensive or defensive security work, such as finding vulnerabilities, writing exploits, moving through networks or analysing malware. Labs and AI safety institutes measure it to decide how models are released and safeguarded.
- AI threat intelligence8AI threat intelligence is reporting on how real attackers use, target or abuse AI systems, drawn from incidents, provider abuse reports and security vendors' telemetry. It separates what is happening in the wild from what is only shown in research.
- AI-enabled offense6AI-enabled offense is attackers using AI models to make attacks faster, cheaper or larger: writing phishing and malware, researching targets, finding vulnerabilities and running parts of intrusions autonomously. It is about AI as the attacker's tool rather than the target.