Why agents are a new kind of security problem
What it is
Three properties make agents different from ordinary software:
- Instructions and data share one channel. A model reads everything as text. It has no built-in way to tell 'this is a command from my operator' from 'this is a sentence inside a document I was asked to read'.
- Agents act. A tricked chatbot says something wrong. A tricked agent sends, deletes, pays or publishes.
- Agents read untrusted content by design: web pages, emails, files, other agents' messages.
Why it matters
In classic software, an attacker needs a bug to turn data into commands. With a language model, text that looks like an instruction may simply be followed.
- This is why the central threat, prompt injection, cannot be fully patched: it follows from how models work.
- So defence is architectural: assume the model will sometimes be fooled, and design so that a fooled model cannot do serious harm.
- The question for every design is not 'can it be tricked?' but 'what happens when it is?'.
In the real world: A security team's first rule for agent reviews: 'assume the model is compromised from the moment it reads untrusted text; show me what stops it then.'
Visual example
TrustOnly the first two rows can give orders. The model cannot tell them apart from the rest unaided.
Believing a clever enough system prompt makes injection impossible. Prompt-level defences reduce the success rate; they never make it zero. Security that rests on the model always obeying is not security.
Practise it: Design an AI Customer Support Agent