Designing AI agents to resist prompt injection

OpenAI Blog · 4 months ago

Researchers are developing methods to make AI agents resistant to prompt injection attacks that try to manipulate their behavior. The approach involves constraining which actions agents can take and implementing safeguards around sensitive data access within agent workflows. This reduces the risk that attackers can redirect agents toward unintended tasks through malicious prompts.