Lessons learned on language model safety and misuse

OpenAI Blog · 4 years ago

Anthropic published guidance on managing safety risks and potential misuse of language models based on their operational experience. The company identified specific attack vectors including prompt injection, model extraction, and jailbreaking attempts across its deployed systems. Their findings are intended to inform industry practices for detecting and mitigating similar harms in other organizations' AI deployments.