Adversarial Attacks on LLMs

Lilian Weng · 2 years ago

Researchers examine adversarial attacks and jailbreak prompts that can circumvent safety measures in large language models despite alignment efforts during training. Adversarial attacks on text-based systems are more challenging than image-based attacks because text operates in discrete space without direct gradient signals. Understanding these attack methods is essential for improving model robustness and maintaining safety guarantees in deployed LLM systems.