Improving instruction hierarchy in frontier LLMs
OpenAI Blog 4 months ago
Researchers developed IH-Challenge, a training method that teaches large language models to prioritize instructions from trusted sources over conflicting inputs. The approach improved instruction hierarchy performance by enabling models to better distinguish between legitimate directives and injected prompts. Models trained with this method showed increased resistance to prompt injection attacks and greater safety control without sacrificing general capabilities.