Automatically Evolving Prompt Guidelines for Task-Specific Optimization

arXiv cs.CL · 6 hours ago

Researchers developed AGOPS, an automated method that generates task-specific prompt guidelines by analyzing reference answers to help users write better-specified prompts for large language models. The technique uses an optimization scheme with multiple LLMs to evolve guidelines that maximize performance on given tasks, recovering performance drops of up to 95.3% caused by underspecified prompts. Implementation across mathematical reasoning, medical question answering, and coding tasks showed performance improvements ranging from 15.5% to 81.7% when users followed the generated guidelines.