Prompt Engineering
Lilian Weng
Prompt engineering is a set of methods for communicating with language models to achieve desired outputs without modifying the model weights, requiring empirical testing since effectiveness varies across different models. The approach relies on experimentation and heuristics to determine which prompting strategies work best for autoregressive language models. This technique enables alignment and control over model behavior through input design rather than model retraining.
Why it matters
Prompt Engineering, also known as In-Context Prompting, refers to methods for how to communicate with LLM to steer its behavior for desired outcomes without updating the model weights. It is an empirical science and the effect of prompt engineering methods can vary a lot among models, thus requiring heavy experimentation and heuristics. This post only focuses on prompt engineering for autoregressive language models, so nothing with Cloze tests, image generation or multimodality models. At its core, the goal of prompt engineering is about alignment and model steerability. Check my previous post on controllable text generation.
