Controllable Neural Text Generation

Lilian Weng · 5 years ago

This article surveys methods for controlling the output of language models during text generation, covering decoding strategies like beam search and nucleus sampling, guided decoding approaches that incorporate feature discriminators, and regularized decoding frameworks that optimize for properties like uniform information density. Key concrete techniques include top-k sampling, nucleus sampling with cumulative probability thresholds, penalized sampling with repetition penalties around theta=1.2, and regularized beam search with lambda parameters optimized between 2 and 5 for translation tasks. These approaches enable users to steer unconditioned language models toward desired attributes such as topic, style, and sentiment without modifying model weights, making them applicable to controlled generation tasks like creating age-appropriate content.