Controlling Language Model Generation with NVIDIA's LogitsProcessorZoo

Hugging Face Blog · 1 year ago

NVIDIA released LogitsProcessorZoo, a library of modular tools for controlling language model text generation by modifying token probability distributions. The library includes processors like GenLengthLogitsProcessor for controlling output length, CiteFromPromptLogitsProcessor for anchoring responses to input text, and ForceLastPhraseLogitsProcessor for enforcing specific closing phrases. Users can install the library via pip and integrate it with Hugging Face's generate method to tailor model outputs for specific tasks like summarization, question-answering, and structured content generation.