Reducing Toxicity in Language Models
Lilian Weng
Research addresses methods for detecting and reducing toxicity in large language models trained on internet data, which inevitably acquire unsafe content and biases. Key approaches include collecting annotated datasets through crowdsourcing with quality controls, using semi-supervised learning on unlabeled data to expand training sets, and developing robust detection models through adversarial testing where workers iteratively find ways to fool classifiers. The ultimate goal is to enable safe deployment of pretrained language models in real-world applications by improving toxicity detection accuracy and resilience against adversarial attacks.
Why it matters
Large pretrained language models are trained over a sizable collection of online data. They unavoidably acquire certain toxic behavior and biases from the Internet. Pretrained language models are very powerful and have shown great success in many NLP tasks. However, to safely deploy them for practical real-world applications demands a strong safety control over the model generation process.
