Reducing Toxicity in Language Models

Lilian Weng · 5 years ago

Research addresses methods for detecting and reducing toxicity in large language models trained on internet data, which inevitably acquire unsafe content and biases. Key approaches include collecting annotated datasets through crowdsourcing with quality controls, using semi-supervised learning on unlabeled data to expand training sets, and developing robust detection models through adversarial testing where workers iteratively find ways to fool classifiers. The ultimate goal is to enable safe deployment of pretrained language models in real-world applications by improving toxicity detection accuracy and resilience against adversarial attacks.