Unsupervised sentiment neuron

OpenAI Blog · 9 years ago

Researchers trained a neural network to predict the next character in Amazon reviews without any labeling, and it automatically developed a single neuron that strongly represents sentiment. The sentiment neuron's predictions correlated with human sentiment ratings across review texts, demonstrating the model learned meaningful structure from character-level prediction alone. This shows that sentiment understanding emerges as a byproduct of language modeling rather than requiring explicit supervision, suggesting useful linguistic features develop naturally during unsupervised training.