Language models can explain neurons in language models
OpenAI Blog
GPT-4 was used to automatically generate and evaluate explanations describing what individual neurons do in large language models. The dataset covers all 1.5 million neurons in GPT-2, with each neuron receiving both an explanation and a quality score. This enables researchers to better understand how neural networks process language without manually analyzing each neuron.
Why it matters
We use GPT-4 to automatically write explanations for the behavior of neurons in large language models and to score those explanations. We release a dataset of these (imperfect) explanations and scores for every neuron in GPT-2.
