Extracting Concepts from GPT-4

OpenAI Blog · 2 years ago

Researchers used scaled sparse autoencoders to extract 16 million distinct computational patterns from GPT-4's operations. The technique identified 16 million individual concepts that the model uses during processing. This capability enables better understanding of how large language models compute internally and may improve interpretability of AI systems.