Extracting Concepts from GPT-4
OpenAI
OpenAI used sparse autoencoders to pull 16 million distinct concepts out of GPT-4's inner workings. That's a peek inside the black box most AI models never let anyone see.
Based on reporting by OpenAI — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
OpenAI just published a chunky bit of interpretability research, and the headline number is hard to ignore: 16 million patterns, automatically extracted from GPT-4's internal computations using scaled-up sparse autoencoders. This isn't a new model or a flashy demo. It's an attempt to answer a question that's been nagging at the field since transformers got big: what is actually happening inside these systems when they generate a sentence.
Sparse autoencoders aren't new, but running them at this scale is. The technique works by taking the dense, tangled activations inside a neural network and forcing them through a bottleneck that encourages sparsity, meaning only a small number of features fire for any given input. Do that at GPT-4 scale and you start pulling out individual concepts, like specific phrases, syntactic patterns, or semantic categories, that were previously smeared across thousands of neurons in ways no human could parse. OpenAI's contribution here is mostly engineering: new methods for training these autoencoders efficiently on a model as large as GPT-4, rather than the toy models interpretability researchers usually settle for.
Why bother? Because right now, nobody, including the people who built GPT-4, can fully explain why it produces any specific output. That's uncomfortable for a technology being deployed in hospitals, courtrooms, and financial systems. Extracting millions of legible features is a step toward being able to audit a model's reasoning rather than just trusting its outputs. It's also directly relevant to safety work: if you can identify the feature that corresponds to, say, deceptive reasoning or a refusal pattern, you have a lever to actually intervene rather than just fine-tune and hope.
The scale of the number is both the achievement and the problem. Sixteen million features is far too many for any team to manually inspect, label, and understand one by one. So the practical value depends on what comes next: clustering these features, building tools that let researchers query them, and figuring out which of the sixteen million actually matter for behavior we care about. OpenAI is open about this being early-stage work, not a solved problem, and the paper reads more like a methods release for other interpretability researchers than a victory lap.
My take — AI-written commentary, not fact-checked reporting
This is the kind of research I want ten times more of, and ten times faster, because interpretability is the difference between deploying AI on faith versus deploying it with actual understanding. My gripe is that OpenAI keeps doing this valuable transparency work on a model that remains otherwise closed, which feels a bit like handing out a partial map of a building you won't let anyone else walk through.
Read more about this at: OpenAI