LLM-Driven Feature Discovery
TLDR Dev
Researchers are using LLMs to help find and label hidden features inside other neural networks. It's a step toward automating the tedious grunt work of AI interpretability.
Mechanistic interpretability has a scaling problem. Sparse autoencoders can pull thousands, sometimes millions, of latent features out of a language model's activations, but someone still has to sit down and figure out what each one actually represents. That someone is usually a human squinting at activation patterns for hours. The AI Alignment Forum post on LLM-driven feature discovery tackles this bottleneck directly: why not let a language model do the labeling?
The approach isn't new in spirit — Anthropic and others have experimented with using models like Claude to auto-generate descriptions for SAE features by feeding them the top-activating text examples. But this piece pushes further into using LLMs not just to describe features after they're found, but to help steer the discovery process itself, effectively closing the loop between extraction and interpretation. Instead of a researcher manually inspecting a dictionary of features and writing captions, the LLM proposes hypotheses about what a given neuron or feature cluster might be doing, then checks those hypotheses against held-out activation data.
What makes this interesting isn't the novelty of asking GPT-4 to summarize text — that's old news. It's the attempt to build a semi-automated pipeline that can keep up with the sheer volume of features modern SAEs produce. When you're dealing with a model that has 16 million or more learned features, as some recent SAE work on GPT-4-scale models has shown, manual review simply doesn't scale. An LLM that can triage, cluster, and pre-label features, even imperfectly, turns an impossible task into a merely large one.
The catch, and the post seems aware of it, is that you're now using a black-box system to interpret another black-box system. If the labeling LLM hallucinates a plausible-sounding but wrong explanation for a feature, that error can quietly propagate into downstream safety research, audits, or claims about what a model "knows" or "wants." The proposed fix is the usual one in this corner of alignment work: cross-validate with statistical tests, human spot-checks, and ablation studies that verify a feature actually causes the behavior the label suggests, rather than trusting the caption at face value.
My take
I like this direction more than most alignment papers I read this month, mostly because it's trying to solve an actual bottleneck instead of proposing another theoretical framework nobody will implement. But using an LLM to interpret an LLM is recursive in a way that should make everyone a little nervous — we're one hallucinated feature label away from someone citing a fake mechanism in a safety paper. Automate the triage, sure. Don't automate the trust.
Read more about this at: TLDR Dev
