New Platform Peers Inside AI’s Black Box
IEEE Spectrum Benjamin Skuse
Goodfire just opened a tool that helps people look inside AI models. It could make these systems easier to debug — and harder to hide behind.
Based on reporting by IEEE Spectrum, Benjamin Skuse — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Ask a popular chatbot a simple question and the answer can still feel like a magic trick. That has always been the problem: the result is there, but the path to it is mostly hidden, even from the people who built the model. When those systems are writing code or doing other serious work, that opacity stops being cute very quickly.
Goodfire is trying to turn that black box into something more like an instrument panel. The San Francisco lab, founded in 2024, has made its Silico platform generally available and is handing out $1 million in free usage for academic and nonprofit interpretability researchers. The pitch is straightforward: put tools that were mostly locked away in elite labs into the hands of startups and research teams that want to build models, adapt open-source ones, or simply understand what they’re dealing with.
The underlying field is mechanistic interpretability, which tries to map the inner machinery of a model — weights, activations, attention patterns, neurons, the routes between them — to specific behaviors. Silico bundles several of those methods and layers AI agents on top. A user can type something plain like “Find out when and why my model is hallucinating,” and the system will plan experiments, run them in parallel, and come back with results or at least something useful to inspect.
That’s not just theory. Goodfire says it worked with UK company Prima Mente on its Pleiades epigenetic foundation model, which had been good at spotting Alzheimer’s from blood samples. The reverse-engineering effort reportedly showed the model was using DNA fragment-length patterns, a signal people hadn’t used before for that disease. Ho says that may be the first major natural-science finding discovered purely by reverse-engineering a foundation model.
The platform is also already being used by Cameron Berg at Reciprocal Research, a New York nonprofit studying AI cognition. Berg says Silico has let him move much faster and, in his words, made him feel like the AI systems are now the research staff. That’s the real shift here: not just making AI smarter, but making it legible enough that humans can finally argue with it on equal footing.
My take — AI-written commentary, not fact-checked reporting
This is the kind of tooling AI keeps pretending it can skip, right up until it embarrasses someone in public. The industry loves shipping mystery boxes and calling it progress; interpretability is the boring, adult part of the job. Open models won’t save the day by themselves, but they do make this sort of scrutiny harder to dodge — which is exactly why it matters.
Read more about this at: IEEE Spectrum
