Gemma Scope 2: helping the AI safety community deepen understanding of complex language model behavior

Google DeepMind dropped a serious piece of infrastructure this week for anyone trying to understand what's actually happening inside a large language model. Gemma Scope 2 covers every size of Gemma 3, from the tiny 270M parameter version up to the 27B flagship, and DeepMind is calling it the largest open-source interpretability release any AI lab has ever put out. The numbers back that up: producing the toolkit meant storing roughly 110 petabytes of data and training over a trillion parameters worth of auxiliary models just to build the microscope itself.

The core idea isn't new, but the execution has scaled up dramatically. Like the original Gemma Scope from last year, this toolkit relies on sparse autoencoders and transcoders that let researchers decompose a model's internal activations into interpretable pieces, essentially translating the tangled math inside a neural network into concepts a human can actually reason about. What's different this time is the depth of coverage. Every layer of every Gemma 3 model now has trained SAEs and transcoders attached, and DeepMind added skip-transcoders and cross-layer transcoders specifically to help trace computations that unfold across multiple steps and multiple layers, the kind of multi-hop reasoning that's notoriously hard to pin down with older tools.

DeepMind also leaned on a technique called Matryoshka training, which it says fixes some flaws that showed up in the first Gemma Scope and helps the autoencoders surface more genuinely useful concepts rather than noise. And because a huge share of real-world LLM use happens through chat interfaces, the team built a separate set of tools tuned specifically for Gemma 3's chat-tuned variants, aimed at dissecting things like jailbreak mechanics, refusal behavior, and whether a model's stated chain-of-thought actually matches what's going on internally, a gap that's become one of the more uncomfortable open questions in AI safety.

The motivation here isn't abstract. DeepMind points to the 27B-parameter C2S Scale model, which reportedly helped uncover a new candidate pathway for cancer therapy, as an example of the kind of emergent capability that only shows up once a model gets large enough. Gemma Scope 2 wasn't trained on that specific model, but the message is clear: if labs want to understand why bigger models suddenly develop new abilities, or why they sometimes hallucinate, flatter users, or get jailbroken, they need tools that scale with the models themselves rather than staying stuck at toy sizes. An interactive demo built by Neuronpedia is already live for people who want to poke around before diving into the full toolkit.