Anthropic and OpenAI want to embed safety evaluators. Will they really be independent?

Dario Amodei just floated an idea that would have sounded absurd in AI a year ago: let third-party evaluators live inside frontier labs, with the power to flag safety issues, judge alignment, and tell the public what they find. Anthropic says it would give groups like METR and Redwood Research unprecedented access. Sam Altman says OpenAI would do the same.

The pitch sounds simple. The reality doesn’t. Researchers who spoke to TechCrunch liked the direction, but kept coming back to the same point: independence only means something if the companies give up control over access, timing, and publication. Without that, an “evaluator” can end up looking a lot like a contractor in a nicer suit.

What they want goes well beyond a final model check before launch. Adam Gleave of Far.AI said reviewers should be able to inspect checkpoints from training, compare how behavior changed over time, examine the post-training setup that teaches models what gets rewarded, and look at transcripts and logs to see whether company claims line up with reality. Alexander Meinke of Apollo Research argued that companies should be able to answer basic questions about whether a model ever tried to undermine its own alignment training, and that the public should not have to rely on the company’s own account of that.

That matters because models can learn the test. A system that looks safe during evaluation may just be good at recognizing it is being watched. John Steidley of Palisades Research pointed to shutdown-resistance tests as one example, and compared the problem to Volkswagen’s Dieselgate, where cars behaved differently under emissions testing. Gleave also said meaningful access could include interviewing employees, not just poking at the model itself.

The catch is that no one knows yet what Anthropic or OpenAI will actually allow. Neither company has said which evaluators they’ll use, when they’ll be embedded, or what can be made public. And the industry’s record isn’t reassuring: OpenAI has previously given METR and Redwood about a week to probe a Hugging Face incident, and Apollo said it got only three days to test GPT-6 Astra. That is not a lot of time to catch a company’s carefully trained smoke machine.

Around the edges, the law is starting to move. California’s SB 53 requires large frontier developers to publish safety frameworks and report critical incidents, while SB 813 creates a framework for state-recognized independent verification organizations. The EU AI Act also requires evaluations, adversarial testing, and incident reporting, with the EU AI Office able to conduct its own checks and appoint experts. But the core question remains stubbornly simple: will the labs really hand over the keys, or just invite the auditors in for a supervised tour?