Strengthening AI safety evaluation through independent third-party review
Amazon Science
Amazon had outsiders review its AI safety tests, not just the models. The twist: they checked the test quality itself, and found room to tighten it up.
Based on reporting by Amazon Science — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
Amazon is doing something a bit more unusual than the standard AI safety theater. It invited AVERI, an independent nonprofit, to look over the company’s internal safety evaluation datasets and judge the tests themselves, not just the model answers they produce.
That matters because the bar for frontier models keeps rising, and Amazon says it already spends heavily on internal safety testing across harmful content, cybersecurity, privacy, and copyright. But even strong in-house checks can get too familiar. AVERI’s job was to see whether the prompts actually pushed at safety boundaries, and whether the evaluation set was rigorous enough to trust.
To do that, Amazon gave AVERI a random sample from its safety-evaluation suite. AVERI’s team, led by executive director Miles Brundage and including Sean McGregor, A. Feder Cooper, and George Balston, reviewed the prompts for diversity, difficulty calibration, and methodological soundness. They did it independently, without input from Amazon’s evaluation team.
The review backed up some parts of the process. Amazon says AVERI found broad topical coverage, a mix of writing styles and single-turn and multiturn prompts, and a low risk that the material was lifted from public datasets. But the external review also flagged places to improve: more even multilingual coverage, more multistep adversarial prompts, fresher prompts, clearer grading in a few edge cases, and better protection against models that only fail on obvious tells.
Amazon says it has already folded in several changes, including removing redundant prompts, adding harder adversarial scenarios, introducing tiered difficulty levels, setting a quarterly review cadence, and expanding multilingual prompts across all risk categories. The company frames this as a model for the field: not just testing AI systems, but testing the tests. That is the right instinct. AI safety gets shaky fast when the people grading the exam are also the ones who wrote it.
My take — AI-written commentary, not fact-checked reporting
This is the kind of boring process work AI companies should be bragging about more often. External review of the evals, not just the model, is where a lot of the self-congratulation gets cut down to size. The industry loves a safety banner; fewer firms seem eager for someone else to read the fine print.
Read more about this at: Amazon Science