Teaching future scientists to interrogate AI tools for scientific discovery
Allen Institute (AI2)
AI2 put its research agent in a UW class, where students used it to hunt for scientific leads. The twist: the tool found ideas, but students had to do the actual thinking.
Based on reporting by Allen Institute (AI2) — read the original for the full story.
Summary, retelling and take written by AI under human oversight; images are AI-generated illustrations. How we work · Report an error
AI2 dropped its AutoDiscovery research agent into a University of Washington classroom this spring, and the exercise was less about flashy automation than about judgment. The system sifted through datasets, proposed hypotheses, ran tests, and ranked results by Bayesian surprise, meaning the gap between what it expected and what the data actually showed.
That produced some intriguing leads. In one project, students saw a possible flaw in how battery aging is measured. In another, a polymer simulation turned up an apparent exception to a long-running rule about how atom charges are estimated. A third group used the tool to study light-sensitive proteins, while a fourth ran it on nuclear-reactor configurations and got a reminder that AI can be very confident and very wrong.
The setup came through the GenAI for Science: Ai2–UW Materials Challenge, organized by Luna Yue Huang, an associate teaching professor in materials science and engineering at UW. Twenty-five teams pitched datasets and research questions. Ten were picked for several weeks with AutoDiscovery, and in some cases instructors had to narrow the scope so the projects were manageable. The selection favored rich data, scientific distinctiveness, and a spread of topics.
What the class found was consistent: AutoDiscovery was useful for surfacing ideas that students might not have pursued on their own, but it didn’t replace the researcher. On synthetic reactor data, roughly half of its hypotheses were illogical and none were worth chasing. On real reactor measurements, it produced about 15 leads that might merit follow-up, but students still had to check plausibility, compare with the literature, and decide what was real, what was noise, and what was just a pretty accident.
The light-protein project showed the best side of the tool. AutoDiscovery found that successful insertions tended to anchor the light-sensitive segment at a few strong points rather than many weak ones, matching the group’s lab results and a long-established idea in protein binding. That’s the pattern here: AI can narrow the search, but it can’t tell you what matters. And in science, that’s the job.
My take — AI-written commentary, not fact-checked reporting
This is the sane version of AI in research: not magic, just a tireless intern with a shaky sense of significance. The real lesson is that better tools make expertise more valuable, not less, which is awkward for everyone selling a future where judgment comes preinstalled.
Read more about this at: Allen Institute (AI2)
