🔬“We have foundation models for language, not for physics” — Anima Anandkumar, Bren Professor of Computing

A few years ago, Caltech professor Anima Anandkumar went looking for an open-source weather model built with AI and met a wall of doubt. Weather is chaotic, the physics is hard, and the usual answer has long been more simulation and more supercomputer time. She ignored the skepticism and built anyway.

Within a year, her team had FourCastNet, a model that matches the best physics-based simulations available. That work, and what came after it, made short-term weather prediction possible on consumer-grade GPUs. For a field that has leaned on massive infrastructure for decades, that is a sharp change in who gets to do the forecasting.

The bigger story is that this kind of science does not behave like language modeling. The source points out that datasets in these domains are often only tens or hundreds of thousands of examples, and the context lengths physics would demand can shoot into the hundreds of billions. So the usual “just scale it up” playbook hits a wall fast. The answer, Anandkumar argues through her work, is structure: inductive biases, physical priors, and models built around the math of the problem.

That led her to Neural Operators, which model functions across scales instead of fixed grids. For global weather, that means using the right basis for the Earth’s shape, not forcing the planet onto a brittle grid. Her Fourier Neural Operator works in the frequency domain, and the spherical version powers FourCastNet 3, which stays stable far into the future.

And the same idea seems to travel. The source says that in fusion, a few thousand samples can be enough to predict plasma disruptions, and can do it a million times faster than traditional simulation. Anandkumar’s goal is still broader: a foundation model for physics that can handle simulation and design across many phenomena. Not by waiting for more data, but by building the rules in from the start.