New course teaches how AI models run on modern accelerators

Most AI engineers still live in PyTorch and think in terms of models, losses, and latency. IBM Research is trying to push them one layer deeper. Its new Coursera specialization, Dataflow Architectures for AI: From Principles to Practice, is built around the messy middle between framework code and silicon, where compilers, memory placement, and scheduling decide whether a model flies or crawls.

The six-course program is taught by Kaoutar El Maghraoui, a principal research scientist at IBM Research and an adjunct professor at Columbia University. It starts with one PyTorch model and then shows how that same nn.Module gets turned into very different execution plans on five systems: GPUs, Google TPUs, Cerebras’ Wafer-Scale Engine, Groq’s Language Processing Unit, and IBM’s Spyre accelerator. The point is not that one system is secretly the winner. It is that each one forces different tradeoffs, and the compiler becomes the place where those tradeoffs get baked in.

That is also why the course spends so much time on compilation and memory. On a GPU, the model runs as kernels across many threads. On TPUs, XLA steers work onto systolic arrays. Cerebras spreads compute across a wafer with more than 40 GB of on-chip SRAM. Groq leans on its compiler to decide when each operation runs and when data moves. Spyre compiles the model into a dataflow graph, with layout and scratchpad placement handled by the compiler and transfers launched by the runtime. A tensor shape that looks harmless on one target can require padding or rearrangement on another, and a fused kernel that helps one system can make another memory-hungry.

The syllabus reads like a practical map of where things go wrong. One course covers control-flow and dataflow execution, another compilation, another memory hierarchies, then debugging, then performance modeling. Students work through an eight-layer failure taxonomy that runs from graph capture errors to DMA timeouts and numerical issues. They also learn to separate kernel time from transfer time, compare measured performance with a compiler’s ideal, and put the remainder on a Roofline chart. IBM even cites a torch-spyre bug where cloning a 65-element fp16 tensor returned the wrong last value because the compiled copy only covered the first 64 elements.

Each course ends with a lab. The core labs run on Google Colab with standard PyTorch, and there is an optional Spyre practice portion for partnered programs. Students build a benchmarking harness, trace transformer blocks, compare eager mode with torch.compile, inspect memory timelines, diagnose planted bugs, and build a Roofline chart for an LLM workload. The specialization is aimed at intermediate to advanced learners, and completion earns a professional certificate in AI Systems Performance Engineering.