Are AI Models Working Harder Than They Need to?

Lizy K. John has a problem with how AI does math. Not the results—she says she's impressed by ChatGPT every time she uses it—but the sheer brute force behind it. Every answer a modern neural network spits out involves millions or billions of multiplications, weights getting multiplied against inputs, over and over. John, an electrical and computer engineering professor at the University of Texas at Austin, thinks that's overkill, and she's spent the last five years building an alternative called weightless neural networks that skip multiplication almost entirely.

The idea isn't hers originally. A commercial product used something similar in the UK back in the 1980s for pattern recognition before it faded away, and a small research group at the Federal University of Rio de Janeiro kept the technique alive quietly for years. John got pulled in through a friend who knew she had hardware chops, and paired her with a student already working on spiking neural networks. Within six months they had a working version running on an FPGA—no GPU required—and the networks came out roughly 1,000 times smaller than comparable models. That was the hook.

Instead of weights, these networks route binary inputs through interconnected lookup tables, essentially checking a stored answer rather than recomputing one. John argues the human brain doesn't multiply its way to an answer either, and does it all on about 20 watts. Her team's results back the pitch on small, specific problems: human-activity recognition, ECG and EEG monitoring, keyword spotting. The best existing model she compared against for one sensor task ran 17 megabytes; hers ran 14 kilobytes. For keyword spotting—the always-listening job before a device catches a wake word—industry models burn over 5,000 nanojoules per inference, while hers land between 42 and 79.

The practical upside isn't just efficiency for its own sake. Small enough to sit directly on a battery-powered sensor, these networks mean raw data doesn't need to stream constantly to a phone or server, which saves energy and keeps the data local—a privacy bonus nobody asked for but got anyway. John's group even built an arrhythmia detector on a bendable plastic substrate that only fits about 10,000 logic gates, something no conventional network could run on, since standard chips need billions of gates to do the same job.

Chatbots are the harder target, and John is upfront that she isn't there yet. Transformer models alternate between attention layers and multilayer perceptrons; her team has swapped out the perceptron half but hasn't touched attention. She doesn't think the rest of the AI stack needs to change to accommodate the approach—same data, potentially less of it, and GPUs could eventually give way to FPGAs, which already have small lookup tables built in. What's missing, by her own account, is scale and attention from the field. People are impressed when she presents the work, she says, but nobody's confident it scales, so few researchers are willing to bet on it over transformers that already work well enough.