Data Machina #252

Time-series forecasting has always been statisticians' turf, all ARIMA and exponential smoothing, but deep learning is muscling in and starting to hold its own. The catch is that the neural approaches have carried real baggage: architectures too complex for their own good, training runs that eat compute for breakfast, inference bills that add up fast, and a nasty habit of missing context that a human forecaster would catch instantly. This week's Data Machina rounds up a wave of research trying to fix exactly that, borrowing tricks from NLP, computer vision, and physics to build leaner, smarter time-series models.

A new survey digs into what the authors call the 3Es — effectiveness, efficiency, explainability — comparing foundation models trained from scratch for time series against the approach of just retrofitting big language models for the job. It comes bundled with a GitHub repo stacked with papers and code, which is more useful than another PDF nobody reads. A parallel survey covers diffusion models applied to time series, spatiotemporal data, and tabular data, complete with a taxonomy that at least tries to organize what's become a sprawling and fast-moving subfield.

The practical stuff is where it gets interesting. IBM Research open-sourced TinyTimeMixers, a family of pre-trained forecasting models that come in under 1 million parameters — tiny by any modern standard, and notable simply because nobody had really pushed compactness this hard before in this space. MambaFormer takes a different tack, fusing Mamba's state-space design (good at long-range dependencies) with a Transformer (good at short-range ones) into a single model that its creators claim beats both parents at long-short range forecasting. Google's TimesFM went bigger instead of smaller, pre-trained on 100 billion real-world time points and showing strong zero-shot results across benchmarks it was never explicitly trained on.

Then there's DeepKAN, which swaps the usual MLP layers for Kolmogorov-Arnold Networks, a mathematically flavored architecture getting attention across the field right now. The team reports KANs beating MLP-based methods on the classic Air Passengers dataset — though that dataset is small and pretty tame, so it's more a proof of concept than a knockout result. Taken together, none of this dethrones classical statistics outright, but it's a clear signal that the toolbox for forecasting is about to get a lot more crowded.