Nums AI Releases Causilo: A Tabular Foundation Model That Tops TabArena Among Single Models

Nums AI has put out Causilo, a pretrained foundation model for tabular data that handles classification and regression. The code is under Apache-2.0, the weights are on Hugging Face, and the model ships with a scikit-learn-style interface. That makes it easy to try. It also comes with a catch: research and evaluation use are fine on CUDA or CPU, but commercial, production and hosted API use need a separate license.

Causilo is built for in-context learning rather than traditional fine-tuning. When you call fit, it does not change the pretrained weights. Instead, it keeps the training rows as context and predicts query rows in a single forward pass. Nums AI says it pretrained the model only on synthetic data, with no TabArena datasets mixed in. It accepts NumPy arrays or pandas DataFrames, including categorical features and missing values. Classification goes up to 10 classes, while regression returns mean predictions by default.

The architecture is split into three stages: refinement, compression and in-context learning. Features are grouped in threes, each value gets 16 learned sine and cosine frequencies, and missing values have their own learned vector. Two column stages summarize the feature groups, with 128 latent slots reading only the training rows. Between those stages, a row stage lets feature groups interact through 4 latent tokens. Nums AI uses cross-attention there instead of full self-attention, which it says keeps cost linear in feature count. A pooling block compresses each row into a fixed 512-dimensional vector before labels are attached and query rows attend to the labeled training rows.

On TabArena, that setup paid off. Nums AI says Causilo has the highest Elo among single models for both classification and regression, and a TabArena maintainer reran the full evaluation and got the same overall Elo of 1794. With system entries included, that rerun placed Causilo 3rd of 88 overall. The margin is not huge, though: the confidence intervals at the top overlap, and TabFM still leads on improvability overall and on classification. Causilo leads on regression.

There’s more than one benchmark story here. On ScoringBench, Nums AI reports first place by CRPS, R² and RMSE after submitting Causilo 1.0.1 on 101 datasets. Speed-wise, its rerun on an H100 80 GB GPU showed it was fastest on both fit and predict among the three models tested, even though TabPFN-3 used much less GPU memory. Version 1.0.1 also adds median and quantile outputs, so regression intervals are available without extra plumbing.