DiScoFormer: One transformer for density and score, across distributions
Allen Institute (AI2) ● Covered by 2 sources
Researchers introduced DiScoFormer, a transformer-based model that estimates both the probability density and score (gradient of log-density) of a distribution from data in a single forward pass without retraining. In 100 dimensions, DiScoFormer achieved 6.5x lower score error and 37x lower density error compared to kernel density estimation while continuing to improve with more samples. The model generalizes across different distribution types and could reduce computational costs for generative modeling, Bayesian inference, and scientific computing by serving as a reusable estimator.
Why it matters
DiScoFormer is a transformer-based density and score estimator that can infer both quantities from a finite sample in one forward pass, generalizing classical KDE while staying accurate in high-dimensional and out-of-distribution settings without retraining for each new distribution.
