Data Machina #256

Substack · 2 years ago · 40

State Space Models (SSMs) are emerging as an alternative to Transformers for sequence modeling, with recent developments including Mamba-2 and applications in time-series forecasting, voice generation, and audio representation learning. Key advances include Chimera achieving superior performance on time-series benchmarks, Cartesian.ai's Sonic voice model using SSM inference for low latency, and Audio Mamba outperforming transformer baselines on self-supervised audio tasks. However, researchers at AllenAI and NYU argue that SSMs lack fundamental advantages over transformers for state tracking, suggesting their practical superiority may be limited despite recent optimism.