Beyond the KV cache: Training and serving i14’s Laplace 3B linear attention model on AWS Trainium2 with native PyTorch
Amazon Web Services Kevin Shabahang
i14’s Laplace 3B linear attention model was trained and served on AWS Trainium2 using native PyTorch via TorchNeuron Native, with i14 adapting its Simple GLA architecture for the accelerator. The model keeps a recurrent state that stays at 29.5 MB per sequence regardless of context length. This replaces the growing KV-cache behavior with fixed-size serving state and enables the reported training and concurrent inference performance on Trainium2.
Why it matters
Learn how i14 trained and served Laplace 3B, a linear attention model, on AWS Trainium2 using native PyTorch. Its recurrent state stays fixed at 29.5 MB per sequence regardless of context length, avoiding the growing key-value cache of transformer models and keeping serving memory predictable.
Related stories
AWS Trainium Frontier competition: Co-design models and kernels on purpose-built AI chips
Amazon Science · 2 months ago ·
11