Train Qwen3 on AWS Trainium with torchtitan
Amazon Web Services Josh Longenecker ● Covered by 5 sources
TorchNeuron registers AWS Trainium as a native PyTorch device so TorchTitan can train and fine-tune Qwen3 via standard PyTorch interfaces in three configurations.
Why it matters
Train Qwen3 on AWS Trainium with torchtitan three ways on a single trn2.48xlarge: the Hugging Face Transformers v5 modeling backend, torchtitan's native Qwen3, and NKI mixture-of-experts kernels. The same torchrun launcher and config structure work unchanged across all three paths, including loading and saving published Hugging Face safetensors.
Qwen3
TorchNeuron
TorchTitan
AWS Trainium
Hugging Face Transformers v5
NKI mixture-of-experts kernels
torchrun
Related stories
Understanding and Implementing Qwen3 From Scratch
Ahead of AI · 1 year ago ·
40
Profiling TorchNeuron (Part 1): System-level profiling on AWS Trainium.
Hugging Face · 6 hours ago ·
47
Beyond the KV cache: Training and serving i14’s Laplace 3B linear attention model on AWS Trainium2 with native PyTorch
Amazon Web Services · 1 day ago ·
48