Google Open-Sources TorchTPU, a Native PyTorch Backend for Cloud TPUs

Google has opened up TorchTPU, a native PyTorch backend for Cloud TPUs that it says was co-developed with the PyTorch team at Meta. The launch landed at PyTorch Conference 2026 on October 20, and Google is already steering new workloads toward it instead of PyTorch/XLA.

The key idea is simple: TPUs now sit alongside GPUs and CPUs as a standard PyTorch device. Existing code can point at a TPU with torch.device("tpu"), and TorchTPU plugs in through PyTorch’s PrivateUse1 interface. That means ordinary PyTorch tensors, no wrapper classes, no weird detours through lazy-tensor graphs. It also keeps tensor views and in-place mutations intact.

Google ships TorchTPU with three eager modes. Strict is the default, Debug adds synchronous execution with bounds checking, and DeferAndFuse batches eager ops to cut HBM traffic. For production, torch.compile(backend="tpu") uses TorchDynamo, then StableHLO, then OpenXLA to emit TPU machine code. Compiled executables are cached across processes, hosts and restarts, which is the sort of thing people notice when jobs are large and expensive.

The stack is not tiny. Google lists four open repos around it: TorchTPU, vLLM-TorchTPU, SGLang-TorchTPU and TorchTitan. It runs on Google Cloud TPUs, has been tested from v5e up through 8,192 Ironwood chips, and supports FP8, MXFP4 and NVFP4 checkpoints for serving. Google also says the system handles DDP, FSDP2 and DTensor, plus both SPMD and MPMD execution.

The performance claims are self-reported, but they are hard to ignore. Google says TorchTitan reached 52% MFU on Qwen 3 8B on Ironwood, and that a fine-tuning step dropped from 11.49 seconds to 2.78 seconds once the stack was tuned. It also says DeferAndFuse alone gave 1.15x and full torch.compile gave 3.80x in that case. Meanwhile, Google says it continuously tests about 5,000 Hugging Face models in eager and compiled modes.