Meta AI Introduces MetaRoCE: A Clean-Sheet RDMA Transport Built for AI-Scale Ethernet

Meta is trying to fix a problem that only shows up when clusters get absurdly large: the network becomes part of the training loop. When thousands of accelerators are tied together for all-reduce and all-to-all work, one slow transfer can hold back the whole job. That’s the backdrop for MetaRoCE, the company’s new RDMA transport for AI workloads on commodity Ethernet.

The pitch is simple, but the engineering shift is not. Standard RoCE assumes the network should hand every frame over in order, with pause-based mechanisms doing a lot of the heavy lifting. MetaRoCE throws that assumption out. It treats the fabric as lossy and moves ordering, path choice, and recovery into the NIC itself. Packets can spray across many routes and arrive out of order by design, while the NIC writes each one directly to its final memory location.

That matters because Meta says this is meant for fabrics that are already complicated, including multi-plane setups and large-scale networks where spraying traffic is part of the point. In MetaRoCE, each path carries its own telemetry — RTT, ECN state, utilization — and its own window estimate. Loss is handled with selective acknowledgments and retransmission of only the missing packet, rather than pretending the network will never drop anything. Congestion control comes from both ends: the sender uses ECN-based AIMD, and the receiver feeds back a fair-share rate hint in every acknowledgment.

Meta is also trying to make the transport less tied to a single kind of network gear. The design asks the fabric for ECN marking and ECMP, and not much else. No pause frames, no credit-based flow control, no switch-side spraying. Meta says that makes it usable even on vendor clouds where the operator doesn’t control the switch configuration. It also collapses connection state: one connection can carry many ordered streams above and many paths below, instead of forcing applications to open lots of queue pairs.

There are numbers behind the claim. Meta implemented MetaRoCE on AMD Pensando programmable NICs and tested it on a 64-node AMD GPU cluster running RCCL collectives. Against RoCEv2, it delivered higher throughput and lower flow completion times on all-reduce and all-to-all. Meta also says it held about 86% throughput at 1% packet loss and kept useful bandwidth even at 10% loss, while 4-plane and 8-plane tests with up to 4,000 concurrent connections scaled linearly with plane count. The spec, a DPDK-optimized reference implementation, and a compliance test suite are slated to come through the Open Compute Project at the 2026 OCP Global Summit, with hardware support still early and more vendor implementations in progress.