Meta has published the full specification, reference software implementation, and compliance test suite for MetaRoCE, a clean-sheet RDMA transport protocol designed for AI workloads on commodity Ethernet. Released through the Open Compute Project (OCP), this initiative marks a paradigm shift: instead of demanding lossless networks, MetaRoCE treats packet loss as the norm and moves intelligence from switches to NICs.

Add AlexTech.ai asPreferred Source on Google

Edge Intelligence, Not Fabric Control

The core insight of MetaRoCE is that "the fabric sees packets, but the NIC sees intent." Unlike standard RoCE which relies on PFC and in-order delivery, MetaRoCE decomposes the network into fine-grained logical paths with real-time telemetry (RTT, ECN state, utilization) per path. This enables native out-of-order delivery: packets are written directly to their final memory location without reorder buffers, eliminating head-of-line blocking.

[PDF] RDMA over Ethernet for Distributed Training at Meta Scale ... — https://www.semanticscholar.org/paper/RDMA-over-Ethernet-for-Distributed-Training-at-Meta-Gangidi-Miao/ec1dca2ac16ce685229cb58c90e07d049d6d5e7a

Loss Tolerance and Dynamic Multipathing

The protocol requires no pause frames or credit-based flow control. Using a 256-bit SACK bitvector per path, MetaRoCE distinguishes loss from reordering and retransmits only missing packets on the specific path that lost them. Multipathing is managed by the NIC, which can change UDP source ports to steer traffic away from congested routes. In case of a plane failure, redistribution happens autonomously without operator intervention.

Performance Validated on AMD Hardware

For validation, Meta collaborated with AMD to implement the protocol on Pensando programmable NICs. On a 64-node GPU cluster, tests showed MetaRoCE outperforming RoCEv2 in throughput and flow completion times. The most significant data point is resilience: at 1% packet loss, MetaRoCE maintains approximately 86% throughput, while RoCEv2 degrades significantly. Even at 10% loss, the protocol continues to deliver useful bandwidth by converging gracefully.

Open Standard for AI Ecosystems

Meta's choice aligns with its strategy to make Ethernet the standard fabric for AI infrastructure, a path already explored with the MTIA 300 chip and in contrast to proprietary architectures like NVIDIA's Spectrum-X Ethernet. MetaRoCE requires only features already present in switches (ECN and ECMP), making it compatible with fat-tree, multiplane topologies, and third-party clouds. Opening the specification via OCP aims to avoid vendor lock-in, allowing any NIC manufacturer to implement the protocol and contribute to standardizing giga-scale AI networking.