Meta Brings Networking Into the Chip to Speed Up Recommendation Model Training

Meta has unveiled the MTIA 300, a custom training accelerator that tackles a fundamentally different problem than the large language models that dominate AI hardware design. While LLM training typically demands raw compute power, recommendation models are constrained by something else: the sheer volume of data moving between chips.

The Communication Bottleneck#

Recommendation models store massive embedding tables that account for over 99% of their parameters. When training these systems across hundreds of accelerators, the constant shuffle of data through collective operations—AllReduce, AllToAll, AllGather—can overwhelm conventional network infrastructure. At scale, moving data becomes as critical as the compute itself.

Networking Baked Into Hardware#

Meta's response was to move the network interface card directly into the accelerator package. MTIA 300 contains two network chiplets with six custom 800 Gbps RDMA NICs each, delivering 1.2 TB/s of total bandwidth without requiring PCIe bus traffic. These twelve NICs flexibly support both intra-rack communication and inter-rack scaling, letting Meta adjust bandwidth allocation on the fly without redesigning the chip architecture.


Beyond proximity, MTIA 300 adds 16 dedicated message engines that handle all collective communication independent of the main compute grid, complete with specialized hardware for reduction operations. This separation means large matrix calculations and network traffic can overlap with less than 0.5% compute degradation—compared to 20% degradation on GPUs when the same operations run concurrently.

Software-Hardware Co-Design#

The accelerator was built hand-in-hand with HCCL, Meta's collective communication library. Rather than having the host CPU orchestrate every message during training, HCCL compiles communication operations into autonomous subgraphs that MTIA 300 can execute directly. Once instructions reach the accelerator, the CPU steps out of the loop.

In practice, HCCL achieves up to 940 GB/s of communication throughput within a single rack. On a 150-billion-parameter recommendation model running across 40 accelerators, Meta reports that MTIA 300 slashed total communication time by 3.9x versus a comparable GPU cluster.