about
HCCL: Collective Communication for Meta Training and Inference Accelerators (arxiv.org)
1 point by matt_d 55 days ago | hide | past | pdf | discuss on HN

In plain words: A new AI chip for training and inference puts networking and helper processors on the chip package, so data-sharing between chips runs off the compute grid while it keeps working. The software reaches 940 GB/s within a rack while slowing computation by under 0.5%.

Abstract

We present HCCL, a collective communication library co-designed with Meta's MTIA 300 accelerator, the first Meta chip to integrate backend networking directly on chip package. MTIA 300 includes dedicated message engines (MEs) with near-memory compute (NMC) that fully offload collective execution from the compute grid, enabling large overlap between computation and communication. HCCL uses a compiled communication model in which the host generates a complete description of each collective including dependencies. We describe the control and data path architecture, topology-aware algorithm selection across MTIA 300's asymmetric scale-up and scale-out network, and optimizations for both training and inference workloads. For training, HCCL achieves up to 940 GB/s on intra-rack collectives while introducing less than 0.5% degradation to concurrent compute throughput. For inference, we leverage one-sided communication primitives that bypass the scheduling path to minimize collective latency and describe collective designs that improve compute-communication pipelining for latency-sensitive workloads.

Wesley Bland, Tiago Antunes, Lars Paul Huse, Chidambaram Muthu, Adel Abouchaev, Rabib Alam, Abdullah Alperen, Alexey Andronov, Jose Anto Akkara, Vineet Badhwar, Pavan Balaji, Daniel Berkovitch, et al.
arXiv:2608.00358 · cs.NI, cs.DC · submitted Aug 1, 2026
abstract · pdf · html · 12 pages, 17 figures, to be published in the proceedings of "SC '26: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis"

add comment on HN