about
Provisioning Networks for AI Supercomputers: A Trace-Driven Study of Performance (arxiv.org)
2 points by matt_d 25 minutes ago | hide | past | pdf | discuss on HN

In plain words: By replaying recorded communication patterns from AI jobs, the study estimates how network delay and speed change runtime, at scales up to 4,096 GPUs. The maps reveal sharp cliffs that differ by workload, and show when extra bandwidth pays off versus when it barely helps.

Abstract · Provisioning Networks for AI Supercomputers: A Trace-Driven Study of Performance Sensitivity at Unprecedented Scale

AI workloads on supercomputers make interconnect performance a primary design constraint. Hardware specifications and microbenchmarks alone do not show how real workloads respond to latency, bandwidth, and jitter. We present a trace-driven study of network sensitivity for training and inference workloads at scales up to 4,096 GPUs. Our methodology combines signature-based trace decomposition with linear programming to make broad network-parameter sweeps feasible. It preserves the recorded NCCL schedules and program dependencies, and estimates a lower bound on runtime under the LogGP model. We validate the compositional analysis against monolithic linear programs, LogGOPSim, and hardware measurements at feasible scales. The resulting latency-bandwidth envelopes identify low-sensitivity regions and performance cliffs that vary substantially across workloads. They also show when bandwidth upgrades provide meaningful returns and when additional network performance has little effect. These results translate execution traces into practical guidance for evaluating network configurations while making the model assumptions explicit.

Tommaso Bonato, Amirreza Rastegari, Daniele De Sensi, Timo Schneider, Siyuan Shen, Ales Kubicek, Prabhat Ram, Torsten Hoefler
arXiv:2610.41700 · cs.NI · submitted Oct 30, 2026
abstract · pdf · html · To appear in the proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC 2026)

add comment on HN