about
Dissecting the Graphcore IPU Architecture via Microbenchmarking (arxiv.org)
1 point by ingve on Dec 11, 2019 | hide | past | pdf | discuss on HN

In plain words: Small hand-written tests measure the Graphcore IPU's memory, data-sharing speed, and math speed for AI work. They show how the chip's design shapes speed versus its theoretical limits, and give simple rules to predict a program's run time from its compute and communication steps.

Abstract

This report focuses on the architecture and performance of the Intelligence Processing Unit (IPU), a novel, massively parallel platform recently introduced by Graphcore and aimed at Artificial Intelligence/Machine Learning (AI/ML) workloads. We dissect the IPU's performance behavior using microbenchmarks that we crafted for the purpose. We study the IPU's memory organization and performance. We study the latency and bandwidth that the on-chip and off-chip interconnects offer, both in point-to-point transfers and in a spectrum of collective operations, under diverse loads. We evaluate the IPU's compute power over matrix multiplication, convolution, and AI/ML primitives. We discuss actual performance in comparison with its theoretical limits. Our findings reveal how the IPU's architectural design affects its performance. Moreover, they offer simple mental models to predict an application's performance on the IPU, on the basis of the computation and communication steps it involves. This report is the natural extension to a novel architecture of a continuing effort of ours that focuses on the microbenchmark-based discovery of massively parallel architectures.

Zhe Jia, Blake Tillman, Marco Maggioni, Daniele Paolo Scarpazza
arXiv:1912.03413 · cs.DC, cs.AR, cs.PF · submitted Dec 7, 2019
abstract · pdf · html · 91 pages, 21 figures

add comment on HN
Also discussed: Jan 2021 (1 point, 0 comments) · Jan 2020 (3 points, 0 comments) · Dec 2019 (1 point, 0 comments) · Dec 2019 (1 point, 0 comments) · Dec 2019 (2 points, 0 comments)