about
The Modern Mathematics of Deep Learning (arxiv.org)
23 points by MAXPOOL on Jun 11, 2021 | hide | past | pdf | 2 comments on HN

In plain words: A survey collects new mathematical tools that explain deep learning puzzles classical learning theory cannot, such as why huge networks generalize, why training finds good solutions despite many traps, and what depth adds. The tools give partial answers to these questions.

Abstract

We describe the new field of mathematical analysis of deep learning. This field emerged around a list of research questions that were not answered within the classical framework of learning theory. These questions concern: the outstanding generalization power of overparametrized neural networks, the role of depth in deep architectures, the apparent absence of the curse of dimensionality, the surprisingly successful optimization performance despite the non-convexity of the problem, understanding what features are learned, why deep architectures perform exceptionally well in physical problems, and which fine aspects of an architecture affect the behavior of a learning task in which way. We present an overview of modern approaches that yield partial answers to these questions. For selected approaches, we describe the main ideas in more detail.

Julius Berner, Philipp Grohs, Gitta Kutyniok, Philipp Petersen
arXiv:2105.04026 · cs.LG, stat.ML · submitted May 9, 2021 · updated Feb 8, 2023
abstract · pdf · html · A version of this review paper appears as a chapter in the book "Mathematical Aspects of Deep Learning" by Cambridge University Press

add comment on HN
Also discussed: Jun 2023 (3 points, 0 comments) · Jun 2021 (276 points, 70 comments) · May 2021 (2 points, 0 comments)

Wow, this looks amazing. There has been a lot of development in this area recently, but without expertise in various areas (functional analysis, statistical physics, manifold theory), reading the papers themselves is difficult if not impossible.

I'm glad someone is taking the time to summarize and integrate the results of these various approaches.

Rather nice presentation, a chapter from the forthcoming "Deep Learning" to be published by CUP.