about
Lecture Notes on Statistical Physics and Neural Networks (arxiv.org)
3 points by Anon84 135 days ago | hide | past | pdf | discuss on HN

In plain words: These notes teach statistical physics as plain probability theory, using simple magnetic models to explain ideas like phase transitions without any physics background. The same energy rules describe memory networks and learning machines, where hiding neurons works like the renormalization group.

Abstract

These lecture notes introduce some topics of classical statistical physics, particularly those that are relevant for neural networks and deep learning. Statistical physics is treated as a branch of probability theory or statistics, with the goal of making concepts such as phase transitions and the renormalization group accessible to readers without prior knowledge of physics. We introduce the Boltzmann-Gibbs distribution and the thermodynamic potentials on a finite configuration space, notably for Ising spins and spin-glass models on a lattice, and then define phase transitions as discontinuities that arise in the limit that the number of lattice points goes to infinity. We further introduce Hopfield networks and Boltzmann machines, which are governed by the same energy function as spin-glass models, and discuss the learning algorithm for restricted Boltzmann machines. In this algorithm hidden neurons are integrated out as in the renormalization group. Finally, modern deep learning is introduced, whose early developments were in part motivated by restricted Boltzmann machines in that they carry many layers of hidden neurons. A description of large language models is given.

Olaf Hohm
arXiv:2605.06394 · cond-mat.dis-nn, cs.LG, hep-th · submitted May 7, 2026
abstract · pdf · html · 56 pages, 7 figures, based on a course given at Humboldt University Berlin

add comment on HN