about
Modern Methods in Associative Memory (arxiv.org)
5 points by liamdgray on Aug 10, 2025 | hide | past | pdf | 1 comment on HN

In plain words: A tutorial explains how associative memories like Hopfield networks store and retrieve patterns in fully connected networks, with modern math and hands-on coding notebooks. It shows how recent theory connects these memories to today's leading AI systems, and how new formulations guide architecture design.

Abstract

Associative Memories like the famous Hopfield Networks are elegant models for describing fully recurrent neural networks whose fundamental job is to store and retrieve information. In the past few years they experienced a surge of interest due to novel theoretical results pertaining to their information storage capabilities, and their relationship with SOTA AI architectures, such as Transformers and Diffusion Models. These connections open up possibilities for interpreting the computation of traditional AI networks through the theoretical lens of Associative Memories. Additionally, novel Lagrangian formulations of these networks make it possible to design powerful distributed models that learn useful representations and inform the design of novel architectures. This tutorial provides an approachable introduction to Associative Memories, emphasizing the modern language and methods used in this area of research, with practical hands-on mathematical derivations and coding notebooks.

Dmitry Krotov, Benjamin Hoover, Parikshit Ram, Bao Pham
arXiv:2507.06211 · cs.LG · submitted Jul 8, 2025 · updated Oct 3, 2025
abstract · pdf · html · Tutorial at ICML 2025

add comment on HN

Abstract: "Associative Memories like the famous Hopfield Networks are elegant models for describing fully recurrent neural networks whose fundamental job is to store and retrieve information. In the past few years they experienced a surge of interest due to novel theoretical results pertaining to their information storage capabilities, and their relationship with SOTA AI architectures, such as Transformers and Diffusion Models. These connections open up possibilities for interpreting the computation of traditional AI networks through the theoretical lens of Associative Memories. Additionally, novel Lagrangian formulations of these networks make it possible to design powerful distributed models that learn useful representations and inform the design of novel architectures. This tutorial provides an approachable introduction to Associative Memories, emphasizing the modern language and methods used in this area of research, with practical hands-on mathematical derivations and coding notebooks."