about
The Little Book of Generative AI Foundations: An Intuitive Mathematical Primer (arxiv.org)
194 points by ibobev 3 hours ago | hide | past | pdf | 31 comments on HN

In plain words: A short textbook that teaches the math behind generative AI by building each idea step by step, from simple data compression to today's image and text generators. It shows how these families grow from the same core ideas, keeping the equations but no code.

Abstract

This book provides a compact, derivation-oriented introduction to the mathematical foundations of modern generative artificial intelligence. Rather than surveying every recent architecture or implementation detail, it develops a coherent route through the ideas connecting major families of generative models, from PCA, probabilistic PCA, variational autoencoders, and diffusion models to normalising flows, autoregressive factorisations, GANs, Wasserstein GANs, and energy-based models. The aim is to make the structure of generative modelling more accessible without removing the mathematical substance needed to understand how these models are derived and related. The book is intended as a foundation-building primer for mathematically curious researchers, practitioners, and students.

Tianhua Chen
arXiv:2605.29713 · cs.LG, cs.AI · submitted May 28, 2026 · updated Sep 8, 2026
abstract · pdf · html · Revised preprint version, 195 pages. Added visual explanations throughout. Comments and corrections are welcome

add comment on HN

My favorite little book is François Fleuret's: The Little Book of Deep Learning https://fleuret.org/francois/lbdl.html
I love how that site offers a phone-formatted PDF as an option. How thoughtful!
You may enjoy the reflow feature on Koreader. It fits a pdfs content into a screenwidth single column layout. That text is also zoomable.
i got duo to increase my reading.
I'd note that the Fleuret books gives the foundation for basic deep learning whereas the OP seems to give the necessary foundations for Transformers and related machinery. These later addition what is now "AI" are in total at least as complicated as the original deep learning concepts. Not that just learning basic neural networks but the two book don't appear to cover the same ground.
It's a nice treatment of the subject, but I wish he would post a conventionally-formatted .PDF that puts every page on a full sheet of standard printer paper.
I immediately wanted the same thing, so I made it. The book is licensed Creative Commons BY-NC-SA 4.0 International License, so here you go.

https://cdn.filepost.dev/file/filepost/uploads/15/1578f21219...

Thanks!
Wow, this book is surprisingly readable! I like how he starts from basics of linear algebra.
Someone should add it to the hacker news curated paper list! Along side the video and book list.
Appreciated, I drop all of these into a RAG folder for when I need them later on in projects.

8Gb of whitepapers/books...

I'm v curious how you're incorporating this into your workflow. Is it that when you want to implement something you can call out to these books as rigorous reference material for your agents? I'd never thought of doing such a thing and am curious how you're doing it and how it's worked.
Can you share what you have so far in that folder?
They spell it out quite clearly: "8Gb of whitepapers/books..."
Interesting!

Is there any one who have recommended books on the attention architecture specifically? I have been interested in that for a long time but never gotten around to actually studying it.

Not a book, but 3blue1brown's video on attention is pretty great: https://www.youtube.com/watch?v=eMlx5fFNoYc
formal algorithms for transformers provides perhaps the most succinct introduction to attention I can imagine https://arxiv.org/pdf/2207.09238
Are there any deep learning books with some problems to solve ?