Stanford CS229 Machine Learning | Spring 2026 | Lecture 11: Diffusion Models
Stanford Online · 1:22:30 · 4 days ago
Diffusion models generate realistic data, such as images, by learning to reverse a multi-step process that gradually transforms clean data into random Gaussian noise.
-
Core mechanism — Models function by learning to reverse a sequence that systematically adds Gaussian noise to clean data until the input becomes indistinguishable from random static .
-
Forward process — The corruption phase is fixed and requires no machine learning; it simply adds noise incrementally according to a set schedule .
-
Reverse process — Neural networks are trained to predict a cleaner state from a noisy one, effectively "denoising" the data one step at a time .
-
Learning objective — Optimization relies on the Evidence Lower Bound (ELBO), a mathematical tool used to maximize the probability of the original data when calculating the direct likelihood is computationally impossible .
-
Gaussian assumptions — The use of Gaussian distributions for these transitions is based on:
- Theoretical results from stochastic processes, which suggest the reverse of continuous noise addition behaves like a Gaussian distribution .
- Practical math requirements, which allow for a closed-form solution when calculating the loss .
-
Loss calculation — Training reduces to minimizing the gap between the predicted noise and the actual noise added at each step, calculated as the squared difference of the means .
-
How does the choice of the noise schedule affect the quality of the generated data?