Stanford CS229 Machine Learning | Spring 2026 | Lecture 10: GMM (EM), PCA

The lecture outlines how to use the Expectation-Maximization (EM) algorithm to identify hidden structures in data and Principal Component Analysis (PCA) to reduce dataset dimensions while retaining the most meaningful information.

• EM purpose — This method manages datasets with hidden variables by iteratively guessing assignments and adjusting parameters to fit those guesses [11:08].
• Clustering differences — Unlike hard-assignment methods like K-means, this approach uses soft probabilities to determine how much a point belongs to a cluster [02:16].
• Mathematical tool — The algorithm uses Jensen’s inequality to create a lower-bound curve that acts as a manageable surrogate for the more complex original function [16:07].
• Iterative cycle — The process alternates between estimating hidden structures (E-step) and updating model parameters (M-step) until it reaches a local peak [34:21].
• PCA function — This technique reduces high-dimensional data into a smaller number of variables by identifying the axes that capture the most variance [53:31].
• Data preparation — Input data must undergo centering, which shifts the mean to zero, and scaling, which ensures different features contribute fairly to the variance calculation [51:41].
• Underlying mechanics — Finding these primary directions is mathematically identical to finding the eigenvectors of the data's covariance matrix [1:11:01].

How does the E-step differ from hard clustering assignments? Why is centering and scaling necessary before performing PCA?