Stanford CS229 Machine Learning | Spring 2026 | Lecture 5: Gaussian Discriminant Analysis
Stanford Online · 1:21:37 · 5 days ago
Generative models learn the underlying data distribution to classify, providing an alternative to discriminative approaches by estimating class probabilities via Bayes' Rule, often enabling closed-form parameter solutions instead of requiring iterative optimization.
-
Generative modeling objective — Generative algorithms learn the data distribution $P(X|Y)$ to classify input, differing from discriminative models that focus directly on the boundary $P(Y|X)$ .
-
Gaussian Discriminant Analysis (GDA) — This approach assumes input features follow a Gaussian distribution and requires learning distinct class means while sharing a single covariance matrix across all classes .
-
Parameter estimation — GDA parameters, including class priors and mean vectors, can be derived through closed-form maximum likelihood estimates rather than computationally intensive gradient descent .
-
Decision boundary geometry — Assuming shared covariance results in a linear decision boundary; allowing independent covariance for each class leads to Quadratic Discriminant Analysis, which creates nonlinear boundaries .
-
Model robustness — Logistic regression is a discriminative method requiring fewer assumptions than GDA, offering greater robustness, while generative methods remain valuable for modeling complex data generation processes .
-
Naive Bayes efficiency — This model handles discrete features by assuming conditional independence between variables, reducing the number of parameters from exponential to linear relative to the input dimension .