Stanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 7: Dynamic Programming
Stanford Online · 1:15:15 · 2 days ago
Dynamic Programming (DP) enables the design of robust closed-loop control policies by decomposing complex control tasks into recursive sub-problems, allowing for computational efficiency by breaking down the decision-making process.
- Control strategies — Open-loop methods compute a fixed sequence of inputs for a nominal path, whereas closed-loop policies map any reachable state to an optimal action, providing robustness against external disturbances or model errors .
- Decomposition logic — The principle of optimality dictates that if an entire path is optimal, its final segment is also the optimal solution for the sub-problem starting at that intermediate point .
- Recursive execution — Solutions are calculated backward in time, starting from the final state where the goal or terminal cost is known and working toward the start, enabling the reuse of previously computed segments .
- Computational scaling — Designing policies becomes exponentially harder as the number of state variables increases, a limitation known as the curse of dimensionality .
- Linear simplification — In Linear Quadratic Regulator (LQR) problems with linear dynamics and quadratic costs, the recursive mathematics simplifies into a matrix update known as the Riccati equation, yielding a linear feedback controller .