Stanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 10: Reachibility Analysis

This lecture covers the transition from discrete-time optimal control to continuous-time differential games, providing the mathematical foundation for analyzing safety and reachability in autonomous systems.

• Infinite horizon methods — Value iteration and policy iteration provide the primary framework for solving infinite horizon problems by computing optimal value functions through successive approximations [05:54].
• Continuous dynamics — Optimal control in continuous time moves from discrete summations to integrals, representing the cost function as a differential evolution [26:07].
• Adversarial modeling — Introducing a second player—acting as a disturbance—converts the control problem into a differential game where the controller aims to maximize a reward while the disturbance acts to minimize it [27:28].
• Mathematical solutions — Solving for the optimal value function requires the Hamilton-Jacobi-Isaacs (HJI) equation, which incorporates a minimax structure to account for competing interests [57:04].
• Safety guarantees — Reachability analysis applies these differential game frameworks to compute sets of states where a system is guaranteed to reach a goal or safely avoid an obstacle despite interference [1:11:52].

How does the HJI equation differ from the standard Bellman equation in a non-adversarial setting? How do planners use backward reachable sets to ensure safety in autonomous vehicles?