AStanford AA203 Optimal and Learning-Based Control | Spring 2026 | Lecture 8: Nonlinearity

LQR-based techniques provide a versatile framework for optimal control that extends beyond simple state regulation to handle trajectory tracking and iterative trajectory optimization. This is achieved by reformulating problems into deviation variables, linearizing nonlinear dynamics, and applying iterative updates to refine control sequences.

• LQR utility — The framework allows for the translation of complex control tasks into standardized problems where the goal is to optimize a cost function relative to a target trajectory [04:12].
• Trajectory tracking — The process uses local models to force the system to follow a reference path:
  • Defining deviation variables as the difference between the actual system trajectory and a precomputed reference path [24:21].
  • Linearizing the dynamics around the reference path to create a time-varying linear model [35:43].
  • Combining a feed-forward term from the nominal control with a closed-loop feedback term to maintain the system on the desired path [28:46].
• Iterative LQR (iLQR) — This algorithm computes optimal open-loop trajectories by refining a control sequence through successive approximations [55:27]:
  • The backward pass solves Riccati-like equations to compute optimal deviation controls [57:10].
  • The forward pass propagates the system states using the true nonlinear dynamics to generate a new reference trajectory [58:00].
• Constraint management — Unlike Successive Convex Programming (SCP), iLQR does not natively handle hard constraints on states or controls, making it less robust in highly constrained environments [1:01:22].
• Differential Dynamic Programming (DDP) — This method extends iLQR by directly approximating the Bellman equation using second-order derivatives of both the system dynamics and the cost function [1:12:31].

How does the backward pass in DDP differ from the one used in iLQR regarding the treatment of dynamics? In what scenarios might SCP be preferred over iLQR despite the computational simplicity of the latter?