Dynamic programming is a powerful method for solving optimization problems, but has a number of drawbacks that limit its use to solving problems of very low dimension. To overcome these limitations, author Rein Luus suggested using it in an iterative fashion. Although this method required vast computer resources, modifications to his original scheme have made the computational procedure feasible.
With iteration, dynamic programming becomes an effective optimization procedure for very high-dimensional optimal control problems and has demonstrated applicability to singular control problems. Recently, iterative dynamic programming (IDP) has been refined to handle inequality state constraints and noncontinuous functions.
Iterative Dynamic Programming offers a comprehensive presentation of this powerful tool. It brings together the results of work carried out by the author and others - previously available only in scattered journal articles - along with the insight that led to its development. The author provides the necessary background, examines the effects of the parameters involved, and clearly illustrates IDP's advantages.
Fundamental Definitions and Notation
Steady-State System Model
Continuous-Time System Model
Discrete-Time System Model
The Performance Index
Interpretation of Results
Examples of Systems for Optimal Control
Solving Algebraic Equations
Solving Ordinary Differential Equations
STEADY-STATE OPTIMIZATION
Linear Programming
LJ Optimization Procedure
References
DYNAMIC PROGRAMMING
Introduction
Examples
Limitations of Dynamic Programming
ITERATIVE DYNAMIC PROGRAMMING
Construction of Time Stages
Construction of Grid for x
Allowable Values for Control
First Iteration
Iterations with Systematic Reduction in Region Size
Example
Use of Accessible States as Grid Points
Algorithm for IDP
Early Applications of IDP
ALLOWABLE VALUES FOR CONTROL
Introduction
Comparison of Uniform Distribution to Random Choice
EVALUATION OF PARAMETERS IN IDP
Number of Grid Points
Multi-Pass Approach
Further Example
PIECEWISE LINEAR CONTINUOUS CONTROL
Problem Formulation
Algorithm for IDP for Piecewise Linear Control
Numerical Examples
TIME-DELAY SYSTEMS
Problem Formulation
Examples
VARIABLE STAGE LENGTHS
Variable Stage-Lengths when Final Time is Free
Problems where Final Time f is not Specified
Systems with Specified Final Time
SINGULAR CONTROL PROBLEMS
Four Simple-Looking Examples
Yeo's Singular Control Problem
Nonlinear Two-Stage CSTR Problem
STATE CONSTRAINTS
Introduction
Final State Constraints
State Inequality Constraints
TIME OPTIMAL CONTROL
Introduction
Time Optimal Control Problem
Direct Approach to Time Optimal Control
Examples
High Dimensional Systems
NONSEPARABLE PROBLEMS
Problem Formulation
Examples
References
SENSITIVITY CONSIDERATIONS
Introduction
Example: Lee-Ramirez Bioreactor
TOWARD PRACTICAL OPTIMAL CONTROL
Optimal Control of Oil Shale Pyrolysis
Future Directions
APPENDICES: Nonlinear Algebraic Equation Solver. Listing of Linear Programming Program. LJ Optimization Programs. Iterative Dynamic Programming Programs. Listing of DVERK.
INDEX
Each chapter also contains an introduction and a References section.
Biography
Rein Luus
"This book provides a working knowledge of IDP with many worked out solutions for a wide range of problems. This is especially useful for graduate students and industrial practitioners because a strong background in mathematical techniques and chemical engineering is not essential for understanding this book. This book can be used in a university as a textbook at the level of seniors or first-year graduate students. Of course, this book is also suitable for academic researchers who need an alternative way to cross-validate their solutions to OCPs with their newly devised methods... It can be concluded that this text is a very good addition to the toolbox for numerical optimal control. It is expected that all engineers, graduate students, researchers who are involved in solving optimal control problems should know IDP - the new powerful OCP solution scheme."
-International Journal of Robust and Nonlinear Control, vol. 11, no. 14, December 15, 2001
