Approximate Dynamic Programming for Nonlinear-Constrained Optimizations
Document Type
Article
Date of Original Version
5-1-2021
Abstract
In this paper, we study the constrained optimization problem of a class of uncertain nonlinear interconnected systems. First, we prove that the solution of the constrained optimization problem can be obtained through solving an array of optimal control problems of constrained auxiliary subsystems. Then, under the framework of approximate dynamic programming, we present a simultaneous policy iteration (SPI) algorithm to solve the Hamilton-Jacobi-Bellman equations corresponding to the constrained auxiliary subsystems. By building an equivalence relationship, we demonstrate the convergence of the SPI algorithm. Meanwhile, we implement the SPI algorithm via an actor-critic structure, where actor networks are used to approximate optimal control policies and critic networks are applied to estimate optimal value functions. By using the least squares method and the Monte Carlo integration technique together, we are able to determine the weight vectors of actor and critic networks. Finally, we validate the developed control method through the simulation of a nonlinear interconnected plant.
Publication Title, e.g., Journal
IEEE Transactions on Cybernetics
Volume
51
Issue
5
Citation/Publisher Attribution
Yang, Xiong, Haibo He, and Xiangnan Zhong. "Approximate Dynamic Programming for Nonlinear-Constrained Optimizations." IEEE Transactions on Cybernetics 51, 5 (2021): 2419-2432. doi: 10.1109/TCYB.2019.2926248.