A three-network architecture for on-line learning and optimization based on adaptive dynamic programming
Document Type
Article
Date of Original Version
2-15-2012
Abstract
In this paper, we propose a novel adaptive dynamic programming (ADP) architecture with three networks, an action network, a critic network, and a reference network, to develop internal goal-representation for online learning and optimization. Unlike the traditional ADP design normally with an action network and a critic network, our approach integrates the third network, a reference network, into the actor-critic design framework to automatically and adaptively build an internal reinforcement signal to facilitate learning and optimization overtime to accomplish goals. We present the detailed design architecture and its associated learning algorithm to explain how effective learning and optimization can be achieved in this new ADP architecture. Furthermore, we test the performance of our architecture both on the cart-pole balancing task and the triple-link inverted pendulum balancing task, which are the popular benchmarks in the community to demonstrate its learning and control performance over time. © 2011 Elsevier B.V.
Publication Title, e.g., Journal
Neurocomputing
Volume
78
Issue
1
Citation/Publisher Attribution
He, Haibo, Zhen Ni, and Jian Fu. "A three-network architecture for on-line learning and optimization based on adaptive dynamic programming." Neurocomputing 78, 1 (2012): 3-13. doi: 10.1016/j.neucom.2011.05.031.