Goal representation heuristic dynamic programming on maze navigation
Document Type
Article
Date of Original Version
7-29-2013
Abstract
Goal representation heuristic dynamic programming (GrHDP) is proposed in this paper to demonstrate online learning in the Markov decision process. In addition to the (external) reinforcement signal in literature, we develop an adaptively internal goal/reward representation for the agent with the proposed goal network. Specifically, we keep the actor-critic design in heuristic dynamic programming (HDP) and include a goal network to represent the internal goal signal, to further help the value function approximation. We evaluate our proposed GrHDP algorithm on two 2-D maze navigation problems, and later on one 3-D maze navigation problem. Compared to the traditional HDP approach, the learning performance of the agent is improved with our proposed GrHDP approach. In addition, we also include the learning performance with two other reinforcement learning algorithms, namely {\rm Sarsa}(\lambda) and Q-learning, on the same benchmarks for comparison. Furthermore, in order to demonstrate the theoretical guarantee of our proposed method, we provide the characteristics analysis toward the convergence of weights in neural networks in our GrHDP approach. © 2012 IEEE.
Publication Title, e.g., Journal
IEEE Transactions on Neural Networks and Learning Systems
Volume
24
Issue
12
Citation/Publisher Attribution
Ni, Zhen, Haibo He, Jinyu Wen, and Xin Xu. "Goal representation heuristic dynamic programming on maze navigation." IEEE Transactions on Neural Networks and Learning Systems 24, 12 (2013): 2038-2050. doi: 10.1109/TNNLS.2013.2271454.