RSMDP-based Robust Q-learning for Optimal Path Planning in a Dynamic Environment

Institute of Advanced Engineering and Science

Yunfei Zhang, Clarence W. de Silva,

IAES International Journal of Robotics and Automation (IJRA), Vol 3, No 1: March 2014 , pp. 1-12


This paper presents arobust Q-learning method for path planningin a dynamic environment. The method consists of three steps: first, a regime-switching Markov decision process (RSMDP) is formed to present the dynamic environment; second a probabilistic roadmap (PRM) is constructed, integrated with the RSMDP and stored as a graph whose nodes correspond to a collision-free world state for the robot; and third, an onlineQ-learning method with dynamic stepsize, which facilitates robust convergence of the Q-value iteration, is integrated with the PRM to determine an optimal path for reaching the goal. In this manner, the robot is able to use past experience for improving its performance in avoiding not only static obstacles but also moving obstacles, without knowing the nature of the obstacle motion. The use ofregime switching in the avoidance of obstacles with unknown motion is particularly innovative.  The developed approach is applied to a homecare robot in computer simulation. The results show that the online path planner with Q-learning is able torapidly and successfully converge to the correct path.

Publisher: Institute of Advanced Engineering and Science

Publish Date: 2014-03-01

DOI: 10.11591/ijra.v3i1.pp1-12

Publish Year: 2014

ipmuGoDigital Library

Copyright © 2021 IpmuGo Digital Library.

All Right Reserved


Help Center

Privacy Policy

Terms of Service