APPLICATION OF REINFORCEMENT LEARNING METHODS FOR PATH PLANNING OF MOBILE ROBOTS

DOI: 10.31673/2412-4338.2024.011625

Abstract

The development and implementation of autonomous mobile robots in various spheres of human life has become an urgent task today. Reinforcement Learning (RL) is a powerful tool for optimizing learning and decision-making by agents in real-world conditions. The use of RL is becoming a key aspect for achieving efficiency and reliability of robotic systems. Reinforcement learning can be used to plan the path of a mobile robot in complex and dynamic environments, to teach a mobile robot to make decisions about direction, speed, and maneuvers based on its sensors, to make decisions about the efficient use of energy resources and maximize operating time. The agent can learn optimal routes, avoid obstacles, and effectively achieve its goals. The article discusses the use of reinforcement learning methods to optimize the path planning of mobile robots. A classification of methods based on the environment model and methods without the environment model is presented. Value-based methods, policy-based methods, and actor-critic methods are considered. In particular, the analysis of such reinforcement learning methods as Q-learning, Deep Q-Networks (DQN), Double Deep Q-Network (DDQN), actor-critic algorithms Advantage Actor-Critic (A2C), Deep Deterministic Policy Gradient (DDPG), Asynchronous Advantage Actor-Critic (A3C), Soft actor-critic (SAC) and Proximal Policy Optimization (PPO) is carried out.) These methods were analyzed in the context of their application to solving the problems of planning the path of a mobile robot in different environments. The advantages and disadvantages of using these reinforcement learning methods in path planning are investigated, taking into account the aspects of efficiency, safety, and adaptability. Particular attention is paid to solving the problems of increasing the speed and sustainability of learning, effective navigation in complex and changing conditions where traditional methods may be ineffective. Prospects for future research and development of this area in scientific works are proposed.

Keywords: machine learning, reinforcement learning methods, mobile robots, path planning, information technology, information system, model, algorithm.

References
1. Kober J, Bagnell JA, Peters J. Reinforcement learning in robotics: A survey. The International Journal of Robotics Research. 2013. Vol. 32, 1
2. Rybczak, M.; Popowniak, N.; Lazarowska, A. A Survey of Machine Learning Approaches for Mobile Robot Control. Robotics. 2024, Vol. 13, No. 1. P.12-22.
3. Gao, J.; Ye, W.; Guo, J.; Li, Z. Deep Reinforcement Learning for Indoor Mobile Robot Path Planning. Sensors 2020. No. 20, 5493.
4. Tai L., Paolo G., Liu M., Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, Canada . 2017, P. 31-36,
5. Zhao Y., Zhang Y., Wang S. A Review of Mobile Robot Path Planning Based on Deep Reinforcement Learning Algorithm. Journal of Physics: Conference Series, Vol. 2138, International Conference on Artificial Intelligence and Big Data Applications (ICAIBD 2021) 24-25
6. Wang W., Wu Zh., Luo H., Zhang B. Path Planning Method of Mobile Robot Using Improved Deep Reinforcement Learning. Journal of Electrical and Computer Engineering, vol. 2022. P. 7.
7. Zheng, J.; Mao, S.; Wu, Z.; Kong, P.; Qiang, H. Improved Path Planning for Indoor Patrol Robot Based on Deep Reinforcement Learning. Symmetry 2022, No. 14, 132.
8. Xin J., Zhao H., Liu D., Li M., Application of deep reinforcement learning in mobile robot path planning, Chinese Automation Congress (CAC), Jinan, China, 2017, p. 7112-7116.
9. Low Ee S., Ong P., Cheah K. Ch., Solving the optimal path planning of a mobile robot using improved Q-learning. Robotics and Autonomous Systems, 2019, Vol. 115, P. 143-161.
10. Jiang Q. Path Planning Method of Mobile Robot Based on Q-learning. Journal of Physics: Conference Series, International Symposium on Artificial Intelligence and Intelligent Manufacturing (AIIM 2021) 12/11/2021 - 14/11/2021 Huzhou 2022. Vol. 2181.
11. Khriji L, Touati F, Benhmed K, Al-Yahmedi A. Mobile Robot Navigation Based on Q-Learning Technique. International Journal of Advanced Robotic Systems. 2011. Vol. 8 No. 1.
12. Singh, R.; Ren, J.; Lin, X. A Review of Deep Reinforcement Learning Algorithms for Mobile Robot Path Planning. Vehicles. 2023. No. 5, P. 1423-1451.
13. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. Proximal policy optimization algorithms. 2017. No. 5. P. 56-67.
14. Srikonda S.; Norris W.R.; Nottage D.; Soylemezoglu A. Deep Reinforcement Learning for Autonomous Dynamic Skid Steer Vehicle Trajectory Tracking. Robotics. 2022, No. 11, P. 95.
15. Xing X., Ding H., Liang Zh., Li B., Yang Zh., Robot path planner based on deep reinforcement learning and the seeker optimization algorithm, Mechatronics, 2022. Vol. 88. P. 102918
16. Zhang, Y.; Chen, P. Path Planning of a Mobile Robot for a Dynamic Indoor Environment Based on an SAC-LSTM Algorithm. Sensors 2023. No. 23, P. 9802.
17. Ganenko L. D., Zhebka V. V. Analytical review of issues of navigation of mobile robots in closed spaces. Telecommunications and information technologies. 2023. No. 3(80). Art. 85-98.
18. Malinov V., Zhebka V., Zolotukhina O., Franchuk T., Chubaievskyi V. Biomining as an Effective Mechanism for Utilizing the Bioenergy Potential of Processing Enterprises in the Agricultural Sector. CEUR Workshop Proceedings. 2023, 3421, p. 223–230