Deep Q-Learning with Custom Reward Shaping for Mobile Robot Navigation in Grid Dynamic Environments

One of the difficulties facing robots is navigating in dynamic and unstable environments. Traditional algorithms often fail to plan paths in environments containing dynamic obstacles. Therefore, this study proposes a framework for deep learning in a grid-based environment containing static and dynamic obstacles. The research contribution is summarized in designing an equivalent function that encourages reaching the goal and reduces unnecessary movements. It also assesses the impact of altering obstacle velocities on navigation performance and compares the DQL and DWA approaches. A deep Q-network with two hidden layers is trained using a greedy policy. Simulated LiDAR sensor used for spatial perception. The model is tested in four simulated environments of increasing difficulty, incorporating both static and dynamic obstacles to mimic realistic conditions. The agent that operates by the approach DQL achieved a complete success rate of 100% in environments A, B, and C, and a 90% success rate in environment D, in contrast to the DWA, which recorded success rates of 100%, 90%, 70%, and 60% respectively. Furthermore, DQL demonstrated sustained high performance despite increases in obstacle speed, achieving success rates of 100%, 90%, 70%, and 40% as the speed escalated to six times that of the robot’s velocity. The results showed that the DQL approach outperformed in success rates and path efficiency and maintained stability in complex environments. In summary, the DQL framework proposed herein presents a robust and scalable approach for mobile robot navigation, clearly demonstrating significant advantages over conventional methodologies in intricate and unpredictable environments.