Performance Comparison of Duel-DDPG and DDPG Algorithms in the Decision-making Phase of Autonomous Vehicles

Abstract

Automated driving (AD) is a developing technology aimed at decreasing traffic accidents and enhancing driving efficiency. This research seeks to create a decision-making approach for self-driving cars, emphasizing actions such as changing lanes, overtaking, and maintaining lane position on highways, through deep reinforcement learning (DRL). In order to achieve this, a driving environment simulating a highway is established in the commercial multi-body simulation software IPG CarMaker 11, allowing the ego vehicle to navigate around other vehicles safely and efficiently. A control framework with a hierarchical structure is established to oversee these vehicles, where the high-level control is tasked with making driving choices. Additionally, the Duel Deep Deterministic Policy Gradient (Duel-DDPG) algorithm, which is a Deep Reinforcement Learning (DRL) method, is employed to create the highway decision-making strategy, which is simulated using MATLAB software. The computational methods of the Duel-DDPG and DDPG algorithms are examined and contrasted. A series of simulation evaluations are performed to evaluate the efficacy of the suggested decision-making policy. The results emphasize the advantages of the proposed framework regarding convergence rate and control effectiveness. The findings indicate that the Duel-DDPG-based approach effectively and safely performs highway driving activities.