An improved proximal policy optimization algorithm for autonomous driving decision-making

Yi Yan; Chunsheng Liu; Faliang Chang; Yiming Huang

doi:10.1117/12.3029265

6 May 2024 An improved proximal policy optimization algorithm for autonomous driving decision-making

Yi Yan, Chunsheng Liu, Faliang Chang, Yiming Huang

Proceedings Volume 13107, Fourth International Conference on Sensors and Information Technology (ICSI 2024); 131073D (2024) https://doi.org/10.1117/12.3029265
Event: Fourth International Conference on Sensors and Information Technology (ICSI 2024), 2024, Xiamen, China

Abstract

In recent years, there has been significant development in research on methods for autonomous driving decision-making based on deep reinforcement learning, including Deep Deterministic Policy Gradient (DDPG) , Proximal Policy Optimization (PPO) , and others. Reinforcement learning strategies based on proximal policy optimization often suffer from poor performance due to getting trapped in local optima and inefficient learning. To address the challenges of proximal policy optimization in autonomous driving, we propose an autonomous driving decision-making algorithm based on adaptive curiosity mechanism and experience replay. Firstly, we introduce a mechanism for adaptive curiosity adjustment based on episode length to address the issue of local optima in proximal policy optimization. By incorporating curiosity mechanisms, the algorithm's exploratory nature is enhanced. Additionally, to ensure safety, we impose constraints on the curiosity coefficient using variations in episode length. Secondly, to overcome the low sample utilization and inefficient learning in PPO, we combine it with an experience replay mechanism, enabling faster learning of superior policies and improving learning efficiency. In simulation experiments, our algorithm efficiently explores a better and safer driving strategy.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Yi Yan, Chunsheng Liu, Faliang Chang, and Yiming Huang "An improved proximal policy optimization algorithm for autonomous driving decision-making", Proc. SPIE 13107, Fourth International Conference on Sensors and Information Technology (ICSI 2024), 131073D (6 May 2024); https://doi.org/10.1117/12.3029265

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available