Adaptive Evolutionary Reinforcement Learning Algorithm with Early Termination Strategy

Evolutionary reinforcement learning algorithms (ERLs), which combine evolutionary algorithms (EAs) with reinforcement learning (RL), have demonstrated significant success in enhancing RL performance. However, most ERLs rely heavily on Gaussian mutation operators to generate new individuals. When the standard deviation is too large or small, this approach will result in the production of poor or highly similar offspring. Such outcomes can be detrimental to the learning process of the RL agent, as too many poor or similar experiences are generated by these individuals. In order to alleviate these issues, this paper proposes an Adaptive Evolutionary Reinforcement Learning (AERL) method that adaptively adjusts both the standard deviation and the evaluation process. By tracking the performance of new individuals, AERL maintains the mutation strength within a suitable range without the need for additional gradient computations. Moreover, the proposed AERL approach early terminates unnecessary evaluations and discards experiences arising from poor individuals, thereby resulting in enhanced learning efficiency. Empirical assessments conducted on a variety of continuous control problems demonstrate the effectiveness of the AERL method.

Authors

• Xiaoqiang Wu Shenzhen University
• Qingling Zhu Shenzhen University
• Qiuzhen Lin Shenzhen University
• Weineng Chen South China University of Technology
• Jianqiang Li College of Computer Science and Software Engineering, Shenzhen University, China National Engineering Laboratory for Big Data System Computing Technology, Shenzhen University, China

Keywords

• Reinforcement Learning
• Evolutionary Algorithm
• Evolutionary
• ∗Corresponding author.
• This work is licensed under a Creative Commons Attribution
• International 4. 0 License.
• Proc. of the 23rd International Conference on Autonomous Agents and Multiagent Systems
• (AAMAS 2024)
• N. Alechina
• V. Dignum
• M. Dastani
• J. S. Sichman (eds.)
• May 6 – 10
• 2024
• Auckland
• New Zealand. © 2024 International Foundation for Autonomous Agents and
• Multiagent Systems (www. ifaamas. org).