Author name cluster

Wei-Neng Chen

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

5 papers

2 author rows

NeurIPS Conference 2025 Conference Paper

DesignX: Human-Competitive Algorithm Designer for Black-Box Optimization

Hongshu Guo
Zeyuan Ma
Yining Ma
Xinglin Zhang
Wei-Neng Chen
Yue-Jiao Gong

Designing effective black‑box optimizers is hampered by limited problem-specific knowledge and manual control that spans months for almost every detail. In this paper, we present DesignX, the first automated algorithm design framework that generates an effective optimizer specific to a given black-box optimization problem within seconds. Rooted in the first principles, we identify two key sub-tasks: 1) algorithm structure generation and 2) hyperparameter control. To enable systematic construction, a comprehensive modular algorithmic space is first built, embracing hundreds of algorithm components collected from decades of research. We then introduce a dual-agent reinforcement learning system that collaborates on structural and parametric design through a novel cooperative training objective, enabling large-scale meta-training across 10k diverse instances. Remarkably, through days of autonomous learning, the DesignX-generated optimizers continuously surpass human-crafted optimizers by orders of magnitude, either on synthetic testbed or on realistic optimization scenarios such as Protein-docking, AutoML and UAV path planning. Further in-depth analysis reveals DesignX's capability to discover non-trivial algorithm patterns beyond expert intuition, which, conversely, provides valuable design insights for the optimization community. We provide DesignX's Python project at~\url{https: //github. com/MetaEvo/DesignX}.

PDF Details

AAAI Conference 2025 Conference Paper

Evolutionary Reinforcement Learning with Parameterized Action Primitives for Diverse Manipulation Tasks

Xianxu Qiu
Haiming Huang
Weiwei Chen
Qiuzhen Lin
Wei-Neng Chen
Fuchun Sun

Reinforcement learning (RL) has shown promising performance in tackling robotic manipulation tasks (RMTs), which require learning a prolonged sequence of manipulation actions to control robots efficiently. However, most RL algorithms often suffer from two problems when solving RMTs: inefficient exploration due to the extremely large action space and catastrophic forgetting due to the poor sampling efficiency. To alleviate these problems, this paper introduces an Evolutionary Reinforcement Learning algorithm with parameterized Action Primitives, called ERLAP, which combines the advantages of an evolutionary algorithm (EA) and hierarchical RL (HRL) to solve diverse RMTs. A library of heterogeneous action primitives is constructed in HRL to enhance the exploration efficiency of robots and dual populations with new evolutionary operators are run in EA to optimize these primitive sequences, which can diversify the distribution of replay buffer and avoid catastrophic forgetting. The experiments show that ERLAP outperforms four state-of-the-art RL algorithms in simulated RMTs with dense rewards and can effectively avoid catastrophic forgetting in a set of more challenging simulated RMTs with sparse rewards.

PDF Details DOI

ECAI Conference 2025 Conference Paper

Population-Based Multi-Objective Reinforcement Learning with Information Sharing and Differentiation

Xiaoqiang Wu
Qingling Zhu
Junkai Ji
Qiuzhen Lin
Wei-Neng Chen
Jianqiang Li 0001

To efficiently tackle problems with multiple conflicting objectives, several Multi-Objective Reinforcement Learning (MORL) algorithms utilize a universal policy network that takes preference weights as input to represent optimal policies for all different preferences. However, it is quite challenging to train such a universal policy as it is easy to forget or fail to learn skills for some preferences. To alleviate this issue, we propose an efficient Population-Based MORL (PB-MORL) method that trains multiple agents with universal policy networks using a shared replay buffer. Each agent is biased towards optimizing specific objectives by applying differentiated weights to the rewards sampled from the buffer. Therefore, the policy of each agent only needs to handle the specific part of the preference space rather than the entire space, simplifying the training task. Meanwhile, the experiences in the common buffer facilitate the information sharing among individuals, which can significantly reduce the number of interaction steps for training multiple agents. Experiments on both continuous and discrete tasks demonstrate the superiority of PB-MORL over several state-of-the-art MORL methods.

Details

AAAI Conference 2024 Conference Paper

ERL-TD: Evolutionary Reinforcement Learning Enhanced with Truncated Variance and Distillation Mutation

Qiuzhen Lin
Yangfan Chen
Lijia Ma
Wei-Neng Chen
Jianqiang Li

Recently, an emerging research direction called Evolutionary Reinforcement Learning (ERL) has been proposed, which combines evolutionary algorithm with reinforcement learning (RL) for tackling the tasks of sequential decision making. However, the recently proposed ERL algorithms often suffer from two challenges: the inaccuracy of policy estimation caused by the overestimation bias in RL and the insufficiency of exploration caused by inefficient mutations. To alleviate these problems, we propose an Evolutionary Reinforcement Learning algorithm enhanced with Truncated variance and Distillation mutation, called ERL-TD. We utilize multiple Q-networks to evaluate state-action pairs, so that multiple networks can provide more accurate evaluations for state-action pairs, in which the variance of evaluations can be adopted to control the overestimation bias in RL. Moreover, we propose a new distillation mutation to provide a promising mutation direction, which is different from traditional mutation generating a large number of random solutions. We evaluate ERL-TD on the continuous control benchmarks from the OpenAI Gym and DeepMind Control Suite. The experiments show that ERL-TD shows excellent performance and outperforms all baseline RL algorithms on the test suites.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Two-Stage Evolutionary Reinforcement Learning for Enhancing Exploration and Exploitation

Qingling Zhu
Xiaoqiang Wu
Qiuzhen Lin
Wei-Neng Chen

The integration of Evolutionary Algorithm (EA) and Reinforcement Learning (RL) has emerged as a promising approach for tackling some challenges in RL, such as sparse rewards, lack of exploration, and brittle convergence properties. However, existing methods often employ actor networks as individuals of EA, which may constrain their exploratory capabilities, as the entire actor population will stop evolution when the critic network in RL falls into local optimal. To alleviate this issue, this paper introduces a Two-stage Evolutionary Reinforcement Learning (TERL) framework that maintains a population containing both actor and critic networks. TERL divides the learning process into two stages. In the initial stage, individuals independently learn actor-critic networks, which are optimized alternatively by RL and Particle Swarm Optimization (PSO). This dual optimization fosters greater exploration, curbing susceptibility to local optima. Shared information from a common replay buffer and PSO algorithm substantially mitigates the computational load of training multiple agents. In the subsequent stage, TERL shifts to a refined exploitation phase. Here, only the best individual undergoes further refinement, while the rest individuals continue PSO-based optimization. This allocates more computational resources to the best individual for yielding superior performance. Empirical assessments, conducted across a range of continuous control problems, validate the efficacy of the proposed TERL paradigm.

PDF Details DOI