Author name cluster

Chenyang Zhu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

6 papers

1 author row

EAAI Journal 2026 Journal Article

Optimizing potential-based reward automata in partially observable reinforcement learning using genetic local search

Zhengwei Zhu
Zhixuan Chen
Chenyang Zhu
Wen Si
Fang Wang

Partially observable reinforcement learning extends the reinforcement learning framework to environments in which agents have limited visibility of the state space, making it particularly relevant for applications in robotics and autonomous vehicle navigation. However, a primary challenge in partially observable reinforcement learning is defining effective reward functions that can guide the learning process despite partial observability. To address this challenge, this paper introduces a novel approach for constructing potential-based reward automata by employing genetic local search methods. Specifically, our method constructs these automata from compressed representations of exploration trajectories, which succinctly capture critical decision points and essential state transitions while eliminating redundant steps. By optimizing trajectory samples and shortening agent trajectories to their crucial transitions, our technique significantly reduces computational overhead. Formally, we define the learning objective as an optimization problem aimed at maximizing the log-likelihood of future observations while simultaneously minimizing the structural complexity of the learned reward automata. Furthermore, by incorporating value-based strategies to estimate potential values within the reward automata, our approach improves learning efficiency and facilitates the identification of optimal reward structures. We empirically evaluate our proposed method on seven partially observable grid-world benchmarks. Experimental results demonstrate that our method achieves superior performance relative to state-of-the-art reward automata-based techniques, exhibiting both accelerated learning speeds and higher accumulated rewards. Additionally, our genetic local search algorithm consistently outperforms comparative heuristic methods in terms of learning curves and reward accumulation.

Details DOI

EAAI Journal 2025 Journal Article

Enhanced cross-domain lithology classification in imbalanced datasets using an unsupervised domain Adversarial Network

Yunxin Xie
Liangyu Jin
Chenyang Zhu
Weibin Luo
Qian Wang

Recent advancements in Artificial Intelligence (AI), particularly deep learning, have significantly improved lithology identification in reservoir exploration by leveraging micrographic rock imagery. Deep neural networks excel in feature extraction, enhancing classification accuracy. However, these models are prone to domain shifts, which often degrade their performance in real-world applications. This paper proposes an unsupervised domain adaptation framework that integrates Fisher linear discriminant analysis and Online Hard Example Mining (OHEM) to mitigate domain shifts and improve classification, particularly in datasets with imbalanced classes. The model employs a ω -balanced global–local domain discriminator to align feature distributions between different domains and introduces focal loss with class-wise weighted factors for better handling of imbalanced data. Additionally, an adapted version of OHEM identifies difficult samples during training, allowing the model to concentrate on challenging cases. The proposed method is validated on micrographic rock imagery from the Tibet, Qinghai, and Xinjiang regions, achieving an average accuracy of 83. 2%, which is 13. 8% higher than ResNet50 and at least 1% superior to other domain adaptation models. This research highlights the potential of AI-driven solutions in geoscientific applications and provides a robust framework for unsupervised lithology classification.

Details DOI

AAAI Conference 2025 Conference Paper

MultiBooth: Towards Generating All Your Concepts in an Image from Text

Chenyang Zhu
Kai Li
Yue Ma
Chunming He
Xiu Li

This paper introduces MultiBooth, a method that generates images from texts containing various concepts from users.Despite diffusion models bringing significant advancements for customized text-to-image generation, existing methods often struggle with multi-concept scenarios due to low concept fidelity and high inference cost. MultiBooth addresses these issues by dividing the multi-concept generation process into two phases: a single-concept learning phase and a multi-concept integration phase. During the single-concept learning phase, we employ a multi-modal image encoder and an efficient concept encoding technique to learn a concise and discriminative representation for each concept. In the multi-concept integration phase, we use bounding boxes to define the generation area for each concept within the cross-attention map. This method enables the creation of individual concepts within their specified regions, thereby facilitating the formation of multi-concept images. This strategy not only improves concept fidelity but also reduces additional inference cost. MultiBooth surpasses various baselines in both qualitative and quantitative evaluations, showcasing its superior performance and computational efficiency.

PDF Details DOI

AAAI Conference 2024 Conference Paper

Decomposing Temporal Equilibrium Strategy for Coordinated Distributed Multi-Agent Reinforcement Learning

Chenyang Zhu
Wen Si
Jinyu Zhu
Zhihao Jiang

The increasing demands for system complexity and robustness have prompted the integration of temporal logic into Multi-Agent Reinforcement Learning (MARL) to address tasks with non-Markovian properties. However, incorporating non-Markovian properties introduces additional computational complexities, as agents are required to integrate historical data into their decision-making process. Also, optimizing strategies within a multi-agent environment presents significant challenges due to the exponential growth of the state space with the number of agents. In this study, we introduce an innovative hierarchical MARL framework that synthesizes temporal equilibrium strategies through parity games and subsequently encodes them as individual reward machines for MARL coordination. More specifically, we reduce the strategy synthesis problem into an emptiness problem concerning parity games with optimized states and transitions. Following this synthesis step, the temporal equilibrium strategy is decomposed into individual reward machines for decentralized MARL. Theoretical proofs are provided to verify the consistency of the Nash equilibrium between the parallel composition of decomposed strategies and the original strategy. Empirical evidence confirms the efficacy of the proposed synthesis technique, showcasing its ability to reduce state space compared to the state-of-the-art tool. Furthermore, our study highlights the superior performance of the distributed MARL paradigm over centralized approaches when deploying decomposed strategies.

PDF Details DOI

AAAI Conference 2023 Conference Paper

Multi-Resolution Monocular Depth Map Fusion by Self-Supervised Gradient-Based Composition

Yaqiao Dai
Renjiao Yi
Chenyang Zhu
Hongjun He
Kai Xu

Monocular depth estimation is a challenging problem on which deep neural networks have demonstrated great potential. However, depth maps predicted by existing deep models usually lack fine-grained details due to convolution operations and down-samplings in networks. We find that increasing input resolution is helpful to preserve more local details while the estimation at low resolution is more accurate globally. Therefore, we propose a novel depth map fusion module to combine the advantages of estimations with multi-resolution inputs. Instead of merging the low- and high-resolution estimations equally, we adopt the core idea of Poisson fusion, trying to implant the gradient domain of high-resolution depth into the low-resolution depth. While classic Poisson fusion requires a fusion mask as supervision, we propose a self-supervised framework based on guided image filtering. We demonstrate that this gradient-based composition performs much better at noisy immunity, compared with the state-of-the-art depth map fusion method. Our lightweight depth fusion is one-shot and runs in real-time, making it 80X faster than a state-of-the-art depth fusion method. Quantitative evaluations demonstrate that the proposed method can be integrated into many fully convolutional monocular depth estimation backbones with a significant performance boost, leading to state-of-the-art results of detail enhancement on depth maps. Codes are released at https://github.com/yuinsky/gradient-based-depth-map-fusion.

PDF Details DOI

AAAI Conference 2021 Conference Paper

Online 3D Bin Packing with Constrained Deep Reinforcement Learning

Hang Zhao
Qijin She
Chenyang Zhu
Yin Yang
Kai Xu

We solve a challenging yet practically useful variant of 3D Bin Packing Problem (3D-BPP). In our problem, the agent has limited information about the items to be packed into a single bin, and an item must be packed immediately after its arrival without buffering or readjusting. The item’s placement also subjects to the constraints of order dependence and physical stability. We formulate this online 3D-BPP as a constrained Markov decision process (CMDP). To solve the problem, we propose an effective and easy-to-implement constrained deep reinforcement learning (DRL) method under the actor-critic framework. In particular, we introduce a prediction-and-projection scheme: The agent first predicts a feasibility mask for the placement actions as an auxiliary task and then uses the mask to modulate the action probabilities output by the actor during training. Such supervision and projection facilitate the agent to learn feasible policies very efficiently. Our method can be easily extended to handle lookahead items, multi-bin packing, and item re-orienting. We have conducted extensive evaluation showing that the learned policy significantly outperforms the state-of-the-art methods. A preliminary user study even suggests that our method might attain a human-level performance.

PDF Details