Hongbin Ren

EAAI Journal 2025 Journal Article

Advancing the safety of intelligent rail transit systems: A segmentation network for efficient end-of-track degradation feature extraction

• Tao Ye
• Haoran Chen
• Guopeng Liu
• Liu Liu
• Hongbin Ren
• Xiaosong Li
• Xi Zhang

Accurate segmentation of rail track is crucial for the safe autonomous driving of intelligent trains. Current train operations struggle with insufficient precision in rail track segmentation, primarily due to poor end-of-track segmentation performance caused by degradation of track-end features. To address these challenges, we propose Rail Track End Wise Network (RTEW-Net), an effective rail track end wise segmentation method. This network utilizes Full-Transformer Module (FTM) for effective track feature extraction and integrates the Global Response Normalization (GRN) module to handle drastic lighting changes. Additionally, we designed the Wise Weigh Maintain (WWM) method to enhance feature learning and retain track features. To validate its effectiveness, we constructed the RailMixed2024 (RM2024) dataset. Our model achieves high-precision global rail track segmentation and optimizes end detection. Experimental results demonstrate that RTEW-Net exhibits outstanding performance on the RM2024 and RailSem19 datasets, establishing it as the state-of-the-art (SOTA) in this field.

EAAI Journal 2025 Journal Article

Erratum to “Advancing the safety of intelligent rail transit systems: A segmentation network for efficient end-of-track degradation feature extraction” [Eng. Appl. Artif. Intellig. 159 (2025) 111527]

• Tao Ye
• Haoran Chen
• Guopeng Liu
• Liu Liu
• Hongbin Ren
• Xiaosong Li
• Xi Zhang

NeurIPS Conference 2025 Conference Paper

Model Merging in Pre-training of Large Language Models

• Yunshui Li
• Yiyuan Ma
• Shen Yan
• Chaoyi Zhang
• Jing Liu
• Jianqiao Lu
• Ziwen Xu
• Mengzhao Chen

Model merging has emerged as a promising technique for enhancing large language models, though its application in large-scale pre-training remains relatively unexplored. In this paper, we present a comprehensive investigation of model merging techniques during the pre-training process. Through extensive experiments with both dense and Mixture-of-Experts (MoE) architectures ranging from millions to over 100 billion parameters, we demonstrate that merging checkpoints trained with constant learning rates not only achieves significant performance improvements but also enables accurate prediction of annealing behavior. These improvements lead to both more efficient model development and significantly lower training costs. Our detailed ablation studies on merging strategies and hyperparameters provide new insights into the underlying mechanisms while uncovering novel applications. Through comprehensive experimental analysis, we offer the open-source community practical pre-training guidelines for effective model merging.