Author name cluster

Rui Pan

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

11 papers

1 author row

TIST Journal 2026 Journal Article

An Improved Autoregressive Evaluation Paradigm for Large Language Models

Jipeng Zhang
Rui Pan
Yuzheng Hu
KaShun Shum
Guanyu Yao
Xiang Liu
Renjie Pi
Hanze Dong

The AI community has witnessed the emergence of various chat-style Large Language Models (LLMs) since the advent of ChatGPT. Despite significant progress in this area, evaluating these models remains a substantial challenge. The evaluations provided by humans or GPT-4 oracles are often taken as the gold standard, but they are neither automatic nor scalable. More recently, a series of (open source) LLM-based judge models have been introduced, yet they often exhibit model-specific biases, e.g., a LLaMA-family judge favors a LLaMA-family model. On the other hand, autoregressive evaluation metrics, which holds the potential to address the aforementioned issues, remains underexplored. Among them, likelihood-based metrics such as perplexity and Negative Log-Likelihood (NLL) are widely adopted and has proven effective in tracking the pre-training progress of LLMs. However, they struggle to evaluate the generation capabilities of fine-tuned models due to exposure bias, a phenomenon where the distribution of the model’s output gradually deviates from the ground-truth during inference. To address this key issue, in this article, we propose a novel autoregressive metric, Normalized Discounted Cumulative Gain (NDCG), to improve the evaluation of fine-tuned LLMs. Our experimental results demonstrate that NDCG significantly outperforms likelihood-based metrics: it shows over 45% improvement in both Spearman and Kendall’s tau correlation coefficients for commonsense QA tasks, and aligns more closely with GPT-4 Elo rankings for instruction-tuned models.

Details DOI

EAAI Journal 2026 Journal Article

Landslide early warning model based on multi-source monitoring data and unsupervised machine learning

Yang Chen
Hongyong Yuan
Jianguo Chen
Rui Pan
Lizheng Deng
Lida Huang
Mingzhi Zhang
Qiang Yang

Existing landslide early warning models, including physical mechanism models, rainfall statistics models and deformation characteristics models, typically compare monitoring values with preset thresholds to determine the warning level. However, when confronted with massive landslide sites exhibiting diverse geological formations, triggering factors and destabilization critical values, these models often encounter difficulties in setting appropriate thresholds. This paper presents a novel landslide early warning model for local system based on multi-source monitoring data using unsupervised machine learning. Various monitoring parameters were subjected to data preprocessing, wavelet decomposition, unsupervised machine learning, and comprehensive evaluation. Three algorithms were compared: Isolation Forest, One-Class Support Vector Machine and Local Outlier Factor. The Liannan landslide was selected as a case study to conduct the entire process, which indicated that the Isolation Forest algorithm outperformed the others in terms of integrated warning; therefore, this algorithm was selected for multi-source data processing. The model was then applied to 14 landslide sites to check its performance. It was proven to be capable of providing quantifiable warning levels at the time of significant deformation, which correlated well with the actual deformation trend and enabled the delivery of accurate warning levels. The proposed model can be applied to a wider range of landslide monitoring sites, and the data processing flow, thresholds and key parameters have a high degree of consistency, helping to address the challenge of setting thresholds for first-failure landslides. Besides, the model enables multiple parameters to be organically combined, and the occurrence of false alarms is visibly reduced.

Details DOI

NeurIPS Conference 2025 Conference Paper

ASGO: Adaptive Structured Gradient Optimization

Kang An
Yuxing Liu
Rui Pan
Yi Ren
Shiqian Ma
Donald Goldfarb
Tong Zhang

Training deep neural networks (DNNs) is a structured optimization problem, because the parameters are naturally represented by matrices and tensors rather than simple vectors. Under this structural representation, it has been widely observed that gradients are low-rank and Hessians are approximately block-wise diagonal. These structured properties are crucial for designing efficient optimization algorithms but may not be utilized by current popular optimizers like Adam. In this paper, we present a novel optimization algorithm ASGO that capitalizes on these properties by employing a preconditioner that is adaptively updated using structured gradients. By fine-grained theoretical analysis, ASGO is proven to achieve superior convergence rates compared to existing structured gradient methods. Based on the convergence theory, we further demonstrate that ASGO can benefit from the low-rank and block-wise diagonal properties. We also discuss practical modifications of ASGO and empirically verify the effectiveness of the algorithm on language model tasks.

PDF Details

TMLR Journal 2025 Journal Article

Entropy-Regularized Process Reward Model

Hanning Zhang
Pengcheng Wang
Shizhe Diao
Yong Lin
Rui Pan
Hanze Dong
Dylan Zhang
Pavlo Molchanov

Large language models (LLMs) have shown promise in performing complex multi-step reasoning, yet they continue to struggle with mathematical reasoning, often making systematic errors. A promising solution is reinforcement learning (RL) guided by reward models, particularly those focusing on process rewards, which score each intermediate step rather than solely evaluating the final outcome. This approach is more effective at guiding policy models towards correct reasoning trajectories. In this work, we propose an entropy-regularized process reward model (ER-PRM) that integrates KL-regularized Markov Decision Processes (MDP) to balance policy optimization with the need to prevent the policy from shifting too far from its initial distribution. We derive a novel reward construction method based on the theoretical results. Our theoretical analysis shows that we could derive the optimal reward model from the initial policy sampling. Our empirical experiments on the MATH and GSM8K benchmarks demonstrate that ER-PRM consistently outperforms existing process reward models, achieving 1% improvement on GSM8K and 2-3% improvement on MATH under best-of-N evaluation, and more than 1% improvement under RLHF. These results highlight the efficacy of entropy-regularization in enhancing LLMs' reasoning capabilities.

PDF Details

NeurIPS Conference 2025 Conference Paper

NegoCollab: A Common Representation Negotiation Approach for Heterogeneous Collaborative Perception

CONGZHANG SHAO
Quan Yuan
Guiyang Luo
Yue Hu
Danni Wang
Liu Yilin
Rui Pan
Bo Chen

Collaborative perception improves task performance by expanding the perception range through information sharing among agents. Immutable heterogeneity poses a significant challenge in collaborative perception, as participating agents may employ different and fixed perception models. This leads to domain gaps in the intermediate features shared among agents, consequently degrading collaborative performance. Aligning the features of all agents to a common representation can eliminate domain gaps with low training cost. However, in existing methods, the common representation is designated as the representation of a specific agent, making it difficult for agents with significant domain discrepancies from this specific agent to achieve proper alignment. This paper proposes NegoCollab, a heterogeneous collaboration method based on the negotiated common representation. It introduces a negotiator during training to derive the common representation from the local representations of each modality's agent, effectively reducing the inherent domain gap with the various local representations. In NegoCollab, the mutual transformation of features between the local representation space and the common representation space is achieved by a pair of sender and receiver. To better align local representations to the common representation containing multimodal information, we introduce structural alignment loss and pragmatic alignment loss in addition to the distribution alignment loss to supervise the training. This enables the knowledge in the common representation to be fully distilled into the sender. The experimental results demonstrate that NegoCollab significantly outperforms existing methods in common representation-based collaboration approaches. The mechanism of obtaining common representations through negotiation provides a more reliable and flexible option for common representations in heterogeneous collaborative perception.

PDF Details

NeurIPS Conference 2025 Conference Paper

Safe RLHF-V: Safe Reinforcement Learning from Multi-modal Human Feedback

Jiaming Ji
Xinyu Chen
Rui Pan
Han Zhu
Jiahao Li
Donghai Hong
Boyuan Chen
Jiayi Zhou

Multimodal large language models (MLLMs) are essential for building general-purpose AI assistants; however, they pose increasing safety risks. How can we ensure safety alignment of MLLMs to prevent undesired behaviors? Going further, it is critical to explore how to fine-tune MLLMs to preserve capabilities while meeting safety constraints. Fundamentally, this challenge can be formulated as a min-max optimization problem. However, existing datasets have not yet disentangled single preference signals into explicit safety constraints, hindering systematic investigation in this direction. Moreover, it remains an open question whether such constraints can be effectively incorporated into the optimization process for multi-modal models. In this work, we present the first exploration of the Safe RLHF-V -- the first multimodal safety alignment framework. The framework consists of: (I) BeaverTails-V, the first open-source dataset featuring dual preference annotations for helpfulness and safety, supplemented with multi-level safety labels (minor, moderate, severe); (II) Beaver-Guard-V, a multi-level guardrail system to proactively defend against unsafe queries and adversarial attacks. Applying the guard model over five rounds of filtering and regeneration significantly enhances the precursor model’s overall safety by an average of 40. 9%. (II) Based on dual preference, we initiate the first exploration of multi-modal safety alignment within a constrained optimization. Experimental results demonstrate that Safe RLHF effectively improves both model helpfulness and safety. Specifically, Safe RLHF-V enhances model safety by 34. 2% and helpfulness by 34. 3%.

PDF Details

NeurIPS Conference 2025 Conference Paper

SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning

Rui Pan
Yinwei Dai
Zhihao Zhang
Gabriele Oliaro
Zhihao Jia
Ravi Netravali

Recent advances in inference-time compute have significantly improved performance on complex tasks by generating long chains of thought (CoTs) using Large Reasoning Models (LRMs). However, this improved accuracy comes at the cost of high inference latency due to the length of generated reasoning sequences and the autoregressive nature of decoding. Our key insight in tackling these overheads is that LRM inference, and the reasoning that it embeds, is highly tolerant of approximations: complex tasks are typically broken down into simpler steps, each of which brings utility based on the semantic insight it provides for downstream steps rather than the exact tokens it generates. Accordingly, we introduce SpecReason, a system that automatically accelerates LRM inference by using a lightweight model to (speculatively) carry out simpler intermediate reasoning steps and reserving the costly base model only to efficiently assess (and potentially correct) the speculated outputs. Importantly, SpecReason's focus on exploiting the semantic flexibility of thinking tokens in preserving final-answer accuracy is complementary to prior speculation techniques, most notably speculative decoding, which demands token-level equivalence at each step. Across a variety of cross-domain reasoning benchmarks, SpecReason achieves 1. 4-3. 0$\times$ speedup over vanilla LRM inference while improving accuracy by 0. 4-9. 0%. Compared to speculative decoding without SpecReason, their combination yields an additional 8. 8-58. 0% latency reduction. We open-source SpecReason at \url{https: //anonymous. 4open. science/r/specreason/}.

PDF Details

NeurIPS Conference 2024 Conference Paper

Image Textualization: An Automatic Framework for Generating Rich and Detailed Image Descriptions

Renjie Pi
Jianshu Zhang
Jipeng Zhang
Rui Pan
Zhekai Chen
Tong Zhang

Image description datasets play a crucial role in the advancement of various applications such as image understanding, text-to-image generation, and text-image retrieval. Currently, image description datasets primarily originate from two sources. One source is the scraping of image-text pairs from the web. Despite their abundance, these descriptions are often of low quality and noisy. Another way is through human labeling. Datasets such as COCO are generally very short and lack details. Although detailed image descriptions can be annotated by humans, the high cost limits their quantity and feasibility. These limitations underscore the need for more efficient and scalable methods to generate accurate and detailed image descriptions. In this paper, we propose an innovative framework termed Image Textualization, which automatically produces high-quality image descriptions by leveraging existing mult-modal large language models (MLLMs) and multiple vision expert models in a collaborative manner. We conduct various experiments to validate the high quality of the descriptions constructed by our framework. Furthermore, we show that MLLMs fine-tuned on our dataset acquire an unprecedented capability of generating richer image descriptions, substantially increasing the length and detail of their output with even less hallucinations.

PDF Details DOI

NeurIPS Conference 2024 Conference Paper

LISA: Layerwise Importance Sampling for Memory-Efficient Large Language Model Fine-Tuning

Rui Pan
Xiang Liu
Shizhe Diao
Renjie Pi
Jipeng Zhang
Chi Han
Tong Zhang

The machine learning community has witnessed impressive advancements since large language models (LLMs) first appeared. Yet, their massive memory consumption has become a significant roadblock to large-scale training. For instance, a 7B model typically requires at least 60 GB of GPU memory with full parameter training, which presents challenges for researchers without access to high-resource environments. Parameter Efficient Fine-Tuning techniques such as Low-Rank Adaptation (LoRA) have been proposed to alleviate this problem. However, in most large-scale fine-tuning settings, their performance does not reach the level of full parameter training because they confine the parameter search to a low-rank subspace. Attempting to complement this deficiency, we investigate the layerwise properties of LoRA on fine-tuning tasks and observe an unexpected but consistent skewness of weight norms across different layers. Utilizing this key observation, a surprisingly simple training strategy is discovered, which outperforms both LoRA and full parameter training in a wide range of settings with memory costs as low as LoRA. We name it Layerwise Importance Sampled AdamW (LISA), a promising alternative for LoRA, which applies the idea of importance sampling to different layers in LLMs and randomly freeze most middle layers during optimization. Experimental results show that with similar or less GPU memory consumption, LISA surpasses LoRA or even full parameter tuning in downstream fine-tuning tasks, where LISA consistently outperforms LoRA by over 10%-35% in terms of MT-Bench score while achieving on-par or better performance in MMLU, AGIEval and WinoGrande. On large models, specifically LLaMA-2-70B, LISA surpasses LoRA on MT-Bench, GSM8K, and PubMedQA, demonstrating its effectiveness across different domains.

PDF Details DOI

IJCAI Conference 2023 Conference Paper

GPLight: Grouped Multi-agent Reinforcement Learning for Large-scale Traffic Signal Control

Yilin Liu
Guiyang Luo
Quan Yuan
Jinglin Li
Lei Jin
Bo Chen
Rui Pan

The use of multi-agent reinforcement learning (MARL) methods in coordinating traffic lights (CTL) has become increasingly popular, treating each intersection as an agent. However, existing MARL approaches either treat each agent absolutely homogeneous, i. e. , same network and parameter for each agent, or treat each agent completely heterogeneous, i. e. , different networks and parameters for each agent. This creates a difficult balance between accuracy and complexity, especially in large-scale CTL. To address this challenge, we propose a grouped MARL method named GPLight. We first mine the similarity between agent environment considering both real-time traffic flow and static fine-grained road topology. Then we propose two loss functions to maintain a learnable and dynamic clustering, one that uses mutual information estimation for better stability, and the other that maximizes separability between groups. Finally, GPLight enforces the agents in a group to share the same network and parameters. This approach reduces complexity by promoting cooperation within the same group of agents while reflecting differences between groups to ensure accuracy. To verify the effectiveness of our method, we conduct experiments on both synthetic and real-world datasets, with up to 1, 089 intersections. Compared with state-of-the-art methods, experiment results demonstrate the superiority of our proposed method, especially in large-scale CTL.

PDF Details DOI

TMLR Journal 2023 Journal Article

RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignment

Hanze Dong
Wei Xiong
Deepanshu Goyal
Yihan Zhang
Winnie Chow
Rui Pan
Shizhe Diao
Jipeng Zhang

Generative foundation models are susceptible to implicit biases that can arise from extensive unsupervised training data. Such biases can produce suboptimal samples, skewed outcomes, and unfairness, with potentially serious consequences. Consequently, aligning these models with human ethics and preferences is an essential step toward ensuring their responsible and effective deployment in real-world applications. Prior research has primarily employed Reinforcement Learning from Human Feedback (RLHF) to address this problem, where generative models are fine-tuned with RL algorithms guided by a human-feedback-informed reward model. However, the inefficiencies and instabilities associated with RL algorithms frequently present substantial obstacles to the successful alignment, necessitating the development of a more robust and streamlined approach. To this end, we introduce a new framework, Reward rAnked FineTuning (RAFT), designed to align generative models effectively. Utilizing a reward model and a sufficient number of samples, our approach selects the high-quality samples, discarding those that exhibit undesired behavior, and subsequently enhancing the model by fine-tuning on these filtered samples. Our studies show that RAFT can effectively improve the model performance in both reward learning and other automated metrics in both large language models and diffusion models.

PDF Details