Author name cluster

Peng Lu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

11 papers

2 author rows

ICML Conference 2025 Conference Paper

Calibrated Language Models and How to Find Them with Label Smoothing

Jerry Huang
Peng Lu
Qiuhao Zeng

Recent advances in natural language processing (NLP) have opened up greater opportunities to enable fine-tuned large language models (LLMs) to behave as more powerful interactive agents through improved instruction-following ability. However, understanding how this impacts confidence calibration for reliable model output has not been researched in full. In this work, we examine various open-sourced LLMs, identifying significant calibration degradation after instruction tuning in each. Seeking a practical solution, we look towards label smoothing, which has been shown as an effective method to regularize for overconfident predictions but has yet to be widely adopted in the supervised fine-tuning (SFT) of LLMs. We first provide insight as to why label smoothing is sufficient to maintain calibration throughout the SFT process. However, settings remain where the effectiveness of smoothing is severely diminished, in particular the case of large vocabulary LLMs (LV-LLMs). We posit the cause to stem from the ability to become over-confident, which has a direct relationship with the hidden size and vocabulary size, and justify this theoretically and experimentally. Finally, we address an outstanding issue regarding the memory footprint of the cross-entropy loss computation in the label smoothed loss setting, designing a customized kernel to dramatically reduce memory consumption without sacrificing speed or performance in comparison to existing solutions for non-smoothed losses.

Details

NeurIPS Conference 2025 Conference Paper

Mamba Modulation: On the Length Generalization of Mamba Models

Peng Lu
Jerry Huang
Qiuhao Zeng
Xinyu Wang
Boxing Chen
Philippe Langlais
Yufei Cui

The quadratic complexity of the attention mechanism in Transformer models has motivated the development of alternative architectures with sub-quadratic scaling, such as state-space models. Among these, Mamba has emerged as a leading architecture, achieving state-of-the-art results across a range of language modeling tasks. However, Mamba’s performance significantly deteriorates when applied to contexts longer than those seen during pre-training, revealing a sharp sensitivity to context length extension. Through detailed analysis, we attribute this limitation to the out-of-distribution behavior of its state-space dynamics, particularly within the parameterization of the state transition matrix $A$. Unlike recent works which attribute this sensitivity to the vanished accumulation of discretization time steps, $\exp(-\sum_{t=1}^N{\Delta}_t)$, we establish a connection between state convergence behavior as the input length approaches infinity and the spectrum of the transition matrix $A$, offering a well-founded explanation of its role in length extension. Next, to overcome this challenge, we propose an approach that applies spectrum scaling to pre-trained Mamba models to enable robust long-context generalization by selectively modulating the spectrum of $A$ matrices in each layer. We show that this can significantly improve performance in settings where simply modulating ${\Delta}_t$ fails, validating our insights and providing avenues for better length generalization of state-space models with structured transition matrices.

PDF Details

AAAI Conference 2025 Conference Paper

Novel View Synthesis Under Large-Deviation Viewpoint for Autonomous Driving

Xin Ma
Jiguang Zhang
Peng Lu
Shibiao Xu
Chengwei Pan

Novel view synthesis is a critical task in autonomous driving. Although 3D Gaussian Splatting (3D-GS) has shown success in generating novel views, it faces challenges in maintaining high-quality rendering when viewpoints deviate significantly from the training set. This difficulty primarily stems from complex lighting conditions and geometric inconsistencies in texture-less regions. To address these issues, we propose an attention-based illumination model that leverages light fields from neighboring views, enhancing the realism of synthesized images. Additionally, we propose a geometry optimization method using planar homography to improve geometric consistency in texture-less regions. Our experiments demonstrate substantial improvements in synthesis quality for large-deviation viewpoints, validating the effectiveness of our approach.

PDF Details DOI

IROS Conference 2025 Conference Paper

Numerical Optimization-based Kinematics with Pose Tracking Control for Continuum Robots

Rui Peng
Ping Deng
Duo Tang
Peng Lu

In this paper, we employ multiple IMUs into a triple-section continuum manipulator to precisely capture the attitude data of each section’s end disk. Leveraging the sensory and mechanical hardware system, we construct a sophisticated coordinate transformation scheme to accurately identify the detailed configuration states of the manipulator. Additionally, we introduce a numerical optimization strategy to develop a unified forward and inverse kinematic modeling framework, ensuring both iterative efficiency and accuracy. Through the IMUs’ real-time attitude feedback, we implement a closed-loop controller, enhancing the manipulator’s operational robustness and agility. In our experimental evaluations, we assess the convergence performance of both forward and inverse kinematics within a simulated environment and validate the precision of these kinematic models through real-time experiments on an actual continuum manipulator. Moreover, we evaluate the performance of the proposed controller by examining its accuracy during the manipulator’s continuous motions and analyzing its response characteristics. In contrast to previous research on continuum robots in the literature, we pioneer a fully integrated kinematic control architecture that is successfully implemented on a physical continuum robotic system.

Details

ICRA Conference 2025 Conference Paper

RM-Planner: Integrating Reinforcement Learning with Whole-Body Model Predictive Control for Mobile Manipulation

Zixuan Zhuang
Le Zheng
Wanyue Li
Renming Liu
Peng Lu
Hui Cheng

Mobile manipulation is a crucial problem in various real-world applications. However, existing methods have demonstrated unsatisfactory training efficiency and sparse rewards, requiring complex coordination strategies between the mobile base and arm. In this paper, we propose RM-Planner, a planning method for mobile manipulation tasks in unknown complex environments. By adopting a two-layer hierarchical framework, we utilize a whole-body Model Predictive Control (MPC)-based low-level planner to track subgoals and generate aggressive but safe joint commands throughout the entire manipulation process, while a Reinforcement Learning (RL)based high-level policy directly uses 3D point cloud representations of the environment, guiding the robot to achieve optimal manipulation postures based on current observations and specific task objectives. We conduct extensive simulations and real-world experiments, where RM-planner significantly outperforms state-of-the-art methods. Our code will be released at https://github.com/SYSU-RoboticsLab/RM-Planner.git.

Details

ICLR Conference 2025 Conference Paper

ZETA: Leveraging Z-order Curves for Efficient Top-k Attention

Qiuhao Zeng
Jerry Huang
Peng Lu
Gezheng Xu
Boxing Chen
Charles X. Ling
Boyu Wang 0004

Over recent years, the Transformer has become a fundamental building block for sequence modeling architectures. Yet at its core is the use of self-attention, whose memory and computational cost grow quadratically with the sequence length $N$, rendering it prohibitively expensive for long sequences. A promising approach is top-$k$ attention, which selects only the $k$ most relevant tokens and achieves performance comparable to vanilla self-attention while significantly reducing space and computational demands. However, causal masks require the current query token to only attend to past tokens, preventing existing top-$k$ attention methods from efficiently searching for the most relevant tokens in parallel, thereby limiting training efficiency. In this work, we propose ZETA, leveraging Z-Order Curves for Efficient Top-k Attention, to enable parallel querying of past tokens for entire sequences. We first theoretically show that the choice of key and query dimensions involves a trade-off between the curse of dimensionality and the preservation of relative distances after projection. In light of this insight, we propose reducing the dimensionality of keys and queries in contrast to values and further leveraging Z-order curves to map low-dimensional keys and queries into one-dimensional space, which permits parallel sorting, thereby largely improving the efficiency for top-$k$ token selection. Experimental results demonstrate that ZETA~matches the performance of standard attention on synthetic tasks Associative Recall and outperforms attention and its variants on Long-Range Arena and WikiText-103 language modeling.

Details

EAAI Journal 2024 Journal Article

AGCVT-prompt for sentiment classification: Automatically generating chain of thought and verbalizer in prompt learning

Xu Gu
XiaoLiang Chen
Peng Lu
Zonggen Li
YaJun Du
XianYong Li

Large language models (LLMs) have revolutionized natural language processing, but they require significant data and hardware resources. Prompt learning offers a solution by enabling a single model for multiple downstream tasks. However, current prompt learning methods rely on costly prompt templates for training. This is a challenge for tasks like sentiment classification, where high-quality templates are hard to create and pseudo-token composed templates can be expensive to train. Recent studies on the chain of thought (COT) have shown that enhancing the presentation of certain aspects of the reasoning process can improve the performance of LLMs. With this in mind, this research introduces the auto-generated COT and verbalizer templates (AGCVT-Prompt) technique, which clusters unlabeled texts according to their identified topic and sentiment. Subsequently, it generates dual verbalizers and formulates both topic and sentiment prompt templates, utilizing the categories discerned within the text and verbalizers. This method significantly improves the transparency and interpretability of the model’s decision-making processes. The AGCVT-Prompt technique was evaluated against conventional prompt learning and advanced sentiment classification methods, using state-of-the-art LLMs on both Chinese and English datasets. The results showed superior performance in all evaluations. Specifically, the AGCVT-Prompt method outperformed previous prompt learning techniques in few-shot learning scenarios, providing higher zero-shot and few-shot learning capabilities. Additionally, AGCVT-Prompt was utilized to analyze network comments about Corona Virus Disease 2019, providing valuable insights. These findings indicate that AGCVT-Prompt is a promising alternative for sentiment classification tasks, particularly in situations where labeled data is scarce.

Details DOI

EAAI Journal 2024 Journal Article

Agent-based modeling of high-rise building fires reveals self-rescue behaviors and better fire protection designs

Peng Lu
Zhuo Zhang
Chiamaka Henrietta Onyebuchi
Lifan Zheng

It is always challenging to seek external rescue assistance in high-rise building fires. Therefore, it is critical for individuals to master survival skills. For crowd dynamics modeling, previous research focused on numerical simulations and building designs with little attention to the self-rescue mechanism. It is critical to understanding crowd evacuations and better response strategies. We modeled the Grenfell Tower (a high-rise building with a complicated structure) case in 2017. Based on the percolation and social force models, we build an agent-based model to simulate individual behaviors inside. We obtain the optimal solution and robust paralleled outcomes under all counterfactual situations based on precisely matching tangible case outcomes (fire duration, deaths, and injuries). For individuals, mastering self-rescue skills is better at reducing social losses (deaths & injuries). In terms of high-rise buildings design, the central alarm system is also useful to reduce them. Besides, the crowd evacuation guided by the social force model also reduces deaths & injuries. This work provides insight into better high-rise building design and practical response strategies for societies. The central alarm system and fire-proof materials should be used in high-rise buildings. The residents should have routine training in social force-based evacuations and survival (self-rescue) skills to better the evacuation process and outcome under natural disasters and social emergencies.

Details DOI

ICRA Conference 2024 Conference Paper

Thermoformed electronic skins for conformal tactile sensor arrays

Peng Lu
Jiaming Liang
Bidan Huang
Sicheng Yang
Wang Wei Lee

Robots and prostheses are increasingly designed with curvilinear surfaces for functional, aesthetic, aerodynamic, and safety reasons. Electronic skins (e-skins) capable of sensing contact location and pressure across complex, non-developable surfaces are essential for empowering next-generation robots with tactile awareness. This will facilitate safe and natural human-machine interactions while enhancing object manipulation capabilities. Despite the evident advantages of conformal e-skins, current fabrication methods face significant challenges in realizing their full potential. In this paper, we introduce thermoforming as a technique to efficiently fabricate tactile sensitive e-skins that conform to curvilinear surfaces. The performance, repeatability and uniformity of the sensors are characterized in detail. We also present a custom calibration pipeline where accurate digital replicas of conformal e-skins are generated for use in simulations. Finally, we demonstrate the benefits of 3D e-skins in a tool manipulation task.

Details

IROS Conference 2024 Conference Paper

TRX-Hand5: An Anthropomorphic Hand with Integrated Tactile Feedback for Grasping and Manipulation in Human Environments

Sicheng Yang
Wang Wei Lee
Zhong Zhang 0015
Youda Xiong
Jiaming Liang
Peng Lu
Yonghui Zhu
Tianliang Liu

Objects of daily life are designed to suit the human hand. Without major modifications to these objects and our environments, robots will need end-effectors with human hand-like configuration and dexterity to efficiently operate on them. Tight integration of tactile and proprioceptive sensors are also critical to ensure robust execution of manipulation policies without sacrificing range-of-motion. Reliability is also key, and a mechanically robust, easy to repair end-effector is important to minimize downtime. To meet these challenges, we designed a 13 degree-of-freedom anthropomorphic hand with over 1000 tactile sensing elements, named TRX-Hand5. Also embedded within are positional encoders and cable tension sensors to provide proprioceptive perception. TRX-Hand5 has a novel biomimetic topology with six small posture motors in the palm to replicate the function of intrinsic hand muscles and five large power motors in the forearm to play the role of forearm flexor muscles. The whole hand weighs 2. 6 kg with its dimensions comparable to those of an adult male’s hand and is capable of actuating its fingertips at over 200°/s while exerting up to 22 N of force. The system can be disassembled in modules for easy maintenance.

Details

ICML Conference 2024 Conference Paper

VinT-6D: A Large-Scale Object-in-hand Dataset from Vision, Touch and Proprioception

Zhaoliang Wan
Yonggen Ling
Senlin Yi
Lu Qi 0001
Wang Wei Lee
Minglei Lu
Sicheng Yang
Xiao Teng

This paper addresses the scarcity of large-scale datasets for accurate object-in-hand pose estimation, which is crucial for robotic in-hand manipulation within the "Perception-Planning-Control" paradigm. Specifically, we introduce VinT-6D, the first extensive multi-modal dataset integrating vision, touch, and proprioception, to enhance robotic manipulation. VinT-6D comprises 2 million VinT-Sim and 0. 1 million VinT-Real entries, collected via simulations in Mujoco and Blender and a custom-designed real-world platform. This dataset is tailored for robotic hands, offering models with whole-hand tactile perception and high-quality, well-aligned data. To the best of our knowledge, the VinT-Real is the largest considering the collection difficulties in the real-world environment so it can bridge the gap of simulation to real compared to the previous works. Built upon VinT-6D, we present a benchmark method that shows significant improvements in performance by fusing multi-modal information. The project is available at https: //VinT-6D. github. io/.

Details