Author name cluster

Xuanbin Peng

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

4 papers

1 author row

ICLR Conference 2025 Conference Paper

3D-Spatial Multimodal Memory

Xueyan Zou
Yuchen Song
Ri-Zhao Qiu
Xuanbin Peng
Jianglong Ye
Sifei Liu
Xiaolong Wang 0004

We present 3D Spatial MultiModal Memory (M3), a multimodal memory system designed to retain information about medium-sized static scenes through video sources for visual perception. By integrating 3D Gaussian Splatting techniques with foundation models, M3 builds a multimodal memory capable of rendering feature representations across granularities, encompassing a wide range of knowledge. In our exploration, we identify two key challenges in previous works on feature splatting: (1) computational constraints in storing high-dimensional features for each Gaussian primitive, and (2) misalignment or information loss between distilled features and foundation model features. To address these challenges, we propose M3 with key components of principal scene components and Gaussian memory attention, enabling efficient training and inference. To validate M3, we conduct comprehensive quantitative evaluations of feature similarity and downstream tasks, as well as qualitative visualizations to highlight the pixel trace of Gaussian memory attention. Our approach encompasses a diverse range of foundation models, including vision-language models (VLMs), perception models, and large multimodal and language models (LMMs/LLMs). Furthermore, to demonstrate real-world applicability, we deploy M3’s feature field in indoor scenes on a quadruped robot. Notably, we claim that M3 is the first work to address the core compression challenges in 3D feature distillation.

Details

ICRA Conference 2025 Conference Paper

WildLMa: Long Horizon Loco-Manipulation in the Wild

Ri-Zhao Qiu
Yuchen Song
Xuanbin Peng
Sai Aneesh Suryadevara
Ge Yang
Minghuan Liu
Mazeyu Ji
Chengzhe Jia

‘In-the-wild’ mobile manipulation aims to deploy robots in diverse real-world environments, which requires the robot to (1) have skills that generalize across object configurations; (2) be capable of long-horizon task execution in diverse environments; and (3) perform complex manipulation beyond pick-and-place. Quadruped robots with manipulators hold promise for extending the workspace and enabling robust locomotion, but existing results do not investigate such a capability. This paper proposes WildLMa with three components to address these issues: (1) adaptation of learned low-level controller for VR-enabled whole-body teleoperation and traversability; (2) WildLMa-Skill - a library of generalizable visuomotor skills acquired via imitation learning or heuristics and (3) WildLMa-Planner - an interface of learned skills that allow LLM planners to coordinate skills for long-horizon tasks. We demonstrate the importance of high-quality training data by achieving higher grasping success rate over existing RL baselines using only tens of demonstrations. WildLMa exploits CLIP for language-conditioned imitation learning that empirically generalizes to objects unseen in training demonstrations. Besides extensive quantitative evaluation, we qualitatively demonstrate practical robot applications, such as cleaning up trash in university hallways or outdoor terrains, operating articulated objects, and rearranging items on a bookshelf.

Details

IROS Conference 2024 Conference Paper

RTTF: Rapid Tactile Transfer Framework for Contact-Rich Manipulation Tasks

Qiwei Wu 0001
Xuanbin Peng
Jiayu Zhou
Zhuoran Sun
Xiaogang Xiong
Yunjiang Lou

An increasing number of robotic manipulation tasks now use optical tactile sensors to provide tactile feedback, making tactile servo control a crucial aspect of robotic operations. This paper presents a rapid tactile transfer framework (RTTF) that achieves optical-tactile image sim2real transfer and robust tactile servo control using limited paired data. The sim2real aspect of RTTF employs a semi-supervised approach, beginning with pretraining the latent space representations of tactile images and subsequently mapping different tactile image domains to a shared latent space within a simulated tactile image domain. This latent space, combined with the proprioceptive information of the robotic arm, is then integrated into a privileged learning framework for policy training, which results in a deployable tactile control policy. Our results demonstrate the robustness of the proposed framework in achieving task objectives across different tactile sensors with varying physical parameters. Furthermore, manipulators equipped with tactile sensors, allow for rapid training and deployment for diverse contact-rich tasks, including object pushing and surface following.

Details

IROS Conference 2024 Conference Paper

Whole-body Compliance Control for Quadruped Manipulator with Actuation Saturation of Joint Torque and Ground Friction

Tianlin Zhang
Xuanbin Peng
Fenghao Lin
Xiaogang Xiong
Yunjiang Lou

In normal operations, when quadruped manipulators with impedance control experience external disturbances, they may become unstable and lose balance due to actuation saturation, affecting their stability, safety, and compliance with the environment. To address this issue, we propose a whole-body compliance controller to prevent unstable behaviors like slip, oscillation, and overshoot, which arise from actuation saturation. The controller includes an admittance scheme with a set-valued operator as the internal feedback, to constrain joint torques within actuators’ limits and ground reaction forces within friction cones to ensure stability against disturbances. Then, it formulates a hierarchical optimization problem using the Hierarchical Quadratic Programming (HQP) to impose the output of the admittance scheme while ensuring physical consistency to maintain compliance behaviors. Unlike traditional compliance control with one-dimensional torque limitations, our approach considers both joints torque limits of manipulator joints and friction cones of quadruped ground reaction as actuation saturation. This ensures overall compliance and stability for the quadruped manipulators, even under significant external forces, regardless of where they are exerted on the robot. We demonstrate through experiments involving variable stiffness environments and external forces during normal operations how effective our approach is in enhancing the safety of quadruped manipulators.

Details