Author name cluster

Cheng Lin

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

5 papers

2 author rows

NeurIPS Conference 2025 Conference Paper

TrackingWorld: World-centric Monocular 3D Tracking of Almost All Pixels

Jiahao Lu
Weitao Xiong
Jiacheng Deng
Peng Li
Tianyu Huang
Zhiyang Dou
Cheng Lin
Sai-Kit Yeung

Monocular 3D tracking aims to capture the long-term motion of pixels in 3D space from a single monocular video and has witnessed rapid progress in recent years. However, we argue that the existing monocular 3D tracking methods still fall short in separating the camera motion from foreground dynamic motion and cannot densely track newly emerging dynamic subjects in the videos. To address these two limitations, we propose TrackingWorld, a novel pipeline for dense 3D tracking of almost all pixels within a world-centric 3D coordinate system. First, we introduce a tracking upsampler that efficiently lifts the arbitrary sparse 2D tracks into dense 2D tracks. Then, to generalize the current tracking methods to newly emerging objects, we apply the upsampler to all frames and reduce the redundancy of 2D tracks by eliminating the tracks in overlapped regions. Finally, we present an efficient optimization-based framework to back-project dense 2D tracks into world-centric 3D trajectories by estimating the camera poses and the 3D coordinates of these 2D tracks. Extensive evaluations on both synthetic and real-world datasets demonstrate that our system achieves accurate and dense 3D tracking in a world-centric coordinate frame.

PDF Details

NeurIPS Conference 2025 Conference Paper

🎧MOSPA: Human Motion Generation Driven by Spatial Audio

Shuyang Xu
Zhiyang Dou
Mingyi Shi
Liang Pan
Leo Ho
Jingbo Wang
Yuan Liu
Cheng Lin

Enabling virtual humans to dynamically and realistically respond to diverse auditory stimuli remains a key challenge in character animation, demanding the integration of perceptual modeling and motion synthesis. Despite its significance, this task remains largely unexplored. Most previous works have primarily focused on mapping modalities like speech, audio, and music to generate human motion. As of yet, these models typically overlook the impact of spatial features encoded in spatial audio signals on human motion. To bridge this gap and enable high-quality modeling of human movements in response to spatial audio, we introduce the first comprehensive "Spatial Audio-Driven Human Motion" (SAM) dataset, which contains diverse and high-quality spatial audio and motion data. For benchmarking, we develop a simple yet effective diffusion-based generative framework for human "MOtion generation driven by SPatial Audio, " termed MOSPA, which faithfully captures the relationship between body motion and spatial audio through an effective fusion mechanism. Once trained, MOSPA can generate diverse realistic human motions conditioned on varying spatial audio inputs. We perform a thorough investigation of the proposed dataset and conduct extensive experiments for benchmarking, where our method achieves state-of-the-art performance on this task. Our code and model are publicly available at https: //github. com/xsy27/Mospa-Acoustic-driven-Motion-Generation. git

PDF Details

NeurIPS Conference 2024 Conference Paper

Era3D: High-Resolution Multiview Diffusion using Efficient Row-wise Attention

Peng Li
Yuan Liu
Xiaoxiao Long
Feihu Zhang
Cheng Lin
Mengfei Li
Xingqun Qi
Shanghang Zhang

In this paper, we introduce Era3D, a novel multiview diffusion method that generates high-resolution multiview images from a single-view image. Despite significant advancements in multiview generation, existing methods still suffer from camera prior mismatch, inefficacy, and low resolution, resulting in poor-quality multiview images. Specifically, these methods assume that the input images should comply with a predefined camera type, e. g. a perspective camera with a fixed focal length, leading to distorted shapes when the assumption fails. Moreover, the full-image or dense multiview attention they employ leads to a dramatic explosion of computational complexity as image resolution increases, resulting in prohibitively expensive training costs. To bridge the gap between assumption and reality, Era3D first proposes a diffusion-based camera prediction module to estimate the focal length and elevation of the input image, which allows our method to generate images without shape distortions. Furthermore, a simple but efficient attention layer, named row-wise attention, is used to enforce epipolar priors in the multiview diffusion, facilitating efficient cross-view information fusion. Consequently, compared with state-of-the-art methods, Era3D generates high-quality multiview images with up to a 512×512 resolution while reducing computation complexity of multiview attention by 12x times. Comprehensive experiments demonstrate the superior generation power of Era3D- it can reconstruct high-quality and detailed 3D meshes from diverse single-view input images, significantly outperforming baseline multiview diffusion methods.

PDF Details DOI

IROS Conference 2023 Conference Paper

Less Than Human: How Different Users of Telepresence Robots Expect Different Social Norms

Cheng Lin
Jimin Rhim
AJung Moon

Does the norm of first-come-first-serve (FCFS) equally apply to those piloting a Mobile Remote Presence (MRP) system as to those who are physically present with it? While telepresence robots could make social interactions more accessible and enjoyable for geographically-constrained individuals, such an outcome requires both pilots and local users of MRPs to share the same social norm expectations that govern their use. To address this question, we conducted an online study $(N=903)$ involving simulated human-MRP interaction scenarios. Our results suggest that those remotely piloting the MRP-rather than local users-assign the robot to a lower social priority; they find it more appropriate when local users ignore queue order than when pilots ignore queue order. Furthermore, we provide significant empirical evidence that local users expect different social norms to be upheld depending on how they perceive the robot. Those who perceive MRPs simply as robots-rather than an extension of a person-do not expect the FCFS norm to be respected for MRPs.

Details

ICRA Conference 2022 Conference Paper

Roboethics as a Design Challenge: Lessons Learned from the Roboethics to Design and Development Competition

Jimin Rhim
Cheng Lin
Alexander Werner
Brandon J. DeHart
Vivian Qiang
Shalaleh Rismani
AJung Moon

How do we make concrete progress towards de-signing robots that can navigate ethically sensitive contexts? Almost two decades after the word ‘roboethics’ was coined, translating interdisciplinary roboethics discussions into techni-cal design still remains a daunting task. This paper describes our first attempt at addressing these challenges through a roboethics-themed design competition. The design competition setting allowed us to (a) formulate ethical considerations as an engineering design task that anyone with basic programming skills can tackle; and (b) develop a prototype evaluation scheme that incorporates diverse normative perspectives of multiple stakeholders. The initial implementation of the competition was held online at the RO-MAN 2021 conference. The competition task involved programming a simulated mobile robot (TIAGo) that delivers items for individuals in the home environment, where many of these tasks involve ethically sensitive con-texts (e. g. , an underage family member asks for an alcoholic drink). This paper outlines our experiences implementing the competition and the lessons we learned. We highlight design competitions as a promising mechanism to enable a new wave of roboethics research equipped with technical design solutions.

Details