Author name cluster

Xiaowei Zhou 0001

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

9 papers

1 author row

ICLR Conference 2025 Conference Paper

Ready-to-React: Online Reaction Policy for Two-Character Interaction Generation

Zhi Cen
Huaijin Pi
Sida Peng
Qing Shuai
Yujun Shen
Hujun Bao
Xiaowei Zhou 0001
Ruizhen Hu

This paper addresses the task of generating two-character online interactions. Previously, two main settings existed for two-character interaction generation: (1) generating one's motions based on the counterpart's complete motion sequence, and (2) jointly generating two-character motions based on specific conditions. We argue that these settings fail to model the process of real-life two-character interactions, where humans will react to their counterparts in real time and act as independent individuals. In contrast, we propose an online reaction policy, called Ready-to-React, to generate the next character pose based on past observed motions. Each character has its own reaction policy as its ``brain'', enabling them to interact like real humans in a streaming manner. Our policy is implemented by incorporating a diffusion head into an auto-regressive model, which can dynamically respond to the counterpart's motions while effectively mitigating the error accumulation throughout the generation process. We conduct comprehensive experiments using the challenging boxing task. Experimental results demonstrate that our method outperforms existing baselines and can generate extended motion sequences. Additionally, we show that our approach can be controlled by sparse signals, making it well-suited for VR and other online interactive environments. Code and data will be made publicly available.

Details

ICLR Conference 2025 Conference Paper

UniRestore3D: A Scalable Framework For General Shape Restoration

Yuang Wang
Yujian Zhang
Sida Peng
Xingyi He
Haoyu Guo
Yujun Shen
Hujun Bao
Xiaowei Zhou 0001

Shape restoration aims to recover intact 3D shapes from defective ones, such as those that are incomplete, noisy, and low-resolution. Previous works have achieved impressive results in shape restoration subtasks thanks to advanced generative models. While effective for specific shape defects, they are less applicable in real-world scenarios involving multiple defect types simultaneously. Additionally, training on limited subsets of defective shapes hinders knowledge transfer across restoration types and thus affects generalization. In this paper, we address the task of general shape restoration, which restores shapes with various types of defects through a unified model, thereby naturally improving the applicability and scalability. Our approach first standardizes the data representation across different restoration subtasks using high-resolution TSDF grids and constructs a large-scale dataset with diverse types of shape defects. Next, we design an efficient hierarchical shape generation model and a noise-robust defective shape encoder that enables effective impaired shape understanding and intact shape generation. Moreover, we propose a scalable training strategy for efficient model training. The capabilities of our proposed method are demonstrated across multiple shape restoration subtasks and validated on various datasets, including Objaverse, ShapeNet, GSO, and ABO.

Details

ICLR Conference 2024 Conference Paper

Neural Polynomial Gabor Fields for Macro Motion Analysis

Chen Geng 0001
Hong-Xing Yu
Sida Peng
Xiaowei Zhou 0001
Jiajun Wu 0001

We study macro motion analysis, where macro motion refers to the collection of all visually observable motions in a dynamic scene. Traditional filtering-based methods on motion analysis typically focus only on local and tiny motions, yet fail to represent large motions or 3D scenes. Recent dynamic neural representations can faithfully represent motions using correspondences, but they cannot be directly used for motion analysis. In this work, we propose Phase-based neural polynomial Gabor fields (Phase-PGF), which learns to represent scene dynamics with low-dimensional time-varying phases. We theoretically show that Phase-PGF has several properties suitable for macro motion analysis. In our experiments, we collect diverse 2D and 3D dynamic scenes and show that Phase-PGF enables dynamic scene analysis and editing tasks including motion loop detection, motion factorization, motion smoothing, and motion magnification. Project page: https://chen-geng.com/phasepgf

Details

ICRA Conference 2023 Conference Paper

Perceiving Unseen 3D Objects by Poking the Objects

Linghao Chen
Yunzhou Song
Hujun Bao
Xiaowei Zhou 0001

We present a novel approach to interactive 3D object perception for robots. Unlike previous perception algorithms that rely on known object models or a large amount of annotated training data, we propose a poking-based approach that automatically discovers and reconstructs 3D objects. The poking process not only enables the robot to discover unseen 3D objects but also produces multi-view observations for 3D reconstruction of the objects. The reconstructed objects are then memorized by neural networks with regular supervised learning and can be recognized in new test images. The experiments on real-world data show that our approach could unsupervisedly discover and reconstruct unseen 3D objects with high quality, and facilitate real-world applications such as robotic grasping. The code and supplementary materials are available at the project page: https://zju3dv.github.io/poking_perception/.

Details

ICRA Conference 2020 Conference Paper

A Low-Rank Matrix Approximation Approach to Multiway Matching with Applications in Multi-Sensory Data Association

Spyridon Leonardos
Xiaowei Zhou 0001
Kostas Daniilidis

Consider the case of multiple visual sensors perceiving the same scene from different viewpoints. In order to achieve consistent visual perception, the problem of data association, in this case establishing correspondences between observed features, must be first solved. In this work, we consider multiway matching which is a specific instance of multi-sensory data association. Multiway matching refers to the problem of establishing correspondences among a set of images from noisy pairwise correspondences, typically by exploiting cycle- consistency. We propose a novel optimization-based formulation of multiway matching problem as a nonconvex low-rank matrix approximation problem. We propose two novel algorithms for numerically solving the problem at hand. The first one is an algorithm based on the Alternating Direction Method of Multipliers (ADMM). The second one is a Riemannian trust- region method on the multinomial manifold, the manifold of strictly positive stochastic matrices, equipped with the Fisher information metric. Experimental results demonstrate that the proposed methods have the state of the art performance in multiway matching while reducing the computational complexity compared to the state of the art.

Details

ICRA Conference 2018 Conference Paper

Human Motion Capture Using a Drone

Xiaowei Zhou 0001
Sikang Liu 0002
Georgios Pavlakos
Vijay Kumar 0001
Kostas Daniilidis

Current motion capture (MoCap) systems generally require markers and multiple calibrated cameras, which can be used only in constrained environments. In this work we introduce a drone-based system for 3D human MoCap. The system only needs an autonomously flying drone with an on-board RGB camera and is usable in various indoor and outdoor environments. A reconstruction algorithm is developed to recover full-body motion from the video recorded by the drone. We argue that, besides the capability of tracking a moving subject, a flying drone also provides fast varying viewpoints, which is beneficial for motion reconstruction. We evaluate the accuracy of the proposed system using our new DroCap dataset and also demonstrate its applicability for MoCap in the wild using a consumer drone.

Details

ICRA Conference 2017 Conference Paper

6-DoF object pose from semantic keypoints

Georgios Pavlakos
Xiaowei Zhou 0001
Aaron Chan
Konstantinos G. Derpanis
Kostas Daniilidis

This paper presents a novel approach to estimating the continuous six degree of freedom (6-DoF) pose (3D translation and rotation) of an object from a single RGB image. The approach combines semantic keypoints predicted by a convolutional network (convnet) with a deformable shape model. Unlike prior work, we are agnostic to whether the object is textured or textureless, as the convnet learns the optimal representation from the available training image data. Furthermore, the approach can be applied to instance- and class-based pose recovery. Empirically, we show that the proposed approach can accurately recover the 6-DoF object pose for both instance- and class-based scenarios with a cluttered background. For class-based object pose estimation, state-of-the-art accuracy is shown on the large-scale PASCAL3D+ dataset.

Details

ICRA Conference 2017 Conference Paper

Distributed consistent data association via permutation synchronization

Spyridon Leonardos
Xiaowei Zhou 0001
Kostas Daniilidis

Data association is one of the fundamental problems in multi-sensor systems. Most current techniques rely on pairwise data associations which can be spurious even after the employment of outlier rejection schemes. Considering multiple pairwise associations at once significantly increases accuracy and leads to consistency. In this work, we propose a fully decentralized method for globally consistent data association from pairwise data associations based on a distributed averaging scheme on the set of doubly stochastic matrices. We demonstrate the effectiveness of the proposed method using theoretical analysis and experimental evaluation.

Details

ICRA Conference 2016 Conference Paper

Articulated motion estimation from a monocular image sequence using spherical tangent bundles

Spyridon Leonardos
Xiaowei Zhou 0001
Kostas Daniilidis

We propose a second order stochastic dynamical model for generic articulated objects whose state space is a Riemannian manifold naturally suggested by the articulation constraints. We derive the equations of a Riemannian Extended Kalman Filter to perform the structure estimation from an image sequence captured by a perspective camera. In order to theoretically validate our approach, we prove that the proposed model is locally weakly observable. Finally, we report quantitative results on both synthetic data and on real sequences from the CMU Mocap dataset.

Details