Multi-Step Deformable Gaussian Splatting for Dynamic Scene Rendering

Reconstructing dynamic scenes has long been a challenging task in 3D vision. Previous mainstream methods based on 3D Gaussian Splatting typically employ a single deformation field to directly model spatiotemporal changes. However, such one-step deformation struggles to capture diverse and complex motion patterns. To address this limitation, we propose decomposing the one-step deformation into a multi-step process, where each step is represented by a deformation layer. Additionally, we introduce a weight prediction mechanism for each layer to control the extent of deformation at every step. We provide two types of deformation layers based on implicit and explicit approaches. Moreover, while the deformation layer is time-conditioned, the Gaussians' behavior may still be influenced by their time-invariant properties. Therefore, we propose a fully time-agnostic scale modulation block to modulate the scaling changes of Gaussians. Extensive experiments on D-NeRF, Neu3D, and HyperNeRF demonstrate that our method achieves state-of-the-art performance.

Authors

• Jiaheng Hu The University of Texas at Austin
• Zhizhong Zhang East China Normal University Shanghai Key Laboratory of Computer Software Evaluating and Testing
• Jingyu Gong East China Normal University
• Lizhuang Ma Shanghai Jiao Tong University, Shanghai, China East China Normal University, Shanghai, China MoE Key Lab of Artificial Intelligence, Shanghai Jiao Tong University, Shanghai, China
• Xin Tan East China Normal University, Shanghai, China
• Yuan Xie East China Normal University, Shanghai, China

Keywords

No keywords are indexed for this paper.