FreeGaussian: Annotation-free Control of Articulated Objects via 3D Gaussian Splats with Flow Derivatives

Reconstructing controllable Gaussian splats for articulated objects from monocular video is especially challenging due to its inherently insufficient constraints. Existing methods address this by relying on dense masks and manually defined control signals, limiting their real-world applications. In this paper, we propose an annotation-free method, FreeGaussian, which mathematically disentangles camera egomotion and articulated movements via flow derivatives. By establishing a connection between 2D flows and 3D Gaussian dynamic flow, our method enables optimization and continuity of dynamic Gaussian motions from flow priors without any control signals. Furthermore, we introduce a 3D spherical vector controlling scheme, which represents the state as a 3D Gaussian trajectory, thereby eliminating the need for complex 1D control signal calculations and simplifying controllable Gaussian modeling. Extensive experiments on articulated objects demonstrate the state-of-the-art visual performance and precise, part-aware controllability of our method.

Authors

• Qizhi Chen
• Delin Qu Shanghai AI Laboratory Fudan University
• Junli Liu Shanghai AI Laboratory Northwest Polytechnical University
• Yiwen Tang Northwestern Polytechnical University Shanghai AI Laboratory
• Haoming Song Shanghai AI Laboratory
• Dong Wang Nanchang University
• Yuan Yuan School of Artifcial Intelligence, Optics and Electronics (iOPEN), Northwestern Polytechnical University, Xi'an 710072, P.R. China
• Bin Zhao Shanghai AI Laboratory Northwestern Polytechnical University

Keywords

No keywords are indexed for this paper.