MoFu: Scale-Aware Modulation and Fourier Fusion for Multi-Subject Video Generation

Multi-subject video generation aims to synthesize videos from textual prompts and multiple reference images, ensuring that each subject preserves natural scale and visual fidelity. However, current methods face two challenges: scale inconsistency, where variations in subject size lead to unnatural generation, and permutation sensitivity, where the order of reference inputs causes subject distortion. In this paper, we propose MoFu, a unified framework that tackles both challenges. For scale inconsistency, we introduce Scale-Aware Modulation (SMO), an LLM-guided module that extracts implicit scale cues from the prompt and modulates features to ensure consistent subject sizes. To address permutation sensitivity, we present a simple yet effective Fourier Fusion strategy that processes the frequency information of reference features via the Fast Fourier Transform to produce a unified representation. Besides, we design a Scale-Permutation Stability Loss to jointly encourage scale-consistent and permutation-invariant generation. To further evaluate these challenges, we establish a dedicated benchmark with controlled variations in subject scale and reference permutation. Extensive experiments demonstrate that MoFu significantly outperforms existing methods in preserving natural scale, subject fidelity, and overall visual quality.

Authors

• Run Ling JD.com, Inc. Northeastern University
• Ke Cao
• Jian Lu School of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, China
• Ao Ma 360 AI Institute
• Haowei Liu
• Runze He University of Chinese Academy of Sciences
• Changwei Wang Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center, Qilu University of Technology (Shandong Academy of Sciences) Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science
• Rongtao Xu MAIS, Institute of Automation, Chinese Academy of Sciences
• Yihua Shao The Hong Kong Polytechnic University Institute of Automation, Chinese Academy of Sciences
• Zhanjie Zhang Zhejiang University
• Peng Wu Northwestern Polytechnical University
• Guibing Guo Northeastern University
• Wei Feng Xi'an Jiaotong University
• Zheng Zhang
• Jingjing Lv JD.com
• Junjie Shen JD.com
• Ching Law JD.com
• Xingwei Wang Northeastern University

Keywords

No keywords are indexed for this paper.