UniGen: Enhanced Training & Test-Time Strategies for Unified Multimodal Understanding and Generation

We introduce UniGen, a unified multimodal large language model (MLLM) capable of image understanding and generation. We study the full training pipeline of UniGen from a data-centric perspective, including multi-stage pre-training, supervised fine-tuning, and direct preference optimization. More importantly, we propose a new Chain-of-Thought Verification (CoT-V) strategy for test-time scaling, which significantly boosts UniGen’s image generation quality using a simple Best-of-N test-time strategy. Specifically, CoT-V enables UniGen to act as both image generator and verifier at test time, assessing the semantic alignment between a text prompt and its generated image in a step-by-step CoT manner. Trained entirely on open-source datasets across all stages, UniGen achieves state-of-the-art performance on a range of image understanding and generation benchmarks, with a final score of 0. 78 on GenEval and 85. 19 on DPG-Bench. Through extensive ablation studies, our work provides actionable insights and addresses key challenges in the full life cycle of building unified MLLMs, contributing meaningful directions to future research. Code is available at https: //github. com/apple/ml-unigen.

Authors

• Rui Tian Harbin Institute of Technology
• Mingfei Gao
• Mingze Xu
• Jiaming Hu
• Jiasen Lu
• Zuxuan Wu Shanghai Key Lab of Intell. Info. Processing, School of CS, Fudan University Shanghai Collaborative Innovation Center of Intelligent Visual Computing
• Yinfei Yang Apple
• Afshin Dehghan

Keywords

No keywords are indexed for this paper.