CycleVTON: A Cycle Mapping Framework for Parser-Free Virtual Try-On

Image-based virtual try-on aims to transfer a target clothing onto a specific person. A significant challenge is arbitrarily matched clothing and person lack corresponding ground truth to supervised learning. A recent pioneering work leveraged an improved cycleGAN to enable one network to generate the desired image for another network during training. However, there is no difference in the result distribution before and after the clothing changes. Therefore, using two different networks is unnecessary and may even increase the difficulty of convergence. Furthermore, the introduced human parsing used to provide body structure information in the input also have a negative impact on the try-on result. How to employ a single network for supervised learning while eliminating human parsing? To tackle these issues, we present a Cycle mapping Virtual Try-On Network (CycleVTON), which can produce photo-realistic try-on results by using a cycle mapping framework without the parser. In particular, we introduce a flow constraint loss to achieve supervised learning of arbitrarily matched clothing and person as inputs to the deformer, thus naturally mimicking the interaction between clothing and the human body. Additionally, we design a skin generation strategy that can adapt to the shape of the target clothing by dynamically adjusting the skin region, i.e., by first removing and then filling skin areas. Extensive experiments conducted on challenging benchmarks demonstrate that our proposed method exhibits superior performance compared to state-of-the-art methods.

Authors

• Chenghu Du Wuhan University of Technology
• Junyin Wang Wuhan University of Technology
• Yi Rong Wuhan University of Technology
• Shuqing Liu Wuhan Textile University
• Kai Liu
• Shengwu Xiong Shanghai Artificial Intelligence Laboratory; Interdisciplinary Artificial Intelligence Research Institute, Wuhan College

Keywords

• APP: Other Applications
• CV: Applications
• CV: Biometrics, Face, Gesture & Pose
• CV: Computational Photography, Image & Video Synthesis
• CV: Representation Learning for Vision
• HAI: Applications
• HAI: Human-in-the-loop Machine Learning
• HAI: User Experience and Usability
• ML: Applications
• ML: Deep Generative Models & Autoencoders
• ML: Deep Learning Algorithms