Reference Guided Face Component Editing

Face portrait editing has achieved great progress in recent years. However, previous methods either 1) operate on pre-defined face attributes, lacking the flexibility of controlling shapes of high-level semantic facial components (e. g. , eyes, nose, mouth), or 2) take manually edited mask or sketch as an intermediate representation for observable changes, but such additional input usually requires extra efforts to obtain. To break the limitations (e. g. shape, mask or sketch) of the existing methods, we propose a novel framework termed r FACE (Reference Guided FAce Component Editing) for diverse and controllable face component editing with geometric changes. Specifically, r-FACE takes an image inpainting model as the backbone, utilizing reference images as conditions for controlling the shape of face components. In order to encourage the framework to concentrate on the target face components, an example-guided attention module is designed to fuse attention features and the target face component features extracted from the reference image. Through extensive experimental validation and comparisons, we verify the effectiveness of the proposed framework.

Authors

• Qiyao Deng
• Jie Cao Lanzhou University of Technology
• Yunfan Liu AI Lab, Research Center for Industries of the Future, Westlake University
• Zhenhua Chai
• Qi Li
• Zhenan Sun School of Artificial Intelligence, University of Chinese Academy of Sciences State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences

Keywords

• Computer Vision: 2D and 3D Computer Vision
• Computer Vision: Biometrics, Face and Gesture Recognition