Zero-Shot Sketch-Based Image Retrieval via Graph Convolution Network

Zero-Shot Sketch-based Image Retrieval (ZS-SBIR) has been proposed recently, putting the traditional Sketch-based Image Retrieval (SBIR) under the setting of zero-shot learning. Dealing with both the challenges in SBIR and zero-shot learning makes it become a more difficult task. Previous works mainly focus on utilizing one kind of information, i. e. , the visual information or the semantic information. In this paper, we propose a SketchGCN model utilizing the graph convolution network, which simultaneously considers both the visual information and the semantic information. Thus, our model can effectively narrow the domain gap and transfer the knowledge. Furthermore, we generate the semantic information from the visual information using a Conditional Variational Autoencoder rather than only map them back from the visual space to the semantic space, which enhances the generalization ability of our model. Besides, feature loss, classi- fication loss, and semantic loss are introduced to optimize our proposed SketchGCN model. Our model gets a good performance on the challenging Sketchy and TU-Berlin datasets.

Authors

• Zhaolong Zhang
• Yuejie Zhang School of Computer Science, Shanghai Key Laboratory of Intelligent Information Processing, Fudan University Shanghai Collaborative Innovation Center of Intelligent Visual Computing
• Rui Feng School of Computer Science, Shanghai Key Laboratory of Intelligent Information Processing, Fudan University Shanghai Collaborative Innovation Center of Intelligent Visual Computing
• Tao Zhang
• Weiguo Fan

Keywords

No keywords are indexed for this paper.