Re-Attention for Visual Question Answering

Visual Question Answering (VQA) requires a simultaneous understanding of images and questions. Existing methods achieve well performance by focusing on both key objects in images and key words in questions. However, the answer also contains rich information which can help to better describe the image and generate more accurate attention maps. In this paper, to utilize the information in answer, we propose a reattention framework for the VQA task. We first associate image and question by calculating the similarity of each objectword pairs in the feature space. Then, based on the answer, the learned model re-attends the corresponding visual objects in images and reconstructs the initial attention map to produce consistent results. Benefiting from the re-attention procedure, the question can be better understood, and the satisfactory answer is generated. Extensive experiments on the benchmark dataset demonstrate the proposed method performs favorably against the state-of-the-art approaches.

Authors

• Wenya Guo Nankai University
• Ying Zhang School of Computer Science, Northwestern Polytechnical University
• Xiaoping Wu
• Jufeng Yang
• Xiangrui Cai College of Computer Science, Nankai University Key Laboratory of Data and Intelligent System Security, Ministry of Education, China Science and Technology on Communication Networks Laboratory, Shijiazhuang, China
• Xiaojie Yuan Nankai University

Keywords

No keywords are indexed for this paper.