VHM: Versatile and Honest Vision Language Model for Remote Sensing Image Analysis

This paper develops a Versatile and Honest vision language Model (VHM) for remote sensing image analysis. VHM is built on a large-scale remote sensing image-text dataset with rich-content captions (VersaD), and an honest instruction dataset comprising both factual and deceptive questions (HnstD). Unlike prevailing remote sensing image-text datasets, in which image captions focus on a few prominent objects and their relationships, VersaD captions provide detailed information about image properties, object attributes, and the overall scene. This comprehensive captioning enables VHM to thoroughly understand remote sensing images and perform diverse remote sensing tasks. Moreover, different from existing remote sensing instruction datasets that only include factual questions, HnstD contains additional deceptive questions stemming from the non-existence of objects. This feature prevents VHM from producing affirmative answers to nonsense queries, thereby ensuring its honesty. In our experiments, VHM significantly outperforms various vision language models on common tasks of scene classification, visual question answering, and visual grounding. Additionally, VHM achieves competent performance on several unexplored tasks, such as building vectorizing, multi-label classification and honest question answering.

Authors

• Chao Pang School of Artificial Intelligence, Wuhan University School of Computer Science, Wuhan University
• Xingxing Weng School of Computer Science, Wuhan University
• Jiang Wu Shanghai Artificial Intelligence Laboratory
• Jiayu Li
• Yi Liu City University of Hong Kong
• Jiaxing Sun Shanghai Artificial Intelligence Laboratory State Key Lab. of LIESMARS, Wuhan University
• Weijia Li Lanzhou University
• Shuai Wang Chalmers University of Technology
• Litong Feng SenseTime Research
• Gui-Song Xia School of Artificial Intelligence, Wuhan University, Wuhan, China
• Conghui He Shanghai AI Lab SenseTime Research

Keywords

No keywords are indexed for this paper.