VProChart: Answering Chart Question Through Visual Perception Alignment Agent and Programmatic Solution Reasoning

Charts are widely used for data visualization across various fields, including education, research, and business. Chart Question Answering (CQA) is an emerging task focused on the automatic interpretation and reasoning of data presented in charts. However, chart images are inherently difficult to interpret, and chart-related questions often involve complex logical and numerical reasoning, which hinders the performance of existing models. This paper introduces VProChart, a novel framework designed to address these challenges in CQA by integrating a lightweight Visual Perception Alignment Agent (VPAgent) and a Programmatic Solution Reasoning approach. VPAgent aligns and models chart elements based on principles of human visual perception, enhancing the understanding of chart context. The Programmatic Solution Reasoning approach leverages large language models (LLMs) to transform natural language reasoning questions into structured solution programs, facilitating precise numerical and logical reasoning. Extensive experiments on benchmark datasets such as ChartQA and PlotQA demonstrate that VProChart significantly outperforms existing methods, highlighting its capability in understanding and reasoning with charts.

Authors

• Muye Huang School of Computer Science and Technology, Xi’an Jiaotong University MOE KLINNS Lab, Xi’an Jiaotong University
• Lingling Zhang
• Han Lai School of Computer Science and Technology, Xi’an Jiaotong University MOE KLINNS Lab, Xi’an Jiaotong University
• Wenjun Wu School of Computer Science and Technology, Xi’an Jiaotong University Shaanxi Province Key Laboratory of Big Data Knowledge Engineering
• Xinyu Zhang School of Computer Science and Technology, Xi’an Jiaotong University Shaanxi Province Key Laboratory of Big Data Knowledge Engineering
• Jun Liu Carnegie Mellon University

Keywords

No keywords are indexed for this paper.