Shaping Human–AI Collaboration in Education: Effects of AI-Assisted Decision-Making Paradigms and Human–AI Decision Consistency on Pre-Service Teachers’ Psychological States and Performance

Artificial intelligence is playing an increasingly important role in supporting decision-making, particularly in educational contexts, where it serves as a critical tool to assist teacher judgment and optimize instructional decisions. However, limited research has examined how different AI-assisted decision-making paradigms influence the Performance of human-AI collaboration, as well as the underlying psychological mechanisms and causal pathways. Therefore, this study investigated 59 pre-service teachers to examine how AI-assisted decision-making paradigms and human-AI consistency influenced their psychological states and task performance. Specifically, this study employed a two-factor mixed experimental design, with the AI-assisted decision-making paradigms as the between-subjects factor and human-AI consistency as the within-subjects factor. Data were analyzed using the Bayesian cumulative link mixed model and structural equation modeling. The results reveal that AI-assisted decision-making paradigms do not have a significant direct effect on task performance. However, when the moderating role of human-AI decision consistency is taken into account, the effect of AI-assisted decision-making paradigms on task performance can exert its influence indirectly through a sequential psychological pathway involving users’ confidence and their trust in the AI. Consistency between human and AI decisions not only significantly enhances users’ trust in AI, confidence, and task performance, but the proportion of consistent decisions also significantly moderates the impact of AI-assisted decision-making paradigms on users’ confidence levels. Notably, our findings indicate that users maintain a moderately level of trust in AI even when their decisions diverge from those of AI. In summary, this study highlights the mediating mechanism by which AI-assisted decision-making paradigms influence task performance through psychological states and identifies the moderating role of human-AI consistency in this pathway. These findings advance the theoretical understanding of human-AI interaction models in educational contexts and offer mechanistic insights to guide the optimization of instructional AI systems.

Authors

• Yingying Wang
• Qin Ni Key Laboratory of Multilingual Education with AI, Shanghai International Studies University
• Haoxin Xu School of Education, Shanghai International Studies University, 1550 Wengxiang Road, Shanghai 201620, China
• Jiaqi Yin School of Education, Hainan Normal University, 99 Longkun South Road, Haikou 571158, China
• Tingjiang Wei Lab of Artificial Intelligence for Education, East China Normal University

Keywords

No keywords are indexed for this paper.