ECAI Conference 2025 Conference Paper
Disentangling Belief and Inference: Adaptive Algorithms for Individual Human Reasoning
- Meghna Bhadra
With the advancement of AI, cooperation between AI systems and humans solving complex problems, akin to human teams, will become increasingly common. The success behind human collaboration may be attributed to our ability to simulate and thus comprehend the cognitive state of a collaborative partner (called the Theory of Mind in Psychology) – something that current AI systems are yet to fully achieve. Furthermore, human information processing is complex, with underlying reasoning mechanisms depending not only on applied inference strategies but also on interactions with one’s beliefs and biases. This paper investigates whether it is possible to automatically develop adaptive cognitive algorithms capable of predicting human information processing, at the level of individual reasoners. Towards this goal, it is necessary to disentangle the impact of an individual’s beliefs from underlying logical or heuristic inference strategies. To date, various works in psychology and cognitive science have not only studied which logical inferences individual reasoners draw, but also how confidently they endorse a given inference (e. g. on a Likert-scale from 1 to 6). The current work uses a large meta-analysis involving 993 individuals solving (un)believable syllogistic problems, a classic and well-studied area of logical reasoning, as a benchmark. It presents predictive algorithms that make individual-specific predictions and have been developed using large language models in a Human-AI collaborative framework. The algorithms surpass the previously highest mean accuracy among existing belief-augmented models by approximately 4%. It also improves upon the best to date predicted ratings, with a mean absolute difference of 1. 13 between prediction and ground truth. The proposed method is generalizable to other domains. Its potential and caveats are also discussed.