RLDM Conference 2013 Conference Abstract
Affective Mechanisms of Reinforcement Learning in Social and Non-Social Decision-Making
- Filippo Rossi
- Ian Fasel
- Marian Bartlett
- Alan Sanfey
Behavioral and neuroscientific evidence shows that mathematical models such as reinforcement learning (RL) can account for very sophisticated dynamic decisions. People adapt their behavior based on gradual adjustments of their beliefs from feedback. However, the motivational mechanisms underlying these adjustments remain poorly understood. We suggest that emotions play an integral role in how we learn from feedback. We collected data from participants playing a multi-armed bandit task, and recorded facial expres- sions during the game. Participants’ behavior was modeled using Kalman filters, and we sought to establish a relationship between RL variables, such as prediction errors, and participants’ emotions assessed using facial expressions. In addition, participants were presented with a “social” version of the same task in order to investigate whether learning and emotional processes differ in social and non-social environments. Our results show that the absolute magnitude of prediction errors (receiving more/less money than expected) is predicted by the facial expressions of surprise and fear. Additionally, in social decisions negative prediction errors (receiving less money than expected) trigger negative emotions such as sadness, anger and fear. These negative emotions may explain the larger volatility that we observe in social behavior – namely, that players are more likely to change strategies when followed by negative prediction errors in a social environment as compared to non-social decisions. These results suggest that our approach can be used to map latent constructs from reinforcement learning onto emotions. Furthermore, our findings contribute to the study of dynamic decision-making by identifying the affective substrate of learning.