RLDM Conference 2015 Conference Abstract
- Nathan Wispinski
- Christopher Madan
- Craig Chapman
When acting in complex environments, humans often need to make decisions involving risky and ambiguous options. That is, decisions frequently involve options that can have multiple outcomes (risk), and information about those outcomes and/or their respective probabilities of occurrence can be uncertain (ambiguity). We investigated biases in a reaching task using both implicit (reaction times and reach trajecto- ries) and explicit (choices, personality inventories, and probability estimates) measures while subjects made decisions involving options for which they were given perfect information, and those for which they only had information about potential outcomes and not their associated probabilities of occurrence. However, participants were given feedback about selected options on every trial, and thus learned about ambiguous options through experience. Overall, we found that each measure revealed distinct results: probability esti- mates were relatively accurate; early choices were biased by novelty-seeking and later choices were biased toward described information; and reaction times and reaching movements were primarily driven by reward and differences in expected value, respectively. Overall, our results demonstrate that how information about options is acquired, how decisions are physically made, and the individual differences between participants are important, though often overlooked, components of learning and decision making, which can reveal important aspects about human cognitive processing when integrated. This research presents novel experi- mental data showing distinct behavioral biases at different levels of cognition during decision making which can be used to constrain plausible models of human reinforcement learning, and also shows that the use of multiple methods may provide valuable information for future human reinforcement-learning research. Poster T28*: Utility-weighted sampling in decisions from experience Falk Lieder*, UC Berkeley; Thomas Griffiths, UC Berkeley; Ming Hsu, UC Berkeley People overweight extreme events in decision-making and overestimate their frequency. Previous theoretical work has shown that this apparently irrational bias could result from utility-weighted sampling-a decision mechanism that makes rational use of limited computational resources (Lieder, Hsu, & Griffiths, 2014). Here, we show that utility-weighted sampling can emerge from a neurally plausible associative learning mechanism. Our model explains the over-weighting of extreme outcomes in repeated decisions from experience (Ludvig, Madan, & Spetch, 2014), as well as the overestimation of their frequency and the underlying memory biases (Madan, Ludvig, & Spetch, 2014). Our results support the conclusion that utility drives probability-weighting by biasing the neural simulation of potential consequences towards extreme values.