Author name cluster

Bradley Doll

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

2 papers

1 author row

RLDM Conference 2013 Conference Abstract

How instructed knowledge shapes aversive learning

Lauren Atlas
Bradley Doll
Nathaniel Daw
Jian Li

In humans, expectations reflect prior experience and instructed knowledge. Most models of aver- sive learning make predictions about brain responses as a function of reinforcement alone. Recent studies of reward learning indicate that striatal learning is modulated when participants are instructed about stimulus contingencies. The aim of this study was to test whether instructed knowledge modulates associative fear learning. Participants performed a Pavlovian aversive learning paradigm. Two cues were presented: One (the CS+) was paired with shock on 30 % of trials, whereas the second (the CS-) was never paired with shock. Fol- lowing 20 trials, contingencies reversed. There were three reversals across the session. Participants were assigned to two groups: The Instructed Group was informed about contingencies prior to learning and upon each reversal, whereas the Feedback Group received no information. We analyzed skin conductance responses (SCRs) and brain responses to cues. Fear expression tracked con- tingency reversals (i. e. larger SCRs for current CS+ than CS-), and the Instructed Group showed stronger differential responses. The Instructed Group showed greater activation in right DLPFC, while the Feedback Group showed greater activation in bilateral striatum. We fit a quantitative model with a dynamic learning rate to SCRs to isolate the timecourse of learning in the Feedback Group, focusing on prediction error and associability. We then tested whether Instructions modulated the neural correlates of feedback-driven sig- nals. We observed group differences in bilateral ventral striatum, such that only the Feedback Group showed striatal prediction errors. These results reveal that instructed knowledge influences aversive learning. Instructions enhance fear acqui- sition and expression, and prediction errors are not observed when instructions are veridical. The DLPFC is likely to play a key role in maintaining instructions, which in turn modulate fear expression.

PDF Details

RLDM Conference 2013 Conference Abstract

Neural correlates of forward planning in model-based reinforcement learning

Bradley Doll
Katherine Duncan
Dylan Simon
Daphna Shohamy
Nathaniel Daw

Incremental learning across species is well described by reinforcement learning (RL) algorithms. The bulk of such demonstrations correlate behavioral and biological signals with signatures of model-free RL. More recently, interest has grown in correlates of model-based RL which can exhibit more cognitive flexibility than model-free RL, though at greater computational cost. Using fMRI, we investigated the neural correlates of learning in a task that dissociates model-based from model-free RL, and permits distinct model- based strategies. Participants navigated to terminal task states from different starting states in search of monetary reward. Model-based behavior in the task may arise from the forward planning typical of these algorithms or from a computational shortcut whereby the representation of actions that produce the same outcomes are joined. The states in this task were represented by different classes of stimuli that activate unique regions of visual cortex. This task feature permitted us to assess RL strategies by decoding brain activations at different task states. Across the population, choice behavior showed evidence of both model-based and model-free learning. Pre- liminary fMRI results permitted closer investigation of the mechanisms by which these classes of learning algorithms are implemented. For each subject, we identified ROIs that showed preferential responses to the stimulus categories used to represent the different task states in an independent functional localizer. We then assessed the activity in these ROIs during the reward learning task start states. We looked for activation of states to be navigated to, as well as representational compression of equivalent start states. Activation related to the former correlated with model-based task behavior, consistent with forward planning in model-based RL.

PDF Details