Author name cluster

William A. Cunningham

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

4 papers

1 author row

NeurIPS Conference 2024 Conference Paper

Temporal-Difference Learning Using Distributed Error Signals

Jonas Guan
Shon E. Verch
Claas Voelcker
Ethan C. Jackson
Nicolas Papernot
William A. Cunningham

A computational problem in biological reward-based learning is how credit assignment is performed in the nucleus accumbens (NAc). Much research suggests that NAc dopamine encodes temporal-difference (TD) errors for learning value predictions. However, dopamine is synchronously distributed in regionally homogeneous concentrations, which does not support explicit credit assignment (like used by backpropagation). It is unclear whether distributed errors alone are sufficient for synapses to make coordinated updates to learn complex, nonlinear reward-based learning tasks. We design a new deep Q-learning algorithm, Artificial Dopamine, to computationally demonstrate that synchronously distributed, per-layer TD errors may be sufficient to learn surprisingly complex RL tasks. We empirically evaluate our algorithm on MinAtar, the DeepMind Control Suite, and classic control tasks, and show it often achieves comparable performance to deep RL algorithms that use backpropagation.

PDF Details DOI

YNIMG Journal 2020 Journal Article

Decomposing the neural pathways in a simple, value-based choice

Timothy R. Koscik
Vincent Man
Andrew Jahn
Christina H. Lee
William A. Cunningham

Understanding the neural implementation of value-based choice has been an important focus of neuroscience for several decades. Although a consensus has emerged regarding the brain regions involved, including ventromedial prefrontal cortex (vmPFC), posterior parietal cortex (PPC), and the ventral striatum (vSTR), the multifaceted nature of decision processes is one cause of persistent debate regarding organization of the value-based choice network. In the current study, we isolate neural activity related to valuation and choice selection using a gambling task where expected gains and losses are dissociated from choice outcomes. We apply multilevel mediation analysis to formally test whether brain regions identified as part of the value-based choice network mediate between perceptions of expected value and choice to accept or decline a gamble. Our approach additionally makes predictions regarding interregional relationships to elucidate the chain of processing events within the value-based decision network. Finally, we use dynamic causal modelling (DCM) to compare plausible models of interregional relationships in value-based choice. We observe that activity in vmPFC does not predict take/pass choices, but rather is highly associated with outcome evaluation. By contrast, both PPC and bilateral vSTR (bilaterally) mediate the relationship between expected value and choice. Interregional mediation analyses reveal that vSTR fully mediates between PPC and choice, and this is supported by DCM. Together these results suggest that vSTR, and not vmPFC nor PPC, functions as an important driver of choice.

Details DOI

YNIMG Journal 2008 Journal Article

The Quadruple Process model approach to examining the neural underpinnings of prejudice

Jennifer S. Beer
Mirre Stallen
Michael V. Lombardo
Karen Gonsalkorale
William A. Cunningham
Jeffrey W. Sherman

In order to investigate the systems underlying the automatic and controlled processes that support social attitudes, we conducted an fMRI study that combined an implicit measure of race attitudes with the Quadruple Process model (Quad model). A number of previous neural investigations have adopted the Implicit Association Test (IAT) to examine the automatic processes that contribute to social attitudes. Application of the Quad model builds on this previous research by permitting measures of distinct automatic and controlled processes that contribute to performance on the IAT. The present research found that prejudiced attitudes of ingroup favoritism were associated with amygdala, medial and right lateral orbitofrontal cortex. In contrast, prejudiced attitudes of outgroup negativity were associated with caudate and left lateral orbitofrontal cortex. Frontal regions found in previous neural research on the IAT, such as anterior cingulate, dorsolateral prefrontal cortex and operculum were associated with detecting appropriate responses in situations in which they conflict with automatic associations. Insula activity was associated with attitudes towards ingroup and outgroup members, as well as detecting appropriate behavior.

Details DOI

YNIMG Journal 2005 Journal Article

Attitudes to the right- and left: Frontal ERP asymmetries associated with stimulus valence and processing goals

William A. Cunningham
Stacey D. Espinet
Colin G. DeYoung
Philip David Zelazo

We used dense-array event-related potentials (ERP) to examine the time course and neural bases of evaluative processing. Participants made good vs. bad (evaluative) and abstract vs. concrete (nonevaluative) judgments of socially relevant concepts (e. g. , “murder, ” “welfare”), and then rated all concepts for goodness and badness. Results revealed a late positive potential (LPP) beginning at about 475 ms post-stimulus and maximal over anterior sites. The LPP was lateralized (higher amplitude and shorter latency) on the right for concepts later rated bad, and on the left for concepts later rated good. Moreover, the degree of lateralization for the amplitude but not the latency was larger when participants were making evaluative judgments than when they were making nonevaluative judgments. These data are consistent with a model in which discrete regions of prefrontal cortex (PFC) are specialized for the evaluative processing of positive and negative stimuli.

Details DOI