Arrow Research search

Author name cluster

Logan Cross

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

2 papers
1 author row

Possible papers

2

NeurIPS Conference 2024 Conference Paper

FactorSim: Generative Simulation via Factorized Representation

  • Fan-Yun Sun
  • S. I. Harini
  • Angela Yi
  • Yihan Zhou
  • Alex Zook
  • Jonathan Tremblay
  • Logan Cross
  • Jiajun Wu

Generating simulations to train intelligent agents in game-playing and robotics from natural language input, user input, or task documentation remains an open-ended challenge. Existing approaches focus on parts of this challenge, such as generating reward functions or task hyperparameters. Unlike previous work, we introduce FACTORSIM that generates full simulations in code from language input that can be used to train agents. Exploiting the structural modularity specific to coded simulations, we propose to use a factored partially observable Markov decision process representation that allows us to reduce context dependence during each step of the generation. For evaluation, we introduce a generative simulation benchmark that assesses the generated simulation code’s accuracy and effectiveness in facilitating zero-shot transfers in reinforcement learning settings. We show that FACTORSIM outperforms existing methods in generating simulations regarding prompt alignment (i. e. , accuracy), zero-shot transfer abilities, and human evaluation. We also demonstrate its effectiveness in generating robotic tasks.

PDF Details DOI

RLDM Conference 2019 Conference Abstract

Combining deep reinforcement learning with fMRI to probe the encoding of state-space in the brain

  • Logan Cross
  • Yisong Yue
  • John P. O’Doherty

Models of reinforcement learning (RL) detail a computational framework for how agents should learn to take actions in order to maximize cumulative reward. Numerous studies have found implemen- tations of components of RL algorithms in the brain. However to date these processes have been studied in simplistic task environments such as bandit tasks or MDPs, which do not capture the high dimensional complexity of environments easily dealt with by humans in the real-world. It is unknown how the brain is capable of extracting state-space representations, nor how the brain computes action-values in these richer environments. A computational approach to solving this problem has involved artificial neural networks such as the deep Q network (DQN), which are capable of learning complex tasks with human level per- formance. Here, we assess the viability of a similar approach as applied to behavioral and fMRI data as a means of providing insight into the brain’s computational strategies for extracting visual features relevant to reward and action. Human subjects freely played three Atari video games during fMRI scanning. Our fMRI analysis identified DQN correlates of action value and state value in premotor regions and vmPFC respectively. Additionally, we utilized the hidden layers of the network as a model for state space represen- tation and used an encoding model analysis to map voxel activity to network activity in the hidden layers. Dorsal visual and motor cortical areas were mapped to the last hidden layer of the DQN. Additionally, asso- ciation cortical areas, such as precuneus, superior parietal lobe, and insula show correspondence with state space representations across multiple games. These results indicate that deep Q networks can effectively capture variance in BOLD activity in sensorimotor and prefrontal cortices related to the brain’s strategy for extracting relevant state-space features and in computing values for actions in complex high dimensional environments.

PDF Details
v2026.09.13