Author name cluster

Neil Dhir

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

6 papers

2 author rows

ICML Conference 2025 Conference Paper

Neurosymbolic World Models for Sequential Decision Making

Leonardo Hernandez Cano
Maxine Perroni-Scharf
Neil Dhir
Arun Ramamurthy
Armando Solar-Lezama

We present Structured World Modeling for Policy Optimization (SWMPO), a framework for unsupervised learning of neurosymbolic Finite State Machines (FSM) that capture environmental structure for policy optimization. Traditional unsupervised world modeling methods rely on unstructured representations, such as neural networks, that do not explicitly represent high-level patterns within the system (e. g. , patterns in the dynamics of regions such as water and land ). Instead, SWMPO models the environment as a finite state machine (FSM), where each state corresponds to a specific region with distinct dynamics. This structured representation can then be leveraged for tasks like policy optimization. Previous works that synthesize FSMs for this purpose have been limited to discrete spaces, not continuous spaces. Instead, our proposed FSM synthesis algorithm operates in an unsupervised manner, leveraging low-level features from unprocessed, non-visual data, making it adaptable across various domains. The synthesized FSM models are expressive enough to be used in a model-based Reinforcement Learning scheme that leverages offline data to efficiently synthesize environment-specific world models. We demonstrate the advantages of SWMPO by benchmarking its environment modeling capabilities in simulated environments.

Details

NeurIPS Conference 2025 Conference Paper

Non-Stationary Structural Causal Bandits

Yeahoon Kwon
Yesong Choe
Soungmin Park
Neil Dhir
Sanghack Lee

We study the problem of sequential decision-making in environments governed by evolving causal mechanisms. Prior work on structural causal bandits--formulations that integrate causal graphs into multi-armed bandit problems to guide intervention selection--has shown that leveraging the causal structure can reduce unnecessary interventions by identifying possibly-optimal minimal intervention sets (POMISs). However, such formulations fall short in dynamic settings where reward distributions may vary over time, as their static, hence myopic, nature focuses on immediate rewards and overlooks the long-term effects of interventions. In this work, we propose a non-stationary structural causal bandit framework that leverages temporal structural causal models to capture evolving dynamics over time. We characterize how interventions propagate over time by developing graphical tools and assumptions, which form the basis for identifying non-myopic intervention strategies. Within this framework, we devise POMIS$^+$, which captures the existence of variables that contribute to maximizing both immediate and long-term rewards. Our framework provides a principled way to reason about temporally-aware interventions by explicitly modeling information propagation across time. Empirical results validate the effectiveness of our approach, demonstrating improved performance over myopic baselines.

PDF Details

NeurIPS Conference 2021 Conference Paper

Dynamic Causal Bayesian Optimization

Virginia Aglietti
Neil Dhir
Javier González
Theodoros Damoulas

We study the problem of performing a sequence of optimal interventions in a dynamic causal system where both the target variable of interest, and the inputs, evolve over time. This problem arises in a variety of domains including healthcare, operational research and policy design. Our approach, which we call Dynamic Causal Bayesian Optimisation (DCBO), brings together ideas from decision making, causal inference and Gaussian process (GP) emulation. DCBO is useful in scenarios where the causal effects are changing over time. Indeed, at every time step, DCBO identifies a local optimal intervention by integrating both observational and past interventional data collected from the system. We give theoretical results detailing how one can transfer interventional information across time steps and define a dynamic causal GP model which can be used to find optimal interventions in practice. Finally, we demonstrate how DCBO identifies optimal interventions faster than competing approaches in multiple settings and applications.

PDF Details

ICRA Conference 2018 Conference Paper

Locomotion Envelopes for Adaptive Control of Powered Ankle Prostheses

Neil Dhir
Houman Dallali
Evandro Maicon Ficanha
Guilherme Aramizo Ribeiro
Mo Rastgaar

In this paper we combine Gaussian process regression and impedance control, to illicit robust, anthropomorphic, adaptive control of a powered ankle prosthesis. We learn the non-linear manifolds which guide how locomotion variables temporally evolve, and regress that surface over a velocity range to create a manifold. The joint set of manifolds, as well as the temporal evolution of the gait-cycle duration is what we term a locomotion envelope. Current powered prostheses have problems adapting across speeds. It is likely that humans rely upon a control strategy which is adaptable, can become more robust and accurate with more data and provides a nonparametric approach which allows the strategy to grow with the number of observations. We demonstrate such a strategy in this study and successfully simulate locomotion well beyond our training data. The method we propose is based on common physical features observed in numerous human subjects walking at different speeds. Based on the derived locomotion envelopes we show that ankle power increases monotonically with speed among all subjects. We demonstrate our methods in simulation and human experiments, on a powered ankle foot prosthesis to demonstrate the effectiveness of the method.

Details

UAI Conference 2017 Conference Paper

Interpreting Lion Behaviour as Probabilistic Programs

Neil Dhir
Matthijs Vákár
Matthew Wijers
Andrew Markham
Frank Wood

We consider the problem of unsupervised learning of meaningful behavioural segments of high-dimensional time-series observations, collected from a pride of African lions1. We demonstrate, by way of a probabilistic programming system (PPS), a methodology which allows for quick iteration over models and Bayesian inferences, which enables us to learn meaningful behavioural segments. We introduce a new Bayesian nonparametric (BNP) state-space model, which extends the hierarchical Dirichlet process (HDP) hidden Markov model (HMM) with an explicit BNP treatment of duration distributions, to deal with different levels of granularity of the latent behavioural space of the lions. The ease with which this is done exemplifies the flexibility that a PPS gives a scientist2. Furthermore, we combine this approach with unsupervised feature learning, using variational autoencoders.

Details

IROS Conference 2016 Conference Paper

Nonparametric Bayesian models for unsupervised activity recognition and tracking

Neil Dhir
Yura N. Perov
Frank Wood

Human locomotion and activity recognition systems form a critical part in a robot's ability to safely and effectively operate in a environment populated with human end users. Previous work in this area relies upon strong assumptions about the labels in the training data; e. g. that are noise-free and that they exist at all. Our approach does not predefine the relevant behaviours or their number, as both are learned directly from observations, similar to real-world human-robot interactions, where labels are neither available. Instead we introduce models that make no assumptions about the state space, by presenting a fully unsupervised nonparametric Bayesian recognition approach, in which we leverage recent advances in state space modelling with automatic inference using probabilistic programming. We demonstrate the utility of full model optimisation using Bayesian optimisation and validate our approach on several challenging problems, using different feature modalities.

Details