Author name cluster

Aldo Faisal

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

4 papers

1 author row

NeurIPS Conference 2025 Conference Paper

Variational Uncertainty Decomposition for In-Context Learning

I. Shavindra Jayasekera
Jacob Si
Filippo Valdettaro
Wenlong Chen
Aldo Faisal
Yingzhen Li

As large language models (LLMs) gain popularity in conducting prediction tasks in-context, understanding the sources of uncertainty in in-context learning becomes essential to ensuring reliability. The recent hypothesis of in-context learning performing predictive Bayesian inference opens the avenue for Bayesian uncertainty estimation, particularly for decomposing uncertainty into epistemic uncertainty due to lack of in-context data and aleatoric uncertainty inherent in the in-context prediction task. However, the decomposition idea remains under-explored due to the intractability of the latent parameter posterior from the underlying Bayesian model. In this work, we introduce a variational uncertainty decomposition framework for in-context learning without explicitly sampling from the latent parameter posterior, by optimising auxiliary inputs as probes to obtain an upper bound to the aleatoric uncertainty of an LLM's in-context learning procedure. Through experiments on synthetic and real-world tasks, we show quantitatively and qualitatively that the decomposed uncertainties obtained from our method exhibit desirable properties of epistemic and aleatoric uncertainty.

PDF Details

NeurIPS Conference 2018 Conference Paper

Representation Balancing MDPs for Off-policy Policy Evaluation

Yao Liu
Omer Gottesman
Aniruddh Raghu
Matthieu Komorowski
Aldo Faisal
Finale Doshi-Velez
Emma Brunskill

We study the problem of off-policy policy evaluation (OPPE) in RL. In contrast to prior work, we consider how to estimate both the individual policy value and average policy value accurately. We draw inspiration from recent work in causal reasoning, and propose a new finite sample generalization error bound for value estimates from MDP models. Using this upper bound as an objective, we develop a learning algorithm of an MDP model with a balanced representation, and show that our approach can yield substantially lower MSE in common synthetic benchmarks and a HIV treatment simulation domain.

PDF Details

RLDM Conference 2015 Conference Abstract

The Moveable Feast of Predictive Reward Discounting in Humans

Bernardo Caldas
Benedikt Schoen-
Guy-Bart Stan
Aldo Faisal

This work investigates the implicit discounting that humans use to compare rewards that may oc- cur at different points in the future. We show that the way discounting is applied is not constant, but changes depending on context and in particular can be influenced by the apparent complexity of the environment. To investigate this, we conduct a series of neurophysics experiments, in which participants perform discrete- time, sequential, 2AC tasks with non-episodic characteristics and varying reward structure. The varying rewards in our games cause participants behaviour to change giving a characteristic signal of their future re- ward discounting. Model-free, model-based and hybrid reinforcement learning models are fit to participant data, as well as a lighter weight model which does not assume a learning mechanism. Results show that the complexity of the task affects the geometric discount factor, relating to the length of time that partici- pants may wait for reward. This in turn indicates that participants may be optimising some hidden objective function that is not dependent on the discount factor.

PDF Details

RLDM Conference 2015 Conference Abstract

Towards Closed-Loop Mortality Prediction and Off-Policy Learning of Medical Decision De- rived from Very Large Scale Intensive Care Unit Databases

Matthieu Komorowski
Aldo Faisal

Introduction As informatisation of medical care continues to progress, increasing amount of healthcare data is being collected. These datasets offer the potential to inform key clinical questions in an objective data driven manner. The intensive care unit (ICU) is a data-intensive environment where patients suffer from high mortality rates (15 to 50%), which gives the opportunity to make a real impact on patient prognosis. Dynamics of a patient’s evolution result from closed loop interactions between the patient and the interventions ordered by the physician. Reinforcement learning approaches lend themselves naturally to capture these interactions. Method and results The analyses were performed on the Multiparameter Intelli- gent Monitoring in Intensive Care II (MIMIC-II) open database, which contains high-resolution ICU data of 32, 536 patients. Data regarding their vitals, lab tests, demographics, and treatments received were collected. The current dataset, comprising over 150, 000 records of patients in 125 dimensions, was clustered into fi- nite states. The treatments (drugs delivered) were grouped into an action-space by expert input. The end outcomes were numerically translated by assigning a reward to discharged patients and a penalty to deaths. These discrete states were then used to run a Markov Decision Process (MDP) model of disease progres- sion. In off-policy learning, the value of the optimal policy is acquired independently of the agent’s actions. In this framework, off-policy reinforcement learning was able to predict outcomes and identify an optimal policy, which maximises the likelihood of discharge. Conclusions MDPs offer an appealing mathemati- cal framework for modelling clinical decision making, because they are able to capture patients’ dynamic through various states. The application of machine learning algorithms to medical data has the potential to lead to the development of meaningful tools capable of improving outcomes and cost-effectiveness.

PDF Details