Author name cluster

Jae Hee Lee

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

14 papers

1 author row

TMLR Journal 2025 Journal Article

Mental Modelling of Reinforcement Learning Agents by Language Models

Wenhao Lu
Xufeng Zhao
Josua Spisak
Jae Hee Lee
Stefan Wermter

Can emergent language models faithfully model the intelligence of decision-making agents? Though modern language models already exhibit some reasoning ability, and theoretically can potentially express any probable distribution over tokens, it remains underexplored how the world knowledge these pre-trained models have memorized can be utilised to comprehend an agent's behaviour in the physical world. This paper empirically examines, for the first time, how well large language models (LLMs) can build a mental model of reinforcement learning (RL) agents, termed agent mental modelling, by reasoning about an agent's behaviour and its effect on states from agent interaction history. This research attempts to unveil the potential of leveraging LLMs for elucidating RL agent behaviour, addressing a key challenge in explainable RL. To this end, we propose specific evaluation metrics and test them on selected RL task datasets of varying complexity, reporting findings on agent mental model establishment. Our results disclose that LLMs are not yet capable of fully realising the mental modelling of agents through inference alone without further innovations. This work thus provides new insights into the capabilities and limitations of modern LLMs, highlighting that while they show promise in understanding agents with a longer history context, preexisting beliefs within LLMs about behavioural optimum and state complexity limit their ability to fully comprehend an agent's behaviour and action effects.

PDF Details

EWRL Workshop 2025 Workshop Paper

Mental Modelling of Reinforcement Learning Agents by Language Models

Wenhao Lu
Xufeng Zhao
Josua Spisak
Jae Hee Lee
Stefan Wermter

PDF

NAI Journal 2024 Journal Article

From neural activations to concepts: A survey on explaining concepts in neural networks

Jae Hee Lee
Sergio Lanza
Stefan Wermter

In this paper, we review recent approaches for explaining concepts in neural networks. Concepts can act as a natural link between learning and reasoning: once the concepts are identified that a neural learning system uses, one can integrate those concepts with a reasoning system for inference or use a reasoning system to act upon them to improve or enhance the learning system. On the other hand, knowledge can not only be extracted from neural networks but concept knowledge can also be inserted into neural network architectures. Since integrating learning and reasoning is at the core of neuro-symbolic AI, the insights gained from this survey can serve as an important step towards realizing neuro-symbolic AI based on explainable concepts.

Details DOI

TMLR Journal 2024 Journal Article

Read Between the Layers: Leveraging Multi-Layer Representations for Rehearsal-Free Continual Learning with Pre-Trained Models

Kyra Ahrens
Hans Hergen Lehmann
Jae Hee Lee
Stefan Wermter

We address the Continual Learning (CL) problem, wherein a model must learn a sequence of tasks from non-stationary distributions while preserving prior knowledge upon encountering new experiences. With the advancement of foundation models, CL research has pivoted from the initial learning-from-scratch paradigm towards utilizing generic features from large-scale pre-training. However, existing approaches to CL with pre-trained models primarily focus on separating class-specific features from the final representation layer and neglect the potential of intermediate representations to capture low- and mid-level features, which are more invariant to domain shifts. In this work, we propose LayUP, a new prototype-based approach to CL that leverages second-order feature statistics from multiple intermediate layers of a pre-trained network. Our method is conceptually simple, does not require access to prior data, and works out of the box with any foundation model. LayUP surpasses the state of the art in four of the seven class-incremental learning benchmarks, all three domain-incremental learning benchmarks and in six of the seven online continual learning benchmarks, while significantly reducing memory and computational requirements compared to existing baselines. Our results demonstrate that fully exhausting the representational capacities of pre-trained models in CL goes well beyond their final embeddings.

PDF Details

IJCAI Conference 2022 Conference Paper

What is Right for Me is Not Yet Right for You: A Dataset for Grounding Relative Directions via Multi-Task Learning

Jae Hee Lee
Matthias Kerzel
Kyra Ahrens
Cornelius Weber
Stefan Wermter

Understanding spatial relations is essential for intelligent agents to act and communicate in the physical world. Relative directions are spatial relations that describe the relative positions of target objects with regard to the intrinsic orientation of reference objects. Grounding relative directions is more difficult than grounding absolute directions because it not only requires a model to detect objects in the image and to identify spatial relation based on this information, but it also needs to recognize the orientation of objects and integrate this information into the reasoning process. We investigate the challenging problem of grounding relative directions with end-to-end neural networks. To this end, we provide GRiD-3D, a novel dataset that features relative directions and complements existing visual question answering (VQA) datasets, such as CLEVR, that involve only absolute directions. We also provide baselines for the dataset with two established end-to-end VQA models. Experimental evaluations show that answering questions on relative directions is feasible when questions in the dataset simulate the necessary subtasks for grounding relative directions. We discover that those subtasks are learned in an order that reflects the steps of an intuitive pipeline for processing relative directions.

PDF Details DOI

IJCAI Conference 2020 Conference Paper

Deep Hurdle Networks for Zero-Inflated Multi-Target Regression: Application to Multiple Species Abundance Estimation

Shufeng Kong
Junwen Bai
Jae Hee Lee
Di Chen
Andrew Allyn
Michelle Stuart
Malin Pinsky
Katherine Mills

A key problem in computational sustainability is to understand the distribution of species across landscapes over time. This question gives rise to challenging large-scale prediction problems since (i) hundreds of species have to be simultaneously modeled and (ii) the survey data are usually inflated with zeros due to the absence of species for a large number of sites. The problem of tackling both issues simultaneously, which we refer to as the zero-inflated multi-target regression problem, has not been addressed by previous methods in statistics and machine learning. In this paper, we propose a novel deep model for the zero-inflated multi-target regression problem. To this end, we first model the joint distribution of multiple response variables as a multivariate probit model and then couple the positive outcomes with a multivariate log-normal distribution. By penalizing the difference between the two distributions’ covariance matrices, a link between both distributions is established. The whole model is cast as an end-to-end learning framework and we provide an efficient learning algorithm for our model that can be fully implemented on GPUs. We show that our model outperforms the existing state-of-the-art baselines on two challenging real-world species distribution datasets concerning bird and fish populations.

PDF Details DOI

JAAMAS Journal 2018 Journal Article

A new distributed algorithm for efficient generalized arc-consistency propagation

Shufeng Kong
Jae Hee Lee
Sanjiang Li

Abstract Generalized arc-consistency propagation is predominantly used in constraint solvers to efficiently prune the search space when solving constraint satisfaction problems. Although many practical applications can be modelled as distributed constraint satisfaction problems, no distributed arc-consistency algorithms so far have considered the privacy of individual agents. In this paper, we propose a new distributed arc-consistency algorithm, called \(\mathsf {DisAC3. 1}\), which leaks less private information of agents than existing distributed arc-consistency algorithms. In particular, \(\mathsf {DisAC3. 1}\) uses a novel termination determination mechanism, which allows the agents to share domains, constraints and communication addresses only with relevant agents. We further extend \(\mathsf {DisAC3. 1}\) to \(\mathsf {DisGAC3. 1}\), which is the first distributed algorithm that enforces generalized arc-consistency on k -ary ( \(k\ge 2\) ) constraint satisfaction problems. Theoretical analyses show that our algorithms are efficient in both time and space. Experiments also demonstrate that \(\mathsf {DisAC3. 1}\) outperforms the state-of-the-art distributed arc-consistency algorithm and that \(\mathsf {DisGAC3. 1}\) ’s performance scales linearly in the number of agents.

Details DOI

AAAI Conference 2018 Conference Paper

Multiagent Simple Temporal Problem: The Arc-Consistency Approach

Shufeng Kong
Jae Hee Lee
Sanjiang Li

The Simple Temporal Problem (STP) is a fundamental temporal reasoning problem and has recently been extended to the Multiagent Simple Temporal Problem (MaSTP). In this paper we present a novel approach that is based on enforcing arc-consistency (AC) on the input (multiagent) simple temporal network. We show that the AC-based approach is suf- ﬁcient for solving both the STP and MaSTP and provide ef- ﬁcient algorithms for them. As our AC-based approach does not impose new constraints between agents, it does not violate the privacy of the agents and is superior to the state-ofthe-art approach to MaSTP. Empirical evaluations on diverse benchmark datasets also show that our AC-based algorithms for STP and MaSTP are signiﬁcantly more efﬁcient than existing approaches.

PDF Details

AAMAS Conference 2017 Conference Paper

A Deterministic Distributed Algorithm for Reasoning with Connected Row-Convex Constraints

Shufeng Kong
Jae Hee Lee
Sanjiang Li

The class of CRC constraints generalizes several tractable classes of constraints and is expressive enough to model problems in domains such as temporal reasoning, geometric reasoning, and scene labelling. This paper presents the first distributed deterministic algorithm for connected row-convex (CRC) constraints. Our distributed (partial) path consistency algorithm efficiently transforms a CRC constraint network into an equivalent constraint network, where all constraints are minimal (i. e. , they are the tightest constraints) and generating all solutions can be done in a backtrackfree manner. When compared with the state-of-the-art distributed algorithm for CRC constraints, which is a randomized one, our algorithm guarantees to generate a solution for satisfiable CRC constraint networks and it is applicable to solve large networks in real distributed systems. The experimental evaluations show that our algorithm outperforms the state-of-the-art algorithm in both practice and theory.

PDF

IJCAI Conference 2016 Conference Paper

Connecting Qualitative Spatial and Temporal Representations by Propositional Closure

Diedrich Wolter
Jae Hee Lee

This paper establishes new relationships between existing qualitative spatial and temporal representations. Qualitative spatial and temporal representation (QSTR) is concerned with abstractions of infinite spatial and temporal domains, which represent configurations of objects using a finite vocabulary of relations, also called a qualitative calculus. Classically, reasoning in QSTR is based on constraints. An important task is to identify decision procedures that are able to handle constraints from a single calculus or from several calculi. In particular the latter aspect is a longstanding challenge due to the multitude of calculi proposed. In this paper we consider propositional closures of qualitative constraints which enable progress with respect to the longstanding challenge. Propositional closure allows one to establish several translations between distinct calculi. This enables joint reasoning and provides new insights into computational complexity of individual calculi. We conclude that the study of propositional languages instead of previously considered purely relational languages is a viable research direction for QSTR leading to expressive formalisms and practical algorithms.

PDF Details

IJCAI Conference 2016 Conference Paper

Trend-Based Prediction of Spatial Change

XiaoYu Ge
Jae Hee Lee
Jochen Renz
Peng Zhang

The capability to predict changes of spatial regions is important for an intelligent system that interacts with the physical world. For example, in a disaster management scenario, predicting potentially endangered areas and inferring safe zones is essential for planning evacuations and countermeasures. Existing approaches usually predict such spatial changes by simulating the physical world based on specific models. Thus, these simulation-based methods will not be able to provide reliable predictions when the scenario is not similar to any of the models in use or when the input parameters are incomplete. In this paper, we present a prediction approach that overcomes the aforementioned problem by using a more general model and by analysing the trend of the spatial changes. The method is also flexible to adopt to new observations and to adapt its prediction to new situations.

PDF Details

IJCAI Conference 2015 Conference Paper

From Raw Sensor Data to Detailed Spatial Knowledge

Peng Zhang
Jae Hee Lee
Jochen Renz

Qualitative spatial reasoning deals with relational spatial knowledge and with how this knowledge can be processed efficiently. Identifying suitable representations for spatial knowledge and checking whether the given knowledge is consistent has been the main research focus in the past two decades. However, where the spatial information comes from, what kind of information can be obtained and how it can be obtained has been largely ignored. This paper is an attempt to start filling this gap. We present a method for extracting detailed spatial information from sensor measurements of regions. We analyse how different sparse sensor measurements can be integrated and what spatial information can be extracted from sensor measurements. Different from previous approaches to qualitative spatial reasoning, our method allows us to obtain detailed information about the internal structure of regions. The result has practical implications, for example, in disaster management scenarios, which include identifying the safe zones in bushfire and flood regions.

PDF Details

IJCAI Conference 2015 Conference Paper

Qualitative Reasoning about Directions in Semantic Spaces

Steven Schockaert
Jae Hee Lee

We introduce a framework for qualitative reasoning about directions in high-dimensional spaces, called EER, where our main motivation is to develop a form of commonsense reasoning about semantic spaces. The proposed framework is, however, more general; we show how qualitative spatial reasoning about points with several existing calculi can be reduced to the realisability problem for EER (or REER for short), including LR and calculi for reasoning about betweenness, collinearity and parallelism. Finally, we propose an efficient but incomplete inference method, and show its effectiveness for reasoning with EER as well as reasoning with some of the aforementioned calculi.

PDF Details

IJCAI Conference 2013 Conference Paper

StarVars—Effective Reasoning about Relative Directions

Jae Hee Lee
Jochen Renz
Diedrich Wolter

Relative direction information is very commonly used. Observers typically describe their environment by specifying the relative directions in which they see other objects or other people from their point of view. Or they receive navigation instructions with respect to their point of view, for example, turn left at the next intersection. However, it is surprisingly hard to integrate relative direction information obtained from different observers, and to reconstruct a model of the environment or the locations of the observers based on this information. Despite intensive research, there is currently no algorithm that can effectively integrate this information: this problem is NP-hard, but not known to be in NP, even if we only use left and right relations. In this paper we present a novel qualitative representation, StarVars, that can solve these problems. It is an extension of the STAR calculus [Renz and Mitra, 2004]) by a VARiable interpretation of the orientation of observers. We show that reasoning in StarVars is in NP and present the first algorithm that allows us to effectively integrate relative direction information from different observers.

PDF Details DOI