Author name cluster

Yoonseon Oh

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

6 papers

2 author rows

AAAI Conference 2025 Conference Paper

GNN-Transformer Task Planning Enhanced with Semantic-Driven Data Augmentation

Soojin Jeong
Seongwan Byeon
Sangwoo Kim
HyeokJun Kwon
Yoonseon Oh

Natural language is the most intuitive means for humans to interact with robots, making task planning based on natural language commands a longstanding area of research. Large language models (LLMs) have significantly improved task planning by enhancing understanding of language and common sense. However, current methods still face several challenges: they lack a deep understanding of physical environments, their performance relies heavily on prompt examples, LLMs are oversized and not customized for specific tasks, and the planning costs remain high. To overcome these issues, we introduce the GNN-Transformer Task Planner (GTTP), designed to predict task-level actions by leveraging the semantic environment and incorporating historical state data. The GTTP architecture is scalable through the use of GNN layers, while transformer layers facilitate understanding task progression. In addition, our model uses a text encoder to embed environments, allowing it to be trained on simulated datasets and applied directly in real-world scenarios. We also propose an automated data generation method that includes semantic augmentation, planning verification, and instruction generation via LLM. This method enables the collection of 14k instruction-annotated tasks in the VirtualHome environment with minimal human effort. The model has been validated across diverse scenes containing up to 715 objects, achieving significantly higher success rates compared to baseline models. It has also been successfully deployed on a physical mobile manipulator, demonstrating its practical applicability and effectiveness.

PDF Details DOI

ICRA Conference 2025 Conference Paper

Task-Aware Semantic Map: Autonomous Robot Task Assignment Beyond Commands

Daewon Choi
Hosung Lee
Soeun Hwang
Yoonseon Oh

With recent advancements in Large Language Models, task planning methods that interpret human commands have garnered significant attention. However, as home robots become more common, specifying every daily task could become impractical. This paper introduces a novel semantic map called the Task-Aware Semantic Map (TASMap), which enables robots to autonomously assign and propose necessary tasks in a scene without explicit human commands. The core innovation of this approach is the ability of TASMap to comprehend the context of objects within a scene and autonomously generate task proposals. This capability significantly advances autonomous robotic assistance, reducing the dependency on specific commands and enhancing interaction with environments. We present two key applications of TASMap: contextual task proposal and spatial task proposal. Our results, verified across 35 diverse and realistically disordered scenes, underscore the effectiveness of TASMap in both simulation and real-world environments.

Details

IROS Conference 2024 Conference Paper

Task Planning for Long-Horizon Cooking Tasks Based on Large Language Models

Jungkyoo Shin
Jieun Han
SeungJun Kim
Yoonseon Oh
Eunwoo Kim

In the field of robot manipulation, learnable task planners are gaining attention, especially for long-horizon tasks such as cooking. However, existing methods that predominantly rely on symbolic representations suffer from limitations in generalization capabilities, particularly in handling unseen objects. Given that objects may vary in real-world environments, this limitation may constrain their practical applicability. To address this issue, we propose a novel task-planning framework that leverages a pretrained large language model (LLM) for environmental interpretation. Our proposed framework extracts semantic features directly from textual data, enabling the planner to accommodate unfamiliar objects. We further incorporate a transformer-based encoder-decoder framework to understand environmental attributes derived from the language model and generate sequential predictions in line with object-oriented subgoals. To validate the effectiveness of our model, we utilize a dataset focused on cooking recipes. Going a step further, we propose a method that automatically generates object-oriented data from natural language description using recurrent LLM, enhancing the framework to manage previously unseen targets as well. Our framework shows an average success rate of 95% when validated with test sets that involve unseen objects. By providing the automatically generated dataset to the framework, we achieve a significant 27% increase in success rate on unknown target recipes. We also provide evidence of the real-world viability of our planner by successfully deploying it on a robot platform.

Details

ICRA Conference 2019 Conference Paper

Multi-Object Search using Object-Oriented POMDPs

Arthur Wandzel
Yoonseon Oh
Michael Fishman 0001
Nishanth Kumar
Lawson L. S. Wong
Stefanie Tellex

A core capability of robots is to reason about multiple objects under uncertainty. Partially Observable Markov Decision Processes (POMDPs) provide a means of reasoning under uncertainty for sequential decision making, but are computationally intractable in large domains. In this paper, we propose Object-Oriented POMDPs (OO-POMDPs), which represent the state and observation spaces in terms of classes and objects. The structure afforded by OO-POMDPs support a factorization of the agent's belief into independent object distributions, which enables the size of the belief to scale linearly versus exponentially in the number of objects. We formulate a novel Multi-Object Search (MOS) task as an OO-POMDP for mobile robotics domains in which the agent must find the locations of multiple objects. Our solution exploits the structure of OO-POMDPs by featuring human language to selectively update the belief at task onset. Using this structure, we develop a new algorithm for efficiently solving OO-POMDPs: Object-Oriented Partially Observable Monte-Carlo Planning (OOPOMCP). We show that OO-POMCP with grounded language commands is sufficient for solving challenging MOS tasks both in simulation and on a physical mobile robot.

Details

ICRA Conference 2016 Conference Paper

Multiple-hypothesis chance-constrained target tracking under identity uncertainty

Yoonseon Oh
Songhwai Oh

We propose a robust target tracking algorithm for a mobile robot under identity uncertainty, which arises in crowded environments. When a mobile robot has a sensor with a fan-shaped field of view and finite sensing region, the proposed algorithm aims to minimize the probability of losing a moving target. We predict the next position of a moving target in a crowded environment using a multiple-hypothesis prediction algorithm which combines the motion model and appearance model of the target. When the distribution of the target's next position follows a Gaussian mixture model, the proposed tracking algorithm can track a target with a guaranteed tracking success probability. If the tracking success probability is sufficiently good, the method minimizes the moving distance of the mobile robot. The performance of the method is extensively validated in simulation and experiments using a Pioneer robot with a Microsoft Kinect sensor.

Details

ICRA Conference 2015 Conference Paper

Chance-constrained target tracking for mobile robots

Yoonseon Oh
Sungjoon Choi
Songhwai Oh

This paper presents a robust target tracking algorithm for a mobile sensor with a fan-shaped field of view and finite sensing range. The goal of the mobile robot is to track a moving target such that the probability of losing the target is minimized. We assume that the distribution of the next position of a moving target can be estimated using a motion prediction algorithm. If the next position of a moving target has the Gaussian distribution, the proposed algorithm can guarantee the tracking success probability. In addition, the proposed method minimizes the moving distance of the mobile robot based on a bound on the tracking success probability. While the problem considered in this paper is a non-convex optimization problem, we derive analytical solutions which can be easily solved in real-time. The performance of the proposed method is evaluated extensively in simulation and validated in pedestrian following experiments using a Pioneer mobile robot with a Microsoft Kinect sensor.

Details