Author name cluster

Heejin Jeong

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

7 papers

2 author rows

ICRA Conference 2021 Conference Paper

Deep Reinforcement Learning for Active Target Tracking

Heejin Jeong
Seyed Hamed Hassani
Manfred Morari
Daniel D. Lee
George J. Pappas

We solve active target tracking, one of the essential tasks in autonomous systems, using a deep reinforcement learning (RL) approach. In this problem, an autonomous agent is tasked with acquiring information about targets of interests using its on-board sensors. The classical challenges in this problem are system model dependence and the difficulty of computing information-theoretic cost functions for a long planning horizon. RL provides solutions for these challenges as the length of its effective planning horizon does not affect the computational complexity, and it drops the strong dependency of an algorithm on system models. In particular, we introduce Active Tracking Target Network (ATTN), a unified deep RL policy that is capable of solving major sub-tasks of active target tracking – in-sight tracking, navigation, and exploration. The policy shows robust behavior for tracking agile and anomalous targets with a partially known target model. Additionally, the same policy is able to navigate in obstacle environments to reach distant targets as well as explore the environment when targets are positioned in unexpected locations.

Details

IROS Conference 2021 Conference Paper

Scalable Reinforcement Learning Policies for Multi-Agent Control

Christopher D. Hsu
Heejin Jeong
George J. Pappas
Pratik Chaudhari

We develop a Multi-Agent Reinforcement Learning (MARL) method to learn scalable control policies for target tracking. Our method can handle an arbitrary number of pursuers and targets; we show results for tasks consisting up to 1000 pursuers tracking 1000 targets. We use a decentralized, partially-observable Markov Decision Process framework to model pursuers as agents receiving partial observations (range and bearing) about targets which move using fixed, unknown policies. An attention mechanism is used to parameterize the value function of the agents; this mechanism allows us to handle an arbitrary number of targets. Entropy-regularized off-policy RL methods are used to train a stochastic policy, and we discuss how it enables a hedging behavior between pursuers that leads to a weak form of cooperation in spite of completely decentralized control execution. We further develop a masking heuristic that allows training on smaller problems with few pursuers-targets and execution on much larger problems. Thorough simulation experiments and comparisons to state of the art algorithms are performed to study the scalability of the approach and robustness of performance to varying numbers of agents and targets.

Details

IJCAI Conference 2019 Conference Paper

Assumed Density Filtering Q-learning

Heejin Jeong
Clark Zhang
George J. Pappas
Daniel D. Lee

While off-policy temporal difference (TD) methods have widely been used in reinforcement learning due to their efficiency and simple implementation, their Bayesian counterparts have not been utilized as frequently. One reason is that the non-linear max operation in the Bellman optimality equation makes it difficult to define conjugate distributions over the value functions. In this paper, we introduce a novel Bayesian approach to off-policy TD methods, called as ADFQ, which updates beliefs on state-action values, Q, through an online Bayesian inference method known as Assumed Density Filtering. We formulate an efficient closed-form solution for the value update by approximately estimating analytic parameters of the posterior of the Q-beliefs. Uncertainty measures in the beliefs not only are used in exploration but also provide a natural regularization for the value update considering all next available actions. ADFQ converges to Q-learning as the uncertainty measures of the Q-beliefs decrease and improves common drawbacks of other Bayesian RL algorithms such as computational complexity. We extend ADFQ with a neural network. Our empirical results demonstrate that ADFQ outperforms comparable algorithms on various Atari 2600 games, with drastic improvements in highly stochastic domains or domains with a large action space.

PDF Details

IROS Conference 2019 Conference Paper

Learning Q-network for Active Information Acquisition

Heejin Jeong
Brent Schlotfeldt
Seyed Hamed Hassani
Manfred Morari
Daniel D. Lee
George J. Pappas

In this paper, we propose a novel Reinforcement Learning approach for solving the Active Information Acquisition problem, which requires an agent to choose a sequence of actions in order to acquire information about a process of interest using on-board sensors. The classic challenges in the information acquisition problem are the dependence of a planning algorithm on known models and the difficulty of computing information-theoretic cost functions over arbitrary distributions. In contrast, the proposed framework of reinforcement learning does not require any knowledge on models and alleviates the problems during an extended training stage. It results in policies that are efficient to execute online and applicable for real-time control of robotic systems. Furthermore, the state-of-the-art planning methods are typically restricted to short horizons, which may become problematic with local minima. Reinforcement learning naturally handles the issue of planning horizon in information problems as it maximizes a discounted sum of rewards over a long finite or infinite time horizon. We discuss the potential benefits of the proposed framework and compare the performance of the novel algorithm to an existing information acquisition method for multi-target tracking scenarios.

Details

IROS Conference 2016 Conference Paper

Efficient learning of stand-up motion for humanoid robots with bilateral symmetry

Heejin Jeong
Daniel D. Lee

Standing up after falling is an essential ability for humanoid robots in order to resume their tasks without help from humans. Although many humanoid robots, especially small-size humanoid robots, have their own stand-up motions, there has not been a generalized method to automatically learn flexible stand-up motions for humanoid robots which can be applied to various fallen positions. In this research, we propose a method for learning stand-up motions for humanoid robots using Q-learning making use of their bilateral symmetry. We implemented this method on DarwIn-OP humanoid robots and learned an optimal policy in simulation. We compared the resulting stand-up motion with manually designed stand-up motions and with stand-up motions learned without considering bilateral symmetry. Both in simulation and on the real robot, the new stand-up motion was successful in most trials while other motions took longer or were not as robust.

Details

AAAI Conference 2016 Conference Paper

Learning Complex Stand-Up Motion for Humanoid Robots

Heejin Jeong
Daniel Lee

In order for humanoid robots to complete various assigned tasks without any human assistance, they must have the ability to stand up on their own. In this abstract, we introduce complex stand-up motion of humanoid robots learned by using Reinforcement Learning.

PDF Details

IROS Conference 2014 Conference Paper

A Robot-Machine Interface for full-functionality automation using a humanoid

Heejin Jeong
David Hyunchul Shim
Sungwook Cho

Humanoid robots can be a highly desirable substitute for humans when it performs various tasks using tools and equipment designed for humans. One of such possible applications is controlling a vehicle. A humanoid robot can sit in the pilot's seat and command the vehicle using the control columns or steering wheels, pedals, switches, levers, and dials. In this paper, we propose a framework of automating a vehicle, an airplane in particular, with a humanoid. In order to perform various tasks of flying an unmodified airplane, the robot needs to perform three levels of tasks - recognition, decision, and action. The robot should collect information of the vehicle by using its own sensors and from various instruments in the cockpit, in addition to possible data link, a privilege of a robot. The robot then decides how to operate the flight control equipment in order to follow a given flight plan. Finally, it directly manipulates the control input equipment by computing the kinematic variables in the presence of various constraints from the surroundings. In order to validate the proposed framework, a piloting robot system is developed using a small low-cost humanoid and a flight simulation equipment designed for humans. The robot showed adequate performance to fly the airplane from cold start to landing to a stop on the runway.

Details