Author name cluster

H. Jin Kim

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

76 papers

2 author rows

ICRA Conference 2025 Conference Paper

Enhancing Feature Tracking Reliability for Visual Navigation Using Real-Time Safety Filter

Dabin Kim
Inkyu Jang
Youngsoo Han
Sunwoo Hwang
H. Jin Kim

Vision sensors are extensively used for localizing a robot's pose, particularly in environments where global localization tools such as GPS or motion capture systems are unavailable. In many visual navigation systems, localization is achieved by detecting and tracking visual features or landmarks, which provide information about the sensor's relative pose. For reliable feature tracking and accurate pose estimation, it is crucial to maintain visibility of a sufficient number of features. This requirement can sometimes conflict with the robot's overall task objective. In this paper, we approach it as a constrained control problem. By leveraging the invariance properties of visibility constraints within the robot's kinematic model, we propose a real-time safety filter based on quadratic programming. This filter takes a reference velocity command as input and produces a modified velocity that minimally deviates from the reference while ensuring the information score from the currently visible features remains above a user-specified threshold. Numerical simulations demonstrate that the proposed safety filter preserves the invariance condition and ensures the visibility of more features than the required minimum. We also validated its real-world performance by integrating it into a visual simultaneous localization and mapping (SLAM) algorithm, where it maintained high estimation quality in challenging environments, outperforming a simple tracking controller.

IROS Conference 2025 Conference Paper

Leveraging Temporally Extended Behavior Sharing for Multi-task Reinforcement Learning

Gawon Lee
Daesol Cho
H. Jin Kim

Multi-task reinforcement learning (MTRL) offers a promising approach to improve sample efficiency and generalization by training agents across multiple tasks, enabling knowledge sharing between them. However, applying MTRL to robotics remains challenging due to the high cost of collecting diverse task data. To address this, we propose MT-Lévy, a novel exploration strategy that enhances sample efficiency in MTRL environments by combining behavior sharing across tasks with temporally extended exploration inspired by Lévy flight [1]. MT-Lévy leverages policies trained on related tasks to guide exploration towards key states, while dynamically adjusting exploration levels based on task success ratios. This approach enables more efficient state-space coverage, even in complex robotics environments. Empirical results demonstrate that MT-Lévy significantly improves exploration and sample efficiency, supported by quantitative and qualitative analyses. Ablation studies further highlight the contribution of each component, showing that combining behavior sharing with adaptive exploration strategies can significantly improve the practicality of MTRL in robotics applications.

NeurIPS Conference 2025 Conference Paper

Periodic Skill Discovery

Jonghae Park
Daesol Cho
Jusuk Lee
Dongseok Shim
Inkyu Jang
H. Jin Kim

Unsupervised skill discovery in reinforcement learning (RL) aims to learn diverse behaviors without relying on external rewards. However, current methods often overlook the periodic nature of learned skills, focusing instead on increasing the mutual dependency between states and skills or maximizing the distance traveled in latent space. Considering that many robotic tasks—particularly those involving locomotion—require periodic behaviors across varying timescales, the ability to discover diverse periodic skills is essential. Motivated by this, we propose Periodic Skill Discovery (PSD), a framework that discovers periodic behaviors in an unsupervised manner. The key idea of PSD is to train an encoder that maps states to a circular latent space, thereby naturally encoding periodicity in the latent representation. By capturing temporal distance, PSD can effectively learn skills with diverse periods in complex robotic tasks, even with pixel-based observations. We further show that these learned skills achieve high performance on downstream tasks such as hurdling. Moreover, integrating PSD with an existing skill discovery method offers more diverse behaviors, thus broadening the agent’s repertoire. Our code and demos are available at https: //jonghaepark. github. io/psd

ICRA Conference 2025 Conference Paper

Safety-Critical Control for Aerial Physical Interaction in Uncertain Environment

Jeonghyun Byun
Yeonjoon Kim
Dongjae Lee
H. Jin Kim

Aerial manipulation for safe physical interaction with their environments is gaining significant momentum in robotics research. In this paper, we present a disturbance-observer-based safety-critical control for a fully actuated aerial manipulator interacting with both static and dynamic structures. Our approach centers on a safety filter that dynamically adjusts the desired trajectory of the vehicle's pose, accounting for the aerial manipulator's dynamics, the disturbance observer's structure, and motor thrust limits. We provide rigorous proof that the proposed safety filter ensures the forward invariance of the safety set—representing motor thrust limits—even in the presence of disturbance estimation errors. To demonstrate the superiority of our method over existing control strategies for aerial physical interaction, we perform comparative experiments involving complex tasks, such as pushing against a static structure and pulling a plug firmly attached to an electric socket. Furthermore, to highlight its repeatability in scenarios with sudden dynamic changes, we perform repeated tests of pushing a movable cart and extracting a plug from a socket. These experiments confirm that our method not only outperforms existing methods but also excels in handling tasks with rapid dynamic variations.

ICRA Conference 2024 Conference Paper

Autonomous aerial perching and unperching using omnidirectional tiltrotor and switching controller

Dongjae Lee
Sunwoo Hwang
Jeonghyun Byun
Seung Jae Lee 0002
H. Jin Kim

Aerial unperching of multirotors has received little attention as opposed to perching that has been investigated to elongate operation time. This study presents a new aerial robot capable of both perching and unperching autonomously on/from a ferromagnetic surface during flight, and a switching controller to avoid rotor saturation and mitigate overshoot during transition between free-flight and perching. To enable stable perching and unperching maneuvers on/from a vertical surface, a lightweight (≈ 1 kg), fully actuated tiltrotor that can hover at 90 ◦ pitch angle is first developed. We design a perching/unperching module composed of a single servomotor and a magnet, which is then mounted on the tiltrotor. A switching controller including exclusive control modes for transitions between free-flight and perching is proposed. Lastly, we propose a simple yet effective strategy to ensure robust perching in the presence of measurement and control errors and avoid collisions with the perching site immediately after unperching. We validate the proposed framework in experiments where the tiltrotor successfully performs perching and unperching on/from a vertical surface during flight. We further show effectiveness of the proposed transition mode in the switching controller by ablation studies where large overshoot and even collision with a perching site occur. To the best of the authors’ knowledge, this work presents the first autonomous aerial unperching framework using a fully actuated tiltrotor.

ICML Conference 2024 Conference Paper

Behavior Generation with Latent Actions

Seungjae Lee 0001
Yibin Wang 0008
Haritheja Etukuru
H. Jin Kim
Nur Muhammad (Mahi) Shafiullah
Lerrel Pinto

Generative modeling of complex behaviors from labeled datasets has been a longstanding problem in decision-making. Unlike language or image generation, decision-making requires modeling actions – continuous-valued vectors that are multimodal in their distribution, potentially drawn from uncurated sources, where generation errors can compound in sequential prediction. A recent class of models called Behavior Transformers (BeT) addresses this by discretizing actions using k-means clustering to capture different modes. However, k-means struggles to scale for high-dimensional action spaces or long sequences, and lacks gradient information, and thus BeT suffers in modeling long-range actions. In this work, we present Vector-Quantized Behavior Transformer (VQ-BeT), a versatile model for behavior generation that handles multimodal action prediction, conditional generation, and partial observations. VQ-BeT augments BeT by tokenizing continuous actions with a hierarchical vector quantization module. Across seven environments including simulated manipulation, autonomous driving, and robotics, VQ-BeT improves on state-of-the-art models such as BeT and Diffusion Policies. Importantly, we demonstrate VQ-BeT’s improved ability to capture behavior modes while accelerating inference speed 5× over Diffusion Policies. Videos can be found https: //sjlee. cc/vq-bet/

ICRA Conference 2024 Conference Paper

Safe Receding Horizon Motion Planning with Infinitesimal Update Interval

Inkyu Jang
Sunwoo Hwang
Jeonghyun Byun
H. Jin Kim

Safety verification in motion planning is known to be computationally burdensome, despite its importance in robotics. In this paper, we investigate the behavior of safe receding horizon motion planners when the update interval becomes infinitesimal. By requiring the trajectory parameters to evolve continuously in time, the trajectory optimization problem is reformulated into a time-derivative form, whose decision variables are their rate of change. This results in a quadratic programming problem which directly provides safe input, and can be regarded as a real-time safety filter. The input expressivity is also enhanced by leveraging the differentiable structure of the parameter space. The proposed safety filter is experimentally validated using a wheeled ground robot in obstacle-cluttered environments. The result shows that the safety filter is capable of generating safe inputs in real-time, while addressing hundreds of constraints simultaneously.

NeurIPS Conference 2023 Conference Paper

CQM: Curriculum Reinforcement Learning with a Quantized World Model

Seungjae Lee
Daesol Cho
Jonghae Park
H. Jin Kim

Recent curriculum Reinforcement Learning (RL) has shown notable progress in solving complex tasks by proposing sequences of surrogate tasks. However, the previous approaches often face challenges when they generate curriculum goals in a high-dimensional space. Thus, they usually rely on manually specified goal spaces. To alleviate this limitation and improve the scalability of the curriculum, we propose a novel curriculum method that automatically defines the semantic goal space which contains vital information for the curriculum process, and suggests curriculum goals over it. To define the semantic goal space, our method discretizes continuous observations via vector quantized-variational autoencoders (VQ-VAE) and restores the temporal relations between the discretized observations by a graph. Concurrently, ours suggests uncertainty and temporal distance-aware curriculum goals that converges to the final goals over the automatically composed goal space. We demonstrate that the proposed method allows efficient explorations in an uninformed environment with raw goal examples only. Also, ours outperforms the state-of-the-art curriculum RL methods on data efficiency and performance, in various goal-reaching tasks even with ego-centric visual inputs.

ICRA Conference 2023 Conference Paper

Decentralized Deadlock-free Trajectory Planning for Quadrotor Swarm in Obstacle-rich Environments

Jungwon Park
Inkyu Jang
H. Jin Kim

This paper presents a decentralized multi-agent trajectory planning (MATP) algorithm that guarantees to generate a safe, deadlock-free trajectory in an obstacle-rich environment under a limited communication range. The proposed algorithm utilizes a grid-based multi-agent path planning (MAPP) algorithm for deadlock resolution, and we introduce the subgoal optimization method to make the agent converge to the waypoint generated from the MAPP without deadlock. In addition, the proposed algorithm ensures the feasibility of the optimization problem and collision avoidance by adopting a linear safe corridor (LSC). We verify that the proposed algorithm does not cause a deadlock in both random forests and dense mazes regardless of communication range, and it outperforms our previous work in flight time and distance. We validate the proposed algorithm through a hardware demonstration with ten quadrotors.

ICML Conference 2023 Conference Paper

Demonstration-free Autonomous Reinforcement Learning via Implicit and Bidirectional Curriculum

Jigang Kim
Daesol Cho
H. Jin Kim

While reinforcement learning (RL) has achieved great success in acquiring complex skills solely from environmental interactions, it assumes that resets to the initial state are readily available at the end of each episode. Such an assumption hinders the autonomous learning of embodied agents due to the time-consuming and cumbersome workarounds for resetting in the physical world. Hence, there has been a growing interest in autonomous RL (ARL) methods that are capable of learning from non-episodic interactions. However, existing works on ARL are limited by their reliance on prior data and are unable to learn in environments where task-relevant interactions are sparse. In contrast, we propose a demonstration-free ARL algorithm via Implicit and Bi-directional Curriculum (IBC). With an auxiliary agent that is conditionally activated upon learning progress and a bidirectional goal curriculum based on optimal transport, our method outperforms previous methods, even the ones that leverage demonstrations.

IROS Conference 2023 Conference Paper

DiffuPose: Monocular 3D Human Pose Estimation via Denoising Diffusion Probabilistic Model

Jeongjun Choi
Dongseok Shim
H. Jin Kim

Thanks to the development of 2D keypoint detectors, monocular 3D human pose estimation (HPE) via 2D-to-3D uplifting approaches have achieved remarkable improvements. Still, monocular 3D HPE is a challenging problem due to the inherent depth ambiguities and occlusions. To handle this problem, many previous works exploit temporal information to mitigate such difficulties. However, there are many real-world applications where frame sequences are not accessible. This paper focuses on reconstructing a 3D pose from a single 2D keypoint detection. Rather than exploiting temporal information, we alleviate the depth ambiguity by generating multiple 3D pose candidates which can be mapped to an identical 2D keypoint. We build a novel diffusion-based framework to effectively sample diverse 3D poses from an off-the-shelf 2D detector. By considering the correlation between human joints by replacing the conventional denoising U-Net with graph convolutional network, our approach accomplishes further performance improvements. We evaluate our method on the widely adopted Human3. 6M and HumanEva-I datasets. Comprehensive experiments are conducted to prove the efficacy of the proposed method, and they confirm that our model outperforms state-of-the-art multi-hypothesis 3D HPE methods.

NeurIPS Conference 2023 Conference Paper

Diversify \& Conquer: Outcome-directed Curriculum RL via Out-of-Distribution Disagreement

Daesol Cho
Seungjae Lee
H. Jin Kim

Reinforcement learning (RL) often faces the challenges of uninformed search problems where the agent should explore without access to the domain knowledge such as characteristics of the environment or external rewards. To tackle these challenges, this work proposes a new approach for curriculum RL called $\textbf{D}$iversify for $\textbf{D}$isagreement \& $\textbf{C}$onquer ($\textbf{D2C}$). Unlike previous curriculum learning methods, D2C requires only a few examples of desired outcomes and works in any environment, regardless of its geometry or the distribution of the desired outcome examples. The proposed method performs diversification of the goal-conditional classifiers to identify similarities between visited and desired outcome states and ensures that the classifiers disagree on states from out-of-distribution, which enables quantifying the unexplored region and designing an arbitrary goal-conditioned intrinsic reward signal in a simple and intuitive way. The proposed method then employs bipartite matching to define a curriculum learning objective that produces a sequence of well-adjusted intermediate goals, which enable the agent to automatically explore and conquer the unexplored region. We present experimental results demonstrating that D2C outperforms prior curriculum RL methods in both quantitative and qualitative aspects, even with the arbitrarily distributed desired outcome examples.

ICRA Conference 2023 Conference Paper

Globally Defined Dynamic Modelling and Geometric Tracking Controller Design for Aerial Manipulator

Byeongjun Kim
Dongjae Lee
Jeonghyun Byun
H. Jin Kim

This study presents a globally defined dynamics for a conventional multirotor equipped with a single $n\mathbf{-DOF}$ manipulator using modified Lagrangian dynamics. This enables the reformulation of entire dynamics directly on $\text{SO}(3)$ without exploiting any local coordinates, and thus problems such as the singularity of Euler angles can be avoided. Since skew-symmetric property of Coriolis matrix $C$ and inertia matrix facilitates stability analysis, we propose a method to compute $C$ which guarantees the skew-symmetric property by considering $C$ as a summation of two sub-matrices. Then, a geometric tracking controller is designed based on decoupled dynamics applying passive decomposition. The proposed controller guarantees almost global region of attraction. We validate our method via consecutive aerial flipping experiments.

IROS Conference 2023 Conference Paper

Minimally Actuated Tiltrotor for Perching and Normal Force Exertion

Dongjae Lee
Sunwoo Hwang
Changhyeon Kim
Seung Jae Lee 0002
H. Jin Kim

This study presents a new hardware design and control of a minimally actuated 5 control degrees of freedom (CDoF) quadrotor-based tiltrotor. The proposed tiltrotor possesses several characteristics distinct from those found in existing works, including: 1) minimal number of actuators for 5 CDoF, 2) large margin to generate interaction force during aerial physical interaction (APhI), and 3) no mechanical obstruction in thrust direction rotation. Thanks to these properties, the proposed tiltrotor is suitable for perching-enabled APhI since it can hover parallel to an arbitrarily oriented surface and can freely adjust its thrust direction. To fully control the 5-CDoF of the designed tiltrotor, we construct an asymptotically stabilizing controller with stability analysis. The proposed tiltrotor design and controller are validated in experiments where the first two experiments of x, y position tracking and pitch tracking show controllability of the added CDoF compared to a conventional quadrotor. Finally, the last experiment of perching and cart pushing demonstrates the proposed tiltrotor's applicability to perching-enabled APhI.

ICRA Conference 2023 Conference Paper

Object-based SLAM utilizing unambiguous pose parameters considering general symmetry types

Taekbeom Lee
Youngseok Jang
H. Jin Kim

Existence of symmetric objects, whose observation at different viewpoints can be identical, can deteriorate the performance of simultaneous localization and mapping (SLAM). This work proposes a system for robustly optimizing the pose of cameras and objects even in the presence of symmetric objects. We classify objects into three categories depending on their symmetry characteristics, which is efficient and effective in that it allows to deal with general objects and the objects in the same category can be associated with the same type of ambiguity. Then we extract only the unambiguous parameters corresponding to each category and use them in data association and joint optimization of the camera and object pose. The proposed approach provides significant robustness to the SLAM performance by removing the ambiguous parameters and utilizing as much useful geometric information as possible. Comparison with baseline algorithms confirms the superior performance of the proposed system in terms of object tracking and pose estimation, even in challenging scenarios where the baseline fails.

ICLR Conference 2023 Conference Paper

Outcome-directed Reinforcement Learning by Uncertainty \& Temporal Distance-Aware Curriculum Goal Generation

Daesol Cho
Seung Jae Lee 0002
H. Jin Kim

Current reinforcement learning (RL) often suffers when solving a challenging exploration problem where the desired outcomes or high rewards are rarely observed. Even though curriculum RL, a framework that solves complex tasks by proposing a sequence of surrogate tasks, shows reasonable results, most of the previous works still have difficulty in proposing curriculum due to the absence of a mechanism for obtaining calibrated guidance to the desired outcome state without any prior domain knowledge. To alleviate it, we propose an uncertainty \& temporal distance-aware curriculum goal generation method for the outcome-directed RL via solving a bipartite matching problem. It could not only provide precisely calibrated guidance of the curriculum to the desired outcome states but also bring much better sample efficiency and geometry-agnostic curriculum goal proposal capability compared to previous curriculum RL methods. We demonstrate that our algorithm significantly outperforms these prior methods in a variety of challenging navigation tasks and robotic manipulation tasks in a quantitative and qualitative way.

ICRA Conference 2023 Conference Paper

Safe and Distributed Multi-Agent Motion Planning under Minimum Speed Constraints

Inkyu Jang
Jungwon Park
H. Jin Kim

The motion planning problem for multiple unstop-pable agents is of interest in many robotics applications, for example, autonomous traffic management for multiple fixed-wing aircraft. Unfortunately, many of the existing algorithms cannot provide safety for such agents, because they require the agents to be able to brake to a complete stop for safety and feasibility insurance. In this paper, we present a distributed multi-agent motion planner that guarantees collision avoidance and persistent feasibility, which can be applied to a team of homogeneous mobile vehicles that cannot stop. The planner is built on top of the idea that a collision-free trajectory in form of a loop can safely accommodate multiple unstoppable agents, while avoiding collisions among them and static obstacles. At every time step, in a distributed manner, the agents generate trajectory-manipulating actions that preserve the loop structure. Then, a deconfliction process selects a conflict-free subset of the generated actions, which are applied at the next time step. Through simulation using an unstoppable Dubins car model, we show that the proposed motion planner is able to provide persistent safety guarantees for such agents in obstacle-cluttered space in real-time.

ICML Conference 2023 Conference Paper

SNeRL: Semantic-aware Neural Radiance Fields for Reinforcement Learning

Dongseok Shim
Seung Jae Lee 0002
H. Jin Kim

As previous representations for reinforcement learning cannot effectively incorporate a human-intuitive understanding of the 3D environment, they usually suffer from sub-optimal performances. In this paper, we present Semantic-aware Neural Radiance Fields for Reinforcement Learning (SNeRL), which jointly optimizes semantic-aware neural radiance fields (NeRF) with a convolutional encoder to learn 3D-aware neural implicit representation from multi-view images. We introduce 3D semantic and distilled feature fields in parallel to the RGB radiance fields in NeRF to learn semantic and object-centric representation for reinforcement learning. SNeRL outperforms not only previous pixel-based representations but also recent 3D-aware representations both in model-free and model-based reinforcement learning.

ICRA Conference 2023 Conference Paper

Stable Contact Guaranteeing Motion/Force Control for an Aerial Manipulator on an Arbitrarily Tilted Surface

Jeonghyun Byun
Byeongjun Kim
Changhyeon Kim
Donggeon David Oh
H. Jin Kim

This study aims to design a motion/force controller for an aerial manipulator which guarantees the tracking of time-varying motion/force trajectories as well as the stability during the transition between free and contact motions. To this end, we model the force exerted on the end-effector as the Kelvin-Voigt linear model and estimate its parameters by recursive least-squares estimator. Then, the gains of the disturbance-observer (DOB)-based motion/force controller are calculated based on the stability conditions considering both the model uncertainties in the dynamic equation and switching between the free and contact motions. To validate the proposed controller, we conducted the time-varying motion/force tracking experiments with different approach speeds and orientations of the surface. The results show that our controller enables the aerial manipulator to track the time-varying motion/force trajectories.

ICRA Conference 2023 Conference Paper

SwinDepth: Unsupervised Depth Estimation using Monocular Sequences via Swin Transformer and Densely Cascaded Network

Dongseok Shim
H. Jin Kim

Monocular depth estimation plays a critical role in various computer vision and robotics applications such as localization, mapping, and 3D object detection. Recently, learning-based algorithms achieve huge success in depth estimation by training models with a large amount of data in a supervised manner. However, it is challenging to acquire dense ground truth depth labels for supervised training, and the unsupervised depth estimation using monocular sequences emerges as a promising alternative. Unfortunately, most studies on unsupervised depth estimation explore loss functions or occlusion masks, and there is little change in model architecture in that ConvNet-based encoder-decoder structure becomes a de-facto standard for depth estimation. In this paper, we employ a convolution-free Swin Transformer as an image feature extractor so that the network can capture both local geometric features and global semantic features for depth estimation. Also, we propose a Densely Cascaded Multi-scale Network (DCMNet) that connects every feature map directly with another from different scales via a top-down cascade pathway. This densely cascaded connectivity reinforces the interconnection between decoding layers and produces high-quality multi-scale depth outputs. The experiments on two different datasets, KITTI and Make3D, demonstrate that our proposed method outperforms existing state-of-the-art unsupervised algorithms.

NeurIPS Conference 2022 Conference Paper

DHRL: A Graph-Based Approach for Long-Horizon and Sparse Hierarchical Reinforcement Learning

Seungjae Lee
Jigang Kim
Inkyu Jang
H. Jin Kim

Hierarchical Reinforcement Learning (HRL) has made notable progress in complex control tasks by leveraging temporal abstraction. However, previous HRL algorithms often suffer from serious data inefficiency as environments get large. The extended components, $i. e. $, goal space and length of episodes, impose a burden on either one or both high-level and low-level policies since both levels share the total horizon of the episode. In this paper, we present a method of Decoupling Horizons Using a Graph in Hierarchical Reinforcement Learning (DHRL) which can alleviate this problem by decoupling the horizons of high-level and low-level policies and bridging the gap between the length of both horizons using a graph. DHRL provides a freely stretchable high-level action interval, which facilitates longer temporal abstraction and faster training in complex tasks. Our method outperforms state-of-the-art HRL algorithms in typical HRL environments. Moreover, DHRL achieves long and complex locomotion and manipulation tasks.

NeurIPS Conference 2022 Conference Paper

S2P: State-conditioned Image Synthesis for Data Augmentation in Offline Reinforcement Learning

Daesol Cho
Dongseok Shim
H. Jin Kim

Offline reinforcement learning (Offline RL) suffers from the innate distributional shift as it cannot interact with the physical environment during training. To alleviate such limitation, state-based offline RL leverages a learned dynamics model from the logged experience and augments the predicted state transition to extend the data distribution. For exploiting such benefit also on the image-based RL, we firstly propose a generative model, S2P (State2Pixel), which synthesizes the raw pixel of the agent from its corresponding state. It enables bridging the gap between the state and the image domain in RL algorithms, and virtually exploring unseen image distribution via model-based transition in the state space. Through experiments, we confirm that our S2P-based image synthesis not only improves the image-based offline RL performance but also shows powerful generalization capability on unseen tasks.

ICRA Conference 2021 Conference Paper

Automated Extrinsic Calibration for 3D LiDARs with Range Offset Correction using an Arbitrary Planar Board

Junha Kim
Changhyeon Kim
Youngsoo Han
H. Jin Kim

This paper proposes an automatic and accuracy- enhanced extrinsic calibration method for 3D LiDARs with a range offset correction, which needs only an arbitrarily-shaped single planar board. One of the most exhaustive parts of existing LiDAR calibration procedures is to manually find target objects from massive point clouds. To obviate user interventions, we propose an automated planar board detection from LiDAR range images. To extract a target completely, we suppress outliers and restore rejected inliers of the target board by introducing a target completion method. We empirically find that range measurements of various LiDARs are mainly skewed by constant offset values. To compensate for this, we suggest a range offset model for each laser channel in calibration procedures. The relative pose between LiDARs and range offsets are jointly estimated by minimizing bi-directional point- to-board distances within the iterative re-weighted least squares (IRLS) framework. To verify the suggested range offset model, we obtain and analyze extensive real-world measurements. By conducting experiments using the various sensor configurations and shapes of boards, we quantitatively and qualitatively confirm accuracy and versatility of the proposed method by comparing with the state-of-the-art LiDAR calibration methods. All the source code and data used in the paper are available at: https://github.com/JunhaAgu/AutoL2LCalib.

ICRA Conference 2021 Conference Paper

Learning a Geometric Representation for Data-Efficient Depth Estimation via Gradient Field and Contrastive Loss

Dongseok Shim
H. Jin Kim

Estimating a depth map from a single RGB image has been investigated widely for localization, mapping, and 3- dimensional object detection. Recent studies on a single-view depth estimation are mostly based on deep Convolutional neural Networks (ConvNets) which require a large amount of training data paired with densely annotated labels. Depth annotation tasks are both expensive and inefficient, so it is inevitable to leverage RGB images which can be collected very easily to boost the performance of ConvNets without depth labels. However, most self-supervised learning algorithms are focused on capturing the semantic information of images to improve the performance in classification or object detection, not in depth estimation. In this paper, we show that existing self- supervised methods do not perform well on depth estimation and propose a gradient-based self-supervised learning algorithm with momentum contrastive loss to help ConvNets extract the geometric information with unlabeled images. As a result, the network can estimate the depth map accurately with a relatively small amount of annotated data. To show that our method is independent of the model structure, we evaluate our method with two different monocular depth estimation algorithms. Our method outperforms the previous state-of-the- art self-supervised learning algorithms and shows the efficiency of labeled data in triple compared to random initialization on the NYU Depth v2 dataset.

ICRA Conference 2021 Conference Paper

Model-based Domain Randomization of Dynamics System with Deep Bayesian Locally Linear Embedding

Jae Hyeon Park
Sungyong Park
H. Jin Kim

Domain randomization (DR) is a powerful tool to make a policy robust to the uncertainty of dynamics caused by unobservable environmental parameters. Conventional DR has adopted model-free reinforcement learning as a policy optimizer. However, the model-free methods in DR demand high time-complexity due to the randomization process where the environment is extremely changed. In this paper, we introduce model-based dynamics and policy learning for efficient DR. A Bayesian model of locally linear embedding is designed to fit the stochastic dynamics in DR. By virtue of locally linear dynamics, model-based optimal control is substituted for the policy optimization. Unlike previous works, our proposed Bayesian model with a MNIW prior allows the locally linear embedding to capture the dynamics in DR as a stochastic model. We show that a training method that combines variational and adversarial approaches is adequate for Bayesian embedding. Finally, a model-based controller is designed on our Bayesian locally linear embedding, and it shows better performance in DR environments compared with the non-Bayesian model of locally linear embedding.

IROS Conference 2021 Conference Paper

Real-Time Motion Planning of a Hydraulic Excavator using Trajectory Optimization and Model Predictive Control

Dongjae Lee
Inkyu Jang
Jeonghyun Byun
Hoseong Seo
H. Jin Kim

Automation of excavation tasks requires real-time trajectory planning satisfying various constraints. To guarantee both constraint feasibility and real-time trajectory re-plannability, we present an integrated framework for real-time optimization-based trajectory planning of a hydraulic excavator. The proposed framework is composed of two main modules: a global planner and a real-time local planner. The global planner computes the entire global trajectory considering excavation volume and energy minimization while the local counterpart tracks the global trajectory in a receding horizon manner, satisfying dynamic feasibility, physical constraints, and disturbance-awareness. We validate the proposed planning algorithm in a simulation environment where two types of operations are conducted in the presence of emulated disturbance from hydraulic friction and soil-bucket interaction: shallow and deep excavation. The optimized global trajectories are obtained in an order of a second, which is tracked by the local planner at faster than 30 Hz. To the best of our knowledge, this work presents the first real-time motion planning framework that satisfies constraints of a hydraulic excavator, such as force/torque, power, cylinder displacement, and flow rate limits.

IROS Conference 2021 Conference Paper

Robust and Recursively Feasible Real-Time Trajectory Planning in Unknown Environments

Inkyu Jang
Dongjae Lee
Seungjae Lee 0001
H. Jin Kim

Motion planners for mobile robots in unknown environments face the challenge of simultaneously maintaining both robustness against unmodeled uncertainties and persistent feasibility of the trajectory-finding problem. That is, while dealing with uncertainties, a motion planner must update its trajectory, adapting to the newly revealed environment in real-time; failing to do so may involve unsafe circumstances. Many existing planning algorithms guarantee these by maintaining the clearance needed to perform an emergency brake, which is itself a robust and persistently feasible maneuver. However, such maneuvers are not applicable for systems in which braking is impossible or risky, such as fixed-wing aircraft. To that end, we propose a real-time robust planner that recursively guarantees persistent feasibility without any need of braking. The planner ensures robustness against bounded uncertainties and persistent feasibility by constructing a loop of sequentially composed funnels, starting from the receding horizon local trajectory’s forward reachable set. We implement the proposed algorithm for a robotic car tracking a speed-fixed reference trajectory. The experiment results show that the proposed algorithm can be run at faster than 16 Hz, while successfully keeping the system away from entering any dead end, to maintain safety and feasibility.

IROS Conference 2021 Conference Paper

Stability and Robustness Analysis of Plug-Pulling using an Aerial Manipulator

Jeonghyun Byun
Dongjae Lee
Hoseong Seo
Inkyu Jang
Jeongjun Choi
H. Jin Kim

In this paper, an autonomous aerial manipulation task of pulling a plug out of an electric socket is conducted, where maintaining the stability and robustness is challenging due to sudden disappearance of a large interaction force. The abrupt change in the dynamical model before and after the separation of the plug can cause destabilization or mission failure. To accomplish aerial plug-pulling, we employ the concept of hybrid automata to divide the task into three operative modes, i. e, wire-pulling, stabilizing, and free-flight. Also, a strategy for trajectory generation and a design of disturbance-observer-based controllers for each operative mode are presented. Furthermore, the theory of hybrid automata is used to prove the stability and robustness during the mode transition. We validate the proposed trajectory generation and control method by an actual wire-pulling experiment with a multirotor-based aerial manipulator.

IROS Conference 2021 Conference Paper

Target-visible Polynomial Trajectory Generation within an MAV Team

Yunwoo Lee
Jungwon Park
Boseong Jeon
H. Jin Kim

Autonomous aerial videography is a challenging task, which involves collision avoidance against obstacles and visibility guaranteed target tracking in unstructured environments. In this paper, we organize a two micro aerial vehicle (MAV) team, which consists of a target agent responsible for a specific mission and a camera agent for filming the target agent. Especially, this paper focuses on trajectory planning of the camera agent to chase without occlusion of target agent. Our trajectory planner module includes two phases of guaranteeing target visibility. In the first phase, we generate homotopic safe flight corridor (SFC) to attain target-visible regions. In the subsequent phase, we generate a safe and smooth trajectory with the continuous visibility constraint based on the SFC, using quadratic programming (QP). Regardless of complexity of map, our planner converts an overall problem to a single QP and generates a steady flight trajectory without undesirable fluctuating motion, while guaranteeing all-time visibility. We validate our approach in Gazebo simulations and a real-world experiment.

IROS Conference 2021 Conference Paper

Topology-Guided Path Planning for Reliable Visual Navigation of MAVs

Dabin Kim
Gyeong Chan Kim
Youngseok Jang
H. Jin Kim

Visual navigation has been widely used for state estimation of micro aerial vehicles (MAVs). For stable visual navigation, MAVs should generate perception-aware paths which guarantee enough visible landmarks. Many previous works on perception-aware path planning focused on sampling-based planners. However, they may suffer from sample inefficiency, which leads to computational burden for finding a global optimal path. To address this issue, we suggest a perception-aware path planner which utilizes topological information of environments. Since the topological class of a path and visible landmarks during traveling the path are closely related, the proposed algorithm checks distinctive topological classes to choose the class with abundant visual information. Topological graph is extracted from the generalized Voronoi diagram of the environment and initial paths with different topological classes are found. To evaluate the perception quality of the classes, we divide the initial path into discrete segments where the points in each segment share similar visual information. The optimal class with high perception quality is selected, and a graph-based planner is utilized to generate path within the class. With simulations and real-world experiments, we confirmed that the proposed method could guarantee accurate visual navigation compared with the perception-agnostic method while showing improved computational efficiency than the sampling-based perception-aware planner.

ICRA Conference 2020 Conference Paper

Aerial Manipulation using Model Predictive Control for Opening a Hinged Door

Dongjae Lee
Hoseong Seo
Dabin Kim
H. Jin Kim

Existing studies for environment interaction with an aerial robot have been focused on interaction with static surroundings. However, to fully explore the concept of an aerial manipulation, interaction with moving structures should also be considered. In this paper, a multirotor-based aerial manipulator opening a daily-life moving structure, a hinged door, is presented. In order to address the constrained motion of the structure and to avoid collisions during operation, model predictive control (MPC) is applied to the derived coupled system dynamics between the aerial manipulator and the door involving state constraints. By implementing a constrained version of differential dynamic programming (DDP), MPC can generate position setpoints to the disturbance observer (DOB)-based robust controller in real-time, which is validated by our experimental results.

IROS Conference 2020 Conference Paper

Detection-Aware Trajectory Generation for a Drone Cinematographer

Boseong Jeon
Dongsuk Shim
H. Jin Kim

This work investigates an efficient trajectory generation for chasing a dynamic target, which incorporates the detectability objective. The proposed method actively guides the motion of a cinematographer drone so that the color of a target is well-distinguished against the colors of the background in the view of the drone. For the objective, we define a measure of color detectability given a chasing path. After computing a discrete path optimized for the metric, we generate a dynamically feasible trajectory. The whole pipeline can be updated on-the- fly to respond to the motion of the target. For the efficient discrete path generation, we construct a directed acyclic graph (DAG) for which a topological sorting can be determined analytically without the depth-first search. The smooth path is obtained in quadratic programming (QP) framework. We validate the enhanced performance of state-of-the-art object detection and tracking algorithms when the camera drone executes the trajectory obtained from the proposed method.

IROS Conference 2020 Conference Paper

Edge-based Visual Odometry with Stereo Cameras using Multiple Oriented Quadtrees

Changhyeon Kim
Junha Kim
H. Jin Kim

We propose an efficient edge-based stereo visual odometry (VO) using multiple quadtrees created according to image gradient orientations. To characterize edges, we classify them into eight orientation groups according to their image gradient directions. Using the edge groups, we construct eight quadtrees and set overlapping areas belonging to adjacent quadtrees for robust and efficient matching. For further acceleration, previously visited tree nodes are stored and reused at the next iteration to warm-start. We propose an edge culling method to extract prominent edgelets and prune redundant edges. The camera motion is estimated by minimizing point-to-edge distances within a re-weighted iterative closest points (ICP) framework, and simultaneously, 3-D structures are recovered by static and temporal stereo settings. To analyze the effects of the proposed methods, we conduct extensive simulations with various settings. Quantitative results on public datasets confirm that our approach has competitive performance with state-of-the-art stereo methods. In addition, we demonstrate the practical values of our system in author-collected modern building scenes with curved edges only.

ICRA Conference 2020 Conference Paper

Efficient Multi-Agent Trajectory Planning with Feasibility Guarantee using Relative Bernstein Polynomial

Jungwon Park
Junha Kim
Inkyu Jang
H. Jin Kim

This paper presents a new efficient algorithm which guarantees a solution for a class of multi-agent trajectory planning problems in obstacle-dense environments. Our algorithm combines the advantages of both grid-based and optimization-based approaches, and generates safe, dynamically feasible trajectories without suffering from an erroneous optimization setup such as imposing infeasible collision constraints. We adopt a sequential optimization method with dummy agents to improve the scalability of the algorithm, and utilize the convex hull property of Bernstein and relative Bernstein polynomial to replace non-convex collision avoidance constraints to convex ones. The proposed method can compute the trajectory for 64 agents on average 6. 36 seconds with Intel Core i7-7700 @ 3. 60GHz CPU and 16G RAM, and it reduces more than 50% of the objective cost compared to our previous work. We validate the proposed algorithm through simulation and flight tests.

ICRA Conference 2020 Conference Paper

Integrated Motion Planner for Real-time Aerial Videography with a Drone in a Dense Environment

Boseong Jeon
Yunwoo Lee
H. Jin Kim

This work suggests an integrated approach for a drone (or multirotor) to perform an autonomous videography task in a 3-D obstacle environment by following a moving object. The proposed system includes 1) a target motion prediction module which can be applied to dense environments and 2) a hierarchical chasing planner. Leveraging covariant optimization, the prediction module estimates the future motion of the target assuming it efforts to avoid the obstacles. The other module, chasing planner, is in a bi-level structure composed of preplanner and smooth planner. In the first phase, we exploit a graph-search method to plan a chasing corridor which incorporates safety and visibility of target. In the subsequent phase, we generate a smooth and dynamically feasible trajectory within the corridor using quadratic programming (QP). We validate our approach with multiple complex scenarios and actual experiments. The source code and the experiment video can be found in https://github.com/icsl-Jeon/traj_gen_vis and https://www.youtube.com/watch?v=_JSwXBwYRl8.

ICRA Conference 2020 Conference Paper

Moving object detection for visual odometry in a dynamic environment based on occlusion accumulation

Haram Kim
Pyojin Kim
H. Jin Kim

Detection of moving objects is an essential capability in dealing with dynamic environments. Most moving object detection algorithms have been designed for color images without depth. For robotic navigation where real-time RGBD data is often readily available, utilization of the depth information would be beneficial for obstacle recognition. Here, we propose a simple moving object detection algorithm that uses RGB-D images. The proposed algorithm does not require estimating a background model. Instead, it uses an occlusion model which enables us to estimate the camera pose on a background confused with moving objects that dominate the scene. The proposed algorithm allows to separate the moving object detection and visual odometry (VO) so that an arbitrary robust VO method can be employed in a dynamic situation with a combination of moving object detection, whereas other VO algorithms for a dynamic environment are inseparable. In this paper, we use dense visual odometry (DVO) as a VO method with a bi-square regression weight. Experimental results show the segmentation accuracy and the performance improvement of DVO in the situations. We validate our algorithm in public datasets and our dataset which also publicly accessible.

IROS Conference 2020 Conference Paper

Navigation-Assistant Path Planning within a MAV team

Youngseok Jang
Yunwoo Lee
H. Jin Kim

In micro aerial vehicle (MAV) operations, the success of a mission is highly dependent on navigation performance, which has raised recent interests on navigation-aware path planning. One of the challenges lies in that optimal motions for successful navigation and the designated mission are often different in unknown, unstructured environments, and only sub-optimality may be obtained in each aspect. We aim to organize a two-MAV team that can effectively execute the mission and simultaneously guarantee navigation quality, which consists of a main-agent responsible for mission and a sub-agent for navigation of the team. Especially, this paper focuses on path planning of the sub-agent to provide navigational assistance to the main-agent using a monocular camera. We adopt a graph-based receding horizon planner to find a dynamically feasible path in order for the sub-agent to help the main-agent's navigation. In this process, we present a metric for evaluating the localization performance utilizing the distribution of the features projected to the image plane. We also design a map management strategy and pose-estimation support mechanism in a monocular camera setup, and validate their effectiveness in two scenarios.

ICRA Conference 2020 Conference Paper

Trajectory Planning with Safety Guaranty for a Multirotor based on the Forward and Backward Reachability Analysis

Hoseong Seo
Clark Youngdong Son
Dongjae Lee
H. Jin Kim

Planning a trajectory with guaranteed safety is a core part for a risk-free flight of a multirotor. If a trajectory planner only aims to ensure safety, it may generate trajectories which overly bypass risky regions and prevent the system from achieving specific missions. This work presents a robust trajectory planning algorithm which simultaneously guarantees the safety and reachability to the target state in the presence of unknown disturbances. We first characterize how the forward and backward reachable sets (FRSs and BRSs) are constructed by using Hamilton-Jacobi reachability analysis. Based on the analysis, we present analytic expressions for the reachable sets and then propose minimal ellipsoids which closely approximate the reachable sets. In the planning process, we optimize the reference trajectory to connect the FRSs and BRSs, while avoiding obstacles. By combining the FRSs and BRSs, we can guarantee that any state inside of the initial set reaches the target set. We validate the proposed algorithm through a simulation of traversing a narrow gap.

ICRA Conference 2019 Conference Paper

Cargo Transportation Strategy using T 3 -Multirotor UAV

Seung Jae Lee 0002
Dongjae Lee
H. Jin Kim

In this paper, we introduce a cargo transportation method with a new type of multi-rotor UAV platform known as T 3 -multirotor, to achieve stable and constant flight performance regardless of the type of cargo attached to the fuselage. The T 3 -multirotor, which consists of the `Thrust Generating Part' and the `Fuselage Part', can directly control the relative attitude between the two parts using the novel servomechanism. By utilizing the servomechanism with the proposed relative attitude control strategy, the T 3 -multirotor with cargo attached to the fuselage part can behave as a multi-rotor with only the moment of inertia of the thrust generating part during entire transportation. This allows the T 3 -multirotor to achieve the reliable performance in the event of any cargo being attached to the fuselage, achieving stable platform motion control. Detailed hardware description and dynamic analysis of T 3 -Multirotor is performed in this paper, and the validity of the proposed control strategy is also analyzed. The feasibility of the proposed control strategy is verified through experimental results with analysis.

IROS Conference 2019 Conference Paper

Fast and Safe Policy Adaptation via Alignment-based Transfer

Jigang Kim
Seungwon Choi
H. Jin Kim

Applying deep reinforcement learning to physical systems, as opposed to learning in simulation, presents additional challenges in terms of sample efficiency and safety. Collecting large amounts of hardware demonstration data is time-consuming and the exploratory behavior of reinforcement learning algorithms may lead the system into dangerous states, especially during the early stages of training. To address these challenges, we apply transfer learning to reuse a previously learned policy instead of learning from scratch. In this paper, we propose a method where given a source policy, policy adaptation is performed via transfer learning to produce a target policy suitable for real-world deployment. For policy adaptation, alignment-based transfer learning is applied to trajectories generated by the source policy and their corresponding safe target trajectories. We apply this method to manipulators and show that the proposed method is applicable to both inter-task and inter-robot transfer whilst considering safety. We also show that the resulting target policy is robust and can be further improved with reinforcement learning.

IROS Conference 2019 Conference Paper

Fast Trajectory Planning for Multiple Quadrotors using Relative Safe Flight Corridor

Jungwon Park
H. Jin Kim

This paper presents a new trajectory planning method for multiple quadrotors in obstacle-dense environments. We suggest a relative safe flight corridor (RSFC) to model safe region between a pair of agents, and it is used to generate linear constraints for inter-collision avoidance by utilizing the convex hull property of relative Bernstein polynomial. Our approach employs a graph-based multi-agent pathfinding algorithm to generate an initial trajectory, which is used to construct a safe flight corridor (SFC) and RSFC. We express the trajectory as a piecewise Bernstein polynomial and formulate the trajectory planning problem into one quadratic programming problem using linear constraints from SFC and RSFC. The proposed method can compute collision-free trajectory for 16 agents within a second and for 64 agents less than a minute, and it is validated both through simulation and indoor flight test.

ICRA Conference 2019 Conference Paper

Networked Operation of a UAV Using Gaussian Process-Based Delay Compensation and Model Predictive Control

Dohyun Jang
Jaehyun Yoo
Clark Youngdong Son
H. Jin Kim
Karl Henrik Johansson

This study addresses an operation of unmanned aerial vehicles (UAVs) in a network environment where there is time-varying network delay. The network delay entails undesirable effects on the stability of the UAV control system due to delayed state feedback and outdated control input. Although several networked control algorithms have been proposed to deal with the network delay, most existing studies have assumed that the plant dynamics is known and simple, or the network delay is constant. These assumptions are improper to multirotor-type UAVs because of their nonlinearity and time-sensitive characteristics. To deal with these problems, we propose a networked control system using model predictive control (MPC) designed under the consideration of multirotor characteristics. We also apply a Gaussian process (GP) to learn an unknown nonlinear model, which increases the accuracy of path planning and state estimation. Flight experiments show that the proposed algorithm successfully compensates the network delay and Gaussian process learning improves the UAV's path tracking performance.

IROS Conference 2019 Conference Paper

Online Trajectory Generation of a MAV for Chasing a Moving Target in 3D Dense Environments

Boseong Jeon
H. Jin Kim

This work deals with a moving target chasing mission of an aerial vehicle equipped with a vision sensor in a cluttered environment. In contrast to obstacle-free or sparse environments, the chaser should be able to handle collision and occlusion together with flight efficiency. In order to tackle these challenges in real-time, we introduce a metric for target visibility and propose a hierarchical chasing planner. In the first phase, we generate a sequence of waypoints and chasing corridors which ensure safety and optimize visibility. In the following phase, the corridors and waypoints are utilized as constraints and objective respectively in quadratic programming from which we complete a dynamically feasible trajectory for chasing. The proposed algorithm is tested in multiple dense environments. The simulator AutoChaser with full code implementation & GUI can be found in https://github.com/icsl-Jeon/traj_gen_vis and video is available at https://youtu.be/-2d3uDlYR_M.

IROS Conference 2019 Conference Paper

Position-based monocular visual servoing of an unknown target using online self-supervised learning

Chungkeun Lee
Hoseong Seo
H. Jin Kim

Visual servoing, i. e. control with visual information, is a valuable capability in many robotic applications. In particular, position based visual servoing (PBVS) estimates position information from the observed image to generate visual servo control. However, the estimation of the position of an unknown target using monocular images is still difficult due to the complexity of the image information. For the target estimation problem, we propose to integrate three complementary techniques for monocular visual servoing. First, to estimate the probability of a target’s existence, the learning model with spatial features from convolution neural network is proposed. Second, the extended Kalman filter based on epipolar geometry estimates the 3D position of the target; moreover, from this 3D position, the perception model is trained online by self-generated virtual ground-truth. Finally, visual servo control is generated, and the resulting movement helps to construct epipolar geometry. Finally. the experimental validation is performed in a challenging setting involving occlusion and target’s shape change.

ICRA Conference 2019 Conference Paper

Real-time Optimal Planning and Model Predictive Control of a Multi-rotor with a Suspended Load

Clark Youngdong Son
Dohyun Jang
Hoseong Seo
Taewan Kim
Hyeonbeom Lee
H. Jin Kim

This paper presents planning and control algorithms for a multi-rotor with a suspended load. The suspended load cannot be controlled easily by the multi-rotor due to severe dynamic coupling between them. Difficulties are exacerbated by under-actuated, highly nonlinear nature of multi-rotor dynamics. Although many studies have been proposed to plan trajectories and control this system, there exist only a few reports on real-time trajectory generation. With this in mind, we propose a planning method which is capable of generating collision-free trajectories real-time and applicable to a high-dimensional nonlinear system. Using a differential flatness property, the system can be linearized entirely with elaborately chosen flat outputs. Convexification of non-convex constraints is carried out, and concave obstacle-avoidance constraints are converted to convex ones. After that, a convex optimization problem is solved to generate an optimal trajectory, but semi-feasible trajectory which considers only some parts of the initial state. We apply model predictive control with a sequential linear quadratic solver to compute a feasible collision-free trajectory and to control the system. Performance of the algorithm is validated by flight experiment.

IROS Conference 2019 Conference Paper

Robust Real-time RGB-D Visual Odometry in Dynamic Environments via Rigid Motion Model

Sangil Lee
Clark Youngdong Son
H. Jin Kim

In the paper, we propose a robust real-time visual odometry in dynamic environments via rigid-motion model updated by scene flow. The proposed algorithm consists of spatial motion segmentation and temporal motion tracking. The spatial segmentation first generates several motion hypotheses by using a grid-based scene flow and clusters the extracted motion hypotheses, separating objects that move independently of one another. Further, we use a dual-mode motion model to consistently distinguish between the static and dynamic parts in the temporal motion tracking stage. Finally, the proposed algorithm estimates the pose of a camera by taking advantage of the region classified as static parts. In order to evaluate the performance of visual odometry under the existence of dynamic rigid objects, we use self-collected dataset containing RGB-D images and motion capture data for ground-truth. We compare our algorithm with state-of-the-art visual odometry algorithms. The validation results suggest that the proposed algorithm can estimate the pose of a camera robustly and accurately in dynamic environments.

IROS Conference 2019 Conference Paper

Robust Trajectory Planning for a Multirotor against Disturbance based on Hamilton-Jacobi Reachability Analysis

Hoseong Seo
Donggun Lee
Clark Youngdong Son
Claire J. Tomlin
H. Jin Kim

Ensuring safety in trajectory planning of multirotor systems is an essential element for risk-free operation. Even if the generated trajectory is known to be safe in the planning phase, unknown disturbance during an actual operation can lead to a dangerous situation. This paper proposes safety-guaranteed receding horizon planning against unknown, but bounded, disturbances. We first characterize forward reachable set (FRS) of the system, the set of states after a certain duration considering all possible disturbances, using Hamilton-Jacobi (HJ) reachability analysis. To compute the FRSs in real-time, we conservatively approximate the true FRS and perform ellipsoidal parameterization on the FRSs. Using the FRSs, we can plan a robust trajectory that avoids risky regions and rapidly re-plan the trajectory when the system encounters sudden disturbance. The proposed method is validated through an experiment of avoiding obstacles in a wind.

IROS Conference 2019 Conference Paper

Sampling-based Motion Planning for Aerial Pick-and-Place

Hyoin Kim
Hoseong Seo
Jongchan Kim
H. Jin Kim

This paper presents a motion planning approach for an aerial pick-and-place task where an aerial manipulator is supposed to pick up or place an object at locations specified as way-points. In particular, we focus on situations where such way-point constraints are imposed on certain partial state variables, rather than on full state variables. Our proposed framework, based on rapidly exploring random trees star (RRT * ) in a bidirectional manner, enables an aerial manipulator to find an optimal trajectory that satisfies way-point constraints with only partial specifications. Here, we suggest an extra merging process to integrate the trees, each originated from the start and goal point. In the merging process, we search various candidate points satisfying a given condition that partially constrains state variables, and select a way-point with full specifications optimal in the perspective of the entire trajectory. Simulation and experiment results are included to validate the proposed framework.

ICRA Conference 2018 Conference Paper

Design, Modeling and Control of T3-Multirotor: A Tilting Thruster Type Multirotor *This work was supported by the Robotics Core Technology Development Project (10080301) funded by the Ministry of Trade Industry and Energy (MoTIE, Korea), National Research Foundation of Korea (NRF) grant funded by the Ministry of Science, ICT and Future Planning (2014M1A3A3A02034854)

Seung Jae Lee 0002
Jaehyun Yoo
H. Jin Kim

This paper presents a new design of multirotor, named as `Tilting Thruster Type' (T 3 )-multirotor. The new platform is equipped with mechanically separated thrusters, which can take any fuselage posture within a specified range regardless of any direction of translational acceleration. A specially designed servo-linkage mechanism is employed for relative attitude control between the thruster and the fuselage. Mathematical modeling and analysis of the new platform are conducted to explore the control method of the dynamically complex system. For demonstrating the potential of the new T 3 -multirotor, an autonomous level flight is performed where the fuselage maintains zero roll and pitch angle during the entire flight. Both simulation and experimental results are provided with detailed analysis.

IROS Conference 2018 Conference Paper

Edge-Based Robust RGB-D Visual Odometry Using 2-D Edge Divergence Minimization

Changhyeon Kim
Pyojin Kim
Sangil Lee
H. Jin Kim

This paper proposes an edge-based robust RGB-D visual odometry (VO) using 2-D edge divergence minimization. Our approach focuses on enabling the VO to operate in more general environments subject to low texture and changing brightness, by employing image edge regions and their image gradient vectors within the iterative closest points (ICP) framework. For more robust and stable ICP-based optimization, we propose a robust edge matching criterion with image gradient vectors. In addition, to reduce a bad effect of outlier residuals, we propose an improved edge registration problem of 2-D edge divergence minimization in the manner of an iterative re-weight least squares (IRLS) motion estimation. To accelerate the proposed approach, a pixel sub-sampling method is employed. We evaluate estimation performance of our method in changing brightness conditions and low-textured scenes. Our approach shows more robust motion estimation than state-of-the-art methods while maintaining comparable accuracy in challenging image sequences at real-time (25 Hz) operation.

IROS Conference 2018 Conference Paper

Learning and Generalization of Dynamic Movement Primitives by Hierarchical Deep Reinforcement Learning from Demonstration

Wonchul Kim
Chungkeun Lee
H. Jin Kim

This paper presents an approach to learn and generalize robotic skills from a demonstration using deep reinforcement learning (deep RL). Dynamic Movement Primitives (DMPs) formulate a nonlinear differential equation and produce the observed movement from a demonstration. However, it is hard to generate new behaviors from using DMPs. Thus, we apply DMPs framework into deep RL as an initial setting for learning the robotic skills. First, we build a network to represent this differential equation, and learn and generalize the movements by optimizing the shape of DMPs with respect to the rewards up to the end of each sequence of movement primitives. In order to do this, we consider a deterministic actor-critic algorithm for deep RL and we also apply a hierarchical strategy. This drastically reduces the search space for a robot by decomposing the task, which allows to solve the sparse reward problem from a complex task. In order to integrate DMPs with hierarchical deep RL, the differential equation is considered as temporal abstraction of option. The overall structure is mainly composed of two controllers: meta-controller and sub-controller. The meta-controller learns a policy over intrinsic goals and a sub-controller learns a policy over actions to accomplish the given goals. We demonstrate our approach on a 6 degree-of-freedom (DOF) arm with a I-DOF gripper and evaluate our approach through a pick-and-place task.

IROS Conference 2018 Conference Paper

Learning-based Path Tracking Control of a Flapping-wing Micro Air Vehicle

Jonggu Lee
Seungwan Ryu
Taewan Kim
Wonchul Kim
H. Jin Kim

Flapping-wing micro air vehicles (FWMAVs) become promising research platforms due to their advantages such as various maneuverability, and concealment. However, unsteady flow at low Reynolds number around the wings makes their dynamics time-varying and highly non-linear. It makes autonomous flight of FWMAV as a big challenge. In this paper, we suggest a model-based control strategy for FWMAV using learning architecture. For this task, we construct a ground station for logging flight data and control inputs, and train dynamics with a neural network. Then, we apply model predictive control (MPC) to the trained model. We validate our method by hardware experiments.

ICRA Conference 2018 Conference Paper

Low-Drift Visual Odometry in Structured Environments by Decoupling Rotational and Translational Motion

Pyojin Kim
Brian Coltin
H. Jin Kim

We present a low-drift visual odometry algorithm that separately estimates rotational and translational motion from lines, planes, and points found in RGB-D images. Previous methods estimate drift-free rotational motion from structural regularities to reduce drift in the rotation estimate, which is the primary source of positioning inaccuracy in visual odometry. However, multiple orthogonal planes are required to be visible throughout the entire motion estimation process; otherwise, these VO approaches fail. We propose a new approach to estimate drift-free rotational motion jointly from both lines and planes by exploiting environmental regularities. We track the spatial regularities with an efficient SO(3)-manifold constrained mean shift algorithm. Once the drift-free rotation is found, we recover the translational motion from all tracked points with and without depth by minimizing the de-rotated reprojection error. We compare the proposed algorithm to other state-of-the-art visual odometry methods on a variety of RGB-D datasets (including especially challenging pure rotations) and demonstrate improved accuracy and lower drift error.

ICRA Conference 2018 Conference Paper

Model Predictive Control of a Multi-Rotor with a Suspended Load for Avoiding Obstacles

Clark Youngdong Son
Hoseong Seo
Taewan Kim
H. Jin Kim

This paper investigates a multi-rotor with a suspended load in perspectives of 1) real-time path planning, 2) obstacle avoidance, and 3) transportation of a suspended object. A suspended load cannot be controlled with conventional controllers designed for nominal multi-rotors due to the dynamic coupling between the multi-rotor and load. Although several control and planning algorithms have been proposed based on elaborately derived dynamic equations, most existing studies separate control and path planning problems by following predefined trajectories after trajectory generation. Moreover, many state-of-the-art trajectory generation algorithms cannot work real-time for a system with high degrees of freedom, which makes it not suitable to operate the system in dynamic environments where obstacles appear abruptly or move unexpectedly. With this in mind, we apply Model Predictive Control (MPC) with Sequential Linear Quadratic (SLQ) solver to compute feasible and optimal trajectory real-time and to operate a multi-rotor with a suspended load in dynamic environments. We design an obstacle-avoidance algorithm suitable for the current platform flying in cluttered environments. Flight experiments shows that the proposed algorithm successfully controls the multi-rotor and allows to avoid obstacles simultaneously.

IROS Conference 2018 Conference Paper

Vision-based Target Tracking for a Skid-steer Vehicle using Guided Policy Search with Field-of-view Constraint

Taewan Kim
Chungkeun Lee
Hoseong Seo
Seungwon Choi
Wonchul Kim
H. Jin Kim

This paper describes a vision-based target tracking method for a skid-steer vehicle. With the development of deep reinforcement learning, many researchers have tried to generate an end-to-end policy to control the mobile robot from a raw pixel image data. However, the action in most research only concerns high-level decisions such as go straight, turn left and right. High-level decisions alone are not sufficient to precisely control platforms such as a skid-steer vehicle due to the lack of steering mechanism. Thus, unlike existing work, we aim to control the motor command for the wheels directly. To this end, we employ guided policy search (GPS) based on the general kinematic slip model for the skid-type robot. Furthermore, to prohibit the target from getting out of the camera field of view (FOV) in the training phase, we update local policy optimization with a FOV constraint and perform a pre-training to make the initial policy more efficient. Our method allows the skid-type robot to automatically acquire the vision-based tracking policy while local policies satisfy the FOV constraint during the training phase. We evaluate our method through both simulation and experiment with a skid-steer mobile robot. Finally, we test the performance of learned policy with a moving target in a new environment.

ICRA Conference 2017 Conference Paper

Aerial grasping of cylindrical object using visual servoing based on stochastic model predictive control

Hoseong Seo
Suseong Kim
H. Jin Kim

This paper concentrates on design of a vision-based guidance command for aerial manipulation of a cylindrical object, using a stochastic model predictive approach. We first develop an image-based cylinder detection algorithm that utilizes a geometric characteristic of perspectively projected circles in 3D space. To enforce the object to be located inside sight of a camera, we formulate a visual servoing problem as a stochastic model predictive control (MPC) framework. By regarding x and y axes rotational velocities as stochastic variables, we guarantee the visibility of the camera considering underactuation of the system. We also provide experimental results that validate effectiveness of the proposed algorithm.

ICRA Conference 2017 Conference Paper

Autonomous swing-angle estimation for stable slung-load flight of multi-rotor UAVs

Seung Jae Lee 0002
H. Jin Kim

This paper presents a practical swing-angle estimation method for slung load operations of the multi-rotor unmanned aerial vehicle (UAV), which is essential to maintain the safety during the operation. In order not to rely on extra sensors for monitoring the swing angle, the proposed method in this paper offers an autonomous swing-angle estimation using only an inertial measurement unit (IMU) and a single load cell attached to the slung load. The disturbance observer (DOB) derived external force estimation is performed to estimate the swing angle. The unique structure of the proposed DOB-based disturbance force estimation technique utilizes the IMU data only. Both simulation and actual experiment are performed to validate the feasibility of the proposed algorithm.

IROS Conference 2017 Conference Paper

Development of a flapping-wing micro air vehicle capable of autonomous hovering with onboard measurements

Seungwan Ryu
H. Jin Kim

Hovering flight is one of the valuable capabilities for unmanned aerial vehicles (UAVs) to perform various missions. It is easily attainable by rotary-wing aircrafts such as a helicopter and multirotor. This flight maneuver, however, remains a significant challenge to a flapping wing micro air vehicle (FWMAV). This paper aims to regulate attitude angles of a FWMAV and maintain an altitude during hovering. For this task, we model dynamic equations of FWMAV to analyze the stability of longitudinal dynamics under the influence of tail wings. In order to examine the effect of the flapping wings on the fuselage, we carry out measurements on the power train. We develop a FWMAV with two pairs of main flapping wings and tail wings, to achieve stable hovering flight. For autonomous control, PID controllers are designed and onboard sensors are used; 9-degree of freedom (DoF) IMU is for attitude regulation and barometer is for an altitude control. For estimating an altitude, we also apply processing filter. Experimental results show that our strategy can accomplish and sustain autonomous hovering of a FWMAV.

IROS Conference 2017 Conference Paper

Locally optimal trajectory planning for aerial manipulation in constrained environments

Hoseong Seo
Suseong Kim
H. Jin Kim

Aerial manipulation tasks necessitate a reliable trajectory planning algorithms to perform complicated tasks. This paper provides a method of developing the locally optimal trajectory for aerial manipulation in constrained environments. We first show differential flatness of the aerial manipulation system when the inertial effect due to equipped robotic arm is compensated with the aid of a robust controller. To find the locally optimal path, we parameterize flat outputs as polynomials of time, and formulate a sequential quadratic programming (SQP) problem. Given a convex mesh representation of environment, we obtain a collision-free trajectory in almost real time, by imposing constraints based on a signed distance metric. We also conduct an experiment of operating an object located in a confined space, which validates effectiveness of the proposed algorithm.

ICRA Conference 2017 Conference Paper

Motion planning with movement primitives for cooperative aerial transportation in obstacle environment

Hyoin Kim
Hyeonbeom Lee
Seungwon Choi
Yung-Kyun Noh
H. Jin Kim

This paper presents a motion planning approach for cooperative transportation using aerial robots. We describe a framework based on Parametric Dynamic Movement Primitives (PDMPs) for coordinating multiple aerial robots and their manipulators quickly in an environment cluttered with obstacles. In order to emulate the optimal motion, we combine PDMPs and Rapidly Exploring Randomized Trees star (RRT*) by using the results of RRT* as demonstrations for PDMPs. For efficient description of the motions corresponding to the environment, we utilize Gaussian Process Regression (GPR) to acquire of the explicit relationship between environmental parameters and style parameters of PDMPs which decide the motions. Simulation and experiment results are attached to validate the proposed framework.

ICRA Conference 2017 Conference Paper

Robust visual localization in changing lighting conditions

Pyojin Kim
Brian Coltin
Oleg Alexandrov
H. Jin Kim

We present an illumination-robust visual localization algorithm for Astrobee, a free-flying robot designed to autonomously navigate on the International Space Station (ISS). Astrobee localizes with a monocular camera and a pre-built sparse map composed of natural visual features. Astrobee must perform tasks not only during the day, but also at night when the ISS lights are dimmed. However, the localization performance degrades when the observed lighting conditions differ from the conditions when the sparse map was built. We investigate and quantify the effect of lighting variations on visual feature-based localization systems, and discover that maps built in darker conditions can also be effective in bright conditions, but the reverse is not true. We extend Astrobee's localization algorithm to make it more robust to changing-light environments on the ISS by automatically recognizing the current illumination level, and selecting an appropriate map and camera exposure time. We extensively evaluate the proposed algorithm through experiments on Astrobee.

IROS Conference 2016 Conference Paper

Autonomous flight and vision-based target tracking for a flapping-wing MAV

Seungwan Ryu
Ukjin Kwon
H. Jin Kim

In this paper, we design an autonomous flight controller for height regulation and bang-bang controller for directional control of a light-weight flapping-wing micro air vehicle (FWMAV) with limited payload. We also present autonomous vision-based target tracking for a FWMAV equipped with a low-cost and light-weight first person view (FPV) camera. We construct a ground station, integrated with control and vision algorithms, which performs the image processing and the computation of control inputs based on the acquired state variables from motion capture system. In addition, we employ a vision algorithm for a low-quality camera to detect a static target with the discussions on the techniques to improve the reliability of visual detection. Experimental results show satisfactory flight performance, achieving the height regulation and directional control, and autonomous vision-based target tracking.

ICRA Conference 2015 Conference Paper

Operating an unknown drawer using an aerial manipulator

Suseong Kim
Hoseong Seo
H. Jin Kim

This paper is about opening and closing an unknown drawer using an aerial manipulator. To accommodate practical applications, it is assumed that the direction of motion and mechanical properties of the drawer are not given beforehand. A multirotor combined with a robotic arm is used for the manipulation task. Typical drawers are allowed to move in only one direction, which constrains the motion of the aerial manipulator while operating a drawer. To analyze this interaction, the dynamic characteristics of the aerial manipulator are modeled. Also, configuration of the aerial manipulator for exerting the desired force to a drawer is presented. To handle the uncertainties associated with the mechanism of a drawer, strategies exploiting velocity of the end effector are employed. The proposed approach is validated with experiments including opening and closing a common drawer, which is detected by a camera mounted in the palm of the end effector.

IROS Conference 2015 Conference Paper

Path planning and control of multiple aerial manipulators for a cooperative transportation

Hyeonbeom Lee
Hyoin Kim
H. Jin Kim

This paper presents planning and control of multiple aerial manipulators for cooperative transportation. Individual aerial manipulators which consist of a hexacopter and 2-DOF robotic arm are controlled by an augmented adaptive sliding mode controller based on a closed-chain robot dynamics. The desired path for each aerial manipulator is obtained by using RRT* to transport an object to the desired position. It also considers the constraints about the grasping point at the end effector. To validate the proposed planning and control algorithm, an experimental result with multiple custom-made aerial manipulators is presented, which involves two aerial manipulators tracking the user-guided command and planned trajectory.

IROS Conference 2015 Conference Paper

Robust visual odometry to irregular illumination changes with RGB-D camera

Pyojin Kim
Hyon Lim
H. Jin Kim

Sensitivity to illumination conditions poses a challenge when utilizing visual odometry (VO) in various applications. To make VO robust with respect to illumination conditions, they need to be considered explicitly. In this paper, we propose a direct visual odometry method which can handle illumination changes by considering an affine illumination model to compensate abrupt, local light variations during direct motion estimation process. The core of our proposed method is to estimate the relative camera pose and the parameters of the illumination changes by minimizing the sum of squared photometric error with efficient second-order minimization. We evaluate the performance of the proposed algorithm on synthetic and real RGB-D datasets with ground-truth. Our result implies that the proposed method successfully estimates 6-DoF pose under significant illumination changes whereas existing direct visual odometry methods either fail or lose accuracy.

ICRA Conference 2014 Conference Paper

Aerodynamic power control for multirotor aerial vehicles

Moses Bangura
Hyon Lim
H. Jin Kim
Robert E. Mahony

In this paper, a new motor control input and controller for small-scale electrically powered multirotor aerial vehicles is proposed. The proposed scheme is based on controlling aerodynamic power as opposed to the rotor speed of each motor-rotor system. Electrical properties of the brushless direct current motor are used to both estimate and control the mechanical power of the motor system which is coupled with aerodynamic power using momentum theory analysis. In comparison to current state-of-the-art motor control for multirotor aerial vehicles, the proposed approach is robust to unmodelled aerodynamic effects such as wind disturbances and ground effects. Theory and experimental results are presented to illustrate the performance of the proposed motor control.

ICRA Conference 2014 Conference Paper

Real-time 6-DOF monocular visual SLAM in a large-scale environment

Hyon Lim
Jongwoo Lim
H. Jin Kim

Real-time approach for monocular visual simultaneous localization and mapping (SLAM) within a large-scale environment is proposed. From a monocular video sequence, the proposed method continuously computes the current 6-DOF camera pose and 3D landmarks position. The proposed method successfully builds consistent maps from challenging outdoor sequences using a monocular camera as the only sensor, while existing approaches have utilized additional structural information such as camera height from the ground. By using a binary descriptor and metric-topological mapping, the system demonstrates real-time performance on a large-scale outdoor environment without utilizing GPUs or reducing input image size. The effectiveness of the proposed method is demonstrated on various challenging video sequences including the KITTI dataset and indoor video captured on a micro aerial vehicle.

IROS Conference 2013 Conference Paper

Aerial manipulation using a quadrotor with a two DOF robotic arm

Suseong Kim
Seungwon Choi
H. Jin Kim

This paper presents aerial manipulation using a quadrotor with a two-DOF robot arm. By considering a quadrotor and robot arm as a combined system, the kinematic and dynamic models are developed, and an adaptive sliding mode controller is designed. With the controller, an autonomous flight experiment is conducted including picking up and delivering an object, which requires accurate control of a quadrotor and robot arm. Overall result shows that the proposed approach demonstrates satisfactory performance as a potential platform which can be utilized in various applications such as inspection, manipulation, or transportation in remote places.

ICRA Conference 2013 Conference Paper

Joint detection and tracking of boundaries using cooperative mobile sensor networks

Woojin Kim
Dong Jun Kwak
H. Jin Kim

This paper considers a boundary tracking problem using mobile sensor networks, in which we design controllers for the mobile sensors to obtain the boundary of physical events. We set the boundary estimation problem as a classification problem of the region in which the physical events occurs, and employ support vector learning (SVL). By using the hyper-dimensional radius function obtained from SVL, we build the hyper-potential field to generate a velocity vector field which is globally attractive to a desired closed path with circulation at the desired speed. We also study stabilizing the collective configuration of the multiple mobile sensors. To coordinate the mobile sensors in the formation that encloses the boundary, we define virtual phases of mobile sensors and compute the desired speed of each mobile sensors minimizing the level of synchrony of the virtual phases. Both a simulation and an experiment is performed and the results demonstrate that this study provides good performance of the collective boundary tracking.

ICRA Conference 2012 Conference Paper

Autonomous landing of a VTOL UAV on a moving platform using image-based visual servoing

Daewon Lee
Tyler Ryan
H. Jin Kim

In this paper we describe a vision-based algorithm to control a vertical-takeoff-and-landing unmanned aerial vehicle while tracking and landing on a moving platform. Specifically, we use image-based visual servoing (IBVS) to track the platform in two-dimensional image space and generate a velocity reference command used as the input to an adaptive sliding mode controller. Compared with other vision-based control algorithms that reconstruct a full three-dimensional representation of the target, which requires precise depth estimation, IBVS is computationally cheaper since it is less sensitive to the depth estimation allowing for a faster method to obtain this estimate. To enhance velocity tracking of the sliding mode controller, an adaptive rule is described to account for the ground effect experienced during the maneuver. Finally, the IBVS algorithm integrated with the adaptive sliding mode controller for tracking and landing is validated in an experimental setup using a quadrotor.

ICRA Conference 2012 Conference Paper

Multi-target tracking using distributed SVM training over wireless sensor networks

Woojin Kim
Jae Hyun Yoo
H. Jin Kim

In this paper, we propose to use distributed support vector machine (SVM) training to solve a multi-target tracking problem in wireless sensor networks. We employ gossip-based incremental SVM to obtain the discriminant function. By gossiping the support vectors with neighboring sensor nodes, the local SVM training results can achieve the agreement of the sub-optimal discriminant planes. After training the local SVM at each node, we can calculate the posterior probability of the existence of the targets using Platt's method. By maximum a posterior (MAP), the target trajectories are estimated. In order to validate the proposed tracking framework in wireless sensor networks, we perform two different target-tracking experiments. The experimental results demonstrate that the proposed procedure provides a good estimator, and supports the feasibility of applying the distributed SVM training to the target tracking problems.

IROS Conference 2012 Conference Paper

Onboard flight control of a micro quadrotor using single strapdown optical flow sensor

Hyon Lim
Hyeonbeom Lee
H. Jin Kim

This paper considers autonomous onboard hovering flight control of a micro quadrotor using a strapdown optical flow sensor which is conventionally used for desktop mice. The vehicle considered in this paper can carry only a few dozen grams of payload, therefore conventional camera-based optical flow methods are not applicable. We present autonomous hovering flight control of the micro quadrotor using a single-chip optical flow sensor, implemented on an 8-bit microprocessor without external positioning sensors. Detailed description of all the system components is provided along with evaluation of the accuracy. Experimental results from flight tests are validated with the ground-truth data provided by a high-accuracy motion capture system.

IROS Conference 2011 Conference Paper

Event-driven Gaussian process for object localization in wireless sensor networks

Jae Hyun Yoo
Woojin Kim
H. Jin Kim

Object localization using wireless sensor networks (WSN) often requires data from many sensor nodes and different types of sensors for position estimation. This incurs a heavy communication load, which can cause packet loss, communication delay and much energy consumption, deteriorating the performance of object localization. Here we employ an event-driven Gaussian process in order to learn the position of an unknown object using WSN with multiple types of sensors. In the event-driven framework, each sensor node transmits data only when decision criteria are satisfied. We consider the error-bounded algorithm as the decision criteria based on the measurement history of each sensor node. The overall communication between sensor nodes is reduced, thus increasing energy-efficiency of the network and relieving the concentration of communication traffic at the base node. Experiments to track the position of a mobile robot are conducted using a multi-sensor WSN, and the comparison is made between the event-driven framework and the conventional approach in which sensors transmit data at a constant sampling rate. Experimental results demonstrate the efficiency and accuracy of the proposed event-driven Gaussian process approach.

IROS Conference 2008 Conference Paper

Utilizing parallax information for collisionavoidancein dynamic environments

Yongsoon Yoon
Jae Mann Park
H. Jin Kim
S. Shankar Sastry

This paper studies an active steering problem of unmanned ground vehicles (UGVs) when avoiding obstacles during sensor based navigation in unknown environments. The overall problem is treated using the nonlinear model predictive framework, in which the sensor information of a limited sensing range is incorporated online. Results show that the introduction of the modified parallax effectively reflects the threat of obstacles and consequently achieves safe navigation in unknown environments satisfying dynamic constraints.

ICRA Conference 2007 Conference Paper

Trajectory Generation for Rendezvous of Unmanned Aerial Vehicles with Kinematic Constraints

Jin-Wook Lee
H. Jin Kim

In this paper, we present an efficient method for finding collision-free trajectory for multiple unmanned aerial vehicles (UAVs) with kinematic constraints and for their rendezvous to form a formation. First, we construct a visibility graph that supports a minimum turning radius constraint when constructing the graph, so that additional smoothing process is not necessary. Second, we modify the standard A* to consider velocity conditions for rendezvous and collision avoidance with obstacles or other UAVs. Permitting velocity decrease only when it is required that the robot slow down the speed, unnecessary node expansions are avoided. This multi-vehicle problem is solved in a decoupled manner. In order to show the effectiveness of this approach, we present simulation results of rendezvous and independent flight for multiple UAVs.

ICRA Conference 2002 Conference Paper

Flying Robots: Modeling, Control and Decision Making

H. Jin Kim
David Hyunchul Shim
S. Shankar Sastry

This paper presents a flight management system (FMS) implemented as on-board intelligence for rotorcraft-based unmanned aerial vehicles (RUAV's), in order to gradually refine given abstract mission commands into real-time control signals for each vehicle. A strategy planner uses the probabilistic decision making algorithms to determine suboptimal action at each time step. A graphical interface on ground station enables human intervention. We derive nonlinear dynamics model upon which we design a tracking control layer using nonlinear model predictive control and integrate with a trajectory generator for logistical action planning. The proposed structure has been implemented on Berkeley RUAVs and validated in probabilistic pursuit-evasion games to show the possibility of intelligent flying robots.