Author name cluster

Dongjun Lee

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

60 papers

2 author rows

AAAI Conference 2026 System Paper

Market-Aware Event Timeline Summarization: Integrating Price Signals to Improve Financial News Understanding

Heesoo Park
Sujin Park
Dongjun Lee

We present a practical system that supports in-depth analysis of cryptocurrency markets through timeline-based event detection and contextual summarization. Our framework processes continuous news streams, identifies price-relevant events, and organizes them into semantic timelines with concise background summaries generated by large language models (LLMs). This design allows traders and analysts to retrospectively explore events alongside price charts, facilitating a deeper understanding of how news developments relate to market fluctuations. By transforming unstructured news data into structured insights, the system provides a valuable tool for market analysis, risk evaluation, and behavioral studies in volatile trading environments.

PDF Details DOI

ICML Conference 2025 Conference Paper

Causality-Aware Contrastive Learning for Robust Multivariate Time-Series Anomaly Detection

HyunGi Kim
Jisoo Mok
Dongjun Lee
Jaihyun Lew
Sungjae Kim
Sungroh Yoon

Utilizing the complex inter-variable causal relationships within multivariate time-series provides a promising avenue toward more robust and reliable multivariate time-series anomaly detection (MTSAD) but remains an underexplored area of research. This paper proposes Causality-Aware contrastive learning for RObust multivariate Time-Series (CAROTS), a novel MTSAD pipeline that incorporates the notion of causality into contrastive learning. CAROTS employs two data augmentors to obtain causality-preserving and -disturbing samples that serve as a wide range of normal variations and synthetic anomalies, respectively. With causality-preserving and -disturbing samples as positives and negatives, CAROTS performs contrastive learning to train an encoder whose latent space separates normal and abnormal samples based on causality. Moreover, CAROTS introduces a similarity-filtered one-class contrastive loss that encourages the contrastive learning process to gradually incorporate more semantically diverse samples with common causal relationships. Extensive experiments on five real-world and two synthetic datasets validate that the integration of causal relationships endows CAROTS with improved MTSAD capabilities. The code is available at https: //github. com/kimanki/CAROTS.

ICRA Conference 2025 Conference Paper

Efficient Gradient-Based Inference for Manipulation Planning in Contact Factor Graphs

Jeongmin Lee
Sunkyung Park
Minji Lee
Dongjun Lee

This paper presents a framework designed to tackle a range of planning problems arise in manipulation, which typically involve complex geometric-physical reasoning related to contact and dynamic constraints. We introduce the Contact Factor Graph (CFG) to graphically model these diverse factors, enabling us to perform inference on the graphs to approximate the distribution and sample appropriate solutions. We propose a novel approach that can incorporate various phenomena of contact manipulation as differentiable factors, and develop an efficient inference algorithm for CFG that leverages this differentiability along with the conditional probabilities arising from the structured nature of contact. Our results demonstrate the capability of our framework in generating viable samples and approximating posterior distributions for various manipulation scenarios.

ICRA Conference 2025 Conference Paper

GPU-Accelerated Subsystem-Based ADMM for Large-Scale Interactive Simulation

Harim Ji
Hyunsu Kim
Jeongmin Lee 0002
Somang Lee
Seoki An
Jinuk Heo
Youngseon Lee
Yongseok Lee

In this paper, we implement the GPU-accelerated subsystem-based Alternating Direction Method of Multipliers (SubADMM) for interactive simulation. The challenging objective for interactive simulations is to deliver realistic results under tight performance, even for large-scale scenarios. We aim to achieve this by exploiting the parallelizable nature of SubADMM to the fullest extent. We introduce a new subsystem division strategy to make SubADMM ‘GPU friendly' along with custom kernel designs and optimization regarding efficient memory access patterns. We successfully implement the GPUaccelerated SubADMM and show the accuracy and speed of the framework for large-scale scenarios, highlighted with an interactive ‘Hand demo’ scenario. We also show improved robustness and accuracy compared to other state-of-the-art interactive simulators with several challenging scenarios that introduce large-scale ill-conditioned dynamics problems.

ICLR Conference 2025 Conference Paper

Learning to Contextualize Web Pages for Enhanced Decision Making by LLM Agents

Dongjun Lee
Juyong Lee
Kyuyoung Kim
Jihoon Tack
Jinwoo Shin
Yee Whye Teh
Kimin Lee

Recent advances in large language models (LLMs) have led to a growing interest in developing LLM-based agents for automating web tasks. However, these agents often struggle with even simple tasks on real-world websites due to their limited capability to understand and process complex web page structures. In this work, we introduce LCoW, a framework for Learning language models to Contextualize complex Web pages into a more comprehensible form, thereby enhancing decision making by LLM agents. LCoW decouples web page understanding from decision making by training a separate contextualization module to transform complex web pages into comprehensible format, which are then utilized by the decision-making agent. We demonstrate that our contextualization module effectively integrates with LLM agents of various scales to significantly enhance their decision-making capabilities in web automation tasks. Notably, LCoW improves the success rates of closed-source LLMs (e.g., Gemini-1.5-flash, GPT-4o, Claude-3.5-Sonnet) by an average of 15.6%, and demonstrates a 23.7% average improvement in success rates for open-source LMs (e.g., Llama-3.1-8B, Llama-3.1-70B) on the WorkArena benchmark. Moreover, the Gemini-1.5-flash agent with LCoW achieves state-of-the-art results on the WebShop benchmark, outperforming human experts. The relevant code materials are available at our project page: https://lcowiclr2025.github.io.

ICRA Conference 2025 Conference Paper

Narrow Passage Path Planning Using Collision Constraint Interpolation

Minji Lee
Jeongmin Lee
Dongjun Lee

Narrow passage path planning is a prevalent problem from industrial to household sites, often facing difficulties in finding feasible paths or requiring excessive computational resources. Given that deep penetration into the environment can cause optimization failure, we propose a framework to ensure feasibility throughout the process using a series of subproblems tailored for narrow passage problem. We begin by decomposing the environment into convex objects and initializing collision constraints with a subset of these objects. By continuously interpolating the collision constraints through the process of sequentially introducing remaining objects, our proposed framework generates subproblems that guide the optimization toward solving the narrow passage problem. Several examples are presented to demonstrate how the proposed framework addresses narrow passage path planning problems.

IROS Conference 2025 Conference Paper

SAC(λ): Efficient Reinforcement Learning for Sparse-Reward Autonomous Car Racing using Imperfect Demonstrations

Heeseong Lee
Sungpyo Sagong
Minhyeong Lee
Jeongmin Lee
Dongjun Lee

Recent advances in Reinforcement Learning (RL) have demonstrated promising results in autonomous car racing. However, two fundamental challenges remain: sparse rewards, which hinder efficient learning process, and the quality of demonstrations, which directly affects the effectiveness of RL from Demonstration (RLfD) approaches. To address these issues, we propose SAC(λ), a novel RLfD algorithm tailored for sparse-reward racing tasks with imperfect demonstrations. SAC(λ) introduces two key components: (1) a discriminator-augmented Q-function, which integrates prior knowledge from demonstrations into value estimation while maintaining off-policy learning benefits, and (2) a Positive-Unlabeled (PU) learning framework with adaptive prior adjustment, which enables the agent to progressively refine its understanding of positive behaviors, while mitigating the overfitting problem. Through extensive experiments in the Assetto Corsa simulator, we demonstrate that SAC(λ) significantly accelerates training, surpasses the provided demonstrations, and achieves superior lap times over existing RL and RLfD approaches. Code and videos are available at https://heesungsung.github.io/AC-RLRacer/.

ICRA Conference 2025 Conference Paper

Time-Correlated Model Predictive Path Integral: Smooth Action Generation for Sampling-Based Control

Minhyeong Lee
Dongjun Lee

In this paper, we introduce time-correlated model predictive path integral (TC-MPPI), a novel approach to mitigate action noise in sampling-based control methods. Unlike conventional smoothing techniques that rely on post-processing or additional state variables, TC-MPPI directly incorporates temporal correlation of actions into stochastic optimal control, effectively enforcing quadratic costs on action derivatives. This reformulation enables us to generate smooth action sequences without extra modifications, using a time-correlated and conditional Gaussian sampling distribution. We demonstrate the effectiveness of our approach through simulations on various robotic platforms, including a pendulum, cart-pole, 2D bicopter, 3D quadcopter, and autonomous vehicle. Simulation videos are available at https://youtu.be/nWfJ2MAV2JI.

IROS Conference 2024 Conference Paper

Collision Detection between Smooth Convex Bodies via Riemannian Optimization Framework

Seoki An
Somang Lee
Jeongmin Lee 0002
Sunkyung Park
Dongjun Lee

Collision detection is a fundamental problem across various fields such as robotics, physical simulation, and computer graphics. While numerous studies have provided efficient solutions, based on the well-known Gilbert, Johnson, and Keerthi (GJK) algorithm and Expanding Polytope Algorithm (EPA), existing methods utilizing GJK-EPA often struggle with smooth strictly convex shapes like ellipsoids. This paper proposes a novel approach to the collision detection problem converting it to a problem compatible with an unconstrained Riemannian optimization problem. Moreover, we presents a specific method of solving the problem based on twice differentiable support functions and the Riemannian trust region (RTR) method. The method exhibits fast and robust convergence rate, leveraging the well-established theory of Riemannian optimization. The evaluation studies comparing our method to GJK-EPA method are done with pre-defined primitive shapes. Additionally, a test result with several more complex shapes is demonstrated exhibiting the method’s effectiveness and applicability.

ICRA Conference 2024 Conference Paper

Efficient Clothoid Tree-Based Local Path Planning for Self-Driving Robots

Minhyeong Lee
Dongjun Lee

In this paper, we propose a real-time clothoid tree-based path planning for self-driving robots. Clothoids, curves that exhibit linear curvature profiles, play an important role in road design and path planning due to their appealing properties. Nevertheless, their real-time applications face considerable challenges, primarily stemming from the lack of a closed-form clothoid expression. To address these challenges, we introduce two innovative techniques: 1) an efficient and precise clothoid approximation using the Gauss-Legendre quadrature; and 2) a data-efficient decoder for interpolating clothoid splines that leverages the symmetry and similarity of clothoids. These techniques are demonstrated with numerical examples. The clothoid approximation ensures an accurate and smooth representation of the curve, and the clothoid spline decoder effectively accelerates the clothoid tree exploration by relaxing the problem constraints and reducing the problem size. Both techniques are integrated into our path planning algorithm and evaluated in various driving scenarios.

NeurIPS Conference 2024 Conference Paper

Introducing Spectral Attention for Long-Range Dependency in Time Series Forecasting

Bong G. Kang
Dongjun Lee
HyunGi Kim
Dohyun Chung
Sungroh Yoon

Sequence modeling faces challenges in capturing long-range dependencies across diverse tasks. Recent linear and transformer-based forecasters have shown superior performance in time series forecasting. However, they are constrained by their inherent inability to effectively address long-range dependencies in time series data, primarily due to using fixed-size inputs for prediction. Furthermore, they typically sacrifice essential temporal correlation among consecutive training samples by shuffling them into mini-batches. To overcome these limitations, we introduce a fast and effective Spectral Attention mechanism, which preserves temporal correlations among samples and facilitates the handling of long-range information while maintaining the base model structure. Spectral Attention preserves long-period trends through a low-pass filter and facilitates gradient to flow between samples. Spectral Attention can be seamlessly integrated into most sequence models, allowing models with fixed-sized look-back windows to capture long-range dependencies over thousands of steps. Through extensive experiments on 11 real-world time series datasets using 7 recent forecasting models, we consistently demonstrate the efficacy of our Spectral Attention mechanism, achieving state-of-the-art results.

PDF Details DOI

ICRA Conference 2024 Conference Paper

Sequential Trajectory Optimization for Externally-Actuated Modular Manipulators with Joint Locking

Jaeu Choe
Jeongseob Lee
Hyunsoo Yang
Hai-Nguyen Nguyen
Dongjun Lee

In this paper, we present a novel trajectory planning method for externally-actuated modular manipulators (EAMMs), consisting of multiple rotor-actuated links with joints that can be either locked or unlocked. This joint-locking feature allows effective balancing of the payload capacity and dexterity of the robot but significantly complicates the planning problem by introducing binary decision variables. To address this challenge, we leverage the problem’s intrinsic structure, i. e. , the payload at the end-effector being enhanced by merely locking its immediate connected links; this allows us to break down the complex planning problem into a series of manageable subproblems and solve them sequentially. Our approach significantly reduces the problem’s complexity: in a serial n-link EAMM with m joint-lock mechanisms, where there could potentially be 2 m distinct configurational dynamics, we require solving only n + 1 trajectory optimization problems for single rigid body dynamics sequentially, thereby rendering the problem tractable. We substantiate the efficacy of our method through various simulation and experimental studies, covering ground-free and ground-bound configurations as well as both motion-only and manipulation tasks.

IROS Conference 2024 Conference Paper

UWB-Based Localization System Considering Antenna Anisotropy and NLOS/Multipath Conditions

Taekyun Kim
Byoungkwon Yoon
Dongjun Lee

Ultra-wideband (UWB) communication technology has gained attention in robotics due to its ability to provide range measurements possibly with centimeter-level accuracy. Nevertheless, practical UWB range measurements are susceptible to disturbances from multiple sources, including the anisotropic characteristics of antennas, non-line-of-sight (NLOS) conditions, and multipath propagation. In this paper, we introduce a UWB range measurement model that addresses these sources of error. To accommodate the effects of antenna anisotropy, we adopt real spherical harmonics to represent directional bias in the UWB range measurement model. To handle delayed measurements induced by NLOS conditions and multipath propagation, an asymmetric heavy-tailed distribution is utilized to model the measurement noise. We calibrate this measurement model based on the maximum likelihood estimation method and propose a UWB-based localization system based on that. Our localization system provides: 1) anchor self-calibration, which identifies anchor placement by fusing visual-inertial-ranging measurements based on continuous-time state representation; and 2) filtering-based state estimation, which applies our measurement model into Kalman filtering framework via an iterative update algorithm. Experimental validation is conducted to demonstrate the effectiveness of the measurement model for our localization system. We open source our implementation of the proposed UWB-based localization system at https://github.com/INRoL/inrol_uwb_localization.

ICRA Conference 2023 Conference Paper

Differentiable Dynamics Simulation Using Invariant Contact Mapping and Damped Contact Force

Minji Lee
Jeongmin Lee 0002
Dongjun Lee

The gradient of typical differentiable simulation is uninformative for two reasons: 1) non-smoothness in contact dynamics not considered properly, and 2) excessive local minima generated from the smoothing procedure. To tackle this issue, we first propose differentiable contact dynamics with an invariant contact set and coordinate differentiation using a signed distance function (SDF). Also, to eliminate the undesirable jittering caused by the smoothing procedure, which induces extra local minima, and to achieve a smooth and informative gradient, we further endow our framework with a novel damped contact model. Various optimization problems are implemented to demonstrate the usefulness and efficacy of our differentiable framework.

ICRA Conference 2023 Conference Paper

Modular and Parallelizable Multibody Physics Simulation via Subsystem-Based ADMM

Jeongmin Lee 0002
Minji Lee
Dongjun Lee

In this paper, we present a new multibody physics simulation framework that utilizes the subsystem-based struc-ture and the Alternating Direction Method of Multiplier (ADMM). The major challenge in simulating complex high degree of freedom systems is a large number of coupled con-straints and large-sized matrices. To address this challenge, we first split the multibody into several subsystems and reformulate the dynamics equation into a subsystem perspective based on the structure of their interconnection. Then we utilize ADMM with our novel subsystem-based variable splitting scheme to solve the equation, which allows parallelizable and modular architecture. The resulting algorithm is fast, scalable, versatile, and converges well while maintaining solution consistency. Sev-eral illustrative examples are implemented with performance evaluation results showing advantages over other state-of-the-art algorithms.

IROS Conference 2023 Conference Paper

Symmetry-Based Modeling and Hybrid Orientation-Force Control of Wearable Cutaneous Haptic Device

Somang Lee
Hyunsu Kim
Dongjun Lee

We propose novel symmetry-based modeling and hybrid orientation-force control frameworks for cutaneous haptic device (CHD) to generate precise three degree-of-freedom (DoF) contact force on the fingertip robustly against user variability. The CHD hardware is designed in a form of an underactuated cable-driven parallel mechanism, with springs placed along the tendon to stabilize the pose. We analyze the kinematics of the CHD and propose a pose estimator by exploiting the symmetrical nature of the mechanism. We then devise a hybrid orientation-force controller to track the direction and magnitude of the desired contact force simultaneously in a feedback manner for control accuracy and robustness. We also adopt a tension regulator to mitigate friction effect during the actuation. Experimental validation and demonstration show the efficacy of the CHD with our proposed estimation and control framework.

IROS Conference 2023 Conference Paper

Wrench Estimation of Modular Manipulator with External Actuation and Joint Locking

Yonghyeok Kim
Hasun Lee
Jeongseob Lee
Dongjun Lee

This paper proposes an external wrench estimation method for modular manipulators, where each link module is driven with external actuation (e. g. , rotors, thrusters) and inter-module joints can be locked to increase end-effector stiffness or workforce of the manipulator. For such systems, the commonly-used momentum-based observer (MBO [1]) is not suitable due to the presence of unknown joint locking (JL) torque and also the degeneracy of Jacobian transpose relation with the system degree-of-freedom (DOF) becoming less than six with the joint locking. To overcome this, we propose two novel external wrench estimation algorithms: a distributed algorithm based on recursive Newton-Euler dynamics and a centralized algorithm based on D'Alembert's principle, both using an F/T (force/torque) sensor at the base. Experiments are conducted to demonstrate the effectiveness of the proposed algorithms.

NeurIPS Conference 2022 Conference Paper

A Multi-Task Benchmark for Korean Legal Language Understanding and Judgement Prediction

Wonseok Hwang
Dongjun Lee
Kyoungyeon Cho
Hanuhl Lee
Minjoon Seo

The recent advances of deep learning have dramatically changed how machine learning, especially in the domain of natural language processing, can be applied to legal domain. However, this shift to the data-driven approaches calls for larger and more diverse datasets, which are nevertheless still small in number, especially in non-English languages. Here we present the first large-scale benchmark of Korean legal AI datasets, LBOX OPEN, that consists of one legal corpus, two classification tasks, two legal judgement prediction (LJP) tasks, and one summarization task. The legal corpus consists of 147k Korean precedents (259M tokens), of which 63k are sentenced in last 4 years and 96k are from the first and the second level courts in which factual issues are reviewed. The two classification tasks are case names (11. 3k) and statutes (2. 8k) prediction from the factual description of individual cases. The LJP tasks consist of (1) 10. 5k criminal examples where the model is asked to predict fine amount, imprisonment with labor, and imprisonment without labor ranges for the given facts, and (2) 4. 7k civil examples where the inputs are facts and claim for relief and outputs are the degrees of claim acceptance. The summarization task consists of the Supreme Court precedents and the corresponding summaries (20k). We also release realistic variants of the datasets by extending the domain (1) to infrequent case categories in case name (31k examples) and statute (17. 7k) classification tasks, and (2) to long input sequences in the summarization task (51k). Finally, we release LCUBE, the first Korean legal language model trained on the legal corpus from this study. Given the uniqueness of the Law of South Korea and the diversity of the legal tasks covered in this work, we believe that LBOX OPEN contributes to the multilinguality of global legal research. LBOX OPEN and LCUBE will be publicly available.

ICRA Conference 2021 Conference Paper

A Parallelized Iterative Algorithm for Real-Time Simulation of Long Flexible Cable Manipulation

Jeongmin Lee 0002
Minji Lee
Jaemin Yoon
Dongjun Lee

We propose a novel real-time physically-accurate simulator for long flexible cable manipulation. We first discretize the cable into multiple rigid link segments, each with complementarity-based contact model and inter-segment compliant coupling; and partition the cable into a number of subsystems, each composed with a number of consecutive links. We then formulate the inter-subsystem consistency constraint as a certain analytical condition among the inter-subsystem coupling and the contact impulses; and solve each subsystem dynamics in parallel with the contact model together with this consistency condition in an iterative manner, achieving both the speed and the accuracy of the simulation. A novel post-regulation scheme is also proposed to further speed up the simulation. Experimental validation/demonstration are also performed to show the theory.

NeurIPS Conference 2021 Conference Paper

KLUE: Korean Language Understanding Evaluation

Sungjoon Park
Jihyung Moon
Sungdong Kim
Won Ik Cho
Ji Yoon Han
Jangwon Park
Chisung Song
Junseong Kim

We introduce Korean Language Understanding Evaluation (KLUE) benchmark. KLUE is a collection of eight Korean natural language understanding (NLU) tasks, including Topic Classification, Semantic Textual Similarity, Natural LanguageInference, Named Entity Recognition, Relation Extraction, Dependency Parsing, Machine Reading Comprehension, and Dialogue State Tracking. We create all of the datasets from scratch in a principled way. We design the tasks to have diverse formats and each task to be built upon various source corpora that respect copyrights. Also, we propose suitable evaluation metrics and organize annotation protocols in a way to ensure quality. To prevent ethical risks in KLUE, we proactively remove examples reflecting social biases, containing toxic content or personally identifiable information (PII). Along with the benchmark datasets, we release pre-trained language models (PLM) for Korean, KLUE-BERT and KLUE-RoBERTa, and find KLUE-Roberta-large outperforms other baselines including multilingual PLMs and existing open-source Korean PLMs. The fine-tuning recipes are publicly open for anyone to reproduce our baseline result. We believe our work will facilitate future research on cross-lingual as well as Korean language models and the creation of similar resources for other languages. KLUE is available at https: //klue-benchmark. com.

IROS Conference 2021 Conference Paper

Real-Time Physically-Accurate Simulation of Robotic Snap Connection Process

Minji Lee
Jeongmin Lee 0002
Jaemin Yoon
Dongjun Lee

We propose a novel real-time physically-accurate simulation framework for the snap connection process. For this, we first notice the peculiarities of the process, namely, small/smooth deformation, stiff connector and segmented contact. We then design our simulation to fully exploit these peculiarities by adopting the following strategies: 1) the technique of passive midpoint integration (PMI [1]), which allows for stable simulation of arbitrarily light/stiff system by enforcing discrete-time passivity; 2) linear finite element method (FEM [2]) modeling, which is adequate to deal with the small snap connector deformation while providing much faster speed as compared to nonlinear FEM; 3) segmentation of the snap connector FEM model and solving of each segment individually with their coupling analytically eliminated, thereby, further speeding up the simulation; 4) balanced model reduction (BMR [3]) to further reduce the dimension of each segment purely analytically without any prior experiment or simulation; and 5) parallelized data-driven collision detection, which turns out to further significantly speed up our simulation. Experimentally-verified simulations are also performed to show the efficacy of our proposed simulation framework.

ICRA Conference 2020 Conference Paper

Distributed Rotor-Based Vibration Suppression for Flexible Object Transport and Manipulation

Hyunsoo Yang
Min-Seong Kim
Dongjun Lee

The RVM (Robot-based Vibration Suppression Modules) is proposed for the manipulation and transport of a large flexible object. Since the RVM is easily attachable/detachable to the object, this RVM allows distributing over the manipulated object so that it is scalable to the object size. The composition of the system is partly motivated by the MAGMaS (Multiple Aerial-Ground Manipulator System) [1]- [3], however, since the quadrotor usage is mechanically too complicated and its design is not optimized for manipulation, thus we overcome these limitations using distributed RVMs and newly developed theory. For this, we first provide a constrained optimization problem of RVM design with the minimum number of rotors, so that the feasible thrust force is maximized while it minimizes undesirable wrench and its own weight. Then, we derive the full dynamics and elucidate a controllability condition with multiple distributed RVMs and show that even if multiple, their structures turn out similar to [2] composed with a single quadrotor. We also elucidate the optimal placement of the RVM via the usage of controllability gramian which is not even alluded in [2] and established for the first time here. Experiments are performed to demonstrate the effectiveness of the proposed theory.

IROS Conference 2020 Conference Paper

Expert-Emulating Excavation Trajectory Planning for Autonomous Robotic Industrial Excavator

Bukun Son
Changu Kim
Changmuk Kim
Dongjun Lee

We propose a novel excavation (i. e. , digging) trajectory planning framework for industrial autonomous robotic excavators, which emulates the strategies of human expert operators to optimize the excavation of (complex/unmodellable) soils while also upholding robustness and safety in practice. First, we encode the trajectory with dynamic movement primitives (DMP), which is known to robustly preserve qualitative shape of the trajectory and attraction to (variable) end-points (i. e. , start-points of swing/dumping), while also being data-efficient due to its structure, thus, suitable for our purpose, where expert data collection is expensive. We further shape this DMPbased trajectory to be expert-emulating, by learning the shaping force of the DMP-dynamics from the real expert excavation data via a neural network (i. e. , MLP (multi-layer perceptron)). To cope with (possibly dangerous) underground uncertainties (e. g. , pipes, rocks), we also real-time modulate the expert-emulating (nominal) trajectory to prevent excessive build-up of excavation force by using the feedback of its online estimation. The proposed framework is then validated/demonstrated by using an industrial-scale autonomous robotic excavator, with the associated data also presented here.

IROS Conference 2020 Conference Paper

Sim-to-Real Transfer of Bolting Tasks with Tight Tolerance

Dongwon Son
Hyunsoo Yang
Dongjun Lee

In this paper, we propose a novel sim-to-real framework to solve bolting tasks with tight tolerance and complex contact geometry which are hard to be modeled. The sim-to-real has desirable features in terms of cost and safety, however, that of the assembly task is rare due to the lack of simulator, which can robustly render multi-contact assembly. We implement the sim-to-real transfer of nut tightening policy which is adaptive to uncertain bolt positions. This can be realized through developing a novel contact model, which is fast and robust to complex assembly geometry, and novel hierarchical controller with reinforcement learning (RL), which can perform the tasks with a narrow and complicated path. The fast and robust contact model is achieved by utilizing configuration space abstraction and passive midpoint integrator (PMI), which render the simulator robust even in a high stiffness contact condition. And we use sampling-based motion planning to construct a path library and design linear quadratic tracking controller as a low-level controller to be compliant and avoid local optima. Additionally, we use the RL agent as a high-level controller to make it possible to adapt to the bolt position uncertainty, thereby realizing sim-to-real. Experiments are performed to verify our proposed sim-to-real framework.

ICRA Conference 2019 Conference Paper

Data-Driven Contact Clustering for Robot Simulation

Myungsin Kim
Jaemin Yoon
Dongwon Son
Dongjun Lee

We propose a novel data-driven learning-based contact clustering (i. e. , of contact points and contact normals) framework for rigid-body robot simulation, with its accuracy established/verified by real experimental data. We first construct an experimental robotic setup with force/torque (F/T) sensors to collect real contact motion/force data. We then design a multilayer perceptron (MLP) network for the contact clustering based on the full motion and force/torque information of the contacts. We also adopt the constraint-based optimization contact solver to facilitate the learning of our MLP network during the training. Our proposed data-driven/learning-based contact clustering framework is then verified against the experimental setup, compared with other techniques/simulators and shown to significantly (or meaningfully) enhance the accuracy of contact simulation as compared to them.

ICRA Conference 2019 Conference Paper

Model-Free Optimal Estimation and Sensor Placement Framework for Elastic Kinematic Chain

Joonmo Ahn
Jaemin Yoon
Jeongseob Lee
Dongjun Lee

We propose a novel model-free optimal estimation and sensor placement framework for a high-DOF (degree-of-freedom) EKC (elastic kinematic chain) with only a limited number of IMU (inertial measurement unit) sensors based on POD (proper orthogonal decomposition) and MAP (maximum a posteriori) estimation. First, we (off-line) excite the system richly enough, collect the data and perform the POD to extract dominant and non-dominant modes. We then decide the minimum number of IMUs according to the dominant modes, and construct the prior distribution of the output (i. e. , top-end position of EKC) based on the singular value of each POD mode. We also formulate the MAP estimation given the prior distribution and different placements of the IMUs and choose the optimal IMU placement to maximize the posterior probability. This optimal placement is then used for real-time output estimation of the EKC. Experiments are also performed to verify the theory.

IROS Conference 2019 Conference Paper

Passive Model Reduction and Switching for Fast Soft Object Simulation with Intermittent Contacts

Jaemin Yoon
Ilkwon Hong
Dongjun Lee

We propose a novel fast simulation framework for soft objects/robots with intermittent contacts, whose contact areas/locations can be varying. We first perform a balanced model reduction of the full-order FEM (finite element method) model for each contact mode with the contact forcing as the input and the shape of the object/robot as the output. We then devise the strategy of passive model reduction and passive model switching of these reduced-order models (each with its contact mode) utilizing the techniques of our recently-proposed passive mid-point integration (for the passivity of each reduced-order model) and simultaneous diagonalization (for the passivity of model reduction and model switching). The efficacy of the theory is then demonstrated with simulation and experimental results.

ICRA Conference 2019 Conference Paper

Pose and Posture Estimation of Aerial Skeleton Systems for Outdoor Flying

Sangyul Park
Yonghan Lee 0001
Jinuk Heo
Dongjun Lee

We present a novel pose and posture estimation framework of aerial skeleton system for outdoor flying. To exploit redundant/independent sensing while rendering the system “modular”, we attach an IMU (inertial measurement unit) sensor and a GNSS (global navigation satellite system) module on each link and perform SE(3)-motion EKF (extended Kalman filtering). We then apply the kinematic constraints of the aerial skeleton system to these EKF estimates of all the links through SCKF (smoothly constrained Kalman filtering), thereby, enforcing the kinematic coherency of the skeleton system and, consequently, significantly enhancing the estimation accuracy and the control performance/stability of the aerial skeleton system. A semi-distributed version of the obtained estimation framework is also presented to address the issue of scalability. The theory is then verified/demonstrated with real outdoor flying experiments and simulation studies of a three-link aerial skeleton system.

ICRA Conference 2018 Conference Paper

LASDRA: Large-Size Aerial Skeleton System with Distributed Rotor Actuation

Hyunsoo Yang
Sangyul Park
Jeongseob Lee
Joonmo Ahn
Dongwon Son
Dongjun Lee

Electrical motor and hydraulic actuation widely-used in robotics are “internal actuation” with their actuators sitting at the joint between two links. This internal actuation is fundamentally limiting to construct a large-size dexterously-articulated robot, since any external force (and its own link weight) is to be accumulated to the base multiplied by the moment arm length, requiring extremely strong/sturdy base actuator/structure as the system size increases. In this paper, we propose a novel robotic system, LASDRA (large-size aerial skeleton with distributed rotor actuation), which, by utilizing distributed rotors as “external actuation”, can overcome this limitation of internal actuation and enables us to realize large-size dexterously-articulated robots. We present its design and modeling, joint locking strategy to increase its loading capability, and also a novel decentralized control scheme to allow for compliant operation with scalability against the number of links. Trajectory tracking and valve turning experiments are also performed to validate the theory.

IROS Conference 2018 Conference Paper

Modeling and Control of Multiple Aerial-Ground Manipulator System (MAGMaS) with Load Flexibility

Hyunsoo Yang
Nicolas Staub
Antonio Franchi
Dongjun Lee

The MAGMaS (Multiple Aerial-Ground Manipulator System) was proposed in [1] as a heterogeneous system composed of multiple ground (mobile) manipulators and aerial robots to collaboratively manipulate a long/large-sized object and demonstrated therein for rigid load manipulation. Here, we extend this result of [1] to the case of load manipulation with flexibility, which is crucial for long/slender object manipulation, yet, not considered in [1]. We first provide a rigorous modeling of the load flexibility and its effects on the MAGMaS dynamics. We then propose a novel collaborative control framework for flexible load-tip pose tracking, where the ground manipulator provides slower nominal pose tracking with overall load weight holding, whereas the aerial robot allows for faster vibration suppression with some load weight sharing. We also discuss the issue of controllability stemming from that the aerial robot provides less number of actuation than the modes of the load flexibility; and elucidate some peculiar conditions for this vibration suppression controllability. Simulations are also performed to demonstrate the effectiveness of the proposed theory.

IROS Conference 2018 Conference Paper

Tire Force Estimation of Dynamic Wheeled Mobile Robots using Tire-Model Based Constrained Kalman Filtering

Sang-Yun Jeon
Rakjoon Chung
Dongjun Lee

We propose a novel real-time algorithm to estimate the full three-dimensional individual tire forces (i. e. , vertical, longitudinal as well as lateral) of a car-like rearwheel-driven four wheel wheeled mobile robots equipped with onboard navigation sensors and wheel encoders. The key enabling idea for this is to utilize the tire model (i. e. , the magic formula) in a feedback manner on the framework of the constrained Kalman filtering to render the tire force estimation: 1) more accurate as compared to the typical tire force estimation techniques neglecting the tire-road interaction; and 2) more robust as compared to the results adopting the tire model, yet, only in an open-loop manner. Our proposed algorithm, while performing this full tire force onboard/real-time estimation, also provides the estimation of: 1) tire-road friction coefficient; and 2) torque inputs of the rear left and right wheels, which are connected via differential gear. Simulations with CarSim and outdoor experiments are performed to validate the proposed estimation algorithm.

ICRA Conference 2017 Conference Paper

On the passivity of mechanical integrators in haptic rendering

Myungsin Kim
Juhyeok Kim
Yongjun Lee
Dongjun Lee

We propose a novel haptic rendering scheme based on passive midpoint integrator (PMI), which is an extension of our prior non-iterative passive mechanical integrator (NPMI) to maximal coordinates for simulation of complex articulated rigid bodies and to multi-point intermittent contact with the linear complementarity problem (LCP) formulation incorporated. The proposed PMI-based haptic rendering then can stably simulate mechanical systems with wide-range of parameters/updaterates (in contrast to semi-implicit Euler integrator), while maintaining losslessness of simple harmonic oscillation (in contrast to implicit Euler integrator), as compared in this paper. Application of this PMI-based haptic rendering is also presented for multi-user virtual co-manipulation of shared rigid object in SE(3) using hand/fingers with multi-point contact over the Internet.

IROS Conference 2017 Conference Paper

Passivity-based control of manipulator-stage systems on vertical flexible beam

Changsu Ha
Hackchan Kim
Dongjun Lee

We develop a novel passivity-based control framework of a manipulator-stage system sitting on a vertical flexible beam. We first model the vertical flexible beam by using Euler-Bernoulli theory and modal approximation while utilizing certain boundary conditions to take into account the effect of the gravity and interaction wrench between the manipulator-stage system and the beam. To separately achieve two control objectives, i. e. the joint tracking and the vibration suppression, we utilize the passive decomposition to split the dynamics into: 1) the stage and beam dynamics; and 2) its orthogonal complement, which converges to the robotic manipulator dynamics as the vibration is subdued. We further show that the linearized stage-beam dynamics is controllable and design an LQR control to stabilize the beam vibration only by using the stage motion. We also design control for the orthogonal complement dynamics in such a way that it can attain the manipulator joint tracking asymptotically as the vibration is suppressed. Experiments are also performed to verify the manipulator joint tracking while the vibration is suppressed internally via the stage motion.

IROS Conference 2016 Conference Paper

Design, modeling and control of omni-directional aerial robot

Sangyul Park
Jongbeom Her
Juhyeok Kim
Dongjun Lee

We propose a novel multi-rotor flying platform, ODAR (omni-directional aerial robot), which is fully-actuated (i. e. , can assume arbitrary motion in SE(3) or generate arbitrary control wrench in se(3)) by six opportunistically distributed rotors, each driven by reversible ESC (electronic speed controller) for bi-directional thrust generation. Due to this omni-directional wrench generation, the ODAR system can realize such powerful behaviors impossible with conventional multi-rotor flying platforms as: 360° photo/video shooting while holding its position for VR scene generation; or resisting sideway gust while keeping its attitude and exerting downward pushing force larger than its own weight for aerial manipulation applications. This paper presents optimal mechanical design, modeling and control, hardware and software implementation, and experimental verification of the ability of the ODAR system to attain those behaviors impossible with the conventional drones as stated above.

IROS Conference 2015 Conference Paper

Aerial tool operation system using quadrotors as Rotating Thrust Generators

Hai-Nguyen Nguyen
Sangyul Park
Dongjun Lee

We propose a new aerial tool operation system consisting of multiple quadrotors connected to a tool by spherical joints to perform tool operation tasks. We model the system and show that the attitude dynamics of each quadrotor is decoupled from the tool dynamics, so that we can consider the quadrotors as thrusters and control the tool by adjusting the orientation and magnitude of these thrusters. We also show that the 6-DOF tool dynamics could be under-actuated or fullyactuated, depending on the number of quadrotors attached to the tool and the geometric configuration of their attachments. We then design control laws for the tool-tip position/orientation tracking of the (under-actuated) tool system with two quadrotors and the (fully-actuated) tool system with three quadrotors. We use Lyapunov approach to find the desired thrust command for each quadrotor while also taking into account the spherical joint limits in a form of constrained optimization. Simulation and implementation results are performed to support the theory.

ICRA Conference 2015 Conference Paper

Hierarchical cooperative control framework of multiple quadrotor-manipulator systems

Hyunsoo Yang
Dongjun Lee

We propose an hierarchical control framework for multiple cooperative quadrotor-manipulator systems, which allows us to endow the common grasped object with a user-specified desired behavior (e. g. , trajectory tracking, compliant interaction, etc.). To achieve this, our control framework consists of the following hierarchical layers: 1) object desired behavior design; 2) optimal cooperative force distribution; and 3) individual quadrotor-manipulator control based on object stiffness model, which can also take into account different dynamics characteristics of the (slower/coarse) quadrotor-platform and the (faster/fine) manipulator. Simulations of object transport and compliant interaction with three quadrotor-manipulator systems are performed to illustrate the theory.

ICRA Conference 2015 Conference Paper

Whole-body multi-modal semi-autonomous teleoperation of mobile manipulator systems

Changsu Ha
Sangyul Park
Jongbeom Her
Inyoung Jang
Yongseok Lee
Gun Rae Cho
Hyoung Il Son
Dongjun Lee

We propose a novel whole-body multi-modal semiautonomous teleoperation framework for mobile manipulator systems, which consists of: 1) Motion capture and whole-body motion mapping to allow the operator to intuitively teleoperate the mobile manipulator without being constrained by the master interface while also fully exploiting whole-body dexterity; 2) Slave robot autonomous control to allow the mobile manipulator to optimally track the operator's whole-body command, while taking into account the user-slave kinematic dissimilarity (slave robot's joint limit, joint velocity limit, and singularity); and 3) Visuo-haptic-vestibular feedback with HMD (Head Mounted Display) for 3D visual information, wearable cutaneous haptic device for manipulation force feedback, and actuated chair for vestibular feedback to reduce HMD-induced motion sickness. Performance of the proposed framework is validated with simulation of a ROV (remotely operated vehicle) manipulator system and some preliminary user studies.

ICRA Conference 2014 Conference Paper

Autonomous dynamic driving control of wheeled mobile robots

Jaemin Yoon
Jonghyun Oh
Joo-Hyun Park
Su-Hwan Kim
Dongjun Lee

We propose a novel control framework to enable nonholonomic wheeled mobile robots (WMRs) to autonomously drive in an environment with the speed fast enough so that the dynamics effect (e. g. , Coriolis effect) is not negligible, yet, still less than a certain threshold to prevent slippage at the wheels. For this, instead of the Newtonian vehicle modeling, we adopt Lagrange-D'Alembert formulation, which then allows us to explicitly relate the system's state/control with the constraint force, so that we can predict/detect possibility of a given motion's violating the no-slip condition. We present a scheme to generate a no-slip/collision-free timed-trajectory for the WMRs using this Lagrange-D'Alembert formulation. We also propose a backstepping-based control law, which enables the WMR to track the generated trajectory while respecting its nonholonomic constraints. Experiment, using a modified commercial radio-controlled car, is performed to verify the theory.

ICRA Conference 2014 Conference Paper

Design and control of a low cost 6 DOF master controller

Tian Qiu
William R. Hamel
Dongjun Lee

This paper provides the design of a low cost master controller for teleoperation systems. The system consists of a center handle and four off the shelf haptic devices. The control law of the four devices coordinates them to provide six degree-of-freedom(DOF) force feedback. The relative position and the dimension of the handle are designed to maximize the range of motion. The system actuation redundancy problem is formulated into a standard convex optimization problem and resolved. The squared sum of the force provided by all the haptic devices are defined as the optimization function and the desired force and torque feedback is used as the linear constraints. Experiments in teleoperation and haptics projects were carried out to verify the theoretical results about system.

ICRA Conference 2014 Conference Paper

Dynamics and control of quadrotor with robotic manipulator

Hyunsoo Yang
Dongjun Lee

We show that the Lagrange dynamics of quadrotor-manipulator systems can be completely decoupled into: 1) the center-of-mass dynamics in E(3), which, similar to the standard quadrotor dynamics, is point-mass dynamics with under-actuation and gravity effect; and 2) the “internal rotational” dynamics of the quadrotor's rotation and manipulator configuration, which assumes the form of standard Lagrange dynamics of robotic manipulator with full-actuation and no gravity effect. Relying on this structure, we propose a novel backstepping-like end-effector tracking control law, which can allow us to assign different roles for the center-of-mass control and for the internal rotational dynamics control according to task objectives. Simulations using a planar quadrotor with a 2-DOF arm are also performed to show the theory.

ICRA Conference 2014 Conference Paper

Preliminary experiments of kinesthetic exploration in a 6 DOF teleoperation system

Tian Qiu
William R. Hamel
Dongjun Lee

A teleoperation system with kinesthetic feedback can provide operators with information so that they can make proper judgments as if they were in direct contact with the slave robot work environment. This paper presents the idea of testing the same haptics exploration tasks in both direct and teleoperation scenarios. This provides the advantage that lessons learned in experimental psychology can be used for future teleoperation systems' control and hardware design. One preliminary experiment is presented in this paper to demonstrate the concept. This experiment comprised of three sets of weight perception tasks. One set of direct hand exploration experiments were carried out first. Two sets of teleoperator based experiments, one with a basic two channel controller and the other with three channel controllers, were used to replicate the same set of experiments. Results from the experiments were compared and analyzed.

ICRA Conference 2013 Conference Paper

Cooperative grasping control of multiple mobile manipulators with obstacle avoidance

Hyunsoo Yang
Dongjun Lee

We present a novel cooperative grasping control framework for multiple kinematic nonholonomic mobile manipulators, which enables them to drive the grasped object with velocity commands, while rigidly maintaining the grasping shape with no dedicated grasp-enforcing fixtures and also avoiding obstacles either via their whole formation maneuver or internal formation reconfiguration. For this, nonholonomic passive decomposition [1], [2] is utilized to split the robots' motion into the three aspects (i. e. , grasping shape; grasped object maneuver; internal motions) so that we can control these aspects simultaneously and separately. Peculiar dynamics of the internal motions is exploited to achieve obstacle avoidance via the formation reconfiguration. Simulations are performed to support the theory.

IROS Conference 2013 Conference Paper

Hybrid force/motion control and internal dynamics of quadrotors for tool operation

Hai-Nguyen Nguyen
Dongjun Lee

This paper presents a hybrid force/motion control framework for quadrotors with a rigid/light tool attached on it. By transforming the quadrotor dynamics into that of the tool-tip position y and applying the passive decomposition to decompose its dynamics into tangential and normal components w. r. t. a contact surface, we design hybrid position/force control. We also elucidate the internal dynamics (i. e. , the dynamics hidden from the tool-tip position and yaw angle output and not directly affected by the control action due to the quadro-tor's underactuation), reveal a (seemingly counter-intuitive) necessary condition for internal stability (i. e. , tool above the quadrotor, not beneath it), and propose a stabilizing control action to ensure the angular rates still be bounded while preventing the finite-time escape. Simulations are performed to support the theory.

ICRA Conference 2013 Conference Paper

Vision-based teleoperation of unmanned aerial and ground vehicles

Changsu Ha
Dongjun Lee

We present a novel vision-based teleoperation control framework for a team of an unmanned aerial vehicle (UAV) and an unmanned ground vehicle (UGV). Our control law allows a remote human user to teleoperate the team with some useful haptic feedback, while also ensuring the UAV-UGV coordination via the camera installed on the UAV (and seeing the UGV) and the velocity limitation of the UGV. For this, we first elucidate a geometric condition for the UAV and UGV velocities to ensure the UAV-UGV coordination by driving the image feature of the UGV to converge to a desired one on the image surface, while also guaranteeing that the UGV's velocity, which is often much slower than that of the UAV, is under a certain specified bound. The UAV is then tele-controlled to track the teleoperation command as close as possible, yet, only to the extent permissible by this geometric condition. Simulation is performed to illustrate the theory.

IROS Conference 2012 Conference Paper

Passive haptic rendering and control of Lagrangian virtual proxy

Dongjun Lee
Myungsin Kim
Tian Qiu

We consider the problem of passive haptic rendering and interfacing of multiple degree-of-freedom (DOF) virtual proxy (VP), which has nonlinear Lagrangian dynamics and interacts with deformable virtual objects. For this, we solve the followings: 1) how to extend our recently-proposed non-iterative passive mechanical integrator (NPMI [1]) to simulate this nonlinear Lagrangian VP haptically-fast and discrete-time passively; 2) how to utilize virtual coupling technique to interface this NPMI-simulated VP and haptic devices while guaranteeing (sampled-data) passivity; and 3) how to passively render the interaction between VP and linear visco-elastic deformable virtual objects, while enhancing passivity at contact-on/off switchings. Some experimental results are also presented to support theory.

ICRA Conference 2011 Conference Paper

Feedback r-passivity of Lagrangian systems for mobile robot teleoperation

Dongjun Lee
Daye Xu

One of the key challenges in mobile robot teleoperation is master-slave kinematic dissimilarity: master device (e. g. , joystick) has a bounded workspace, while slave mobile robot can/should cover an unbounded workspace. A direct master-position/slave-velocity coupling has been frequently used for this, which, yet, in general, violates passivity of the closed-loop system, since the master system with position output cannot be passive. In this paper, we propose a new notion of feedback r-passivity of general multi-dimensional Lagrangian systems, which allows us to achieve such master-position/slave-velocity coupling while enforcing passivity. Its applications to tele-driving of dynamic and kinematic WMRs with constant communication delay are also presented.

ICRA Conference 2011 Conference Paper

Haptic teleoperation of multiple unmanned aerial vehicles over the internet

Dongjun Lee
Antonio Franchi
Paolo Robuffo Giordano
Hyoung Il Son
Heinrich H. Bülthoff

We propose a novel haptic teleoperation control framework for multiple unmanned aerial vehicles (UAVs) over the Internet, consisting of the three control layers: 1) UAV control layer, where each UAV is abstracted by, and is controlled to follow the trajectory of, its own kinematic virtual point (VP); 2) VP control layer, which modulates each VP's motion according to the teleoperation commands and local artificial potentials (for inter-VP/VP-obstacle collision avoidance and inter-VP connectivity preservation); and 3) teleoperation layer, through which a remote human user can command all (or some) of the VPs' velocity while haptically perceiving the state of all (or some) of the UAVs over the Internet. Master-passivity/slave-stability and some asymptotic performance measures are proved. Semi-experiment results are presented to validate the theory.

IROS Conference 2011 Conference Paper

Hybrid virtual-proxy based control framework for passive bilateral teleoperation over the internet

Ke Huang
Dongjun Lee

Let us consider the control problem of bilateral teleoperation over the Internet. One of its important, yet, often overlooked, aspects is its hybrid nature, that is, the (nonlinear) master and slave robots are continuous-time systems with sampled-data controls, while the Internet between them is a discrete-time packet-switching communication channel with the communication unreliability (e. g. , varying-delay, packet-loss, data duplication/swapping, etc.). To tackle this problem, a hybrid proportional-derivative (PD) type control framework has recently been proposed in [1], which, by utilizing the hybrid nature, can enforce closed-loop passivity and asymptotic position/force coordination, by compensating for the Internet's communication unreliability solely via device viscous damping. This scheme of [1], yet, requires large device damping when the communication delay/loss is large, thus, would not be so suitable if the device's un-tunable damping is low, but, the communication unreliability is severe. In this paper, extending the results of [1], [2], [3], we propose a novel virtual-proxy based hybrid teleoperation control framework, which can passify the Internet's communication unreliability through the virtual proxy's tunable discrete damping, thus, removing the requirement of large device damping of [1] and rendering a more flexible framework than [1], while providing explicit position feedback via the PD-action. Simulation is also performed to verify the theory.

IROS Conference 2011 Conference Paper

Measuring an operator's maneuverability performance in the haptic teleoperation of multiple robots

Hyoung Il Son
Lewis L. Chuang
Antonio Franchi
Junsuk Kim
Dongjun Lee
Seong-Whan Lee
Heinrich H. Bülthoff
Paolo Robuffo Giordano

In this paper, we investigate the maneuverability performance of human teleoperators on multi-robots. First, we propose that maneuverability performance can be assessed by a frequency response function that jointly considers the input force of the operator and the position errors of the multi-robot system that is being maneuvered. Doing so allows us to evaluate maneuverability performance in terms of the human teleoperator's interaction with the controlled system. This allowed us to effectively determine the suitability of different haptic cue algorithms in improving teleoperation maneuverability. Performance metrics based on the human teleoperator's frequency response function indicate that maneuverability performance is best supported by a haptic feedback algorithm which is based on an obstacle avoidance force.

ICRA Conference 2010 Conference Paper

Peer-to-peer control architecture for multiuser haptic collaboration over undirected delayed packet-switching network

Dongjun Lee
Ke Huang

We propose a novel peer-to-peer distributed control architecture for shared haptic collaboration among remotely-located users over undirected packet-switching network (e. g. Internet) with inter-user communication delay. The proposed architecture is distributed, in that each user simulates and interacts with its own local copy of the shared virtual environment. Spring connection among the local copies and local damping are used, which, together, under a certain condition, achieve configuration synchronization among the local copies while enforcing discrete-time passivity of the total peer-to-peer architecture, thereby, rendering the architecture portable/scalable for any (passive) users/devices and ensuring its interaction stability be user/device-invariant. The issue of optimizing communication network is also addressed with some relevant experimental results.

IROS Conference 2009 Conference Paper

Extension of colgate's passivity condition for variable-rate haptics

Dongjun Lee

Colgate's passivity conditions for the virtual coupling and the virtual wall have been one of the most important results in haptics. However, due to its development being rooted in the frequency-domain, its theoretical justification has been limited to only uniform-rate haptics. In this paper, relying on analysis/derivation in the time-domain, we extend this Colgate's passivity condition to variable-rate haptics. We also present a similar passivity condition for the haptic rendering based on the recently proposed passive set-position modulation (PSPM).

IROS Conference 2009 Conference Paper

Implementation and experiments of passive set-position modulation for internet teleoperation and slow/varying-rate haptics

Ke Huang
Dongjun Lee

In previous papers we proposed passive set- position modulation (PSPM) framework which enables us to connect continuous-time robot's position to a sequence of discrete set-position signals via spring coupling with damping injection while enforcing passivity. In this paper we present experimental results to show its stability and performance. We will also discuss some practical implementation details.

ICRA Conference 2007 Conference Paper

Time-Scaling Trajectories of Passive-Dynamic Bipedal Robots

Jonathan K. Holm
Dongjun Lee
Mark W. Spong

This paper presents a control law that time-scales reference trajectories of dynamical systems, yielding arbitrary velocities in arbitrary time. It is shown that, for unforced (passive) reference trajectories, constant time-scaling results in a potential energy-shaping control. Application to walking trajectories of bipedal robots is shown, extending the use of the control beyond purely continuous dynamical systems to a class of hybrid dynamical systems with discontinuities that are linear in velocity. Two biped models are used to demonstrate the control law: for the compass-gait biped, we illustrate time-scaling of a passive reference trajectory; for the biped with a torso, we show time-scaling of a semi-passive reference trajectory.

ICRA Conference 2006 Conference Paper

Bilateral Teleoperation of a Wheeled Mobile Robot over Delayed Communication Network

Dongjun Lee
Oscar Martinez-Palafox
Mark W. Spong

We consider bilateral teleoperation of a wheeled mobile robot over communication channels with constant delays. Our main objective is to enable humans to control the mobile robot much as they drive a car: i. e. by operating a master haptic joystick, they can control the linear velocity and heading angle of the mobile robot, much like they do so with the gas pedal and steering wheel. Passivity of the closed-loop system is also enforced so that, even with communication delays, humans can stably and safely teleoperate the wheeled mobile robot with force-reflection. A semi-experiment (i. e. real master/simulated slave) is performed to validate the proposed framework

IROS Conference 2006 Conference Paper

Bilateral Teleoperation of Mobile Robot over Delayed Communication Network: Implementation

Oscar Martinez-Palafox
Dongjun Lee
Mark W. Spong
Ivan Lopez
Chaouki T. Abdallah

In a previous paper we proposed a bilateral teleoperation framework of a wheeled mobile robot over communication channel with constant time delay. In this paper we present experimental results. Our goal is to illustrate and validate the properties of the proposed scheme as well as to present practical implementation issues and the adopted solutions. In particular, the bilaterally teleoperated system is passive and the system is stable in the presence of time delay. Internet has been used as the communication channel and a buffer has been implemented to maintain a constant time delay and to handle packet order

ICRA Conference 2006 Conference Paper

Passive Bilateral Teleoperation with Constant Time Delays

Dongjun Lee
Mark W. Spong

We propose a novel control framework for bilateral teleoperation of a pair of multi-degree-of-freedom (DOF) nonlinear robotic systems under constant communication delays. The proposed framework utilizes the simple proportional-derivative (PD) control, i. e. the master and slave robots are directly connected via spring and damper over the delayed communication channels. Using the controller passivity concept, the Lyapunov-Krasovskii technique, and Parseval's identity, we can passify the combination of the delayed communication and control blocks altogether robustly, as long as the delays are finite constants and an upper-bound for the round-trip delay is known. Having explicit position feedback through the delayed P-action, the proposed framework enforces master-slave position coordination which is often compromised in the velocity-based schemes (e. g. conventional scattering-based teleoperation). The proposed control framework provides humans with extended physiological proprioception so that s/he can affect and sense the remote slave environments mainly relying on her/his musculoskeletal systems. Experiments are performed to validate the proposed control framework

ICRA Conference 2005 Conference Paper

Bilateral Teleoperation of Multiple Cooperative Robots over Delayed Communication Networks: Application

Dongjun Lee
Oscar Martinez-Palafox
Mark W. Spong

In a companion paper [1], we propose a control framework for the bilateral teleoperation between a single master robot and multiple cooperative slave robots over delayed communication network. In this paper, we perform simulation and semi-experiment (i. e. real master and simulated slaves) to illustrate and validate properties of the proposed control scheme. In particular, the three key properties of the proposed control framework are highlighted in this paper: 1) cooperative fixtureless grasping and manipulation of inertial and deformable objects by multiple slave robots; 2) passive teleoperation of the overall behavior of the multiple slave robots (and the grasped object) over the delayed communication with force reflection; and 3) grasping safety (i. e. secure grasping) and interaction stability regardless of the communication delay and human command.

ICRA Conference 2005 Conference Paper

Bilateral Teleoperation of Multiple Cooperative Robots over Delayed Communication Networks: Theory

Dongjun Lee
Mark W. Spong

We propose a control framework for the bilateral teleoperation between a single master robot and multiple cooperative slave robots with communication-delay in the master-slave communication channel. Using passive decomposition, we first decompose the dynamics of multiple slaves into two decoupled systems while preserving energetic passivity: the shape system describing cooperative grasping aspect, and the locked system representing overall behavior of the multiple slaves. Then, by locally controlling the decoupled shape system with the cancellation of disturbances on it, secure and tight (and possibly fixtureless) cooperative grasping can be achieved regardless of the communication-delay and human command. We also construct a bilateral teleoperation loop between the master and the locked system s. t. , by operating the master, a human operator can control the overall behavior of the multiple slaves and the grasped object while perceiving environmental forces acting on them. Scattering-based communication is used to passify the master-slave communication-delay. By exploiting the passivity property of the decomposition and scattering-based communication, energetic passivity of the closed-loop system can be ensured, thus, interaction stability and safety are improved significantly. In a companion paper [1], simulation and semi-experiment (i. e. real master and simulated slaves) are performed to illustrate properties of this proposed framework.

ICRA Conference 2002 Conference Paper

Passive Coordination Control of Nonlinear Bilateral Teleoperated Manipulators

Dongjun Lee
Perry Y. Li

In this paper, a teleoperation controller is proposed for a pair of nonlinear master and slave manipulators. The control law enables the nonlinear robots to be perfectly coordinated despite arbitrary environment and human forcings, while ensuring that the closed loop teleoperator system is passive with respect to a supply rate that includes a user specified bilateral power scaling factor. After perfect coordination has been attained, the control law constrains n degrees of freedom (DOF). The dynamics of the remaining n degrees of freedom resembles those of a usual rigid robot. Thus, the teleoperator becomes a common rigid mechanical tool with which both the human operator and the physical environment interact.

ICRA Conference 2002 Conference Paper

Passive Tool Dynamics Rendering for Nonlinear Bilateral Teleoperated Manipulators

Dongjun Lee
Perry Y. Li

In a previous paper (see ibid. , p. 3278-83), a control law renders a 2n-degree of freedom (DOF) nonlinear teleoperator (consisting of n-DOF master and slave robots) as a n-DOF common passive mechanical tool which has the usual robotic dynamics. In this paper, we develop a control methodology to endow the resulting n-DOF common passive mechanical tool with useful passive tool dynamics which incorporates inertia scaling, guidance/avoidance system while preserving energetic passivity of the closed-loop system. A fictitious energy storage is used to scale the apparent inertia of the teleoperator. The passive velocity field control for a velocity field tracking and artificial potential function are utilized for guidance/avoidance system. Thus, with the control law proposed previously, the control law renders the 2n-DOF teleoperator as a n-DOF common passive mechanical tool which has programmable apparent inertia and moves under the effects of velocity/potential field tailored to task objectives and obstacles in the workspace.