Jun Morimoto

IROS Conference 2025 Conference Paper

VecNav: Vector Goal Robot Navigation from In-the-wild Videos

• Ruixiang Cao
• Satoshi Yagi
• Satoshi Yamamori
• Jun Morimoto

We propose VecNav, a novel approach that trains a monocular navigation model through self-supervision using uncalibrated, human-captured videos. These videos, characterized by unknown camera intrinsics and extrinsics, are readily available from video-sharing platforms (e. g. YouTube) and are referred to as "in-the-wild" videos due to their unregulated capture conditions. Our approach involves estimating ground truth trajectories from these videos using monocular visual odometry. We then train a transformer-based diffusion policy that takes a goal specified by a vector and RGB images as input and generates action predictions. Our method leverages a significantly larger and more diverse dataset compared to existing monocular visual navigation approaches. This diversity holds the potential to develop a generalist navigation model capable of guiding various types of robots in unfamiliar environments. We evaluated our method on a differential drive robot, demonstrating its capability to effectively navigate using solely "in-the-wild" videos for training. Our experiments demonstrate that VecNav successfully learned to act based on visual affordances, relying solely on uncalibrated "in-the-wild" data.

IROS Conference 2021 Conference Paper

Computationally Affordable Hierarchical Framework for Humanoid Robot Control

• Koji Ishihara
• Jun Morimoto

We propose a hierarchical control framework for generating versatile motions by a humanoid robot. The central feature of our framework is computational affordability: a large amount of computation time is allowable in the upper-level hierarchy. Consequently, whole-body trajectory optimization for a long time horizon becomes feasible. To ensure such affordability, a fast feedback loop is established in the lower-level hierarchy to increase the robustness against the large latency in the upper level. We experimentally examined the advantages of the achieved computational affordability. Our framework allowed a large computational time of 100 ms in each control cycle. This enables online trajectory optimization to predict 50 time steps ahead while taking full-body dynamics into account. Due to such a long prediction range, 20 motions were successfully generated in real time with our computation-ally affordable framework.

ICRA Conference 2019 Conference Paper

Exploiting Human and Robot Muscle Synergies for Human-in-the-loop Optimization of EMG-based Assistive Strategies

• Masashi Hamaya
• Takamitsu Matsubara
• Jun-ichiro Furukawa
• Yuting Sun
• Satoshi Yagi
• Tatsuya Teramae
• Tomoyuki Noda
• Jun Morimoto

In this study, we propose a novel human-in-the-loop optimization approach for exoskeleton robot control. We develop a method to optimize widely-used Electromyography (EMG)-based assistive strategies. If we use multiple EMG channels to control multi-DoF robots, optimization process becomes complex and requires a large amount of data. To make the optimization tractable, we exploit the synergies both of the human muscles and artificial muscles of the exoskeleton robots to reduce the number of parameters of the assistive strategies. We show that we can extract the synergies not only from the user's muscle activities but from pneumatic artificial muscle (PAMs) contractions of the exoskeleton robot. Then, we adopt a Bayesian optimization method to acquire the parameters for assisting human movements by iteratively identifying the user's preferences of the assistive strategies. We conducted experiments to evaluate our proposed method with a PAMs-driven upper-limb exoskeleton robot. Our method successfully learned assistive strategies from the human-in-theloop optimization with a practicable number of interactions.

ICRA Conference 2019 Conference Paper

Learning to Write Anywhere with Spatial Transformer Image-to-Motion Encoder-Decoder Networks

• Barry Ridge
• Rok Pahic
• Ales Ude
• Jun Morimoto

Learning to recognize and reproduce handwritten characters is already a challenging task both for humans and robots alike, but learning to do the same thing for characters that can be transformed arbitrarily in space, as humans do when writing on a blackboard for instance, significantly ups the ante from a robot vision and control perspective. In previous work we proposed various different forms of encoder-decoder networks that were capable of mapping raw images of digits to dynamic movement primitives (DMPs) such that a robot could learn to translate the digit images into motion trajectories in order to reproduce them in written form. However, even with the addition of convolutional layers in the image encoder, the extent to which these networks are spatially invariant or equivariant is rather limited. In this paper, we propose a new architecture that incorporates both an image-to-motion encoder-decoder and a spatial transformer in a fully differentiable overall network that learns to rectify affine transformed digits in input images into canonical forms, before converting them into DMPs with accompanying motion trajectories that are finally transformed back to match up with the original digit drawings such that a robot can write them in their original forms. We present experiments with various challenging datasets that demonstrate the superiority of the new architecture compared to our previous work and demonstrate its use with a humanoid robot in a real writing task.

ICRA Conference 2018 Conference Paper

Deep Encoder-Decoder Networks for Mapping Raw Images to Dynamic Movement Primitives

• Rok Pahic
• Andrej Gams
• Ales Ude
• Jun Morimoto

In this paper we propose a new approach for learning perception-action couplings. We show that by collecting a suitable set of raw images and the associated movement trajectories, a deep encoder-decoder network can be trained that takes raw images as input and outputs the corresponding dynamic movement primitives. We propose suitable cost functions for training the network and describe how to calculate their gradients to enable effective training by back-propagation. We tested the proposed approach both on a synthetic dataset and on a widely used MNIST database to generate handwriting movements from raw images of digits. The calculated movements were also applied for digit writing with a real robot.

ICRA Conference 2018 Conference Paper

Robotizing Double-Bar Ankle-Foot Orthosis

• Tomoyuki Noda
• Asuka Takai
• Tatsuya Teramae
• Eiko Hirookai
• Kimitaka Hase
• Jun Morimoto

This paper introduces an approach that robotizes an ankle-foot orthosis (AFO). In particular, toward post-stroke gait rehabilitation, we robotize a double-bar AFO, which is widely used in rehabilitation facilities, by newly designing a modular joint, a pneumatic actuator, and a Bowden cable force-transmission system. Our modular joint system, called the Modular Exoskeletal Joint (MEJ), has a hollow shaft for simple attachment to an AFO's pivot. We designed MEJ to compactly house an encoder that is built in a bearing in a pulley. We adopted Bowden cables to transmit contraction forces from an actuator to the MEJ. As an actuation scheme, we developed the Nested-cylinder Pneumatic Artificial Muscle (NcPAM) system. Even though PAMs are mechanically compliant and lightweight, they can still generate a large force. Therefore, they can provide an ideal actuation system for exoskeletal robots. The nested-cylinder in NcPAM houses a cable-tensioning spring to properly maintain small cable tension for passive movements and a cable stopper to connect the PAM and the cable for properly transmitting the large force generated by PAM. We show the ankle-joint trajectory tracking performances of this integrated system using iterative learning control.

RLDM Conference 2017 Conference Abstract

A Forward and Inverse Optimal Control Framework to Generate Humanoid Robot Movements with Hierarchical MPC

• Koji Ishihara
• Jun Morimoto

Humans can easily learn a policy to generate a wide variety of dynamic movements. On the other hand, it is still difficult for humanoid robots to acquire a control system for such dynamic movements in a real environment. Model Predictive Control (MPC) is a candidate for such robot control because a wide variety of robot motions can be derived by specifying high-level task goals as objective functions. However, designing the objective functions for high-dimensional robots, such as humanoid robots, is time- consuming because the appropriate objective functions have to be designed through trial and error. In this study, we derive a hierarchical architecture in both forward and inverse optimal control so that the policy can be derived in real time using MPC. In the proposed hierarchical architecture, the control objectives for MPC are estimated via Inverse Optimal Control (IOC), and the learned objectives are utilized to generate the movements of a humanoid robot. By using captured human expert movements, human movement skills are transferred to a humanoid robot model through the estimated objective function. To evaluate our proposed method, we applied the proposed framework to a humanoid robot model. We showed that two different movements, jumping and squatting, can be generated with different objective functions estimated with IOC. Furthermore, we showed that both movements can be generated in real time with our hierarchical MPC approach.

ICRA Conference 2017 Conference Paper

Learning task-parametrized assistive strategies for exoskeleton robots by multi-task reinforcement learning

• Masashi Hamaya
• Takamitsu Matsubara
• Tomoyuki Noda
• Tatsuya Teramae
• Jun Morimoto

Recent studies suggest that reinforcement learning has great potential for generating assistive strategies in exoskeletons through physical interactions between a user and a robot. Previous methods focused on a task-specific assistive strategy, where for every single task (situation/context), the user needs to interact with a robot to learn an appropriate assistive strategy. Therefore, the learned strategies cannot be generalized for a new task. Since the sampling cost is expensive for such human-in-the-loop systems as exoskeletons, generalization must be enabled. In this paper, we propose to learn task-parametrized assistive strategies for exoskeleton robots. Our method employs an assistive strategy, which depends on the task parameter and the state variable, that can be learned from multiple sets of human-robot interaction data across different tasks and generalized even for an unseen task, given the task parameter without additional learning. To alleviate the user's burden in the learning process across multiple tasks, we exploit a data-efficient multi-task reinforcement learning framework. To verify the effectiveness of our method, we developed an experimental platform with an exoskeleton robot. We conducted a series of experiments whose experimental results show that our method can learn such a task-parametrized assistive strategy and be generalized for unseen tasks to reduce the user's electromyography signals (EMGs) during tasks.

ICRA Conference 2017 Conference Paper

Power-augmentation control approach for arm exoskeleton based on human muscular manipulability

• Rok Goljat
• Jan Babic
• Tadej Petric
• Luka Peternel
• Jun Morimoto

The paper presents a novel control method for the arm exoskeletons that takes into account the muscular force manipulability of the human arm. In contrast to classical controllers that provide assistance without considering the biomechanical properties of the human arm, we propose a control method that takes into account the configuration of the arm and the direction of the motion to effectively compensate the anisotropic property of the muscular manipulability of the human arm. Consequently, the proposed control method effectively maintains a spherical endpoint manipulability in the entire workspace of the arm. As a result, the proposed method allows the human using the exoskeleton to efficiently perform tasks in arm configurations that are normally unsuitable due to the low manipulability. We evaluated the proposed approach by a preliminary experimental study where a subject wearing a 2 DOF arm-exoskeleton had to move a 4 kg weight between several locations. The results of our study demonstrate that the proposed approach effectively augments the ability of human motor control to perform tasks equally well in the whole arm workspace that include configurations with low intrinsic manipulability.

IROS Conference 2017 Conference Paper

User-robot collaborative excitation for PAM model identification in exoskeleton robots

• Masashi Hamaya
• Takamitsu Matsubara
• Tomoyuki Noda
• Tatsuya Teramae
• Jun Morimoto

Pneumatic Artificial Muscle (PAM) actuators have been used as exoskeletons because of their inherited compliance and high power-weight ratio. However, creating accurate models remains difficult mainly due to the compliance issue; the model can be changed by the force applied by the user. Therefore, both user and robot actions need to be considered for sufficient excitation of PAMs that are equipped in exoskeleton robots, unlike typical rigid actuators that can only be sufficiently excited by robot actions. In this paper, we propose a user-robot collaborative excitation approach for PAM model identification as an active learning framework for sequentially collecting data by deriving and executing optimal user and robot actions at each step with Gaussian processes. The optimal actions, which are executed by the robot, are displayed on a monitor that enables the user to execute them. We conducted experiments using a powered elbow exoskeleton with a PAM actuator. Experimental results show that our method can more efficiently identify the PAM model than a standard model identification method that does not use any data acquired through user-robot collaboration.

ICRA Conference 2016 Conference Paper

Dry-wireless EEG and asynchronous adaptive feature extraction towards a plug-and-play co-adaptive brain robot interface

• Giuseppe Lisi
• Masashi Hamaya
• Tomoyuki Noda
• Jun Morimoto

This paper introduces a novel asynchronous adaptive brain machine interface (BMI), based on a dry-wireless headset, to trigger the movement of a lower limb exoskeleton robot by foot motor imagery. Specifically, it addresses two issues that are critical for the development of a plug-and-play brain robot interface (BRI): setup-time and the nonstationarity of the electroencephalogram (EEG). The former is solved by a dry-wireless headset that reduces setup-time compared to gel-based systems, and removes the nuisance of cables. The latter has been extensively studied in the literature, leading to effective adaptive algorithms in synchronous BMI. However, asynchronous BMI has received little attention. We propose an extension of state-of-the-art adaptive methods by defining the forgetting factors according to the time constant of the exponential moving average. In addition, we propose feature adaptation as opposed to the standard bias adaptation of a linear classifier. After calibrating the decoder, the subject with a reliable classification of sensorimotor rhythms was asked to trigger robot squatting. The motion was successfully initialized by foot motor imagery; with an essential contribution of the proposed adaptive BMI, which makes features less prone to nonstationarities and improves classification performance compared to standard adaptive methods. The ultimate goal of our research is to develop a plug-and-play co-adaptive BRI for neuromotor rehabilitation.

ICRA Conference 2016 Conference Paper

Learning assistive strategies from a few user-robot interactions: Model-based reinforcement learning approach

• Masashi Hamaya
• Takamitsu Matsubara
• Tomoyuki Noda
• Tatsuya Teramae
• Jun Morimoto

Designing an assistive strategy for exoskeletons is a key ingredient in movement assistance and rehabilitation. While several approaches have been explored, most studies are based on mechanical models of the human user, i. e. , rigid-body dynamics or Center of Mass (CoM)-Zero Moment Point (ZMP) inverted pendulum moECenter of Massdel, or only focus on periodic movements with using oscillator models. On the other hand, the interactions between the user and the robot are often not considered explicitly because of its difficulty in modeling. In this paper, we propose to learn the assistive strategies directly from interactions between the user and the robot. We formulate the learning problem of assistive strategies as a policy search problem. To alleviate heavy burdens to the user for data acquisition, we exploit a data-efficient model-based reinforcement learning framework. To validate the effectiveness of our approach, an experimental platform composed of a real subject, an electromyography (EMG)-measurement system, and a simulated robot arm is developed. Then, a learning experiment with the assistive control task of the robot arm is conducted. As a result, proper assistive strategies that can achieve the robot control task and reduce EMG signals of the user are acquired only by 30 seconds interactions.

IROS Conference 2016 Conference Paper

Trajectory representation by nonlinear scaling of dynamic movement primitives

• Ales Ude
• Rok Vuga
• Bojan Nemec
• Jun Morimoto

An effective robot trajectory representation should encode all relevant aspects of the desired motion. For kinematic representations, this means that both the spatial course of the trajectory and its speed profile must be specified. The concept of dynamic movement primitives (DMP) provides a kinematic representation that fully specifies these two aspects of motion. They are, however, not separated from each other within the DMP representation. This can be problematic when movements with significant speed variations are compared within movement recognition and skill learning algorithms. In such comparisons it is often important to distinguish between the spatial and temporal aspects of motion. In this paper we propose a new representation based on dynamic movement primitives, where spatial and temporal aspects are well separated. We demonstrate the effectiveness of the proposed representation for statistical learning of robot skills and movement recognition and compare the performance with standard DMPs, where temporal and spatial aspects of motion are intertwined.

IROS Conference 2015 Conference Paper

Accelerating synchronization of movement primitives: Dual-arm discrete-periodic motion of a humanoid robot

• Andrej Gams
• Ales Ude
• Jun Morimoto

Human-demonstrated motion transferred to a robotic platform often needs to be adapted to the current state of the environment or to modified task requirements. Adaptation, i. e. learning of a modified behavior, needs to be fast to enable quick utilization of the robot either in industry or in future household-assistant tasks. In this paper we show how to accelerate trajectory adaptation based on learning of coupling terms in the framework of dynamic movement primitives (DMPs). Our method applies ideas from feedback error learning to iterative learning control (ILC). By taking into account the actual physical constraints of the synchronous motion - through synchronization of both positions (or forces) and velocities - it is not only a more faithful representation of actual real-world processes, but it also accelerates the speed of convergence. To show the applicability of the approach in the framework of DMPs, we tested it on a formulation which encodes an initial discrete motion, followed by a periodic behavior, all in a single system. Modifications of the original discrete-periodic formulation now also allow for the use of DMP temporal scaling property. In the paper we also show how the DMP coupling can be implemented in joint space, whereas the measured forces and previous approaches always remained in the task space. We applied our approach to an example dual-arm synchronization task on Sarcos humanoid robot CB-i.

IROS Conference 2015 Conference Paper

Development of a pneumatic-electromagnetic hybrid linear actuator with an integrated structure

• Yoshihiro Nakata
• Tomoyuki Noda
• Jun Morimoto
• Hiroshi Ishiguro

Conventional hybrid actuators can achieve a better force/torque bandwidth than a single principle actuator without losing back-drivability. However, hybrid actuators occupy space at least equal to the sum of the volumes of two or more actuators and multiple transmissions consisting of timing belts, gears, links, tendons, or Bowden-cables. Designs in which space and structural elements of a hybrid actuator are integrated have not been sufficiently studied. This paper presents an original development work on the design of an integrated pneumatic-electromagnetic hybrid linear actuator (iPEHLA). In the integrated design, hybrid components of an air cylinder and a linear motor were arranged around a single shaft, and the pneumatic and electromagnetic actuators shared the same moving parts (i. e. , piston and moving part) and internal cylindrical spaces (i. e. , cylinder and stator). Consequently, space required was less than the sum of the volumes of the two actuators. To verify the feasibility of this novel integrated design, the iPEHLA was designed and prototyped by modifying parts of a commercially available pneumatic cylinder integrated with electromagnetic components. The performances of the iPEHLA under pneumatic actuation and hybrid actuation were compared with respect to a feedforward friction compensation. The experimental results demonstrated that the novel design maintained its hybrid properties.

ICRA Conference 2015 Conference Paper

Estimating joint movements from observed EMG signals with multiple electrodes under sensor failure situations toward safe assistive robot control

• Jun-ichiro Furukawa
• Tomoyuki Noda
• Tatsuya Teramae
• Jun Morimoto

In this paper, we propose an estimation method of human joint movements from measured EMG signals for assistive robot control. We focus on how to estimate joint movements using multiple EMG electrodes even under sensor failure situations. In real world applications, EMG sensor electrodes might become disconnected or detached from skin surfaces. If we consider EMG-based robot control for assistive robots, such sensor failures lead to significant errors in the estimation of user joint movements. To cope with these sensor failures, we propose a state estimation model that takes uncertain observations into account. Sensor channel anomalies are found by checking the covariance of the EMG signals measured by multiple EMG electrodes. To validate the proposed control framework, we artificially disconnect an EMG electrode or detach one side of an EMG probe from the skin surface during elbow joint movement estimation. We show proper control of a one-DOF exoskeleton robot based on the estimated joint torque using our proposed method even when one EMG electrode has a sensor problem; a standard method with no tolerability against uncertain observations was unable to deal with these fault situations. Furthermore, the errors of the estimated joint torque with our proposed method were smaller than the standard method or a method with a conventional sensor fault detection algorithm.

IROS Conference 2015 Conference Paper

Torque and variable stiffness control for antagonistically driven pneumatic muscle actuators via a stable force feedback controller

• Barkan Ugurlu
• Paolo Forni
• Corinne Doppmann
• Jun Morimoto

This paper describes a novel controller that is capable of simultaneously controlling torque and variable stiffness in real-time, for actuators with antagonistically driven pneumatic artificial muscles (PAMs). To this end, two contributions are presented: i) A stable force feedback controller that can cope with inherent PAM nonlinearities is synthesized using the dissipativity theory, for each PAM unit. ii) On top of this force feedback controller, a mathematical formulation is developed to compute reference force inputs that correspond to desired joint torque and joint stiffness inputs, concerning both agonist and antagonist PAMs. This strategy enables us to introduce real-time sensory feedback; torque and stiffness control is addressed by means of PAM force feedback control with guaranteed stability. To validate the proposed control scheme, a series of experiments were conducted on an experimental setup. As the result, the controller exhibited favorable torque and stiffness tracking in real-time, demonstrating that it could meet the performance criteria to power exoskeleton systems.

ICRA Conference 2015 Conference Paper

Towards balance recovery control for lower body exoskeleton robots with Variable Stiffness Actuators: Spring-loaded flywheel model

• Corinne Doppmann
• Barkan Ugurlu
• Masashi Hamaya
• Tatsuya Teramae
• Tomoyuki Noda
• Jun Morimoto

This paper presents a biologically-inspired real-time balance recovery control strategy that is applied to a lower body exoskeleton with variable physical stiffness actuators at its ankle joints. For this purpose, a torsional spring-loaded flywheel model is presented to encapsulate both approximated angular momentum and variable physical stiffness, which are crucial parameters in describing the postural balance. In particular, the incorporation of physical compliance enables us to provide three main contributions: i) A mathematical formulation is developed to express the relation between the dynamic balance criterion ZMP and the physical ankle joint stiffness. Therefore, balancing control can be interpreted in terms of ankle joint stiffness regulation. ii) ‘Variable physical’ stiffness is utilized in the bipedal robot balance control task for the first time in the literature, to the authors' knowledge. iii) The variable physical stiffness strategy is compared with the optimal constant stiffness strategy by conducting experiments on our exoskeleton robot. The results indicate that the proposed method provides a favorable balancing control performance to cope with unperceived perturbations, in terms of center of mass position regulation, ZMP error and mechanical power.

IROS Conference 2014 Conference Paper

Development of an upper limb exoskeleton powered via pneumatic electric hybrid actuators with bowden cable

• Tomoyuki Noda
• Tatsuya Teramae
• Barkan Ugurlu
• Jun Morimoto

In this paper, we introduce our ongoing work on the development of an upper body exoskeleton robot, driven by a pneumatic-electric hybrid actuation system. Since the limb of an exoskeleton robot needs to have small inertia to achieve agility and safety, using a heavy actuator is not preferable. Furthermore, we need to use backdrivable actuators that can generate sufficiently large torques to support user movements. These two requirements may seem contradictory. In order to cope with this development problem, we use a hybrid actuation system composed of Pneumatic Artificial Muscles (PAMs) and small-size electromagnetic motors. Although we and other research groups have already presented the advantage of the hybrid actuation system, we newly propose the usage of Bowden cable in a hybrid actuator to transmit the force generated by the PAMs to joints of our exoskeleton robot so that we can design a compact upper limb with small inertia. In addition, small size electric motors are mechanically connected to joints in order to compensate uncertainty generated by the PAM dynamics and the Bowden cable. We demonstrate that the proposed joint is backdrivable with the capability of large torque generation for the gravity compensation task both in One-DOF system with a dummy weight and right arm of the upper body exoskeleton with a mannequin arm. We also show the right arm exoskeleton can be moved using a torque input, extracted from sensory information via a goniometer.

ICRA Conference 2014 Conference Paper

Online approach for altering robot behaviors based on human in the loop coaching gestures

• Tadej Petric
• Andrej Gams
• Leon Zlajpah
• Ales Ude
• Jun Morimoto

The creation and adaptation of motor behaviors is an important capability for autonomous robots. In this paper we propose an approach for altering existing robot behaviors online, where a human coach interactively changes the robot motion to achieve the desired outcome. Using hand gestures, the human coach can specify the desired modifications to the previously acquired behavior. To preserve a natural posture while performing the task, the movement is encoded in the robot's joint space using periodic dynamic movement primitives. The coaching gestures are mapped to the robot joint space via robot Jacobian and used to create a virtual force field affecting the movement. A recursive least squares technique is used to modify the existing movement with respect to the virtual force field. The proposed approach was evaluated on a simulated three degrees of freedom planar robot and on a real humanoid robot, where human coaching gestures were captured by an RGB-D sensor. Although our focus was on rhythmic movements, the developed approach is also applicable to discrete (point-to-point) movements.

ICRA Conference 2014 Conference Paper

Optimal control approach for pneumatic artificial muscle with using pressure-force conversion model

• Tatsuya Teramae
• Tomoyuki Noda
• Jun Morimoto

In this paper, we propose an optimal control framework for pneumatic actuators. In particular, we consider using Pneumatic Artificial Muscle (PAM) as a part of Pneumatic-Electric (PE) hybrid actuation system. An optimal control framework can be useful for PE hybrid system to properly distribute desired torque outputs to the actuators that have different characteristics. In the optimal control framework, the standard choice to represent control cost is squared force or torque outputs. However, since the control input for PAM is pressure rather than the force or the torque, we should explicitly consider the pressure of PAM as the control cost in an objective function of the optimal control method. We show that we are able to use pressure input as the control cost for PAM by explicitly considering the model which represents a relationship between the pressure input and the force output of PAM. We demonstrate that one-DOF robot with the PE hybrid actuation system can generate pressure-optimized ball throwing movements by using the optimal control method.

ICRA Conference 2014 Conference Paper

Orientation in Cartesian space dynamic movement primitives

• Ales Ude
• Bojan Nemec
• Tadej Petric
• Jun Morimoto

Dynamic movement primitives (DMPs) were proposed as an efficient way for learning and control of complex robot behaviors. They can be used to represent point-to-point and periodic movements and can be applied in Cartesian or in joint space. One problem that arises when DMPs are used to define control policies in Cartesian space is that there exists no minimal, singularity-free representation of orientation. In this paper we show how dynamic movement primitives can be defined for non minimal, singularity free representations of orientation, such as rotation matrices and quaternions. All of the advantages of DMPs, including ease of learning, the ability to include coupling terms, and scale and temporal invariance, can be adopted in our formulation. We have also proposed a new phase stopping mechanism to ensure full movement reproduction in case of perturbations.

ICRA Conference 2013 Conference Paper

An electromyogram based force control coordinated in assistive interaction

• Tomoyuki Noda
• Jun-ichiro Furukawa
• Tatsuya Teramae
• Sang-Ho Hyon
• Jun Morimoto

This study proposes the design of electromyography (EMG)-based force feedback controller which explicitly considers human-robot interaction for the exoskeletal assistive robot. Conventional approaches have been only consider one-directional mapping from EMG to control input for assistive robot control. However, EMG and force generated by the assistive robot interfere each other, e. g. , amplitude of EMG decreases if limb movements are assisted by the robot. In our proposed method, we first derive the nonlinear mapping from EMG signal to muscle force for estimating human joint torque, and convert it to assistive force using human musculoskeletal model and robot kinematic model. Additionally the feedforward interaction torque is feedback into torque controller to acquire the necessity loads. To validate the feasibility of the proposed method, assistive One-DOF system was developed as the real equipment and the simulator. We compared the proposed method with conventional approaches using both the simulated and the real One-DOF systems. As the result, we found that the proposed model was able to estimate the necessary torque adequately to achieve stable human-robot interaction.

ICRA Conference 2013 Conference Paper

Canonical correlation analysis for muscle synergies organized by sensory-motor interactions in musculoskeletal arm movements

• Hiromichi Suetani
• Jun Morimoto

Synergy is a key concept for understanding smooth and dexterous body movements generated in biological systems. In this paper, we propose an approach based on canonical correlation analysis (CCA) for identifying synergies that capture coherences organized between motor activations and sensory signals in movements. Using a musculoskeletal planar arm model with muscle and joint redundancies, we show that synergies identified by CCA give more natural and tractable activation patterns in virtual controller space with lower dimensionality when the arm performs reaching and a 3D pole balancing movement tasks.

IROS Conference 2013 Conference Paper

Design of hybrid drive exoskeleton robot XoR2

• Sang-Ho Hyon
• Takuya Hayashi 0003
• Atsutoshi Yagi
• Tomoyuki Noda
• Jun Morimoto

The paper reports on a novel hybrid drive lower-extremity exoskeleton research platform, XoR2, an improved version of XoR. Its design concept, details of the new hardware and basic experimental results are presented. The robot is designed so that it does not interfere with the user's normal walking and supports a 30-kg payload in addition to its own weight of 20 kg. The robot has a total of 14 joints; among them six flexion/extension joints are powered. Pneumatic artificial muscles are combined with small high-response servo motors for the hip and knee joints, and arranged antagonistically at the hip and ankle joints to provide passive stability and variable stiffness. The preliminary experimental results on position and torque control demonstrate that the proposed mechanisms, sensors and control systems are effective, and hybrid drive is promising for torque-controllable, high-speed, backdrivable, mobile (but non-power-autonomous) exoskeleton robots.

IROS Conference 2013 Conference Paper

Modeling and control of a Pneumatic-Electric hybrid system

• Tatsuya Teramae
• Tomoyuki Noda
• Sang-Ho Hyon
• Jun Morimoto

We introduce our Pneumatic-Electric (PE) hybrid actuator model and propose to use the model to derive a controller for the hybrid actuation system by an optimal control method. Our PE hybrid actuator is composed of Pneumatic Artificial Muscle (PAM) and an electric motor. The PE hybrid actuator is light and can generate large torque. These properties are desirable for assistive devices such as exoskeleton robots. However, to maximally take advantage of PE hybrid system, we need to reasonably distribute necessary torque to these redundant actuators by properly taking distinctive characteristics of a pneumatic actuator and an electric motor into account. To do this, in this study, we use an optimal control method called iterative LQG to reasonably distribute the necessary torque to the PAM and the electric motor. The crucial issue to apply the optimal control method to the PE hybrid system is PAM modeling. We built a PAM model composed of three elements: 1) an (air)pressure-force conversion model, 2) a contraction rate model, 3) time delay of the air valve, and 4) the upper limit of force generation that depends on the contraction rate and the movable range. We apply our proposed method to a one degree of freedom (one-DoF) arm with PE hybrid actuator. The one-DoF arm successfully swing tasks 0. 5 Hz, 2 Hz and 4 Hz and swing up and stability task by reasonably distributing necessary torque to the two different actuators in a simulated and a real environments.

ICRA Conference 2013 Conference Paper

Motion capture and reinforcement learning of dynamically stable humanoid movement primitives

• Rok Vuga
• Matjaz Ogrinc
• Andrej Gams
• Tadej Petric
• Norikazu Sugimoto
• Ales Ude
• Jun Morimoto

Direct transfer of human motion trajectories to humanoid robots does not result in dynamically stable robot movements due to the differences in human and humanoid robot kinematics and dynamics. We developed a system that converts human movements captured by a low-cost RGB-D camera into dynamically stable humanoid movements. The transfer of human movements occurs in real-time. As need arises, the developed system can smoothly transition between unconstrained movement imitation and imitation with balance control, where movement reproduction occurs in the null space of the balance controller. The developed balance controller is based on an approximate model of the robot dynamics, which is sufficient to stabilize the robot during on-line imitation. However, the resulting movements cannot be guaranteed to be optimal because the model of the robot dynamics is not exact. The initially acquired movement is therefore subsequently improved by model-free reinforcement learning, both with respect to the accuracy of reproduction and balance control. We present experimental results in simulation and on a real humanoid robot.

ICRA Conference 2013 Conference Paper

Off-line path integral reinforcement learning using stochastic robot dynamics approximated by sparse pseudo-input Gaussian processes: Application to humanoid robot motor learning in the real environment

• Norikazu Sugimoto
• Jun Morimoto

We develop fast reinforcement learning (RL) framework using the approximated dynamics of a humanoid robot. Although RL is a useful non-linear optimizer, applying it to real robotic systems is usually difficult due to the large number of iterations required to acquire suitable policies. In this study, we approximate the dynamics using data from a real robot with sparse pseudo-input Gaussian processes (SPGPs). By using SPGPs, we estimated the probability distribution considering both the input vector and output signal variances. In real environments, since the observations from robotic sensors include large noise, SPGPs can suitably approximate the stochastic dynamics of a real humanoid robot. We use the approximated dynamics to improve the performance of a movement task in a path integral RL framework, which updates a policy from the sampled trajectories of the state and action vectors and the cost. We implemented our proposed method on a real humanoid robot and tested on a via-point reaching task. The robot achieved successful performance with fewer number of interactions with the real environment by using the proposed method than a conventional approach which dose not use the simulated dynamics.

ICRA Conference 2012 Conference Paper

Extraction of latent kinematic relationships between human users and assistive robots

• Jun Morimoto
• Tomoyuki Noda
• Sang-Ho Hyon

In this study, we propose a control method for movement assistive robots using measured signals from human users. Some of the wearable assistive robots have mechanisms that can be adjusted to human kinematics (e. g. , adjustable link length). However, since the human body has a complicated joint structure, it is generally difficult to design an assistive robot which mechanically well fits human users. We focus on the development of a control algorithm to generate corresponding movements of wearable assistive robots to that of human users even when the kinematic structures of the assistive robot and the human user are different. We first extract the latent kinematic relationship between a human user and the assistive robot. The extracted relationship is then used to control the assistive robot by converting human behavior into the corresponding joint angle trajectories of the robot. The proposed approach is evaluated by a simulated robot model and our newly developed exoskeleton robot.

IROS Conference 2012 Conference Paper

Full-body exoskeleton robot control for walking assistance by style-phase adaptive pattern generation

• Takamitsu Matsubara
• Akimasa Uchikata
• Jun Morimoto

We propose an adaptive walking assistance strategy to control an exoskeleton robot. In our proposed framework, we explicitly consider the following: 1) the diversity of user motions (style) and 2) the interactions among a user, a robot, and an environment. To spatially coordinate a wide variety of user motions and robot behaviors, we estimated style parameters from observed user movements. To temporally coordinate the interactions among the user, the robot, and the environment, we synchronized the phases of these three systems with a coupled oscillator model. The estimated style parameters and the phase of the user motion can be used to predict future user movements. We investigated how movement prediction and phase synchronization can be beneficial to control an exoskeleton robot. To evaluate our adaptive walking assistance strategy, we developed simulated user and exoskeleton models. The physical interactions among the user, the exoskeleton, and the ground models are introduced in the simulated system. We show that the necessary torque for the user walking movement was reduced around 40% by using our proposed method to control the exoskeleton model.

ICRA Conference 2012 Conference Paper

Integrating surface-based hypotheses and manipulation for autonomous segmentation and learning of object representations

• Ales Ude
• David Schiebener
• Norikazu Sugimoto
• Jun Morimoto

Learning about new objects that a robot sees for the first time is a difficult problem because it is not clear how to define the concept of object in general terms. In this paper we consider as objects those physical entities that are comprised of features which move consistently when the robot acts upon them. Among the possible actions that a robot could apply to a hypothetical object, pushing seems to be the most suitable one due to its relative simplicity and general applicability. We propose a methodology to generate and apply pushing actions to hypothetical objects. A probing push causes visual features to move, which enables the robot to either confirm or reject the initial hypothesis about existence of the object. Furthermore, the robot can discriminate the object from the background and accumulate visual features that are useful for training of state of the art statistical classifiers such as bag of features.

ICRA Conference 2012 Conference Paper

Spatio-temporal synchronization of periodic movements by style-phase adaptation: Application to biped walking

• Takamitsu Matsubara
• Akimasa Uchikata
• Jun Morimoto

In this paper, we propose a framework for generating coordinated periodic movements of robotic systems with external inputs. We developed an adaptive pattern generator model that is composed of a two-factor observation model with a style parameter and phase dynamics with a phase variable. The style parameter controls the spatial patterns of the generated trajectories, and the phase variable controls its temporal profiles. To validate the effectiveness of our proposed method, we applied it to a simulated humanoid model to perform biped walking behaviors coordinated with observed walking patterns and the environment. The robot successfully performed stable biped walking behaviors even when the style of the observed walking pattern and the period were suddenly changed.

ICRA Conference 2012 Conference Paper

Using basin ruins and co-moving low-dimensional latent coordinates for dynamic programming of biped walkers on roughing ground

• Hiromichi Suetani
• Aiko M. Ideta
• Jun Morimoto

Disturbance rejection is one of the most important abilities required for biped walkers. In this study, we propose a method for dynamic programming of biped walking and apply it to a simple passive dynamic walker (PDW) on an irregular slope. The key of the proposed approach is to employ the transient dynamics of the walker just before approaching the falling state in the absence of any controlling input, and to derive the optimal control policy in the low-dimensional latent space. In recent our study, we found that such transient dynamics deeply relates to the basin of attraction for a stable gait. By patching latent coordinates to such a structures in each Poincaré section and defining the reward function according to the survive time of the transient dynamics, so-called escape-times, we construct a Markov decision process (MDP) for the PDW and obtain an optimal policy using a dynamic programming (DP). We will show that the proposed method actually succeeds in controlling the PDW even if the degree of disturbance is relatively large and the dimensionality of coordinates is reduced to lower ones.

IROS Conference 2011 Conference Paper

Nonlinear structure of escape-times to falls for a passive dynamic walker on an irregular slope: Anomaly detection using multi-class support vector machine and latent state extraction by canonical correlation analysis

• Hiromichi Suetani
• Aiko M. Ideta
• Jun Morimoto

Falls that occur during walking are a significant problem from the viewpoints of both medicine and robotics engineering. It is very important to predict falls in order to prevent the falls or minimize the ensuing damage from them. In this study, we investigate the structure of the escape-times from walking to falling of a passive dynamic biped walker on a slope in a 2D plane with irregularities. We find that the structure lies on a manifold with high nonlinearity in state space that cannot be analyzed by linear methods under the assumption of a Gaussian distribution. Therefore, we first apply an extension of the support vector machine (SVM) to characterize its nonlinear structure, which enables us to predict imminent falls. Next, we find a latent space which describes the essential dynamics of the passive walker in a lower-dimensional space using canonical correlation analysis (CCA). There is wide applicability of this work for monitoring walking anomalies of both robots and human beings.

IROS Conference 2011 Conference Paper

Switching multiple LQG controllers based on Bellman's optimality principle: Using full-state feedback to control a humanoid robot

• Norikazu Sugimoto
• Jun Morimoto

In this study, we propose novel modular architecture to control a robot with many degrees of freedom, such as a humanoid robot. High-degree-of-freedom (DOF) robots tend to have highly nonlinear dynamics. In general, deriving a nonlinear controller for high-dimensional systems is intractable. In our approach, we adopt multiple Linear Quadratic Gaussian (LQG) controllers to cope with nonlinear dynamics. Switching criteria for the modular architecture is provided by Bellman's optimality. The proposed method is applied to a simulated 10-DOF biped model and a 51-DOF humanoid robot called CB-i.

IROS Conference 2011 Conference Paper

XoR: Hybrid drive exoskeleton robot that can balance

• Sang-Ho Hyon
• Jun Morimoto
• Takamitsu Matsubara
• Tomoyuki Noda
• Mitsuo Kawato

We propose a novel exoskeleton robot prototype aimed at a brain-machine interface and rehabilitation for postural control for elderly people, people with spinal cord injury, stroke patients, and others with similar needs. By arranging pneumatic muscles with electric motors in a optimal way, one can achieve both weight-reduction and torque-controllability. Its anthropomorphic design and torque-controllability enable users to implement and test various rehabilitation/compensation programs consistent with human motor control and learning mechanism. Hybrid drive itself is not new, but its specialized application to lightweight exoskeleton is novel. This paper reports the design and development of the robot, particularly addressing a hybrid drive for load-bearing tasks such as standing and postural maintenance. The experimental data as well as the attached videos demonstrate the effectiveness of the proposed system.

ICRA Conference 2010 Conference Paper

From compliant balancing to dynamic walking on humanoid robot: Integration of CNS and CPG

• Sang-Ho Hyon
• Jun Morimoto
• Mitsuo Kawato

We present our ongoing effort to achieve compliant balancing to dynamic walking on our torque-controlled, human-sized, biped humanoid robot. Inspired by human musculoskeletal systems, our approach integrates full-body task-space force controllers with joint-space pattern generators on the commanded joint torque output to facilitate robust control performance, as well as the efficient online learning. With this approach various compliant and stable motions have been created in a constructive manner. We demonstrate the effectiveness of our approach by two folds of experiments: 1) Compliant double / single-support balancing and quasi-static walking on uneven terrain, which do not require any joint patters, 2) Fast and stable squat and dynamic walking by introducing joint-space pattern generators.

IROS Conference 2010 Conference Paper

Learning Stylistic Dynamic Movement Primitives from multiple demonstrations

• Takamitsu Matsubara
• Sang-Ho Hyon
• Jun Morimoto

In this paper, we propose a novel concept of movement primitives called Stylistic Dynamic Movement Primitives (SDMPs) for motor learning and control in humanoid robotics. In the SDMPs, a diversity of styles in human motion observed through multiple demonstrations can be compactly encoded in a movement primitive, and this allows style manipulation of motion sequences generated from the movement primitive by a control variable called a style parameter. Focusing on discrete movements, a model of the SDMPs is presented as an extension of Dynamic Movement Primitives (DMPs) proposed by Ijspeert et al. . A novel learning procedure of the SDMPs from multiple demonstrations, including a diversity of motion styles, is also described. We present two practical applications of the SDMPs, i. e. , stylistic table tennis swings and obstacle avoidance with an anthropomorphic manipulator.

IROS Conference 2008 Conference Paper

Behavior recognition with ground reaction force estimation and its application to imitation learning

• Yuka Ariki
• Jun Morimoto
• Sang-Ho Hyon

In this paper, we propose an imitation learning framework to generate multiple behaviors with balance control by recognizing human behaviors while estimating the ground reaction force. In our proposed method, a part of captured human motion data is recognized as one particular behavior that is represented by a linear dynamical model. Therefore, our method has small dependence on a classification criteria defined by an experimenter. Based on the behavior recognition method with the ground reaction force estimation and by combining the different recognized behaviors, it is possible to generate many different motion sequences while taking balance into account. First, we approximate a human motion pattern by using linear dynamical models. Then, we can recognize and generate different behavior sequences by switching linear dynamical models. We apply the proposed method to a four-link simulated robot model. Two different squat motions are recognized from motion capture data and the four-link robot generated four different combined squat behaviors from two different squat motions. To show generalization performance, we apply our imitation learning framework to the four-link robot models that have different weights.

ICRA Conference 2008 Conference Paper

CB: Exploring neuroscience with a humanoid research platform

• Gordon Cheng
• Sang-Ho Hyon
• Ales Ude
• Jun Morimoto
• Joshua G. Hale
• Joseph Hart
• Jun Nakanishi
• Darrin C. Bentivegna

In this video presentation we introduce a 50 degrees of freedom humanoid robot, CB - Computational Brain [1]. CB is a humanoid robot created for exploring the underlying processing of the human brain while dealing with the real world. We place our investigations within real world contexts, as humans do. In so doing, we focus on utilising a system that is closer to humans - in sensing, kinematics configuration and performance.

ICRA Conference 2008 Conference Paper

Hierarchical motor learning and synthesis with passivity-based controller and phase oscillator

• Sang-Ho Hyon
• Jun Morimoto
• Gordon Cheng

In this paper, we propose a simple framework for learning and synthesis of fast and complex motor tasks. Where a passivity-based task-space controller acts not only as a full-body force control module, but also as an important module to generate phasic joint patterns. The generated joint patterns are encoded into the paramaters of phase oscillators and form the synergy of the task. Then, similar and/or faster motions are synthesized by superposing the task space controller output and the oscillator output with the modified oscillator amplitudes and/or frequencies. We present some examples of whole-body motion synthesis on a human-sized biped humanoid robot including squatting, dancing and stepping while bipedal balancing. The simulation and experimental videos are supplemented.

ICRA Conference 2008 Conference Paper

Low-dimensional feature extraction for humanoid locomotion using kernel dimension reduction

• Jun Morimoto
• Sang-Ho Hyon
• Christopher G. Atkeson
• Gordon Cheng

We propose using the kernel dimension reduction (KDR) to extract a low-dimensional feature space for humanoid locomotion tasks. Although humanoids have many degrees of freedom, task relevant feature spaces can be much smaller than the number of dimension of the original state space. We consider an application of the proposed approach to improve the locomotive performance of humanoid robots using an extracted low-dimensional state space. To improve the locomotive performance, we use a reinforcement learning (RL) framework. While RL is a useful non-linear optimizer, it is usually difficult to apply RL to real robotic systems — due to the large number of iterations required to acquire suitable policies. In this study, we use the extracted low-dimensional feature space for RL so that the learning system can improve task performance quickly. The kernel dimension reduction method allows us to extract the feature space even if the task relevant mapping is non-linear. This is an essential property to improve humanoid locomotive performance since stepping or walking dynamics involves highly nonlinear dynamics. We show that we can improve stepping and walking policies by using a RL method on an extracted feature space by using KDR.

IROS Conference 2007 Conference Paper

Improving humanoid locomotive performance with learnt approximated dynamics via Gaussian processes for regression

• Jun Morimoto
• Christopher G. Atkeson
• Gen Endo
• Gordon Cheng

We propose to improve the locomotive performance of humanoid robots by using approximated biped stepping and walking dynamics with reinforcement learning (RL). Although RL is a useful non-linear optimizer, it is usually difficult to apply RL to real robotic systems - due to the large number of iterations required to acquire suitable policies. In this study, we first approximated the dynamics by using data from a real robot, and then applied the estimated dynamics in RL in order to improve stepping and walking policies. Gaussian processes were used to approximate the dynamics. By using Gaussian processes, we could estimate a probability distribution of a target function with a given covariance function. Thus, RL can take the uncertainty of the approximated dynamics into account throughout the learning process. We show that we can improve stepping and walking policies by using a RL method with the approximated models both in simulated and real environments. Experimental validation on a real humanoid robot of the proposed

ICRA Conference 2007 Conference Paper

Learning to acquire whole-body humanoid CoM movements to achieve dynamic tasks

• Takamitsu Matsubara
• Jun Morimoto
• Jun Nakanishi
• Sang-Ho Hyon
• Joshua G. Hale
• Gordon Cheng

This paper presents a novel approach to acquire dynamic whole-body movements on humanoid robots focused on learning a control policy for the center of mass. A policy-gradient method is used to acquire a CoM movement as a control policy for achieving a desired dynamic task. A CoM-Jacobian-based redundancy resolution is then used to compute angular velocities for all joints in order to achieve a whole-body movement consistent with the CoM movement acquired through learning. To demonstrate the effectiveness of our method, we apply it in simulation to the learning of a strong punching movement on the Fujitsu humanoid robot, Hoap-2.

ICRA Conference 2006 Conference Paper

Modulation of Simple Sinusoidal Patterns by a Coupled Oscillator Model for Biped Walking

• Jun Morimoto
• Gen Endo
• Jun Nakanishi
• Sang-Ho Hyon
• Gordon Cheng
• Darrin C. Bentivegna
• Christopher G. Atkeson

We show that a humanoid robot can step and walk using simple sinusoidal desired joint trajectories with their phase adjusted by a coupled oscillator model. We use the center of pressure location and velocity to detect the phase of the lateral robot dynamics. This phase information is used to modulate the desired joint trajectories. We applied the proposed control approach to our newly developed human sized humanoid robot and a small size humanoid robot developed by Sony, enabling them to generate successful stepping and walking patterns

ICRA Conference 2005 Conference Paper

Experimental Studies of a Neural Oscillator for Biped Locomotion with QRIO

• Gen Endo
• Jun Nakanishi
• Jun Morimoto
• Gordon Cheng

Recently, there has been a growing interest in biologically inspired biped locomotion control with Central Pattern Generator (CPG). However, few experimental attempts on real hardware 3D humanoid robots have yet been made. Our goal in this paper is to present our achievement of 3D biped locomotion using a neural oscillator applied to a humanoid robot, QRIO. We employ reduced number of neural oscillators as the CPG model, along with a task space Cartesian coordinate system and utilizing entrainment property to establish stable walking gait. We verify robustness against lateral perturbation, through numerical simulation of stepping motion in place along the lateral plane. We then implemented it on the QRIO. It could successfully cope with unknown 3mm bump by autonomously adjusting its stepping period. Sagittal motion produced by a neural oscillator is introduced, and then overlapped with the lateral motion generator in realizing 3D biped locomotion on a QRIO humanoid robot.

ICRA Conference 2005 Conference Paper

Learning Sensory Feedback to CPG with Policy Gradient for Biped Locomotion

• Takamitsu Matsubara
• Jun Morimoto
• Jun Nakanishi
• Masa-aki Sato
• Kenji Doya

This paper proposes a learning framework for a CPG-based biped locomotion controller using a policy gradient method. Our goal in this study is to develop an efficient learning algorithm by reducing the dimensionality of the state space used for learning. We demonstrate that an appropriate feedback controller in the CPG-based controller can be acquired using the proposed method within a few thousand trials by numerical simulations. Furthermore, we implement the learned controller on the physical biped robot to experimentally show that the learned controller successfully works in the real environment.

ICRA Conference 2005 Conference Paper

Poincaré-Map-Based Reinforcement Learning For Biped Walking

• Jun Morimoto
• Jun Nakanishi
• Gen Endo
• Gordon Cheng
• Christopher G. Atkeson
• Garth Zeglin

We propose a model-based reinforcement learning algorithm for biped walking in which the robot learns to appropriately modulate an observed walking pattern. Via-points are detected from the observed walking trajectories using the minimum jerk criterion. The learning algorithm modulates the via-points as control actions to improve walking trajectories. This decision is based on a learned model of the Poincaré map of the periodic walking pattern. The model maps from a state in the single support phase and the control actions to a state in the next single support phase. We applied this approach to both a simulated robot model and an actual biped robot. We show that successful walking policies are acquired.

ICRA Conference 2004 Conference Paper

A Simple Reinforcement Learning Algorithm for Biped Walking

• Jun Morimoto
• Gordon Cheng
• Christopher G. Atkeson
• Garth Zeglin

We propose a model-based reinforcement learning algorithm for biped walking in which the robot learns to appropriately place the swing leg. This decision is based on a learned model of the Poincare map of the periodic walking pattern. The model maps from a state at the middle of a step and foot placement to a state at next middle of a step. We also modify the desired walking cycle frequency based on online measurements. We present simulation results, and are currently implementing this approach on an actual biped robot.

ICRA Conference 2004 Conference Paper

An Empirical Exploration of a Neural Oscillator for Biped Locomotion Control

• Gen Endo
• Jun Morimoto
• Jun Nakanishi
• Gordon Cheng

Humanoid research has made remarkable progress during the past 10 years. However, currently most humanoids use the target ZMP (zero moment point) control algorithm for bipedal locomotion, which requires precise modeling and actuation with high control gains. On the contrary, humans do not rely on such precise modeling and actuation. Our aim is to examine biologically related algorithms for bipedal locomotion that resemble human-like locomotion. This paper describes an empirical study of a neural oscillator for the control of biped locomotion. We propose a new neural oscillator arrangement applied to a compass-like biped robot. Dynamic simulations and experiments with a real biped robot were carried out and the controller performs steady walking for over 50 steps. Gait variations resulting in energy efficiency was made possible through the adjustment of only a single neural activity parameter. Aspects of adaptability and robustness of our approach are shown by allowing the robot to walk over terrains with varying surfaces with different frictional properties. Initial results suggesting optimal amplitude for dealing with perturbation are also presented.

IROS Conference 2004 Conference Paper

An empirical exploration of phase resetting for robust biped locomotion with dynamical movement primitives

• Jun Nakanishi
• Jun Morimoto
• Gen Endo
• Gordon Cheng
• Stefan Schaal
• Mitsuo Kawato

We propose a framework for learning biped locomotion using dynamical movement primitives based on nonlinear oscillators. In our previous work, we suggested dynamical movement primitives as a central pattern generator (CPG) to learn biped locomotion from demonstration. We introduced an adaptation algorithm for the frequency of the oscillators based on phase resetting at the instance of heel strike and entrainment between the phase oscillator and mechanical system using feedback from the environment. In this paper, we empirically explore the role of phase resetting in the proposed algorithm for robust biped locomotion. We demonstrate that phase resetting contributes to robustness against external perturbations and environmental changes by numerical simulations and experiments with a physical biped robot.

IROS Conference 2003 Conference Paper

Minimax differential dynamic programming: application to a biped walking robot

• Jun Morimoto
• Garth Zeglin
• Christopher G. Atkeson

We developed a robust control policy design method in high-dimensional state space by using differential dynamic programming with a minimax criterion. As an example, we applied our method to a simulated five link biped robot. The results show lower joint torques from the optimal control policy compared to a hand-tuned PD servo controller. Results also show that the simulated biped robot can successfully walk with unknown disturbances that cause controllers generated by standard differential dynamic programming and the hand-tuned PD servo to fail. Learning to compensate for modeling error and previously unknown disturbances in conjunction with robust control design is also demonstrated. We also applied proposed method to a real biped robot for optimizing swing leg trajectories.

NeurIPS Conference 2002 Conference Paper

Minimax Differential Dynamic Programming: An Application to Robust Biped Walking

• Jun Morimoto
• Christopher Atkeson

We developed a robust control policy design method in high-dimensional state space by using differential dynamic programming with a minimax criterion. As an example, we applied our method to a simulated five link biped robot. The results show lower joint torques from the optimal con- trol policy compared to a hand-tuned PD servo controller. Results also show that the simulated biped robot can successfully walk with unknown disturbances that cause controllers generated by standard differential dy- namic programming and the hand-tuned PD servo to fail. Learning to compensate for modeling error and previously unknown disturbances in conjunction with robust control design is also demonstrated.

NeurIPS Conference 2002 Conference Paper

Nonparametric Representation of Policies and Value Functions: A Trajectory-Based Approach

• Christopher Atkeson
• Jun Morimoto

A longstanding goal of reinforcement learning is to develop non- parametric representations of policies and value functions that support rapid learning without suffering from interference or the curse of di- mensionality. We have developed a trajectory-based approach, in which policies and value functions are represented nonparametrically along tra- jectories. These trajectories, policies, and value functions are updated as the value function becomes more accurate or as a model of the task is up- dated. We have applied this approach to periodic tasks such as hopping and walking, which required handling discount factors and discontinu- ities in the task dynamics, and using function approximation to represent value functions at discontinuities. We also describe extensions of the ap- proach to make the policies more robust to modeling error and sensor noise.

ICML Conference 2000 Conference Paper

Acquisition of Stand-up Behavior by a Real Robot using Hierarchical Reinforcement Learning

• Jun Morimoto
• Kenji Doya

NeurIPS Conference 2000 Conference Paper

Robust Reinforcement Learning

• Jun Morimoto
• Kenji Doya

This paper proposes a new reinforcement learning (RL) paradigm that explicitly takes into account input disturbance as well as mod(cid: 173) eling errors. The use of environmental models in RL is quite pop(cid: 173) ular for both off-line learning by simulations and for on-line ac(cid: 173) tion planning. However, the difference between the model and the real environment can lead to unpredictable, often unwanted results. Based on the theory of H oocontrol, we consider a differential game in which a 'disturbing' agent (disturber) tries to make the worst possible disturbance while a 'control' agent (actor) tries to make the best control input. The problem is formulated as finding a min(cid: 173) max solution of a value function that takes into account the norm of the output deviation and the norm of the disturbance. We derive on-line learning algorithms for estimating the value function and for calculating the worst disturbance and the best control in refer(cid: 173) ence to the value function. We tested the paradigm, which we call "Robust Reinforcement Learning (RRL), " in the task of inverted pendulum. In the linear domain, the policy and the value func(cid: 173) tion learned by the on-line algorithms coincided with those derived analytically by the linear H ootheory. For a fully nonlinear swing(cid: 173) up task, the control by RRL achieved robust performance against changes in the pendulum weight and friction while a standard RL control could not deal with such environmental changes.

IROS Conference 1998 Conference Paper

Reinforcement learning of dynamic motor sequence: learning to stand up

• Jun Morimoto
• Kenji Doya

We propose a learning method for implementing human-like sequential movements in robots. As an example of dynamic sequential movement, we consider the "stand-up" task for a two-joint, three-link robot. In contrast to the case of steady walking or standing, the desired trajectory for such a transient behavior is very difficult to derive. The goal of the task is to find a path that links a lying state to an upright state under the constraints of the system dynamics. The geometry of the robot is such that there is no static solution; the robot has to stand up dynamically utilizing the momentum of its body. We use reinforcement learning, in particular, a continuous time and state temporal difference (TD) learning method. For successful results, we use 1) an efficient method of value function approximation in a high-dimensional state space, and 2) a hierarchical architecture which divides a large state space into a few smaller pieces.