Christopher G. Atkeson

ICRA Conference 2025 Conference Paper

One-Shot Video Imitation via Parameterized Symbolic Abstraction Graphs

• Jianren Wang
• Kangni Liu
• Dingkun Guo
• Zhou Xian
• Christopher G. Atkeson

Learning to manipulate dynamic and deformable objects from a single demonstration video holds great promise in terms of scalability. Previous approaches have predominantly focused on either replaying object relationships or actor trajectories. The former often struggles to generalize across diverse tasks, while the latter suffers from data inefficiency. Moreover, both methodologies encounter challenges in capturing invisible physical attributes, such as forces. In this paper, we propose to interpret video demonstrations through a series of Parameterized Symbolic Abstraction Graphs (PSAGs), where nodes represent objects and edges denote relationships between objects. We further ground geometric constraints through simulation to estimate non-geometric, visually imperceptible attributes. The augmented PSAGs are then applied in real robot experiments. Our approach has been validated across a range of tasks, such as Cutting Avocado, Cutting Vegetable, Pouring Liquid, Rolling Dough, and Slicing Pizza. We demonstrate successful generalization to novel objects with distinct visual and physical properties. For visualizations of the learned policies please check: https://www.jianrenw.com/PSAG/

ICRA Conference 2025 Conference Paper

Skills Made to Order: Efficient Acquisition of Robot Cooking Skills Guided by Multiple Forms of Internet Data

• Mrinal Verghese
• Christopher G. Atkeson

This study explores the utility of various internet data sources to select among a set of template robot behaviors to perform skills. Learning contact-rich skills involving tool use from internet data sources has typically been challenging due to the lack of physical information such as contact existence, location, areas, and force in this data. Prior works have generally used internet data and foundation models trained on this data to generate low-level robot behavior. We hypothesize that these data and models may be better suited to selecting among a set of basic robot behaviors to perform these contact-rich skills. We explore three methods of template selection: querying large language models, comparing video of robot execution to retrieved human video using features from a pretrained video encoder common in prior work, and performing the same comparison using features from an optic flow encoder trained on internet data. Our results show that LLMs are surprisingly capable template selectors despite their lack of visual information, optical flow encoding significantly outperforms video encoders trained with an order of magnitude more data, and important synergies exist between various forms of internet data for template selection. By exploiting these synergies, we create a template selector using multiple forms of internet data that achieves a 79% success rate on a set of 16 different cooking skills involving tool-use.

ICRA Conference 2025 Conference Paper

Soft Robotic Dynamic in-Hand Pen Spinning

• Yunchao Yao
• Uksang Yoo
• Jean Oh
• Christopher G. Atkeson
• Jeffrey Ichnowski

Dynamic in-hand manipulation remains challenging for soft robotic systems, which have demonstrated advantages in safe, compliant interactions but struggle with highspeed dynamic tasks. In this work, we present SWIFT, a system for learning dynamic tasks using a soft and compliant robotic hand. Unlike previous works that rely on simulation, quasistatic actions, and precise object models, SWIFT learns to spin a pen through trial and error using only real-world data and without requiring explicit knowledge of the pen's physical attributes. With self-labeled trials sampled from the real world, SWIFT discovers the set of pen grasping and spinning primitive parameters that enables a soft hand to spin a pen reliably. After 130 sampled actions per object, SWIFT achieves 10/10 success rate across three pens with different weights and weight distributions, demonstrating generalizability and robustness to changes in object properties. The results highlight the potential for soft robotic end-effectors to perform dynamic tasks. We also demonstrate generalization to different shapes and weights, such as a brush and a screwdriver, with 10/10 and 5/10 success rates, respectively. Videos, data, and code are available at https://soft-spin.github.io.

ICRA Conference 2023 Conference Paper

Learning Exploration Strategies to Solve Real-World Marble Runs

• Alisa Allaire
• Christopher G. Atkeson

Tasks involving locally unstable or discontinuous dynamics (such as bifurcations and collisions) remain challenging in robotics, because small variations in the environment can have a significant impact on task outcomes. For such tasks, learning a robust deterministic policy is difficult. We focus on structuring exploration with multiple stochastic policies based on a mixture of experts (MoE) policy representation that can be efficiently adapted. The MoE policy is composed of stochastic sub-policies that allow exploration of multiple distinct regions of the action space (or strategies) and a high-level selection policy to guide exploration towards the most promising regions. We develop a robot system to evaluate our approach in a real-world physical problem solving domain. After training the MoE policy in simulation, online learning in the real world demonstrates efficient adaptation within just a few dozen attempts, with a minimal sim2real gap. Our results confirm that representing multiple strategies promotes efficient adaptation in new environments and strategies learned under different dynamics can still provide useful information about where to look for good strategies.

ICRA Conference 2023 Conference Paper

Using Memory-Based Learning to Solve Tasks with State-Action Constraints

• Mrinal Verghese
• Christopher G. Atkeson

Tasks where the set of possible actions depend discontinuously on the state pose a significant challenge for current reinforcement learning algorithms. For example, a locked door must be first unlocked, and then the handle turned before the door can be opened. The sequential nature of these tasks makes obtaining final rewards difficult, and transferring information between task variants using continuous learned values such as weights rather than discrete symbols can be inefficient. Our key insight is that agents that act and think symbolically are often more effective in dealing with these tasks. We propose a memory-based learning approach that leverages the symbolic nature of constraints and temporal ordering of actions in these tasks to quickly acquire and transfer high-level information. We evaluate the performance of memory-based learning on both real and simulated tasks with approximately discontinuous constraints between states and actions, and show our method learns to solve these tasks an order of magnitude faster than both model-based and model-free deep reinforcement learning methods.

ICRA Conference 2022 Conference Paper

Learning to Navigate by Pushing

• Cornelia Bauer
• Dominik Bauer
• Alisa Allaire
• Christopher G. Atkeson
• Nancy S. Pollard

In this work, we investigate a form of dynamic contact-rich locomotion in which a robot pushes off from obstacles in order to move through its environment. We present a reflex-based approach that switches between optimized hand-crafted reflex controllers and produces smooth and predictable motions. In contrast to previous work, our approach does not rely on periodic movements, complex models of robot and contact dynamics, or extensive hand tuning. We demonstrate the effectiveness of our approach and evaluate its performance compared to a standard model-free RL algorithm. We identify continuous clusters of similar behaviours, which allows us to successfully transfer different push-off motions directly from simulation to a physical robot without further retraining.

ICRA Conference 2019 Conference Paper

Using Deep Reinforcement Learning to Learn High-Level Policies on the ATRIAS Biped

• Tianyu Li 0005
• Hartmut Geyer
• Christopher G. Atkeson
• Akshara Rai

Learning controllers for bipedal robots is a challenging problem, often requiring expert knowledge and extensive tuning of parameters that vary in different situations. Recently, deep reinforcement learning has shown promise at automatically learning controllers for complex systems in simulation. This has been followed by a push towards learning controllers that can be transferred between simulation and hardware, primarily with the use of domain randomization. However, domain randomization can make the problem of finding stable controllers even more challenging, especially for under actuated bipedal robots. In this work, we explore whether policies learned in simulation can be transferred to hardware with the use of high-fidelity simulators and structured controllers. We learn a neural network policy which is a part of a more structured controller. While the neural network is learned in simulation, the rest of the controller stays fixed, and can be tuned by the expert as needed. We show that using this approach can greatly speed up the rate of learning in simulation, as well as enable transfer of policies between simulation and hardware. We present our results on an ATRIAS robot and explore the effect of action spaces and cost functions on the rate of transfer between simulation and hardware. Our results show that structured policies can indeed be learned in simulation and implemented on hardware successfully. This has several advantages, as the structure preserves the intuitive nature of the policy, and the neural network improves the performance of the hand-designed policy. In this way, we propose a way of using neural networks to improve expert designed controllers, while maintaining ease of understanding.

JMLR Journal 2019 Journal Article

Using Simulation to Improve Sample-Efficiency of Bayesian Optimization for Bipedal Robots

• Akshara Rai
• Rika Antonova
• Franziska Meier
• Christopher G. Atkeson

Learning for control can acquire controllers for novel robotic tasks, paving the path for autonomous agents. Such controllers can be expert-designed policies, which typically require tuning of parameters for each task scenario. In this context, Bayesian optimization (BO) has emerged as a promising approach for automatically tuning controllers. However, sample-efficiency can still be an issue for high-dimensional policies on hardware. Here, we develop an approach that utilizes simulation to learn structured feature transforms that map the original parameter space into a domain-informed space. During BO, similarity between controllers is now calculated in this transformed space. Experiments on the ATRIAS robot hardware and simulation show that our approach succeeds at sample-efficiently learning controllers for multiple robots. Another question arises: What if the simulation significantly differs from hardware? To answer this, we create increasingly approximate simulators and study the effect of increasing simulation-hardware mismatch on the performance of Bayesian optimization. We also compare our approach to other approaches from literature, and find it to be more reliable, especially in cases of high mismatch. Our experiments show that our approach succeeds across different controller types, bipedal robot models and simulator fidelity levels, making it applicable to a wide range of bipedal locomotion problems. [abs] [ pdf ][ bib ] &copy JMLR 2019. ( edit, beta )

ICRA Conference 2018 Conference Paper

Bayesian Optimization Using Domain Knowledge on the ATRIAS Biped

• Akshara Rai
• Rika Antonova
• Seungmoon Song
• William C. Martin
• Hartmut Geyer
• Christopher G. Atkeson

Robotics controllers often consist of expert-designed heuristics, which can be hard to tune in higher dimensions. Simulation can aid in optimizing these controllers if parameters learned in simulation transfer to hardware. Unfortunately, this is often not the case in legged locomotion, necessitating learning directly on hardware. This motivates using data-efficient learning techniques like Bayesian Optimization (BO) to minimize collecting expensive data samples. BO is a black-box data-efficient optimization scheme, though its performance typically degrades in higher dimensions. We aim to overcome this problem by incorporating domain knowledge, with a focus on bipedal locomotion. In our previous work, we proposed a feature transformation that projected a 16-dimensional locomotion controller to a 1-dimensional space using knowledge of human walking. When optimizing a human-inspired neuromuscular controller in simulation, this feature transformation enhanced sample efficiency of BO over traditional BO with a Squared Exponential kernel. In this paper, we present a generalized feature transform applicable to non-humanoid robot morphologies and evaluate it on the ATRIAS bipedal robot, in both simulation and hardware. We present three different walking controllers and two are evaluated on the real robot. Our results show that this feature transform captures important aspects of walking and accelerates learning on hardware and simulation, as compared to traditional BO.

ICRA Conference 2016 Conference Paper

A distributed MEMS gyro network for joint velocity estimation

• X. Xinjilefu 0001
• Siyuan Feng 0003
• Christopher G. Atkeson

This paper is about improving joint and actuator velocity estimates on a human-sized hydraulic humanoid robot by adding a network of inexpensive microelectromechanical systems (MEMS) gyroscopes. Due to the lack of joint velocity sensors on the majority of humanoid robots, the joint velocity estimates often become a limiting factor on the controller performance. The distributed gyroscopes serve as indirect sensors of the joint velocities that can be estimated by a Kalman filter. Using this framework, we achieve higher velocity gains at the center of mass level on an Atlas hydraulic humanoid robot, which translates into better control performance.

ICRA Conference 2016 Conference Paper

Neural networks and differential dynamic programming for reinforcement learning problems

• Akihiko Yamaguchi
• Christopher G. Atkeson

We explore a model-based approach to reinforcement learning where partially or totally unknown dynamics are learned and explicit planning is performed. We learn dynamics with neural networks, and plan behaviors with differential dynamic programming (DDP). In order to handle complicated dynamics, such as manipulating liquids (pouring), we consider temporally decomposed dynamics. We start from our recent work [1] where we used locally weighted regression (LWR) to model dynamics. The major contribution of this paper is making use of deep learning in the form of neural networks with stochastic DDP, and showing the advantages of neural networks over LWR. For this purpose, we extend neural networks for: (1) modeling prediction error and output noise, (2) computing an output probability distribution for a given input distribution, and (3) computing gradients of output expectation with respect to an input. Since neural networks have nonlinear activation functions, these extensions were not easy. We provide an analytic solution for these extensions using some simplifying assumptions. We verified this method in pouring simulation experiments. The learning performance with neural networks was better than that of LWR. The amount of spilled materials was reduced. We also present early results of robot experiments using a PR2. Accompanying video: https://youtu.be/aM3hE1J5W98

IROS Conference 2016 Conference Paper

Robust dynamic walking using online foot step optimization

• Siyuan Feng 0003
• X. Xinjilefu 0001
• Christopher G. Atkeson
• Joohyung Kim

To enable robust dynamic walking on the Atlas robot, we extend our previous work by adding a receding-horizon component. The new controller consists of three hierarchies: a center of mass (CoM) trajectory planner that follows a sequence of desired foot steps, a receding-horizon controller that optimizes the next foot placement to minimize future CoM tracking errors, and an inverse dynamics based full body controller that generates instantaneous joint commands to track these motions while obeying physical constraints. An approximate value function is generated by the CoM planner, and is used to guide the foot placement and inverse dynamics optimizations. The proposed controller is implemented and tested on the Atlas robot. It is capable of walking with strong external perturbations such as recovering from large pushes and traversing unstructured terrain.

IROS Conference 2015 Conference Paper

Humanoid full-body manipulation planning with multiple initial guesses and key postures

• Bowei Tang
• Tianyu Chen
• Christopher G. Atkeson

We present an optimization method to solve coupled redundant inverse kinematics problems and generate trajectories for humanoid robot full-body manipulation. The basic idea of our algorithm is to divide a manipulation task into a series of key postures, generate multiple diverse initial guesses for each key posture, and use optimization to find inverse kinematics solutions based on these initial guesses. We then find an optimal series of key postures and form a continuous trajectory. Our approach is implemented in a Gazebo simulation using the Atlas humanoid robot from Boston Dynamics.

ICRA Conference 2015 Conference Paper

Online Bayesian changepoint detection for articulated motion models

• Scott Niekum
• Sarah Osentoski
• Christopher G. Atkeson
• Andrew G. Barto

We introduce CHAMP, an algorithm for online Bayesian changepoint detection in settings where it is difficult or undesirable to integrate over the parameters of candidate models. CHAMP is used in combination with several articulation models to detect changes in articulated motion of objects in the world, allowing a robot to infer physically-grounded task information. We focus on three settings where a changepoint model is appropriate: objects with intrinsic articulation relationships that can change over time, object-object contact that results in quasi-static articulated motion, and assembly tasks where each step changes articulation relationships. We experimentally demonstrate that this system can be used to infer various types of information from demonstration data including causal manipulation models, human-robot grasp correspondences, and skill verification tests.

ICRA Conference 2014 Conference Paper

Decoupled state estimation for humanoids using full-body dynamics

• X. Xinjilefu 0001
• Siyuan Feng 0003
• Weiwei Huang
• Christopher G. Atkeson

We propose a framework to use full-body dynamics for humanoid state estimation. The main idea is to decouple the full body state vector into several independent state vectors. Some decoupled state vectors can be estimated very efficiently with a steady state Kalman Filter. In a steady state Kalman Filter, state covariance is computed only once during initialization. Furthermore, decoupling speeds up numerical linearization of the dynamic model. We demonstrate that these state estimators are capable of handling walking on flat ground and on rough terrain.

IROS Conference 2014 Conference Paper

Dynamic state estimation using Quadratic Programming

• X. Xinjilefu 0001
• Siyuan Feng 0003
• Christopher G. Atkeson

We propose a framework for using full-body dynamics for humanoid state estimation. It is formulated as an optimization problem and solved with Quadratic Programming (QP). This formulation provides two main advantages over a nonlinear Kalman filter for dynamic state estimation. QP does not require the dynamic system to be written in the state space form, and it handles equality and inequality constraints naturally. The QP state estimator considers modeling error as part of the optimization vector and includes it in the cost function. The proposed QP state estimator is tested on a Boston Dynamics Atlas humanoid robot.

ICRA Conference 2014 Conference Paper

Versatile and robust 3D walking with a simulated humanoid robot (Atlas): A model predictive control approach

• Salman Faraji
• Soha Pouya
• Christopher G. Atkeson
• Auke Jan Ijspeert

In this paper, we propose a novel walking method for torque controlled robots. The method is able to produce a wide range of speeds without requiring off-line optimizations and re-tuning of parameters. We use a quadratic whole-body optimization method running online which generates joint torques, given desired Cartesian accelerations of center of mass and feet. Using a dynamics model of the robot inside this optimizer, we ensure both compliance and tracking, required for fast locomotion. We have designed a foot-step planner that uses a linear inverted pendulum as simplified robot internal model. This planner is formulated as a quadratic convex problem which optimizes future steps of the robot. Fast libraries help us performing these calculations online. With very few parameters to tune and no perception, our method shows notable robustness against strong external pushes, relatively large terrain variations, internal noises, model errors and also delayed communication.

ICRA Conference 2013 Conference Paper

Energy-based optimal step planning for humanoids

• Weiwei Huang
• Junggon Kim
• Christopher G. Atkeson

Step planning is becoming an increasingly important research topic for humanoid robots. Most cost functions for step planning in the literature are designed based on terrain information. The energy cost to perform each step action is usually ignored. In walking, energy consumption depends on gait features such as step length and width. In this paper, we use three simple and intuitive energy cost functions for different step lengths, widths, and the turning angle. These functions are inspired by literature on human walking energy analysis, and the function parameters are tuned to match computed costs for optimal humanoid walking motions obtained by simulation. The energy cost and the terrain cost are combined to obtain an optimal step planning sequence using A* search.

IROS Conference 2012 Conference Paper

State estimation of a walking humanoid robot

• X. Xinjilefu 0001
• Christopher G. Atkeson

This paper compares two approaches to designing Kalman Filters for walking systems. The first design uses Linear Inverted Pendulum Model (LIPM) dynamics, and the other design uses a more complete Planar dynamics. The filter based on the simpler LIPM design is more robust to modeling error. The more complex design estimates center of mass height and joint velocities, and tracks horizontal center of mass translation more accurately. We also investigate different ways of handling contact states and using force sensing in state estimation. In the LIPM filter, force sensing is used to determine contact states and tune filter parameters. In the Planar filter, force sensing is used to select the proper measurement equation.

ICRA Conference 2010 Conference Paper

An optimization approach to rough terrain locomotion

• Matthew Zucker 0001
• J. Andrew Bagnell
• Christopher G. Atkeson
• James J. Kuffner

We present a novel approach to legged locomotion over rough terrain that is thoroughly rooted in optimization. This approach relies on a hierarchy of fast, anytime algorithms to plan a set of footholds, along with the dynamic body motions required to execute them. Components within the planning framework coordinate to exchange plans, cost-to-go estimates, and “certificates” that ensure the output of an abstract high-level planner can be realized by deeper layers of the hierarchy. The burden of careful engineering of cost functions to achieve desired performance is substantially mitigated by a simple inverse optimal control technique. Robustness is achieved by real-time re-planning of the full trajectory, augmented by reflexes and feedback control. We demonstrate the successful application of our approach in guiding the LittleDog quadruped robot over a variety of rough terrains.

IROS Conference 2010 Conference Paper

Dynamic Balance Force Control for compliant humanoid robots

• Benjamin J. Stephens
• Christopher G. Atkeson

This paper presents a model-based method, called Dynamic Balance Force Control (DBFC), for determining full body joint torques based on desired COM motion and contact forces for compliant humanoid robots. The center of mass (COM) dynamics are affected directly through contact force control to achieve stable balance. This idea is used to formulate DBFC considering the full rigid-body dynamics of the robot to produce desired contact forces. To achieve generic force control tasks, a virtual model controller, DBFC-VMC, is presented. Results presented from experiments on a force-controlled humanoid robot and simulation demonstrate the general purpose use of this control.

IROS Conference 2010 Conference Paper

Gain scheduled control of perturbed standing balance

• Dengpeng Xing
• Christopher G. Atkeson
• Jianbo Su
• Benjamin J. Stephens

This paper develops full-state parametric controllers for standing balance of humanoid robots in response to impulsive and constant pushes. We also explore a hypothesis that postural feedback gains in standing balance should change with perturbation size. From an engineering point of view this is known as gain scheduling. We use an optimization approach to see if feedback gains should scale with the perturbation for a simulated robot. We simulate models in the sagittal and lateral plane and in 3-dimensions, use a horizontal push of a given size, direction and location as a perturbation, and optimize parametric controllers for different push sizes, directions and locations. During a simulated perturbation experiment, the appropriate controller is continuously selected based on the current push. For an impulse, the simulated robot recovers back to the initial state; for a constant push, the robot moves to an equilibrium position which leans into the push and has zero joint torques. We show the performance of optimized parametric controllers in response to different external pushes.

IROS Conference 2009 Conference Paper

Robots with inflatable links

• Siddharth Sanan
• Justin B. Moidel
• Christopher G. Atkeson

The use of robots in assistive roles will be an increasingly significant application for robotics. Assistive robots need to physically interact with humans in a safe manner. We propose the use of inflatable robot links as structural members instead of traditional rigid links. We believe such links would allow the development of inherently safe robots. For these robots to be useful in tasks such as assisting humans, it is essential that we be able to control contact forces with these robots. In this paper, we propose a model for force control with a single inflatable link, investigate the dynamics of the model, and present experimental results.

IROS Conference 2009 Conference Paper

Standing balance control using a trajectory library

• Chenggang Liu
• Christopher G. Atkeson

This paper presents a standing balance controller that explicitly handles pushes. We employ a library of optimal trajectories and the neighboring optimal control method to generate local approximations to the optimal control. We take advantage of a parametric nonlinear optimization method, SNOPT, to generate initial trajectories and then use Differential Dynamic Programming (DDP) to further refine them and get their neighboring optimal control. A library generation method is proposed, which keeps the trajectory library to a reasonable size. We compare the proposed controller with an optimal controller and an LQR based gain scheduling controller using the same optimization criterion. Simulation results demonstrate the performance of the proposed method.

ICRA Conference 2008 Conference Paper

CB: Exploring neuroscience with a humanoid research platform

• Gordon Cheng
• Sang-Ho Hyon
• Ales Ude
• Jun Morimoto
• Joshua G. Hale
• Joseph Hart
• Jun Nakanishi
• Darrin C. Bentivegna

In this video presentation we introduce a 50 degrees of freedom humanoid robot, CB - Computational Brain [1]. CB is a humanoid robot created for exploring the underlying processing of the human brain while dealing with the real world. We place our investigations within real world contexts, as humans do. In so doing, we focus on utilising a system that is closer to humans - in sensing, kinematics configuration and performance.

ICRA Conference 2008 Conference Paper

Low-dimensional feature extraction for humanoid locomotion using kernel dimension reduction

• Jun Morimoto
• Sang-Ho Hyon
• Christopher G. Atkeson
• Gordon Cheng

We propose using the kernel dimension reduction (KDR) to extract a low-dimensional feature space for humanoid locomotion tasks. Although humanoids have many degrees of freedom, task relevant feature spaces can be much smaller than the number of dimension of the original state space. We consider an application of the proposed approach to improve the locomotive performance of humanoid robots using an extracted low-dimensional state space. To improve the locomotive performance, we use a reinforcement learning (RL) framework. While RL is a useful non-linear optimizer, it is usually difficult to apply RL to real robotic systems — due to the large number of iterations required to acquire suitable policies. In this study, we use the extracted low-dimensional feature space for RL so that the learning system can improve task performance quickly. The kernel dimension reduction method allows us to extract the feature space even if the task relevant mapping is non-linear. This is an essential property to improve humanoid locomotive performance since stepping or walking dynamics involves highly nonlinear dynamics. We show that we can improve stepping and walking policies by using a RL method on an extracted feature space by using KDR.

IROS Conference 2007 Conference Paper

Improving humanoid locomotive performance with learnt approximated dynamics via Gaussian processes for regression

• Jun Morimoto
• Christopher G. Atkeson
• Gen Endo
• Gordon Cheng

We propose to improve the locomotive performance of humanoid robots by using approximated biped stepping and walking dynamics with reinforcement learning (RL). Although RL is a useful non-linear optimizer, it is usually difficult to apply RL to real robotic systems - due to the large number of iterations required to acquire suitable policies. In this study, we first approximated the dynamics by using data from a real robot, and then applied the estimated dynamics in RL in order to improve stepping and walking policies. Gaussian processes were used to approximate the dynamics. By using Gaussian processes, we could estimate a probability distribution of a target function with a given covariance function. Thus, RL can take the uncertainty of the approximated dynamics into account throughout the learning process. We show that we can improve stepping and walking policies by using a RL method with the approximated models both in simulated and real environments. Experimental validation on a real humanoid robot of the proposed

IROS Conference 2007 Conference Paper

Transfer of policies based on trajectory libraries

• Martin Stolle
• Hanns Tappeiner
• Joel E. Chestnutt
• Christopher G. Atkeson

Libraries of trajectories are a promising way of creating policies for difficult problems. However, often it is not desirable or even possible to create a new library for every task. We present a method for transferring libraries across tasks, which allows us to build libraries by learning from demonstration on one task and apply them to similar tasks. Representing the libraries in a feature-based space is key to supporting transfer. We also search through the library to ensure a complete path to the goal is possible. Results are shown for the Little Dog task. Little Dog is a quadruped robot that has to walk across rough terrain at reasonably fast speeds.

IROS Conference 2006 Conference Paper

Learning Similar Tasks From Observation and Practice

• Darrin C. Bentivegna
• Christopher G. Atkeson
• Gordon Cheng

This paper presents a case study of learning to select behavioral primitives and generate subgoals from observation and practice. Our approach uses local features to generalize across tasks and global features to learn from practice. We demonstrate this approach applied to the marble maze task. Our robot uses local features to initially learn primitive selection and subgoal generation policies from observing a teacher maneuver a marble through a maze. The robot then uses this information as it tries to traverse another maze, and refines the information during learning from practice

ICRA Conference 2006 Conference Paper

Modulation of Simple Sinusoidal Patterns by a Coupled Oscillator Model for Biped Walking

• Jun Morimoto
• Gen Endo
• Jun Nakanishi
• Sang-Ho Hyon
• Gordon Cheng
• Darrin C. Bentivegna
• Christopher G. Atkeson

We show that a humanoid robot can step and walk using simple sinusoidal desired joint trajectories with their phase adjusted by a coupled oscillator model. We use the center of pressure location and velocity to detect the phase of the lateral robot dynamics. This phase information is used to modulate the desired joint trajectories. We applied the proposed control approach to our newly developed human sized humanoid robot and a small size humanoid robot developed by Sony, enabling them to generate successful stepping and walking patterns

ICRA Conference 2006 Conference Paper

Policies based on Trajectory Libraries

• Martin Stolle
• Christopher G. Atkeson

We present a control approach that uses a library of trajectories to establish a global control law or policy. This is an alternative to methods for finding global policies based on value functions using dynamic programming and also to using plans based on a single desired trajectory. Our method has the advantage of providing reasonable policies much faster than dynamic programming can provide an initial policy. It also has the advantage of providing more robust and global policies than following a single desired trajectory. Trajectory libraries can be created for robots with many more degrees of freedom than what dynamic programming can be applied to as well as for robots with dynamic model discontinuities. Results are shown for the "Labyrinth" marble maze, both in simulation as well as a real world version. The marble maze is a difficult task which requires both fast control as well as planning ahead

ICRA Conference 2005 Conference Paper

Dynamic Programming in Reduced Dimensional Spaces: Dynamic Planning For Robust Biped Locomotion

• Mike Stilman
• Christopher G. Atkeson
• James J. Kuffner
• Garth Zeglin

We explore the use of computational optimal control techniques for automated construction of policies in complex dynamic environments. Our implementation of dynamic programming is performed in a reduced dimensional subspace of a simulated four-DOF biped robot with point feet. We show that a computed solution to this problem can be generated and yield empirically stable walking that can handle various types of disturbances.

ICRA Conference 2005 Conference Paper

Poincaré-Map-Based Reinforcement Learning For Biped Walking

• Jun Morimoto
• Jun Nakanishi
• Gen Endo
• Gordon Cheng
• Christopher G. Atkeson
• Garth Zeglin

We propose a model-based reinforcement learning algorithm for biped walking in which the robot learns to appropriately modulate an observed walking pattern. Via-points are detected from the observed walking trajectories using the minimum jerk criterion. The learning algorithm modulates the via-points as control actions to improve walking trajectories. This decision is based on a learned model of the Poincaré map of the periodic walking pattern. The model maps from a state in the single support phase and the control actions to a state in the next single support phase. We applied this approach to both a simulated robot model and an actual biped robot. We show that successful walking policies are acquired.

ICRA Conference 2004 Conference Paper

A Simple Reinforcement Learning Algorithm for Biped Walking

• Jun Morimoto
• Gordon Cheng
• Christopher G. Atkeson
• Garth Zeglin

We propose a model-based reinforcement learning algorithm for biped walking in which the robot learns to appropriately place the swing leg. This decision is based on a learned model of the Poincare map of the periodic walking pattern. The model maps from a state at the middle of a step and foot placement to a state at next middle of a step. We also modify the desired walking cycle frequency based on online measurements. We present simulation results, and are currently implementing this approach on an actual biped robot.

IROS Conference 2003 Conference Paper

Combining peripheral and foveal humanoid vision to detect, pursue, recognize and act

• Ales Ude
• Christopher G. Atkeson
• Gordon Cheng

In this paper we present a humanoid system that can integrate information provided by its foveal and peripheral cameras. We use peripheral vision to detect and pursue objects of interest based on simple shape and color models. A detection event triggers the robot to direct its eyes towards the object, thus making a more detailed analysis of the observed objects in higher resolution foveal images feasible. The recognition is based on principal component analysis and is performed while the robot actively pursues the detected object. The classification results are inferred using information from a video stream rather than just a single image. Once the desired object is recognized, the robot reaches for it while ignoring other objects.

ICRA Conference 2003 Conference Paper

Enabling real-time full-body imitation: a natural way of m-ansferring human movement to humanoids

• Marcia Riley
• Ales Ude
• Keegan Wade
• Christopher G. Atkeson

We seek intuitive, efficient ways to create and direct human-like behaviors for humanoid robots. Here we present a method to enable humanoid robots to acquire movements by imitation. The robot uses 3D vision to perceive the movements of a human teacher, and then estimates the teacher's body postures using a fast full-body inverse kinematics method that incorporates a kinematic model of the teacher. This solution is then mapped to the robot and reproduced in real-time. The robustness of the method is tested on a 30-degree-of-freedom Sarcos humanoid robot located at ATR using 3D vision data from external cameras and from head-mounted cameras.

IROS Conference 2003 Conference Paper

Learning to select primitives and generate sub-goals from practice

• Darrin C. Bentivegna
• Christopher G. Atkeson
• Gordon Cheng

This paper focuses on learning to select behavioral primitives and generate sub-goals from practicing a task. We present a novel algorithm that combines Q-learning and a locally weighted learning method to improve primitive selection and sub-goal generation. We demonstrate this approach applied to the tilt maze task. Our robot initially learns to perform this task using learning from observation, and then learns from practice.

IROS Conference 2003 Conference Paper

Minimax differential dynamic programming: application to a biped walking robot

• Jun Morimoto
• Garth Zeglin
• Christopher G. Atkeson

We developed a robust control policy design method in high-dimensional state space by using differential dynamic programming with a minimax criterion. As an example, we applied our method to a simulated five link biped robot. The results show lower joint torques from the optimal control policy compared to a hand-tuned PD servo controller. Results also show that the simulated biped robot can successfully walk with unknown disturbances that cause controllers generated by standard differential dynamic programming and the hand-tuned PD servo to fail. Learning to compensate for modeling error and previously unknown disturbances in conjunction with robust control design is also demonstrated. We also applied proposed method to a real biped robot for optimizing swing leg trajectories.

ICRA Conference 2002 Conference Paper

Adapting Human Motion for the Control of a Humanoid Robot

• Nancy S. Pollard
• Jessica K. Hodgins
• Marcia Riley
• Christopher G. Atkeson

Using the pre-recorded human motion and trajectory tracking, we can control the motion of a humanoid robot for free-space, upper body gestures. However, the number of degrees of freedom, range of joint motion, and achievable joint velocities of today's humanoid robots are far more limited than those of the average human subject. In this paper, we explore a set of techniques for limiting human motion of upper body gestures to that achievable by a Sarcos humanoid robot located at ATR. We assess the quality of the results by comparing the motion of the human actor to that of the robot, both visually and quantitatively.

IROS Conference 2002 Conference Paper

Humanoid robot learning and game playing using PC-based vision

• Darrin C. Bentivegna
• Ales Ude
• Christopher G. Atkeson
• Gordon Cheng

This paper describes humanoid robot learning from observation and game playing using information provided by a real-time PC-based vision system. To cope with extremely fast motions that arise in the environment, a visual system capable of perceiving the motion of several objects at 60 fields per second was developed. We have designed a suitable error recovery scheme for our vision system to ensure successful game playing over longer periods of time. To increase the learning rate of the robot it is given domain knowledge in the form of primitives. The robot learns how to perform primitives from data collected while observing a human. The robot control system and primitive use strategy are also explained.

ICRA Conference 2001 Conference Paper

Learning From Observation Using Primitives

• Darrin C. Bentivegna
• Christopher G. Atkeson

This paper describes the rise of task primitives in robot learning from observation. A framework is developed that uses observed data to initially learn a task and the agent then goes on to increase its performance through repeated task performance (learning from practice). Data that is collected while the human performs a task is parsed into small parts of the task called primitives. Modules are created for each primitive that encode the movements required during the performance of the primitive, and when and where the primitives are performed. The feasibility of this method is currently being tested with agents that learn to play a virtual and an actual air hockey game.

IROS Conference 2001 Conference Paper

Real-time visual system for interaction with a humanoid robot

• Ales Ude
• Christopher G. Atkeson

We describe a real-time visual system that enables a humanoid robot to learn from and interact with humans. The core of the visual system is a probabilistic tracker that uses shape and color information to find relevant objects in the scene. Multiscale representations, windowing and masking are employed to accelerate the data processing. The perception system is directly coupled with the motor control system of our humanoid robot DB. We present an example of on-line interaction with a humanoid robot: mimicking of human hand motion. The generation of humanoid robot motion based on the human motion is accomplished in real-time. The study is supported by experimental results on DB.

ICRA Conference 2000 Conference Paper

Planning of Joint Trajectories for Humanoid Robots Using B-Spline Wavelets

• Ales Ude
• Christopher G. Atkeson
• Marcia Riley

The formulation and optimization of joint trajectories for humanoid robots is quite different from this same task for standard robots because of the complexity of the humanoid robots' kinematics. We exploit the similarity between the movements of a humanoid robot and human movements to generate joint trajectories for such robots. In particular we show how to transform human motion information captured by an optical tracking device into a high dimensional trajectory of a humanoid robot. We utilize B-spline wavelets to efficiently represent the joint trajectories and to automatically select the density of the basis functions on the time axis. We applied our method to the task of teaching a humanoid robot how to make a dance movement.

ICRA Conference 2000 Conference Paper

Real-Time Robot Learning with Locally Weighted Statistical Learning

• Stefan Schaal
• Christopher G. Atkeson
• Sethu Vijayakumar

Locally weighted learning (LWL) is a class of statistical learning techniques that provides useful representations and training algorithms for learning about complex phenomena during autonomous adaptive control of robotic systems. This paper introduces several LWL algorithms that have been tested successfully in real-time learning of complex robot tasks. We discuss two major classes of LWL, memory-based LWL and purely incremental LWL that does not need to remember any data explicitly. In contrast to the traditional beliefs that LWL methods cannot work well in high-dimensional spaces, we provide new algorithms that have been tested in up to 50 dimensional learning problems. The applicability of our LWL algorithms is demonstrated in various robot learning examples, including the learning of devil-sticking, pole-balancing of a humanoid robot arm, and inverse-dynamics learning for a seven degree of-freedom robot.

ICRA Conference 1997 Conference Paper

A comparison of direct and model-based reinforcement learning

• Christopher G. Atkeson
• Juan Carlos Santamaría

This paper compares direct reinforcement learning (no explicit model) and model-based reinforcement learning on a simple task: pendulum swing up. We find that in this task model-based approaches support reinforcement learning from smaller amounts of training data and efficient handling of changing goals.

ICRA Conference 1997 Conference Paper

Learning tasks from a single demonstration

• Christopher G. Atkeson
• Stefan Schaal

Learning a complex dynamic robot manoeuvre from a single human demonstration is difficult. This paper explores an approach to learning from demonstration based on learning an optimization criterion from the demonstration and a task model from repeated attempts to perform the task, and using the learned criterion and model to compute an appropriate robot movement. A preliminary version of the approach has been implemented on an anthropomorphic robot arm using a pendulum swing up task as an example.

ICML Conference 1997 Conference Paper

Robot Learning From Demonstration

• Christopher G. Atkeson
• Stefan Schaal

ICRA Conference 1994 Conference Paper

Memory-Based Robot Learning

• Stefan Schaal
• Christopher G. Atkeson

We present a memory-based local modeling approach to robot learning using a nonparametric regression technique, locally weighted regression. The model of the task to be performed is represented by infinitely many local linear models, the (hyper-) tangent planes at every query point. This is in contrast to other methods using finite set of linear models to accomplish a piecewise linear model. Architectural parameters of our approach, such as distance metrics, are a function of the current query point instead of being global. Statistical tests are presented for when a local model is good enough such that it can be reliably used to build a local controller. These statistical measures also direct the exploration of the robot. We explicitly deal with the case where prediction accuracy requirements exist during exploration: by gradually shifting a center of exploration and controlling the speed of the shift with local prediction accuracy, a goal-directed exploration of state space takes place along the fringes of the current data support until the task goal is achieved. We illustrate this approach by describing how it has been used to enable a robot to learn a juggling task. >

IROS Conference 1994 Conference Paper

Robot learning by nonparametric regression

• Stefan Schaal
• Christopher G. Atkeson

We present an approach to robot learning based on a nonparametric regression technique, locally weighted regression. The model of the task to be performed is represented by infinitely many local linear models, i. e. , the (hyper-) tangent planes at every query point. Such a model, however, is only generated when a query performed and is not retained. The architectural parameters of our approach, such as distance metrics, are also a function of the current query point instead of being global. Statistical tests are presented for when a local model is good enough such that it can be reliably used to build a local controller. These statistical measures also direct the exploration of the robot. We explicitly deal with the case where prediction accuracy requirements exist during exploration. By gradually shifting a center of exploration and controlling the speed of the shift with local prediction accuracy, a goal-directed exploration of state space takes place along the fringes of the current data support until the task goal is achieved. We illustrate this approach by describing how it has been used to enable a robot to learn a challenging juggling task. >

ICRA Conference 1991 Conference Paper

Using locally weighted regression for robot learning

• Christopher G. Atkeson

The use of locally weighted regression in memory-based robot learning is explored. A local model is formed to answer each query, using a weighted regression in which close points (similar experiences) are weighted more than distant points (less relevant experiences). This approach implements a philosophy of modeling a complex function with many simple local models. The author explains how an appropriate distance metric or measure of similarity can be found, and how the distance metric is used. How irrelevant input variables and terms in the local model are detected is also explained. An example from the control of a robot arm is used to compare this approach with other robot control and learning techniques. >

ICRA Conference 1989 Conference Paper

Task-level robot learning: juggling a tennis ball more accurately

• Eric W. Aboaf
• Steven Mark Drucker
• Christopher G. Atkeson

Results are presented from a preliminary investigation of task-level learning, an approach to learning from practice. The authors programmed a robot to juggle a single ball in three dimensions by batting it upwards with a large paddle. The robot uses a real-time binary vision system to track the ball and measure its performance. Task-level learning consists of building a model of performance errors at the task level during practice, and using that model to refine task-level commands. A polynomial surface was fitted to the errors in the path which the ball took after each hit, and this task model is used to refine how the ball is hit. This application of task-level learning dramatically increased the number of consecutive hits the robot could execute before the ball was hit out of range of the paddle. >

ICRA Conference 1989 Conference Paper

Using associative content-addressable memories to control robots

• Christopher G. Atkeson
• David J. Reinkensmeyer

The use of an associative content-addressable memory to model a robot and the world the robot interacts with is discussed. The model can be learned by storing experiences in the memory. To make predictions, the memory is searched for relevant experience. An initial implementation of such a memory-based modeling scheme has been made on a parallel computer, the Connection Machine. The implementation machine was used to model and control a simulated planar two-joint arm and a simulated running machine. The issues and problems that arose in the preliminary work are described. It is found that the use of parallel search in the implementation of an associative content-addressable memory allows quick searching of stored experiences, and reasonable retrieval is obtained using a simple distance metric and a simple generalized scheme. The memory is able to generalize after storing only a small number of relevant experiences. The use of search by parallel processors also avoids many of the problems of previous memory-based or tubular approaches to robot modeling (such as search speed and memory requirements). >

ICRA Conference 1988 Conference Paper

Model-based control of a direct drive arm. I. Building models

• Chae H. An
• Christopher G. Atkeson
• John M. Hollerbach

Work on building robot models to be used in designing model-based controllers is described. Various algorithms are presented estimating the kinematic, link-inertial, and load-inertial parameters. It is shown experimentally that accurate estimates can be obtained automatically using sensor data taken during movement. The algorithms have been implemented on the MIT Serial Link Direct Drive Arm. >

ICRA Conference 1988 Conference Paper

Model-based control of a direct drive arm. II. Control

• Chae H. An
• Christopher G. Atkeson
• John M. Hollerbach

For pt. I see ibid. , p. 1374-9 (1988). Work on model-based control with the MIT Serial Link Direct Drive Arm is described. It is shown that model-based control leads to performance superior to control not based on carefully constructed models. Trajectory control, trajectory learning, and force control are treated. A new type of trajectory learning is considered in which a robot fine-tunes one particular trajectory through repetition. Various experiments with the direct drive arm are reported to validate the importance of model-based control. >

ICRA Conference 1988 Conference Paper

Task-level robot learning

• Eric W. Aboaf
• Christopher G. Atkeson
• David J. Reinkensmeyer

The functionality of robots can be improved by programming them to learn tasks from practice. Task-level learning can compensate for the structural modeling errors of the robot's lower-level control systems and can speed up the learning process by reducing the degrees of freedom of the models to be learned. The authors demonstrate two general learning procedures-fixed-model learning and refined-model learning-on a ball-throwing robot system. Both learning approaches refine the task command based on the performance error of the system, while they ignore the intermediate variables separation the lower-level systems. The authors also provide experimental and theoretical evidence that task-level learning can improve the functionality of robots. >

ICRA Conference 1987 Conference Paper

Experimental evaluation of feedforward and computed torque control

• Chae H. An
• Christopher G. Atkeson
• John D. Griffiths
• John M. Hollerbach

Trajectory tracking errors resulting from the application of various controllers have been experimentally determined on the MIT Serial Link Direct Drive Arm. The controllers range from simple analog PD control applied independently at each joint to feedforward and computed torque methods incorporating full dynamics. It was found that trajectory tracking errors decreased as more dynamic compensation terms were incorporated. There was no significant difference in trajectory tracking performance between the feedforward controller using independent digital servos and the full computed torque controller.

ICRA Conference 1986 Conference Paper

Experimental determination of the effect of feedforward control on trajectory tracking errors

• Chae H. An
• Christopher G. Atkeson
• John M. Hollerbach

Trajectory tracking errors resulting from the application of various controllers have been experimentally determined on the MIT Serial Link Direct Drive Arm. The controllers range from simple PD control applied independently at each joint to feedforward control incorporating full dynamics followed by a separate PD control loop. It was found that trajectory tracking errors decreased as more feedforward terms were incorporated.

ICRA Conference 1986 Conference Paper

Robot trajectory learning through practice

• Christopher G. Atkeson
• Joe McIntyre

We present an algorithm that uses trajectory following errors to improve a feedforward command to a robot. This approach to robot learning is based on explicit modeling of the robot; and uses an inverse of the robot model as part of a learning operator which processes the trajectory errors. Results are presented from a successful implementation of this procedure on the MIT Serial Link Direct Drive Arm. The major point of this paper is that more accurate robot models improve trajectory learning performance, and learning algorithms do not reduce the need for good models in robot control.