Cooperative coaching in robot learning

Many closed loop learning algorithms perform gradient descent on a cost function with respect to the parameters of a learning controller. The authors observe that both local closed loop learners, which consider only the cost of the current time step, and optimal control based closed loop learners, which consider the future effects of control actions, can become stuck in sub-optimal local minima in the cost function. The authors propose the use of "cooperating coaches" to deal with this problem. Each coach attempts gradient descent based on its own cost function and they work together to avoid getting stuck in local minima. When one coach has achieved the best result it can (the gradient for its cost function is zero), another coach takes over to guide the search through the parameter space. The authors demonstrate cooperative coaching on the problem of curve tracking with an inverted pendulum and show that it yields faster, smoother tracking of target curves by combining the best aspects of two different coaches.

Authors

• Jeff G. Schneider
• Christopher M. Brown

Keywords

• Cost function
• Training data
• Control systems
• Error correction
• Optimal control
• Target tracking
• Tracking loops
• Robot control
• Orbital robotics
• Three-term control
• Coaching
• Robot Learning
• Learning Algorithms
• Gradient Descent
• Control Parameters
• Local Minima
• Learning Control
• Important Problem
• Performance Metrics
• Horizontal Axis
• Control Signal
• Points In Space
• Equation Of State
• Friction Coefficient
• Control Efforts
• Selection Algorithm
• Function Approximation
• Reference Curve
• Part Of The Task
• Learning Rule
• Tracking Speed
• Inverse Dynamics
• Running Trials
• Actual Output
• Reference Signal