Reinforcement learning for sensing strategies

Since sensors have limited range and coverage, mobile robots often have to make decisions on where to point their sensors. A good sensing strategy allows a robot to collect information that is useful for its tasks. Most existing solutions to this active sensing problem choose the direction that maximally reduces the uncertainty in a single state variable. In more complex problem domains, however, uncertainties exist in multiple state variables, and they affect the performance of the robot in different ways. The robot thus needs to have more sophisticated sensing strategies in order to decide which uncertainties to reduce, and to make the correct trade-offs. In this work, we apply a least squares reinforcement learning method to solve this problem. We implemented and tested the learning approach in the RoboCup domain, where the robot attempts to reach a ball and accurately kick it into the goal. We present experimental results that suggest our approach is able to learn highly effective sensing strategies.

Authors

• Cody C. T. Kwok
• Dieter Fox

Keywords

• Learning
• Robot sensing systems
• Uncertainty
• Orbital robotics
• State estimation
• State-space methods
• Least squares methods
• Testing
• Robot vision systems
• Cameras
• State Variables
• Linear Function
• State Space
• Simulation Experiments
• Linear Approximation
• Optimal Policy
• Thick Line
• Thin Line
• Particle Filter
• Markov Decision Process
• Policy Learning
• Robot Motion
• Robot Model
• Uncertainty Variables
• Sensor Information
• Real Robot
• Model-free Approach
• Model-free Reinforcement Learning
• Legged Robots
• Ball Position
• Goal Scoring
• State-space Model
• Policy Improvement
• Kalman Filter
• Optimal Control Policy