Reinforcement Learning for Network Offloading in Cloud Robotics

We apply deep reinforcement learning to a central decision-making problem in robotics - when should a robot use its on-board compute model or, in cases of local uncertainty, query a compute-intensive model in “the cloud”? Today’s robotic systems are increasingly turning to computationally expensive mod- els such as deep neural networks (DNNs) for tasks like object detection, perception and planning. However, resource-constrained robots, like low-power drones, often have insufficient on-board compute resources or power reserves to scalably run the most accurate, state-of-the art neural network compute models. Cloud robotics allows mobile robots to offload compute to centralized servers if they are uncertain locally or want to run more accurate, compute-intensive models. However, cloud robotics comes with a key, often un- derstated cost: communicating with the cloud over congested wireless networks may result in latency and increase network congestion. In fact, sending high data-rate video or LIDAR from multiple robots over congested networks can lead to prohibitive delay for real-time applications. We formulate a novel Robot Offloading Problem — how and when should robots offload sensing tasks, especially if they are uncertain, to improve accuracy while minimizing the cost of cloud communication? We formulate offloading as a sequential decision making problem for robots, and propose a solution using deep reinforcement learning. In both simulations and practical hardware experiments using state-of-the art vision DNNs, our offloading strategy improves vision task performance by between 1. 3-2. 6x of benchmark offloading strategies. We conclude by showing how cloud offloading has an inherent exploration vs. exploitation trade-off since a robot must balance use of a known local model (exploitation) with learning context-dependent utility of the cloud (exploration). Accordingly, we discuss how our model is widely applicable beyond cloud robotics.

Authors

• Sandeep Chinchali The University of Texas at Austin
• Apoorva Sharma
• Amine Elhafsi
• Daniel Kang
• Evgenya Perga-
• Eyal Cidon
• Sachin Katti

Keywords

No keywords are indexed for this paper.