Harmonic Mobile Manipulation

Recent advancements in robotics have enabled robots to navigate complex scenes or manipulate diverse objects independently. However, robots are still impotent in many household tasks requiring coordinated behaviors such as opening doors. The factorization of navigation and manipulation, while effective for some tasks, fails in scenarios requiring coordinated actions. To address this challenge, we introduce, HarmonicMM, an end-to-end learning method that optimizes both navigation and manipulation, showing notable improvement over existing techniques in everyday tasks. This approach is validated in simulated and real-world environments and adapts to novel unseen settings without additional tuning. Our contributions include a new benchmark for mobile manipulation and the successful deployment with only RGB visual observation in a real unseen apartment, demonstrating the potential for practical indoor robot deployment in daily life.

Authors

• Ruihan Yang
• Yejin Kim 0003
• Rose Hendrix
• Aniruddha Kembhavi
• Xiaolong Wang 0004
• Kiana Ehsani

Keywords

• Learning systems
• Visualization
• Navigation
• Robot kinematics
• Pipelines
• Layout
• Kinematics
• Benchmark testing
• Robots
• Tuning
• Mobile Manipulator
• Benchmark
• Apartment
• Simulation Environment
• Visual Observation
• Open Door
• Everyday Tasks
• Advanced Robotics
• Semantic
• Task Completion
• Heuristic Algorithm
• Target Object
• Manipulation Tasks
• Arm Movements
• Reward Function
• Daily Tasks
• Robot Manipulator
• RGB Camera
• Robot Navigation
• Average Success Rate
• Navigation Skills
• Start Of Episode
• Mobility Tasks
• Successful Task Completion
• Task Progress