Multi-Floor Zero-Shot Object Navigation Policy

Object navigation in multi-floor environments presents a formidable challenge in robotics, requiring sophisticated spatial reasoning and adaptive exploration strategies. Traditional approaches have primarily focused on single-floor scenarios, overlooking the complexities introduced by multi-floor structures. To address these challenges, we first propose a Multi-floor Navigation Policy (MFNP) and implement it in Zero-Shot object navigation tasks. Our framework comprises three key components: (i) Multi-floor Navigation Policy, which enables an agent to explore across multiple floors; (ii) Multi-modal Large Language Models (MLLMs) for reasoning in the navigation process; and (iii) Inter-Floor Navigation, ensuring efficient floor transitions. We evaluate MFNP on the Habitat-Matterport 3D (HM3D) and Matterport 3D (MP3D) datasets, both include multi-floor scenes. Our experiment results demonstrate that MFNP significantly outperforms all the existing methods in Zero-Shot object navigation, achieving higher success rates and improved exploration efficiency. Ablation studies further highlight the effectiveness of each component in addressing the unique challenges of multi-floor navigation. Meanwhile, we conducted real-world experiments to evaluate the feasibility of our policy. Upon deployment of MFNP, the Unitree quadruped robot demonstrated successful multi-floor navigation and found the target object in a completely unseen environment. By introducing MFNP, we offer a new paradigm for tackling complex, multi-floor environments in object navigation tasks, opening avenues for future research in vision-based navigation in realistic, multi-floor settings.

Authors

• Lingfeng Zhang
• Hao Wang
• Erjia Xiao
• Xinyao Zhang
• Qiang Zhang 0029
• Zixuan Jiang
• Renjing Xu

Keywords

• Three-dimensional displays
• Navigation
• Large language models
• Imitation learning
• Reinforcement learning
• Cognition
• Complexity theory
• Quadrupedal robots
• Floors
• Periodic structures
• Navigation Policy
• Object Navigation
• Target Object
• Navigation Task
• Multimodal Model
• Navigation In Environments
• Robotics Challenge
• Time Step
• Point Cloud
• Confidence Score
• Indoor Environments
• Capability Of Model
• Semantic Segmentation
• Path Planning
• Machine Vision
• Semantic Map
• Candidate Points
• RGB-D Images
• Beginning Of Episode
• Decisions Of Agents
• Navigation Problem
• Pose Information
• Navigation Path