CloudTrack: Scalable UAV Tracking with Cloud Semantics

Nowadays, unmanned aerial vehicles (UAVs) are commonly used in search and rescue scenarios to gather information in the search area. The automatic identification of the person searched for in aerial footage could increase the autonomy of such systems, reduce the search time, and thus increase the missed person's chances of survival. In this paper, we present a novel approach to perform semantically conditioned open vocabulary object tracking that is specifically designed to cope with the limitations of UAV hardware. Our approach has several advantages: It can run with verbal descriptions of the missing person, e. g. , the color of the shirt, it does not require dedicated training to execute the mission, and can efficiently track a potentially moving person. Our experimental results demonstrate the versatility and efficacy of our approach. We publish the methods source code at https://github.com/utn-blei/CloudTrack.

Authors

• Yannik Blei
• Michael Krawez
• Nisarga Nilavadi
• Tanja Katharina Kaiser
• Wolfram Burgard

Keywords

• Training
• Vocabulary
• Source coding
• Semantics
• Color
• Autonomous aerial vehicles
• Hardware
• Object tracking
• Robotics and automation
• Unmanned Aerial Vehicles
• Unmanned Aerial Vehicle Tracking
• Benchmark
• Convolutional Neural Network
• Object Detection
• Frame Rate
• Semantic Information
• Part Of Work
• Bounding Box
• Object Of Interest
• Target Object
• Multiple Frames
• Raspberry Pi
• Tracking Approach
• Foundation Model
• Hardware Setup
• Semantic Understanding
• Multiple Unmanned Aerial Vehicles
• Injured Person