DiffVAS: Diffusion-Guided Visual Active Search in Partially Observable Environments

Visualactivesearch(VAS)hasbeenintroducedasamodelingframework that leverages visual cues to direct aerial (e. g. , UAV-based) exploration and pinpoint areas of interest within extensive geospatialregions. PotentialapplicationsofVASincludedetectinghotspots for rare wildlife poaching, aiding search-and-rescue missions, and uncovering illegal trafficking of weapons, among other uses. Previous VAS approaches assume that the entire search space is known upfront, which is often unrealistic due to constraints such as a restricted field of view and high acquisition costs, and they typically learn policies tailored to specific target objects, which limits their ability to search for multiple target categories simultaneously. In this work, we propose DiffVAS, a target-conditioned policy that searches for diverse objects simultaneously according to task requirements in partially observable environments, which advances the deployment of visual active search policies in real-world applications. DiffVAS leverages a diffusion model to reconstruct the entire geospatial area from sequentially observed partial glimpses, which enables a target-conditioned reinforcement learning-based planning module to effectively reason and guide subsequent search steps. Extensive experiments demonstrate that DiffVAS excels in searching diverse objects in partially observable environments, significantly surpassing state-of-the-art methods on several datasets. Code and models are available at this link.

Authors

• Anindya Sarkar Washington University in Saint Louis
• Srikumar Sastry Washington University in Saint Louis
• Aleksis Pirinen RISE Research Institutes of Sweden
• Nathan Jacobs Washington University in Saint Louis
• Yevgeniy Vorobeychik Washington University, St. Louis

Keywords

• Visual Active Search
• Geospatial
• UAV