ViewActive: Active viewpoint optimization from a single image

When observing objects, humans benefit from their spatial visualization and mental rotation ability to envision potential optimal viewpoints based on the current observation. This capability is crucial for enabling robots to achieve efficient and robust scene perception during operation, as optimal viewpoints provide essential and informative features for accurately representing scenes in 2D images, thereby enhancing downstream tasks. To endow robots with this human-like active viewpoint optimization capability, we propose ViewActive, a modernized machine learning approach drawing inspiration from aspect graph, which provides viewpoint optimization guidance based solely on the current 2D image input. Specifically, we introduce the 3D Viewpoint Quality Field (VQF), a compact and consistent representation for viewpoint quality distribution similar to an aspect graph, composed of three general-purpose viewpoint quality metrics: self-occlusion ratio, occupancy-aware surface normal entropy, and visual entropy. We utilize pre-trained image encoders to extract robust visual and semantic features, which are then decoded into the 3D VQF, allowing our model to generalize effectively across diverse objects, including unseen categories. The lightweight ViewActive network (72 FPS on a single GPU) significantly enhances the performance of state-of-the-art object recognition pipelines and can be integrated into real-time motion planning for robotic applications. Our code and dataset are available here https://github.com/jiayi-wu-umd/ViewActive.

Authors

• Jiayi Wu
• Xiaomin Lin 0002
• Botao He
• Cornelia Fermüller
• Yiannis Aloimonos

Keywords

• Measurement
• Visualization
• Solid modeling
• Three-dimensional displays
• Semantics
• Feature extraction
• Entropy
• Object recognition
• Robots
• Optimization
• Single Image
• Viewpoint Optimization
• Machine Learning
• Visual Features
• 2D Images
• Quality Metrics
• Path Planning
• Semantic Features
• Variety Of Objects
• Mental Rotation
• 3D Field
• Extract Semantic Features
• Training Set
• Amount Of Information
• Normal Vector
• Geometric Features
• Total Surface Area
• Object Features
• Azimuth Angle
• Unseen Classes
• Human Pose Estimation
• Visible Surface
• Polar Angle
• Single Observation
• Pose Estimation
• 3D Pose
• Machine Learning Paradigm
• Self-supervised Learning
• Object Recognition Task