Learning where to search using visual attention

One of the central tasks for a household robot is searching for specific objects. It does not only require localizing the target object but also identifying promising search locations in the scene if the target is not immediately visible. As computation time and hardware resources are usually limited in robotics, it is desirable to avoid expensive visual processing steps that are exhaustively applied over the entire image. The human visual system can quickly select those image locations that have to be processed in detail for a given task. This allows us to cope with huge amounts of information and to efficiently deploy the limited capacities of our visual system. In this paper, we therefore propose to use human fixation data to train a top-down saliency model that predicts relevant image locations when searching for specific objects. We show that the learned model can successfully prune bounding box proposals without rejecting the ground truth object locations. In this aspect, the proposed model outperforms a model that is trained only on the ground truth segmentations of the target object instead of fixation data.

Authors

• Alina Kloss
• Daniel Kappler
• Hendrik P. A. Lensch
• Martin V. Butz
• Stefan Schaal
• Jeannette Bohg

Keywords

• Predictive models
• Computational modeling
• Data models
• Search problems
• Training
• Feature extraction
• Proposals
• Visual Attention
• Visual Processing
• Top-down Approach
• Bounding Box
• Target Object
• Human Visual System
• Huge Amount Of Information
• Saliency Models
• Low Resolution
• Convolutional Neural Network
• Feature Maps
• Object Detection
• Image Regions
• Data Augmentation
• Intersection Over Union
• Final Prediction
• Semantic Segmentation
• Object Of Interest
• Area Under Curve
• Target Search
• Saliency Map
• Bottom-up Model
• Search Task
• Region Proposal
• Fixation Patterns
• Promising Regions
• COCO Dataset
• Target Category
• Correlation Score
• High Values