Author name cluster

Aravindh Mahendran

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

9 papers

2 author rows

ICML Conference 2023 Conference Paper

Invariant Slot Attention: Object Discovery with Slot-Centric Reference Frames

Ondrej Biza
Sjoerd van Steenkiste
Mehdi S. M. Sajjadi
Gamaleldin Fathy Elsayed
Aravindh Mahendran
Thomas Kipf

Automatically discovering composable abstractions from raw perceptual data is a long-standing challenge in machine learning. Recent slot-based neural networks that learn about objects in a self-supervised manner have made exciting progress in this direction. However, they typically fall short at adequately capturing spatial symmetries present in the visual world, which leads to sample inefficiency, such as when entangling object appearance and pose. In this paper, we present a simple yet highly effective method for incorporating spatial symmetries via slot-centric reference frames. We incorporate equivariance to per-object pose transformations into the attention and generation mechanism of Slot Attention by translating, scaling, and rotating position encodings. These changes result in little computational overhead, are easy to implement, and can result in large gains in terms of data efficiency and overall improvements to object discovery. We evaluate our method on a wide range of synthetic object discovery benchmarks namely CLEVR, Tetrominoes, CLEVRTex, Objects Room and MultiShapeNet, and show promising improvements on the challenging real-world Waymo Open dataset.

Details

ICLR Conference 2023 Conference Paper

Iterative Patch Selection for High-Resolution Image Recognition

Benjamin Bergner
Christoph Lippert
Aravindh Mahendran

High-resolution images are prevalent in various applications, such as autonomous driving and computer-aided diagnosis. However, training neural networks on such images is computationally challenging and easily leads to out-of-memory errors even on modern GPUs. We propose a simple method, Iterative Patch Selection (IPS), which decouples the memory usage from the input size and thus enables the processing of arbitrarily large images under tight hardware constraints. IPS achieves this by selecting only the most salient patches, which are then aggregated into a global representation for image recognition. For both patch selection and aggregation, a cross-attention based transformer is introduced, which exhibits a close connection to Multiple Instance Learning. Our method demonstrates strong performance and has wide applicability across different domains, training regimes and image sizes while using minimal accelerator memory. For example, we are able to finetune our model on whole-slide images consisting of up to 250k patches (>16 gigapixels) with only 5 GB of GPU VRAM at a batch size of 16.

Details

ICML Conference 2023 Conference Paper

Scaling Vision Transformers to 22 Billion Parameters

Mostafa Dehghani 0001
Josip Djolonga
Basil Mustafa
Piotr Padlewski
Jonathan Heek
Justin Gilmer
Andreas Peter Steiner
Mathilde Caron

The scaling of Transformers has driven breakthrough capabilities for language models. At present, the largest large language models (LLMs) contain upwards of 100B parameters. Vision Transformers (ViT) have introduced the same architecture to image and video modelling, but these have not yet been successfully scaled to nearly the same degree; the largest dense ViT contains 4B parameters (Chen et al. , 2022). We present a recipe for highly efficient and stable training of a 22B-parameter ViT (ViT-22B) and perform a wide variety of experiments on the resulting model. When evaluated on downstream tasks (often with a lightweight linear model on frozen features), ViT-22B demonstrates increasing performance with scale. We further observe other interesting benefits of scale, including an improved tradeoff between fairness and performance, state-of-the-art alignment to human visual perception in terms of shape/texture bias, and improved robustness. ViT-22B demonstrates the potential for "LLM-like" scaling in vision, and provides key steps towards getting there.

Details

ICLR Conference 2022 Conference Paper

Conditional Object-Centric Learning from Video

Thomas Kipf
Gamaleldin Fathy Elsayed
Aravindh Mahendran
Austin Stone
Sara Sabour
Georg Heigold
Rico Jonschkowski
Alexey Dosovitskiy

Object-centric representations are a promising path toward more systematic generalization by providing flexible abstractions upon which compositional world models can be built. Recent work on simple 2D and 3D datasets has shown that models with object-centric inductive biases can learn to segment and represent meaningful objects from the statistical structure of the data alone without the need for any supervision. However, such fully-unsupervised methods still fail to scale to diverse realistic data, despite the use of increasingly complex inductive biases such as priors for the size of objects or the 3D geometry of the scene. In this paper, we instead take a weakly-supervised approach and focus on how 1) using the temporal dynamics of video data in the form of optical flow and 2) conditioning the model on simple object location cues can be used to enable segmenting and tracking objects in significantly more realistic synthetic data. We introduce a sequential extension to Slot Attention which we train to predict optical flow for realistic looking synthetic scenes and show that conditioning the initial state of this model on a small set of hints, such as center of mass of objects in the first frame, is sufficient to significantly improve instance segmentation. These benefits generalize beyond the training distribution to novel objects, novel backgrounds, and to longer video sequences. We also find that such initial-state-conditioning can be used during inference as a flexible interface to query the model for specific objects or parts of objects, which could pave the way for a range of weakly-supervised approaches and allow more effective interaction with trained models.

Details

NeurIPS Conference 2022 Conference Paper

Object Scene Representation Transformer

Mehdi S. M. Sajjadi
Daniel Duckworth
Aravindh Mahendran
Sjoerd van Steenkiste
Filip Pavetic
Mario Lucic
Leonidas J. Guibas
Klaus Greff

A compositional understanding of the world in terms of objects and their geometry in 3D space is considered a cornerstone of human cognition. Facilitating the learning of such a representation in neural networks holds promise for substantially improving labeled data efficiency. As a key step in this direction, we make progress on the problem of learning 3D-consistent decompositions of complex scenes into individual objects in an unsupervised fashion. We introduce Object Scene Representation Transformer (OSRT), a 3D-centric model in which individual object representations naturally emerge through novel view synthesis. OSRT scales to significantly more complex scenes with larger diversity of objects and backgrounds than existing methods. At the same time, it is multiple orders of magnitude faster at compositional rendering thanks to its light field parametrization and the novel Slot Mixer decoder. We believe this work will not only accelerate future architecture exploration and scaling efforts, but it will also serve as a useful tool for both object-centric as well as neural scene representation learning communities.

PDF Details

NeurIPS Conference 2022 Conference Paper

SAVi++: Towards End-to-End Object-Centric Learning from Real-World Videos

Gamaleldin Elsayed
Aravindh Mahendran
Sjoerd van Steenkiste
Klaus Greff
Michael C. Mozer
Thomas Kipf

The visual world can be parsimoniously characterized in terms of distinct entities with sparse interactions. Discovering this compositional structure in dynamic visual scenes has proven challenging for end-to-end computer vision approaches unless explicit instance-level supervision is provided. Slot-based models leveraging motion cues have recently shown great promise in learning to represent, segment, and track objects without direct supervision, but they still fail to scale to complex real-world multi-object videos. In an effort to bridge this gap, we take inspiration from human development and hypothesize that information about scene geometry in the form of depth signals can facilitate object-centric learning. We introduce SAVi++, an object-centric video model which is trained to predict depth signals from a slot-based video representation. By further leveraging best practices for model scaling, we are able to train SAVi++ to segment complex dynamic scenes recorded with moving cameras, containing both static and moving objects of diverse appearance on naturalistic backgrounds, without the need for segmentation supervision. Finally, we demonstrate that by using sparse depth signals obtained from LiDAR, SAVi++ is able to learn emergent object segmentation and tracking from videos in the real-world Waymo Open dataset.

PDF Details

NeurIPS Conference 2020 Conference Paper

Object-Centric Learning with Slot Attention

Francesco Locatello
Dirk Weissenborn
Thomas Unterthiner
Aravindh Mahendran
Georg Heigold
Jakob Uszkoreit
Alexey Dosovitskiy
Thomas Kipf

Learning object-centric representations of complex scenes is a promising step towards enabling efficient abstract reasoning from low-level perceptual features. Yet, most deep learning approaches learn distributed representations that do not capture the compositional properties of natural scenes. In this paper, we present the Slot Attention module, an architectural component that interfaces with perceptual representations such as the output of a convolutional neural network and produces a set of task-dependent abstract representations which we call slots. These slots are exchangeable and can bind to any object in the input by specializing through a competitive procedure over multiple rounds of attention. We empirically demonstrate that Slot Attention can extract object-centric representations that enable generalization to unseen compositions when trained on unsupervised object discovery and supervised property prediction tasks.

PDF Details

IROS Conference 2013 Conference Paper

Heterogeneous UGV-MAV exploration using integer programming

Ayush Dewan
Aravindh Mahendran
Nikhil Soni
K. Madhava Krishna

This paper presents a novel exploration strategy for coordinated exploration between unmanned ground vehicles (UGV) and micro-air vehicles (MAV). The exploration is modeled as an Integer Programming (IP) optimization problem and the allocation of the vehicles(agents) to frontier locations is modeled using binary variables. The formulation is also studied for distributed system, where agents are divided into multiple teams using graph partitioning. Optimization seamlessly integrates several practical constraints that arise in exploration between such heterogeneous agents and provides an elegant solution for assigning task to agents. We have also presented comparison with previous methods based on distance traversed and computational time to signify advantages of presented method. We also show practical realization of such an exploration where an UGV-MAV team efficiently builds a map of an indoor environment.

Details

AAMAS Conference 2013 Conference Paper

Optimization Based Coordinated UGV-MAV Exploration for 2D Augmented Mapping

Ayush Dewan
Aravindh Mahendran
Nikhil Soni
Madhava Krishna

This paper presents a novel optimization formulation for coordinated exploration between unmanned ground vehicles (UGV) and micro-aerial vehicles (MAV). The exploration is posed as an Integer Programming (IP) problem and the allotment of these vehicles(agents) to frontier locations is speciﬁed as an integer constraint. The optimization provides a one shot solution for the allotment of all such active agents to possible frontier locations thereby guaranteeing substantial performance gain over previous approaches where the allotment proceeds in an incremental fashion. We also show a practical realization of such an exploration where an UGV- MAV team eﬃciently builds a map of an indoor environment.

PDF