Author name cluster

Rico Jonschkowski

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

6 papers

2 author rows

ICLR Conference 2022 Conference Paper

Conditional Object-Centric Learning from Video

Thomas Kipf
Gamaleldin Fathy Elsayed
Aravindh Mahendran
Austin Stone
Sara Sabour
Georg Heigold
Rico Jonschkowski
Alexey Dosovitskiy

Object-centric representations are a promising path toward more systematic generalization by providing flexible abstractions upon which compositional world models can be built. Recent work on simple 2D and 3D datasets has shown that models with object-centric inductive biases can learn to segment and represent meaningful objects from the statistical structure of the data alone without the need for any supervision. However, such fully-unsupervised methods still fail to scale to diverse realistic data, despite the use of increasingly complex inductive biases such as priors for the size of objects or the 3D geometry of the scene. In this paper, we instead take a weakly-supervised approach and focus on how 1) using the temporal dynamics of video data in the form of optical flow and 2) conditioning the model on simple object location cues can be used to enable segmenting and tracking objects in significantly more realistic synthetic data. We introduce a sequential extension to Slot Attention which we train to predict optical flow for realistic looking synthetic scenes and show that conditioning the initial state of this model on a small set of hints, such as center of mass of objects in the first frame, is sufficient to significantly improve instance segmentation. These benefits generalize beyond the training distribution to novel objects, novel backgrounds, and to longer video sequences. We also find that such initial-state-conditioning can be used during inference as a flexible interface to query the model for specific objects or parts of objects, which could pave the way for a range of weakly-supervised approaches and allow more effective interaction with trained models.

Details

ICRA Conference 2020 Conference Paper

Differentiable Mapping Networks: Learning Structured Map Representations for Sparse Visual Localization

Péter Karkus
Anelia Angelova
Vincent Vanhoucke
Rico Jonschkowski

Mapping and localization, preferably from a small number of observations, are fundamental tasks in robotics. We address these tasks by combining spatial structure (differentiable mapping) and end-to-end learning in a novel neural network architecture: the Differentiable Mapping Network (DMN). The DMN constructs a spatially structured view-embedding map and uses it for subsequent visual localization with a particle filter. Since the DMN architecture is end-to-end differentiable, we can jointly learn the map representation and localization using gradient descent. We apply the DMN to sparse visual localization, where a robot needs to localize in a new environment with respect to a small number of images from known viewpoints. We evaluate the DMN using simulated environments and a challenging real-world Street View dataset. We find that the DMN learns effective map representations for visual localization. The benefit of spatial structure increases with larger environments, more viewpoints for mapping, and when training data is scarce. Project website: https://sites.google.com/view/differentiable-mapping.

Details

IROS Conference 2019 Conference Paper

State Representation Learning with Robotic Priors for Partially Observable Environments

Marco Morik
Divyam Rastogi
Rico Jonschkowski
Oliver Brock

We introduce Recurrent State Representation Learning (RSRL) to tackle the problem of state representation learning in robotics for partially observable environments. To learn low-dimensional state representations, we combine a Long Short Term Memory network with robotic priors. RSRL introduces new priors with landmarks and combines them with existing robotics priors from the literature to train the representations. To evaluate the quality of the learned state representation, we introduce validation networks that help us better visualize and quantitatively analyze the learned state representations. We show that the learned representations are low-dimensional, locally consistent, and can approximate the underlying true state for robot localization in simulated 3D maze environments. We use the learned representations for reinforcement learning and show that we achieve similar performance as training with the true state. The learned representations are robust to landmark misclassification errors.

Details

IJCAI Conference 2017 Conference Paper

Lessons from the Amazon Picking Challenge: Four Aspects of Building Robotic Systems

Clemens Eppner
Sebastian Höfer
Rico Jonschkowski
Roberto Martín-Martín
Arne Sieverling
Vincent Wall
Oliver Brock

We describe the winning entry to the Amazon Picking Challenge 2015. From the experience of building this system and competing, we derive several conclusions: (1) We suggest to characterize robotic system building along four key aspects, each of them spanning a spectrum of solutions - modularity vs. integration, generality vs. assumptions, computation vs. embodiment, and planning vs. feedback. (2) To understand which region of each spectrum most adequately addresses which robotic problem, we must explore the full spectrum of possible approaches. (3) For manipulation problems in unstructured environments, certain regions of each spectrum match the problem most adequately, and should be exploited further. This is supported by the fact that our solution deviated from the majority of the other challenge entries along each of the spectra. This is an abridged version of a conference publication.

PDF Details

IROS Conference 2016 Conference Paper

Probabilistic multi-class segmentation for the Amazon Picking Challenge

Rico Jonschkowski
Clemens Eppner
Sebastian Höfer
Roberto Martín-Martín
Oliver Brock

We present a method for multi-class segmentation from RGB-D data in a realistic warehouse picking setting. The method computes pixel-wise probabilities and combines them to find a coherent object segmentation. It reliably segments objects in cluttered scenarios, even when objects are translucent, reflective, highly deformable, have fuzzy surfaces, or consist of loosely coupled components. The robust performance results from the exploitation of problem structure inherent to the warehouse setting. The proposed method proved its capabilities as part of our winning entry to the 2015 Amazon Picking Challenge. We present a detailed experimental analysis of the contribution of different information sources, compare our method to standard segmentation techniques, and assess possible extensions that further enhance the algorithm's capabilities. We release our software and data sets as open source.

Details

EWRL Workshop 2013 Workshop Paper

Temporal Abstraction in Reinforcement Learning with Proximity Statistics

Rico Jonschkowski
Marc Toussaint

Automatic discovery of temporal abstractions is a key problem in hierarchical reinforcement learning. We propose a new approach to this problem by learning a time marginalized transition probability which we call proximity statistics from successful trajectories for related tasks. Viewing the proximity statistics as state values allows the agent to generate greedy policies from them. Making the statistics sparse and combining proximity estimates by proximity propagation can substantially accelerate planning compared to value iteration while keeping the size of the statistics manageable.

PDF Details