Michael Spranger

ICLR Conference 2023 Conference Paper

MECTA: Memory-Economic Continual Test-Time Model Adaptation

• Junyuan Hong
• Lingjuan Lyu
• Jiayu Zhou
• Michael Spranger

Continual Test-time Adaptation (CTA) is a promising art to secure accuracy gains in continually-changing environments. The state-of-the-art adaptations improve out-of-distribution model accuracy via computation-efficient online test-time gradient descents but meanwhile cost about times of memory versus the inference, even if only a small portion of parameters are updated. Such high memory consumption of CTA substantially impedes wide applications of advanced CTA on memory-constrained devices. In this paper, we provide a novel solution, dubbed MECTA, to drastically improve the memory efficiency of gradient-based CTA. Our profiling shows that the major memory overhead comes from the intermediate cache for back-propagation, which scales by the batch size, channel, and layer number. Therefore, we propose to reduce batch sizes, adopt an adaptive normalization layer to maintain stable and accurate predictions, and stop the back-propagation caching heuristically. On the other hand, we prune the networks to reduce the computation and memory overheads in optimization and recover the parameters afterward to avoid forgetting. The proposed MECTA is efficient and can be seamlessly plugged into state-of-the-art CTA algorithms at negligible overhead on computation and memory. On three datasets, CIFAR10, CIFAR100, and ImageNet, MECTA improves the accuracy by at least 6% with constrained memory and significantly reduces the memory costs of ResNet50 on ImageNet by at least 70% with comparable accuracy. Our codes can be accessed at https://github.com/SonyAI/MECTA.

ICLR Conference 2023 Conference Paper

MocoSFL: enabling cross-client collaborative self-supervised learning

• Jingtao Li
• Lingjuan Lyu
• Daisuke Iso
• Chaitali Chakrabarti
• Michael Spranger

Existing collaborative self-supervised learning (SSL) schemes are not suitable for cross-client applications because of their expensive computation and large local data requirements. To address these issues, we propose MocoSFL, a collaborative SSL framework based on Split Federated Learning (SFL) and Momentum Contrast (MoCo). In MocoSFL, the large backbone model is split into a small client-side model and a large server-side model, and only the small client-side model is processed locally on the client's local devices. MocoSFL has three key components: (i) vector concatenation which enables the use of small batch size and reduces computation and memory requirements by orders of magnitude; (ii) feature sharing that helps achieve high accuracy regardless of the quality and volume of local data; (iii) frequent synchronization that helps achieve better non-IID performance because of smaller local model divergence. For a 1,000-client case with non-IID data (each client only has data from 2 random classes of CIFAR-10), MocoSFL can achieve over 84% accuracy with ResNet-18 model. Next we present TAResSFL module that significantly improves the resistance to privacy threats and communication overhead with small sacrifice in accuracy for a MocoSFL system. On a Raspberry Pi 4B device, the MocoSFL-based scheme requires less than 1MB of memory and less than 40MB of communication, and consumes less than 5W power. The code is available at https://github.com/SonyAI/MocoSFL.

ICAART Conference 2023 Invited Paper

Outracing Champion Gran Turismo Drivers with Deep Reinforcement Learning

• Michael Spranger

Many potential applications of artificial intelligence involve making real-time decisions in physical systems while interacting with humans. Automobile racing represents an extreme example of these conditions; drivers must execute complex tactical manoeuvres to pass or block opponents while operating their vehicles at their traction limits. Racing simulations, such as the PlayStation game Gran Turismo, faithfully reproduce the non-linear control challenges of real race cars while also encapsulating the complex multi-agent interactions. Here we describe how we trained agents for Gran Turismo that can compete with the world’s best e-sports drivers using Reinforcement Learning. We demonstrate the capabilities of our agent, Gran Turismo Sophy, by winning a head-to-head competition against four of the world’s best Gran Turismo drivers. By describing how we trained championship-level racers, we demonstrate the possibilities and challenges to control complex dynamical systems in domains where agents must respect imprecisely defined human norms.

IS Journal 2022 Journal Article

Challenges and Opportunities of Applying Reinforcement Learning to Autonomous Racing

• Peter R. Wurman
• Peter Stone
• Michael Spranger

Simulated motorsports are an exciting environment in which to explore the power and limitations of deep reinforcement learning. Racing requires precise control of a vehicle that is operating at its traction limits while competing wheel-to-wheel with other drivers. We recently demonstrated an agent that can beat the best drivers in the world at the racing game Gran Turismo. In this article, we briefly discuss some of the lessons learned and some of the remaining open research challenges.

AIJ Journal 2022 Journal Article

Logic Tensor Networks

• Samy Badreddine
• Artur d'Avila Garcez
• Luciano Serafini
• Michael Spranger

Attempts at combining logic and neural networks into neurosymbolic approaches have been on the increase in recent years. In a neurosymbolic system, symbolic knowledge assists deep learning, which typically uses a sub-symbolic distributed representation, to learn and reason at a higher level of abstraction. We present Logic Tensor Networks (LTN), a neurosymbolic framework that supports querying, learning and reasoning with both rich data and abstract knowledge about the world. LTN introduces a fully differentiable logical language, called Real Logic, whereby the elements of a first-order logic signature are grounded onto data using neural computational graphs and first-order fuzzy logic semantics. We show that LTN provides a uniform language to represent and compute efficiently many of the most important AI tasks such as multi-label classification, relational learning, data clustering, semi-supervised learning, regression, embedding learning and query answering. We implement and illustrate each of the above tasks with several simple explanatory examples using TensorFlow 2. The results indicate that LTN can be a general and powerful framework for neurosymbolic AI.

NeurIPS Conference 2022 Conference Paper

Outsourcing Training without Uploading Data via Efficient Collaborative Open-Source Sampling

• Junyuan Hong
• Lingjuan Lyu
• Jiayu Zhou
• Michael Spranger

As deep learning blooms with growing demand for computation and data resources, outsourcing model training to a powerful cloud server becomes an attractive alternative to training at a low-power and cost-effective end device. Traditional outsourcing requires uploading device data to the cloud server, which can be infeasible in many real-world applications due to the often sensitive nature of the collected data and the limited communication bandwidth. To tackle these challenges, we propose to leverage widely available open-source data, which is a massive dataset collected from public and heterogeneous sources (e. g. , Internet images). We develop a novel strategy called Efficient Collaborative Open-source Sampling (ECOS) to construct a proximal proxy dataset from open-source data for cloud training, in lieu of client data. ECOS probes open-source data on the cloud server to sense the distribution of client data via a communication- and computation-efficient sampling process, which only communicates a few compressed public features and client scalar responses. Extensive empirical studies show that the proposed ECOS improves the quality of automated client labeling, model compression, and label outsourcing when applied in various learning scenarios. Source codes will be released.

NeSy Conference 2021 Conference Paper

Extending Real Logic with Aggregate Functions

• Samy Badreddine
• Michael Spranger

Real Logic is a recently introduced first-order language where formulas have fuzzy truth values in the interval [0, 1] and semantics are defined concretely with real domains. The Logic Tensor Networks (LTN) framework has applied Real Logic to many important AI tasks through querying, learning, and reasoning. Motivated by real-life relational database applications, we study adding aggregate functions, such as averaging elements of a relation table, to Real Logic. The key contribution of this paper is the formalization of such functions within Real Logic. This extension is straightforward and fits coherently in the end-to-end differentiable language that Real Logic is. We illustrate it on FooDB, a food chemistry database, and query foods and their nutrients. The resulting framework combines strengths of descriptive statistics modeled by fuzzy predicates, FOL to write complex queries and formulas, and SQL-like expressiveness to aggregate insights from data tables.

NeurIPS Conference 2020 Conference Paper

Assessing SATNet's Ability to Solve the Symbol Grounding Problem

• Oscar Chang
• Lampros Flokas
• Hod Lipson
• Michael Spranger

SATNet is an award-winning MAXSAT solver that can be used to infer logical rules and integrated as a differentiable layer in a deep neural network. It had been shown to solve Sudoku puzzles visually from examples of puzzle digit images, and was heralded as an impressive achievement towards the longstanding AI goal of combining pattern recognition with logical reasoning. In this paper, we clarify SATNet's capabilities by showing that in the absence of intermediate labels that identify individual Sudoku digit images with their logical representations, SATNet completely fails at visual Sudoku (0% test accuracy). More generally, the failure can be pinpointed to its inability to learn to assign symbols to perceptual phenomena, also known as the symbol grounding problem, which has long been thought to be a prerequisite for intelligent agents to perform real-world logical reasoning. We propose an MNIST based test as an easy instance of the symbol grounding problem that can serve as a sanity check for differentiable symbolic solvers in general. Naive applications of SATNet on this test lead to performance worse than that of models without logical reasoning capabilities. We report on the causes of SATNet’s failure and how to prevent them.

NeurIPS Conference 2020 Conference Paper

Temporal Positive-unlabeled Learning for Biomedical Hypothesis Generation via Risk Estimation

• Uchenna Akujuobi
• Jun Chen
• Mohamed Elhoseiny
• Michael Spranger
• Xiangliang Zhang

Understanding the relationships between biomedical terms like viruses, drugs, and symptoms is essential in the fight against diseases. Many attempts have been made to introduce the use of machine learning to the scientific process of hypothesis generation (HG), which refers to the discovery of meaningful implicit connections between biomedical terms. However, most existing methods fail to truly capture the temporal dynamics of scientific term relations and also assume unobserved connections to be irrelevant (i. e. , in a positive-negative (PN) learning setting). To break these limits, we formulate this HG problem as future connectivity prediction task on a dynamic attributed graph via positive-unlabeled (PU) learning. Then, the key is to capture the temporal evolution of node pair (term pair) relations from just the positive and unlabeled data. We propose a variational inference model to estimate the positive prior, and incorporate it in the learning of node pair embeddings, which are then used for link prediction. Experiment results on real-world biomedical term relationship datasets and case study analyses on a COVID-19 dataset validate the effectiveness of the proposed model.

ICRA Conference 2019 Conference Paper

Continuous Value Iteration (CVI) Reinforcement Learning and Imaginary Experience Replay (IER) For Learning Multi-Goal, Continuous Action and State Space Controllers

• Andreas Gerken
• Michael Spranger

This paper presents a novel model-free Reinforcement Learning algorithm for learning behavior in continuous action, state, and goal spaces. The algorithm approximates optimal value functions using non-parametric estimators. It is able to efficiently learn to reach multiple arbitrary goals in deterministic and nondeterministic environments. To improve generalization in the goal space, we propose a novel sample augmentation technique. Using these methods, robots learn faster and overall better controllers. We benchmark the proposed algorithms using simulation and a real-world voltage controlled robot that learns to maneuver in a non-observable Cartesian task space.

NeSy Conference 2019 Conference Paper

Injecting Prior Knowledge for Transfer Learning into Reinforcement Learning Algorithms using Logic Tensor Networks

• Samy Badreddine
• Michael Spranger

Human ability at solving complex tasks is helped by priors on object and event semantics of their environment. This paper investigates the use of similar prior knowledge for transfer learning in Reinforcement Learning agents. In particular, the paper proposes to use a first-order-logic language grounded in deep neural networks to represent facts about objects and their semantics in the real world. Facts are provided as background knowledge a priori to learning a policy for how to act in the world. The priors are injected with the conventional input in a single agent architecture. As proof-of-concept, the paper tests the system in simple experiments that show the importance of symbolic abstraction and flexible fact derivation. The paper shows that the proposed system can learn to take advantage of both the symbolic layer and the image layer in a single decision selection module.

FLAP Journal 2019 Journal Article

Neural-symbolic Computing: An Effective Methodology for Principled Integration of Machine Learning and Reasoning.

• Artur S. d'Avila Garcez
• Marco Gori
• Luís C. Lamb
• Luciano Serafini
• Michael Spranger
• Son N. Tran

IROS Conference 2018 Conference Paper

Online Learning of Body Orientation Control on a Humanoid Robot Using Finite Element Goal Babbling

• Pontus Loviken
• Nikolas Hemion
• Alban Laflaquière
• Michael Spranger
• Angelo Cangelosi

How can high dimensional robots learn general sets of skills from experience in the real world? Many previous approaches focus on maximizing a single utility function and require large datasets of experience to do this, something that is not possible to collect outside of simulation as every data point is expensive both in time and in a potential wear down of the robot. This paper addresses this question using a newly developed framework called Finite Element Goal Babbling (FEGB). FEGB is an online learning method that aims at providing general control over some measurable feature, in contrast to optimizing it to some given utility function. It generalizes standard goal babbling by breaking down the full learning problem into local sub-problems, and combining it with a planner that learns how to navigate between these subproblems. We test FEGB using a real humanoid robot Nao, and find that it could quickly learn to robustly control its body orientation. After only 20-30 minutes of training, the robot could freely move into any body orientation between lying on either side and on its back. Rapid learning of body orientation control in high dimensional real robots is largely an unexplored field of robotics, and although many challenges remain, FEGB shows a feasible approach to the problem.

IJCAI Conference 2016 Conference Paper

Robust Natural Language Processing - Combining Reasoning, Cognitive Semantics, and Construction Grammar for Spatial Language

• Michael Spranger
• Jakob Suchan
• Mehul Bhatt

We present a system for generating and understanding of dynamic and static spatial relations in robotic interaction setups. Robots describe an environment of moving blocks using English phrases that include spatial relations such as "across" and "in front of. " We evaluate the system in robot-robot interactions and show that the system can robustly deal with visual perception errors, language omissions and ungrammatical utterances.

IJCAI Conference 2015 Conference Paper

Co-Acquisition of Syntax and Semantics - An Investigation in Spatial Language

• Michael Spranger
• Luc Steels

This paper reports recent progress on modeling the grounded co-acquisition of syntax and semantics of locative spatial language in developmental robots. We show how a learner robot can learn to produce and interpret spatial utterances in guided-learning interactions with a tutor robot (equipped with a system for producing English spatial phrases). The tutor guides the learning process by simplifying the challenges and complexity of utterances, gives feedback, and gradually increases the complexity of the language to be learnt. Our experiments show promising results towards long-term, incremental acquisition of natural language in a process of codevelopment of syntax and semantics.

IROS Conference 2015 Conference Paper

Procedural semantics for autonomous robots - a case study in locative spatial language

• Michael Spranger

This paper discusses the application of a procedural semantics framework to autonomous robots. We model insights from decades of research in cognitive linguistics and empirical research to develop a system capable of autonomously producing and interpreting German locative utterances. The system is tested in the real world using a population of robots that talk to each other via natural language.

ECAI Conference 2010 Conference Paper

Acquisition of Grammar in Autonomous Artificial Systems

• Kateryna Gerasymova
• Michael Spranger

Over the past several decades, psycholinguists have gained countless insights into the process of child language acquisition. Can these findings be used for the development of language competence in autonomous artificial systems? This paper reports on our attempt to apply insights from developmental psychology in order to enable artificial systems to acquire language. We consider a comprehensive chain of computational processes, starting from conceptualization and extending through language generation and interpretation, and show how they can be intertwined to allow for acquisition of complex aspects of grammar.

ECAI Conference 2010 Conference Paper

Open-ended Grounded Semantics

• Michael Spranger
• Martin Loetzsch
• Simon Pauw

Artificial agents trying to achieve communicative goals in situated interactions in the real-world need powerful computational systems for conceptualizing their environment. In order to provide embodied artificial systems with rich semantics reminiscent of human language complexity, agents need ways of both conceptualizing complex compositional semantic structure and actively reconstructing semantic structure, due to uncertainty and ambiguity in transmission. Furthermore, the systems must be open-ended and adaptive and allow agents to adjust their semantic inventories in order to reach their goals. This paper presents recent progress in modeling open-ended, grounded semantics through a unified software system that addresses these problems.

IJCAI Conference 2009 Conference Paper

• Luc Steels
• Michael Spranger

IROS Conference 2008 Conference Paper

Using reference objects to improve vision-based bearing measurements

• Heinrich Mellmann
• Matthias Jüngel
• Michael Spranger

Robots perceiving its environment using cameras usually need a good representation of how the camera is aligned to the body and how the camera is rotated relative to the ground. This is especially important for bearing-based distance measurement. In this paper we show how to use reference objects to improve vision-based distance measurements to objects of unknown size. Several methods for different kinds of reference objects are introduced. These are objects of known size (like a ball), objects extending over the horizon (like goals and beacons), and objects with known shape on the ground (like field lines). We give a detailed description how to determine the rotation of the robot’s camera relative to the ground, provide an error-estimation for all methods and describe the experiments we performed on an Aibo robot.

ICRA Conference 2006 Conference Paper

Further Studies on the Use of Negative Information in Mobile Robot Localization

• Jan Hoffmann 0001
• Michael Spranger
• Daniel Goehring
• Matthias Jüngel
• Hans-Dieter Burkhard

This paper deals with how the absence of an expected sensor reading can be used to improve Markov localization. Negative information has not been used for robot localization for various reasons like sensor imperfections, and occlusions that make it hard to determine if a missing sensor reading is really caused by the absence of a feature. We address these difficulties by carefully modeling the robot's main sensor, its camera. Taking into account the viewing frustum and detected obstacles, the absence of a sensor reading can be associated with the absence of that particular feature. This information can then be integrated into the localization process. We show the positive effect on robot localization in various experiments. (a) In a specific setup, the robot is able to localize using negative information where without it, it is unable to localize. (b) We demonstrate the importance of modeling occlusions and the impact of false negatives on localization. (c) We show the positive impact in a typical run

IROS Conference 2005 Conference Paper

Making use of what you don't see: negative information in Markov localization

• Jan Hoffmann 0001
• Michael Spranger
• Daniel Goehring
• Matthias Jüngel

This paper explores how the absence of an expected sensor reading can be used to improve Markov localization. This negative information usually is not being used in localization, because it yields less information than positive information (i. e. sensing a landmark), and a sensor often fails to detect a landmark, even if it falls within its sensing range. We address these difficulties by carefully modeling the sensor to avoid false negatives. This can also be thought of as adding an additional sensor that detects the absence of an expected landmark. We show how such modeling is done and how it is integrated into Markov localization. In real world experiments, we demonstrate that a robot is able to localize in positions where otherwise it could not and quantify our findings using the entropy of the particle distribution. Exploiting negative information leads to a greatly improved localization performance and reactivity.