LIMT: Language-Informed Multi-Task Visual World Models

Most recent successes in robot reinforcement learning involve learning a specialized single-task agent. However, robots capable of performing multiple tasks can be much more valuable in real-world applications. Multi-task reinforcement learning can be very challenging due to the increased sample complexity and the potentially conflicting task objectives. Previous work on this topic is dominated by model-free approaches. The latter can be very sample inefficient even when learning specialized single-task agents. In this work, we focus on model-based multi-task reinforcement learning. We propose a method for learning multi-task visual world models, leveraging pre-trained language models to extract semantically meaningful task representations. These representations are used by the world model and policy to reason about task similarity in dynamics and behavior. Our results highlight the benefits of using language-driven task representations for world models and a clear advantage of model-based multi-task learning over the more common model-free paradigm.

Authors

• Elie Aljalbout
• Nikolaos Sotirakis
• Patrick van der Smagt
• Maximilian Karl
• Nutan Chen

Keywords

• Training
• Visualization
• Reinforcement learning
• Multitasking
• Data models
• Complexity theory
• Robots
• Visual Model
• Multiple Tasks
• Language Model
• Multi-task Learning
• Task Representations
• Multi-task Model
• Pre-trained Language Models
• Learning Methods
• Language Teaching
• Multiple Users
• Sampling Efficiency
• Actor Network
• Model-based Approach
• Individual Tasks
• End-effector
• Policy Learning
• Policy Network
• Latent Vector
• Reinforcement Learning Methods
• Model-based Reinforcement Learning
• Multi-task Training
• Tokenized
• Model-free Reinforcement Learning
• Model-free Methods
• Robot Learning
• Perceptual Loss
• Representative Observations
• Joint Training
• Robotic Tasks