Learning Generalizable Manipulation Policy with Adapter-Based Parameter Fine-Tuning

This study investigates the use of adapters in reinforcement learning for robotic skill generalization across multiple robots and tasks. Traditional methods are typically reliant on robot-specific retraining and face challenges such as efficiency and adaptability, particularly when scaling to robots with varying kinematics. We propose an alternative approach where a disembodied (virtual) hand manipulator learns a task (i. e. , an abstract skill) and then transfers it to various robots with different kinematic constraints without retraining the entire model (i. e. , the concrete, physical implementation of the skill). Whilst adapters are commonly used in other domains with strong supervision available, we show how weaker feedback from robotic control can be used to optimize task execution by preserving the abstract skill dynamics whilst adapting to new robotic domains. We demonstrate the effectiveness of our method with experiments conducted in the SAPIEN ManiSkill environment, showing improvements in generalization and task success rates. All code, data, and additional videos are at this GitHub link: https://kl-research.github.io/genrob.

Authors

• Kai Lu 0003
• Kim Tien Ly
• William Hebberd
• Kaichen Zhou
• Ioannis Havoutis
• Andrew Markham

Keywords

• Training
• Hands
• Adaptation models
• Kinematics
• Reinforcement learning
• Manipulators
• Robots
• Intelligent robots
• Videos
• Software development management
• Manipulation Policies
• Robot Control
• Kinematic Constraints
• Multiple Robots
• Joint Position
• Original Network
• Reward Function
• Real-world Experiments
• Domain Adaptation
• Markov Decision Process
• Linear Layer
• Robot Manipulator
• Policy Learning
• Roll Angle
• Policy Network
• Inverse Kinematics
• Adaptive Modulation
• Foundation Model
• Imitation Learning
• Joint Limits
• Mobile Manipulator
• End-effector Pose
• Robot Learning
• Kinematic Properties
• Mobile Robot
• Robotic Platform
• Reinforcement Learning Policy
• Reinforcement Learning Agent
• Original Parameters
• Visual Domain