SoftNeRF: A Self-Modeling Soft Robot Plugin for Various Tasks

Building a self-model for robots, enabling them to simulate their physical selves and predict future states without direct interaction with the physical world, is crucial for robot motion planning and control. Existing self-modeling methods primarily focus on rigid robots and typically require significant time, effort, and resources to gather training data. In this study, we introduce SoftNeRF, a self-supervised visual self-model designed for soft robots. We use a hybrid neural shape representation based on the Signed Distance Function (SDF) to capture both the geometry and complex nonlinear motion of soft robots. By leveraging differentiable rendering, our method learns a self-model from readily available RGB images, similar to how humans understand their physical state through reflection. To improve training efficiency and model accuracy, we propose an error-guided adaptive sampling strategy. SoftNeRF can serve as a plug-in for various downstream tasks, even when trained with data unrelated to those tasks. We demonstrate SoftNeRF’s ability to support shape prediction and motion planning for robots in both simulated and real-world environments. Furthermore, SoftNeRF excels in detecting and recovering from damage, thereby enhancing machine resilience. Code is available at: https://github.com/irmvlab/soft-nerf.

Authors

• Jiwei Shan
• Yirui Li
• Qiyu Feng
• Ditao Li
• Lijun Han
• Hesheng Wang 0001

Keywords

• Training
• Visualization
• Adaptation models
• Accuracy
• Shape
• Training data
• Soft robotics
• Rendering (computer graphics)
• Sampling methods
• Planning
• Soft Robots
• Physical Conditions
• Motor Control
• Simulation Environment
• RGB Images
• Path Planning
• Physical World
• Training Efficiency
• Robot Motion
• Signed Distance Function
• Actuator
• Prediction Error
• Probability Density Function
• Control Signal
• Point Cloud
• Multilayer Perceptron
• Body Shape
• Lifelong Learning
• 3D Point
• Volume Rendering
• Multi-view Images
• Color Segmentation
• Real Robot
• Length Set
• Target Shape
• Error Map
• Masked Images
• Damage Detection