Differentiable Factor Graph Optimization for Learning Smoothers

A recent line of work has shown that end-to-end optimization of Bayesian filters can be used to learn state estimators for systems whose underlying models are difficult to hand-design or tune, while retaining the core advantages of probabilistic state estimation. As an alternative approach for state estimation in these settings, we present an end-to-end approach for learning state estimators modeled as factor graph-based smoothers. By unrolling the optimizer we use for maximum a posteriori inference in these probabilistic graphical models, our method is able to learn probabilistic system models in the full context of an overall state estimator, while also taking advantage of the distinct accuracy and runtime advantages that smoothers offer over recursive filters. We study our approach using two fundamental state estimation problems, object tracking and visual odometry, where we demonstrate a significant improvement over existing baselines. Our work comes with an extensive code release, which includes training and evaluation scripts, as well as Python libraries for Lie theory and factor graph optimization: https://sites.google.com/view/diffsmoothing/.

Authors

• Brent Yi
• Michelle A. Lee
• Alina Kloss
• Roberto Martín-Martín
• Jeannette Bohg

Keywords

• Training
• Smoothing methods
• Simultaneous localization and mapping
• Tactile sensors
• Probabilistic logic
• Filtering theory
• State estimation
• Smoothing
• Factor Graph
• Factor Graph Optimization
• Probability Estimates
• Maximum A Posteriori
• Probabilistic Graphical Models
• State Estimation Problem
• Visual Odometry
• Time Step
• Long Short-term Memory
• Angular Velocity
• Conditional Independence
• Visual Task
• Raw Images
• Kalman Filter
• Nonlinear Programming
• Learnable Parameters
• Noise Model
• Particle Filter
• Differential Filter
• Constant Noise
• Virtual Sensors
• Sensor Model
• Task Goal
• Posterior Mode
• Variable Nodes
• Ground Truth Trajectory
• Sensor Observations
• Robot Learning