Author name cluster

Hongkai Dai

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

6 papers

1 author row

ICLR Conference 2025 Conference Paper

Diffusion Policy Policy Optimization

Allen Z. Ren
Justin Lidard
Lars Ankile
Anthony Simeonov
Pulkit Agrawal 0001
Anirudha Majumdar
Benjamin Burchfiel
Hongkai Dai

We introduce Diffusion Policy Policy Optimization, DPPO, an algorithmic framework including best practices for fine-tuning diffusion-based policies (e.g. Diffusion Policy) in continuous control and robot learning tasks using the policy gradient (PG) method from reinforcement learning (RL). PG methods are ubiquitous in training RL policies with other policy parameterizations; nevertheless, they had been conjectured to be less efficient for diffusion-based policies. Surprisingly, we show that DPPO achieves the strongest overall performance and efficiency for fine-tuning in common benchmarks compared to other RL methods for diffusion-based policies and also compared to PG fine-tuning of other policy parameterizations. Through experimental investigation, we find that DPPO takes advantage of unique synergies between RL fine-tuning and the diffusion parameterization, leading to structured and on-manifold exploration, stable training, and strong policy robustness. We further demonstrate the strengths of DPPO in a range of realistic settings, including simulated robotic tasks with pixel observations, and via zero-shot deployment of simulation-trained policies on robot hardware in a long-horizon, multi-stage manipulation task.

Details

ICML Conference 2024 Conference Paper

Lyapunov-stable Neural Control for State and Output Feedback: A Novel Formulation

Lujie Yang
Hongkai Dai
Zhouxing Shi
Cho-Jui Hsieh
Russ Tedrake
Huan Zhang 0001

Learning-based neural-network (NN) control policies have shown impressive empirical performance in a wide range of tasks in robotics and control. However, formal (Lyapunov) stability guarantees over the region-of-attraction (ROA) for NN controllers with nonlinear dynamical systems are challenging to obtain, and most existing approaches rely on expensive solvers for sums-of-squares (SOS), mixed-integer programming (MIP), or satisfiability modulo theories (SMT). In this paper, we demonstrate a new framework for learning NN controllers together with Lyapunov certificates using fast empirical falsification and strategic regularizations. We propose a novel formulation that defines a larger verifiable region-of-attraction (ROA) than shown in the literature, and refines the conventional restrictive constraints on Lyapunov derivatives to focus only on certifiable ROAs. The Lyapunov condition is rigorously verified post-hoc using branch-and-bound with scalable linear bound propagation-based NN verification techniques. The approach is efficient and flexible, and the full training and verification procedure is accelerated on GPUs without relying on expensive solvers for SOS, MIP, nor SMT. The flexibility and efficiency of our framework allow us to demonstrate Lyapunov-stable output feedback control with synthesized NN-based controllers and NN-based observers with formal stability guarantees, for the first time in literature.

Details

IROS Conference 2019 Conference Paper

A Convex-Combinatorial Model for Planar Caging

Bernardo Aceituno
Hongkai Dai
Alberto Rodriguez 0003

Caging is a promising tool which allows a robot to manipulate an object without directly reasoning about the contact dynamics involved. Furthermore, caging also provides useful guarantees in terms of robustness to uncertainty, and often serves as a way-point to a grasp. However, caging is traditionally difficult to integrate as part of larger manipulation frameworks, where caging is not the goal but an intermediate condition. In this paper, we develop a convex-combinatorial model to characterize caging from an optimization perspective. More specifically, we derive a set of sufficient constraints to enclose the configuration of the object in a compact-connected component of its free-space. The convex-combinatorial nature of this approach provides guarantees on optimality and convergence, and its optimization nature makes it versatile for further applications on robot manipulation tasks. To the best of our knowledge, this is the first optimization-based approach to formulate the caging condition.

Details

IROS Conference 2017 Conference Paper

A mixed-integer convex optimization framework for robust multilegged robot locomotion planning over challenging terrain

Bernardo Aceituno
Hongkai Dai
José Cappelletto
Juan C. Grieco
Gerardo Fernández-López

This paper introduces an optimization-based framework for robust multilegged walking motion planning. Previous approaches use fixed gait sequences, and rely on Zero Moment Point (ZMP) to guarantee dynamic stability. While this combination works well on flat ground, it does not generalize to uneven terrain requiring aggressive gait or gait transition. To overcome such difficulties, in this paper, we present an optimization framework, that can plan both the contact location and gait sequence simultaneously in a mixed-integer convex optimization program. Moreover, we rely on the Contact Wrench Cone (CWC) stability criterion, which generalizes the ZMP criterion to uneven terrain with friction cone constraints, and we plan the walking motion together with the angular momentum through a convex optimization program. Our approach is successfully tested on a LittleDog quadruped over simulated scenarios. We show that on the flat ground, our planner generates a periodic gait, same as Central Pattern Generator + ZMP planner; while on uneven terrain, our planner can successfully generate a motion containing different gaits, with a center-of-mass motion that respects the friction cone constraints, which are violated by ZMP planners. This improvement clearly demonstrates the advantage of our approach over traditional planning strategies.

Details

ICRA Conference 2014 Conference Paper

A summary of team MIT's approach to the virtual robotics challenge

Russ Tedrake
Maurice F. Fallon
Sisir Karumanchi
Scott Kuindersma
Matthew E. Antone
Toby Schneider
Thomas M. Howard
Matthew R. Walter

The attached video describes the system developed by researchers from MIT for the Defense Advanced Research Projects Agency's (DARPA) Virtual Robotics Challenge (VRC), held in June 2013. The VRC was the first competition in the DARPA Robotics Challenge (DRC), a program that aims to “develop ground robotic capabilities to execute complex tasks in dangerous, degraded, human-engineered environments” [1]. The VRC required teams to guide a model of Boston Dynamics' humanoid robot, Atlas, through driving, walking, and manipulation tasks in simulation.

Details

ICRA Conference 2013 Conference Paper

L2-gain optimization for robust bipedal walking on unknown terrain

Hongkai Dai
Russ Tedrake

In this paper we seek to quantify and explicitly optimize the robustness of a control system for a robot walking on terrain with uncertain geometry. Geometric perturbations to the terrain enter the equations of motion through a relocation of the hybrid event “guards” which trigger an impact event; these perturbations can have a large effect on the stability of the robot and do not fit into the traditional robust control analysis and design methodologies without additional machinery. We attempt to provide that machinery here. In particular, we quantify the robustness of the system to terrain perturbations by defining an L 2 gain from terrain perturbations to deviations from the nominal limit cycle. We show that the solution to a periodic dissipation inequality provides a sufficient upper bound on this gain for a linear approximation of the dynamics around the limit cycle, and we formulate a semidefinite programming problem to compute the L 2 gain for the system with a fixed linear controller. We then use either binary search or an iterative optimization method to construct a linear robust controller and to minimize the L 2 gain. The simulation results on canonical robots suggest that the L 2 gain is closely correlated to the actual number of steps traversed on the rough terrain, and our controller can improve the robot's robustness to terrain disturbances.

Details