Author name cluster

John M. Dolan

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

65 papers

2 author rows

ICRA Conference 2025 Conference Paper

A Generalized Control Revision Method for Autonomous Driving Safety

Zehang Zhu
Yuning Wang
Tianqi Ke
Zeyu Han
Shaobing Xu
Qing Xu 0010
John M. Dolan
Jianqiang Wang 0003

Safety is one of the most crucial challenges of autonomous driving vehicles, and one solution to guarantee safety is to employ an additional control revision module after the planning backbone. Control Barrier Function (CBF) has been widely used because of its strong mathematical foundation on safety. However, the incompatibility with heterogeneous perception data and incomplete consideration of traffic scene elements make existing systems hard to be applied in dynamic and complex real-world scenarios. In this study, we introduce a generalized control revision method for autonomous driving safety, which adopts both vectorized perception and occupancy grid map as inputs and comprehensively models multiple types of traffic scene constraints based on a new proposed barrier function. Traffic elements are integrated into one unified framework, decoupled from specific scenario settings or rules. Experiments on CARLA, SUMO, and OnSite simulator prove that the proposed algorithm could realize safe control revision under complicated scenes, adapting to various planning backbones, road topologies, and risk types. Physical platform validation also verifies the real-world application feasibility.

ICRA Conference 2025 Conference Paper

Agile Mobility with Rapid Online Adaptation via Meta-Learning and Uncertainty-Aware MPPI

Dvij Kalaria
Haoru Xue
Wenli Xiao
Tony Tao
Guanya Shi
John M. Dolan

Modern non-linear model-based controllers require an accurate physics model and model parameters to be able to control mobile robots at their limits. Also, due to surface slipping at high speeds, the friction parameters may continually change (like tire degradation in autonomous racing), and the controller may need to adapt rapidly. Many works derive a task-specific robot model with a parameter adaptation scheme that works well for the task but requires a lot of effort and tuning for each platform and task. In this work, we design a full model-learning-based controller based on meta pretraining that can very quickly adapt using few-shot dynamics data to any wheel-based robot with any model parameters, while also reasoning about model uncertainty. We demonstrate our results in small-scale numeric simulation, the large-scale Unity simulator, and on a medium-scale hardware platform with a wide range of settings. We show that our results are comparable to domain-specific well-engineered controllers, and have excellent generalization performance across all scenarios.

ICRA Conference 2025 Conference Paper

AnyCar to Anywhere: Learning Universal Dynamics Model for Agile and Adaptive Mobility

Wenli Xiao
Haoru Xue
Tony Tao
Dvij Kalaria
John M. Dolan
Guanya Shi

Recent works in the robot learning community have successfully introduced generalist models capable of controlling various robot embodiments across a wide range of tasks, such as navigation and locomotion. However, achieving agile control, which pushes the limits of robotic performance, still relies on specialist models that require extensive parameter tuning. To leverage generalist-model adaptability and flexibility while achieving specialist-level agility, we propose AnyCar, a transformer-based generalist dynamics model designed for agile control of various wheeled robots. To collect training data, we unify multiple simulators and leverage different physics backends to simulate vehicles with diverse sizes, scales, and physical properties across various terrains. With robust training and real-world fine-tuning, our model enables precise adaptation to different vehicles, even in the wild and under large state estimation errors. In real-world experiments, AnyCar shows both few-shot and zero-shot generalization across a wide range of vehicles and environments, where our model, combined with a sampling-based MPC, outperforms specialist models by up to 54%. These results represent a key step toward building a foundation model for agile wheeled robot control. AnyCar is fully open-source to support further research.

IROS Conference 2025 Conference Paper

Disturbance Observer-based Control Barrier Functions with Residual Model Learning for Safe Reinforcement Learning

Dvij Kalaria
Qin Lin 0001
John M. Dolan

Reinforcement learning (RL) agents need to explore their environment to learn optimal behaviors and achieve maximum rewards. However, exploration can be risky when training RL directly on real systems, while simulation-based training introduces the tricky issue of the sim-to-real gap. Recent approaches have leveraged safety filters, such as control barrier functions (CBFs), to penalize unsafe actions during RL training. However, the strong safety guarantees of CBFs rely on a precise dynamic model. In practice, uncertainties always exist, including internal disturbances from the errors of dynamics and external disturbances such as wind. In this work, we propose a novel safe RL framework built on a robust CBF, where the discrepancy between the nominal and true dynamic models is quantified through a combination of disturbance observation and residual model learning. We demonstrate our results on the Safety-gym benchmark for Point and Car robots on all tasks where we can outperform state-of-the-art approaches that use only residual model learning or a disturbance observer (DOB). We further validate the efficacy of our framework using a physical F1/10 racing car. Videos: https://sites.google.com/view/res-dob-cbf-rl

IROS Conference 2025 Conference Paper

LLA-MPC: Fast Adaptive Control for Autonomous Racing

Maitham F. AL-Sunni
Hassan Almubarak
Katherine Horng
John M. Dolan

We present Look-Back and Look-Ahead Adaptive Model Predictive Control (LLA-MPC), a real-time adaptive control framework for autonomous racing that addresses the challenge of rapidly changing tire-surface interactions. Unlike existing approaches requiring substantial data collection or offline training, LLA-MPC employs parallelization over a bank of models for rapid adaptation with no training. It integrates two key mechanisms: a look-back window that uses recent vehicle behavior to optimize the model used in a look-ahead stage for trajectory optimization and control. The optimized model and its associated parameters are then incorporated into an adaptive path planner to optimize reference racing paths in real time. Experiments across diverse racing scenarios demonstrate that LLA-MPC outperforms state-of-the-art methods in adaptation speed and handling, even during sudden friction transitions. Its learning-free, computationally efficient design enables rapid adaptation, making it ideal for high-speed autonomous racing in multi-surface environments.

ICRA Conference 2025 Conference Paper

Model-Free Safety Filter for Soft Robots: A Q-Learning Approach

Guo Ning Sue
Yogita Choudhary
Richard Desatnik
Carmel Majidi
John M. Dolan
Guanya Shi

Ensuring safety via safety filters in real-world robotics presents significant challenges, particularly when the system dynamics is complex or unavailable. To handle this issue, learning-based safety filters recently gained popularity, which can be classified as model-based and model-free methods. Existing model-based approaches requires various assumptions on system model (e. g. , control-affine), which limits their application in complex systems, and existing model-free approaches need substantial modifications to standard RL algorithms and lack versatility. This paper proposes a simple, plugin-and-play, and effective model-free safety filter learning framework. We introduce a novel reward formulation and use Q-learning to learn Q-value functions to safeguard arbitrary task specific nominal policies via filtering out their potentially unsafe actions. Due to its model-free nature and simplicity, our framework can be seamlessly integrated with various RL algorithms. We validate the proposed approach through simulations on double integrator and Dubin's car systems and demonstrate its effectiveness in real-world experiments with a soft robotic limb.

ICRA Conference 2025 Conference Paper

Real-Time Whole-Body Control of Legged Robots with Model-Predictive Path Integral Control

Juan Alvarez-Padilla
John Z. Zhang
Sofia Kwok
John M. Dolan
Zachary Manchester

This paper presents a system for enabling real-time synthesis of whole-body locomotion and manipulation policies for real-world legged robots. Motivated by recent advancements in robot simulation, we leverage the efficient parallelization capabilities of the MuJoCo simulator on a multi-core CPU to achieve fast sampling over the robot state and action trajectories. Our results show surprisingly effective real-world locomotion and manipulation capabilities with a very simple control strategy. We demonstrate our approach on several hardware and simulation experiments: robust locomotion over flat and uneven terrains, climbing over a box whose height is comparable to the robot, and pushing a box to a goal position. To our knowledge, this is the first successful deployment of whole-body sampling-based MPC on real-world legged robot hardware. Experiment videos and code can be found at: whole-body-mppi.github.io.

ICRA Conference 2025 Conference Paper

Safe Control of Quadruped in Varying Dynamics via Safety Index Adaptation

Kai S. Yun
Rui Chen 0030
Chase Dunaway
John M. Dolan
Changliu Liu

Varying dynamics pose a fundamental difficulty when deploying safe control laws in the real world. Safety Index Synthesis (SIS) deeply relies on the system dynamics and once the dynamics change, the previously synthesized safety index becomes invalid. In this work, we show the real-time efficacy of Safety Index Adaptation (SIA) in varying dynamics. SIA enables real-time adaptation to the changing dynamics so that the adapted safe control law can still guarantee 1) forward invariance within a safe region and 2) finite time convergence to that safe region. This work employs SIA on a packagecarrying quadruped robot, where the payload weight changes in real-time. SIA updates the safety index when the dynamics change, e. g. , a change in payload weight, so that the quadruped can avoid obstacles while achieving its performance objectives. Numerical study provides theoretical guarantees for SIA and a series of hardware experiments demonstrate the effectiveness of SIA in real-world deployment in avoiding obstacles under varying dynamics.

ICRA Conference 2024 Conference Paper

Adaptive Planning and Control with Time-Varying Tire Models for Autonomous Racing Using Extreme Learning Machine

Dvij Kalaria
Qin Lin 0001
John M. Dolan

Autonomous racing is a challenging problem, as the vehicle needs to operate at the friction or handling limits in order to achieve minimum lap times. Autonomous race cars require highly accurate perception, state estimation, planning, and control. Adding to this complexity is the need to accurately identify vehicle model parameters governing lateral tire slip effects, which can evolve over time due to factors such as tire wear and tear. Current approaches to this problem typically either propose offline model identification methods or rely on initial parameters within a narrow range (typically within 15-20% of the actual values). However, these approaches fall short in accounting for significant changes in tire models that can occur during actual races, particularly when pushing the vehicle to its handling limits. We present a unified framework that not only learns the tire model in real time from collected data but also adapts the model to environmental changes, even when the model parameters exhibit substantial deviations. The friction estimation, obtained as a byproduct from the learning results, facilitates the selection of the optimal racing line from a library for adaptive speed planning. We validate our approach through testing in simulators, encompassing a 1: 43 scale race car and a full-size car, and also through experiments with a physical F1/10 autonomous race car.

ICRA Conference 2024 Conference Paper

Hierarchical Learned Risk-Aware Planning Framework for Human Driving Modeling

Nathan Ludlow
Yiwei Lyu 0002
John M. Dolan

This paper presents a novel approach to modeling human driving behavior, designed for use in evaluating autonomous vehicle control systems in a simulation environments. Our methodology leverages a hierarchical forward-looking, risk-aware estimation framework with learned parameters to generate human-like driving trajectories, accommodating multiple driver levels determined by model parameters. This approach is grounded in multimodal trajectory prediction, using a deep neural network with LSTM-based social pooling to predict the trajectories of surrounding vehicles. These trajectories are used to compute forward-looking risk assessments along the ego vehicle’s path, guiding its navigation. Our method aims to replicate human driving behaviors by learning parameters that emulate human decision-making during driving. We ensure that our model exhibits robust generalization capabilities by conducting simulations, employing real-world driving data to validate the accuracy of our approach in modeling human behavior. The results reveal that our model effectively captures human behavior, showcasing its versatility in modeling human drivers in diverse highway scenarios.

ICRA Conference 2024 Conference Paper

Learning Model Predictive Control with Error Dynamics Regression for Autonomous Racing

Haoru Xue
Edward L. Zhu
John M. Dolan
Francesco Borrelli

This work presents a novel Learning Model Predictive Control (LMPC) strategy for autonomous racing at the handling limit that can iteratively explore and learn unknown dynamics in high-speed operational domains. We start from existing LMPC formulations and modify the system dynamics learning method. In particular, our approach uses a nominal, global, nonlinear, physics-based model with a local, linear, data-driven learning of the error dynamics. We conducted experiments in simulation and on 1/10th scale hardware, and deployed the proposed LMPC on a full-scale autonomous race car used in the Indy Autonomous Challenge (IAC) with closed loop experiments at the Putnam Park Road Course in Indiana, USA. The results show that the proposed control policy exhibits improved robustness to parameter tuning and data scarcity. Incremental and safety-aware exploration toward the limit of handling and iterative learning of the vehicle dynamics in high-speed domains is observed both in simulations and experiments.

ICLR Conference 2024 Conference Paper

Reasoning with Latent Diffusion in Offline Reinforcement Learning

Siddarth Venkatraman
Shivesh Khaitan
Ravi Tej Akella
John M. Dolan
Jeff G. Schneider
Glen Berseth

Offline reinforcement learning (RL) holds promise as a means to learn high-reward policies from a static dataset, without the need for further environment interactions. However, a key challenge in offline RL lies in effectively stitching portions of suboptimal trajectories from the static dataset while avoiding extrapolation errors arising due to a lack of support in the dataset. Existing approaches use conservative methods that are tricky to tune and struggle with multi-modal data or rely on noisy Monte Carlo return-to-go samples for reward conditioning. In this work, we propose a novel approach that leverages the expressiveness of latent diffusion to model in-support trajectory sequences as compressed latent skills. This facilitates learning a Q-function while avoiding extrapolation error via batch-constraining. The latent space is also expressive and gracefully copes with multi-modal data. We show that the learned temporally-abstract latent space encodes richer task-specific information for offline RL tasks as compared to raw state-actions. This improves credit assignment and facilitates faster reward propagation during Q-learning. Our method demonstrates state-of-the-art performance on the D4RL benchmarks, particularly excelling in long-horizon, sparse-reward tasks.

ICRA Conference 2024 Conference Paper

Safe Deep Policy Adaptation

Wenli Xiao
Tairan He
John M. Dolan
Guanya Shi

A critical goal of autonomy and artificial intelligence is enabling autonomous robots to rapidly adapt in dynamic and uncertain environments. Classic adaptive control and safe control provide stability and safety guarantees but are limited to specific system classes. In contrast, policy adaptation based on reinforcement learning (RL) offers versatility and generalizability but presents safety and robustness challenges. We propose SafeDPA, a novel RL and control framework that simultaneously tackles the problems of policy adaptation and safe reinforcement learning. SafeDPA jointly learns adaptive policy and dynamics models in simulation, predicts environment configurations, and fine-tunes dynamics models with few-shot real-world data. A safety filter based on the Control Barrier Function (CBF) on top of the RL policy is introduced to ensure safety during real-world deployment. We provide theoretical safety guarantees of SafeDPA and show the robustness of SafeDPA against learning errors and extra perturbations. Comprehensive experiments on (1) classic control problems (Inverted Pendulum), (2) simulation benchmarks (Safety Gym), and (3) a real-world agile robotics platform (RC Car) demonstrate great superiority of SafeDPA in both safety and task performance, over state-of-the-art baselines. Particularly, SafeDPA demonstrates notable generalizability, achieving a 300% increase in safety rate compared to the baselines, under unseen disturbances in real-world experiments.

ICRA Conference 2024 Conference Paper

Tractable Joint Prediction and Planning over Discrete Behavior Modes for Urban Driving

Adam Villaflor
Brian Yang
Huangyuan Su
Katerina Fragkiadaki
John M. Dolan
Jeff G. Schneider

Significant progress has been made in training multimodal trajectory forecasting models for autonomous driving. However, effectively integrating these models with downstream planners and model-based control approaches is still an open problem. Although these models have conventionally been evaluated for open-loop prediction, we show that they can be used to parameterize autoregressive closed-loop models without retraining. We consider recent trajectory prediction approaches which leverage learned anchor embeddings to predict multiple trajectories, finding that these anchor embeddings can parameterize discrete and distinct modes representing high-level driving behaviors. We propose to perform fully reactive closed-loop planning over these discrete latent modes, allowing us to tractably model the causal interactions between agents at each step. We validate our approach on a suite of more dynamic merging scenarios, finding that our approach avoids the frozen robot problem which is pervasive in conventional planners. Our approach also outperforms the previous state-of-the-art in CARLA on challenging dense traffic scenarios when evaluated at realistic speeds.

ICRA Conference 2023 Conference Paper

Active Probing and Influencing Human Behaviors Via Autonomous Agents

Shuangge Wang
Yiwei Lyu 0002
John M. Dolan

Autonomous agents (robots) face tremendous challenges while interacting with heterogeneous human agents in close proximity. One of these challenges is that the autonomous agent does not have an accurate model tailored to the specific human that the autonomous agent is interacting with, which could sometimes result in inefficient human-robot interaction and suboptimal system dynamics. Developing an online method to enable the autonomous agent to learn information about the human model is therefore an ongoing research goal. Existing approaches position the robot as a passive learner in the environment to observe the physical states and the associated human response. This passive design, however, only allows the robot to obtain information that the human chooses to exhibit, which sometimes doesn't capture the human's full intention. In this work, we present an online optimization-based probing procedure for the autonomous agent to clarify its belief about the human model in an active manner. By optimizing an information radius, the autonomous agent chooses the action that most challenges its current conviction. This procedure allows the autonomous agent to actively probe the human agents to reveal information that's previously unavailable to the autonomous agent. With this gathered information, the autonomous agent can interactively influence the human agent for some designated objectives. Our main contributions include a coherent theoretical framework that unifies the probing and influence procedures and two case studies in autonomous driving that show how active probing can help to create better participant experience during influence, like higher efficiency or less perturbations.

AAMAS Conference 2023 Conference Paper

Decentralized Safe Navigation for Multi-agent Systems via Risk-aware Weighted Buffered Voronoi Cells

Yiwei Lyu
John M. Dolan
Wenhao Luo

In this paper, we propose Risk-aware Weighted Buffered Voronoi tessellation, a variant of Generalized Voronoi tessellation, for decentralized multi-agent collision-free navigation. Inherited from the traditional Voronoi tessellation, a safety guarantee in terms of inter-robot collision avoidance is achieved by partitioning the joint state space of the multi-agent system into individual cells that constrain each individual agent’s motion in a distributed manner. Different from many existing Voronoi tessellations-based collision avoidance approaches, our Risk-aware Weighted Buffered Voronoi Cell (Risk-aware WBVC) partition not only takes agent positional information into account, but also the motion information when determining the cell boundaries between pairwise robots. Our riskaware WBVC relies on the novel use of Control Barrier Functions (CBF) as a measure of risk evaluation that captures to what extent the safety constraints are satisfied between pairwise robots. With that, the cell boundaries of risk-aware WBVC are determined by (1) the varying levels of relative efforts between pairwise agents to respond to potential collisions, and (2) the accumulated risk each agent experiences that is caused by the surrounding agents. This allows for an adaptive constrained space partition among robots that balances between individual’s efforts in respecting the safety constraints and the overall threats due to other agents in the environment, e. g. an aggressive robot moving with higher speed requires a relatively larger space for responding to potential collisions, and a less-threatened robot may be expected to yield and make more room for those exposed to higher risk. Rigorous proofs of formal safety guarantees are provided and simulations are demonstrated on up to 16 robots to show the effectiveness of our method.

ICRA Conference 2023 Conference Paper

Reinforcement Learning with Probabilistically Safe Control Barrier Functions for Ramp Merging

Soumith Udatha
Yiwei Lyu 0002
John M. Dolan

Prior work has looked at applying reinforcement learning (RL) approaches to autonomous driving scenarios, but the safety of the algorithm is often compromised due to instability or the presence of ill-defined reward functions. With the use of control barrier functions embedded into the RL policy, we arrive at safe policies to optimize the performance of the autonomous driving vehicle through the advantage of a safety layer over the RL methods to ease the design of reward functions. However, control barrier functions need a good approximation of the model of the system. We use probabilistic control barrier functions [4] to account for model uncertainty. Our Safety-Assured Policy Optimization - Ramp Merging (SAPO-RM) algorithm is implemented online in the CARLA [1] Simulator and offline on the US I-80 dataset extracted from the NGSIM Database provided by NHTSA [2]. We further test the algorithm and perform ablation studies of it on the US-101 and exi-D datasets to compare the approaches. The proposed algorithm can also be applied to other driving scenarios by changing the reward and safety constraints.

AAAI Conference 2023 Short Paper

Risk-Aware Decentralized Safe Control via Dynamic Responsibility Allocation (Student Abstract)

Yiwei Lyu
Wenhao Luo
John M. Dolan

In this work, we present a novel risk-aware decentralized Control Barrier Function (CBF)-based controller for multi-agent systems. The proposed decentralized controller is composed based on pairwise agent responsibility shares (a percentage), calculated from the risk evaluation of each individual agent faces in a multi-agent interaction environment. With our proposed CBF-inspired risk evaluation framework, the responsibility portions between pairwise agents are dynamically updated based on the relative risk they face. Our method allows agents with lower risk to enjoy a higher level of freedom in terms of a wider action space, and the agents exposed to higher risk are constrained more tightly on action spaces, and are therefore forced to proceed with caution.

PDF Details DOI

IROS Conference 2023 Conference Paper

Risk-Aware Safe Control for Decentralized Multi-Agent Systems via Dynamic Responsibility Allocation

Yiwei Lyu 0002
Wenhao Luo
John M. Dolan

Decentralized control schemes are increasingly favored in various domains that involve multi-agent systems due to the need for computational efficiency as well as general applicability to large-scale systems. However, in the absence of an explicit global coordinator, it is hard for distributed agents to determine how to efficiently interact with others. In this paper, we present a risk-aware decentralized control framework that provides guidance on how much relative responsibility share (a percentage) an individual agent should take to avoid collisions with others while moving efficiently without direct communications. We propose a novel Control Barrier Function (CBF)-inspired risk measurement to characterize the aggregate risk agents face from potential collisions under motion uncertainty. We use this measurement to allocate responsibility shares among agents dynamically and develop risk-aware decentralized safe controllers. In this way, we are able to leverage the flexibility of robots with lower risk to improve the motion flexibility for those with higher risk, thus achieving improved collective safety. We demonstrate the validity and efficiency of our proposed approach through two examples: ramp merging in autonomous driving and a multi-agent position-swapping game.

AAAI Conference 2023 Short Paper

Tackling Safe and Efficient Multi-Agent Reinforcement Learning via Dynamic Shielding (Student Abstract)

Wenli Xiao
Yiwei Lyu
John M. Dolan

Multi-agent Reinforcement Learning (MARL) has been increasingly used in safety-critical applications but has no safety guarantees, especially during training. In this paper, we propose dynamic shielding, a novel decentralized MARL framework to ensure safety in both training and deployment phases. Our framework leverages Shield, a reactive system running in parallel with the reinforcement learning algorithm to monitor and correct agents' behavior. In our algorithm, shields dynamically split and merge according to the environment state in order to maintain decentralization and avoid conservative behaviors while enjoying formal safety guarantees. We demonstrate the effectiveness of MARL with dynamic shielding in the mobile navigation scenario.

PDF Details DOI

AAAI Conference 2022 Short Paper

Adaptive Safe Behavior Generation for Heterogeneous Autonomous Vehicles Using Parametric-Control Barrier Functions (Student Abstract)

Yiwei Lyu
Wenhao Luo
John M. Dolan

Control Barrier Functions have been extensively studied to ensure guaranteed safety during inter-robot interactions. In this paper, we introduce the Parametric-Control Barrier Function (Parametric-CBF), a novel variant of the traditional Control Barrier Function to extend its expressivity in describing different safe behaviors among heterogeneous robots. A parametric-CBF based framework is presented to enable the ego robot to model the neighboring robots behavior and further improve the coordination efficiency during interaction while enjoying formally provable safety guarantees. We demonstrate the usage of Parametric-CBF in behavior prediction and adaptive safe control in the ramp merging scenario.

ICML Conference 2022 Conference Paper

Addressing Optimism Bias in Sequence Modeling for Reinforcement Learning

Adam Villaflor
Zhe Huang
Swapnil Pande
John M. Dolan
Jeff G. Schneider

Impressive results in natural language processing (NLP) based on the Transformer neural network architecture have inspired researchers to explore viewing offline reinforcement learning (RL) as a generic sequence modeling problem. Recent works based on this paradigm have achieved state-of-the-art results in several of the mostly deterministic offline Atari and D4RL benchmarks. However, because these methods jointly model the states and actions as a single sequencing problem, they struggle to disentangle the effects of the policy and world dynamics on the return. Thus, in adversarial or stochastic environments, these methods lead to overly optimistic behavior that can be dangerous in safety-critical systems like autonomous driving. In this work, we propose a method that addresses this optimism bias by explicitly disentangling the policy and world models, which allows us at test time to search for policies that are robust to multiple possible futures in the environment. We demonstrate our method’s superior performance on a variety of autonomous driving tasks in simulation.

IROS Conference 2022 Conference Paper

Motion Planning by Search in Derivative Space and Convex Optimization with Enlarged Solution Space

Jialun Li
Xiaojia Xie
Qin Lin 0001
Jianping He 0001
John M. Dolan

To efficiently generate safe trajectories for an autonomous vehicle in dynamic environments, a layered motion planning method with decoupled path and speed planning is widely used. This paper studies speed planning, which mainly deals with dynamic obstacle avoidance given a planned path. The main challenges lie in the optimization in a non-convex space and the trade-off between safety, comfort, and efficiency. First, this work proposes to conduct a search in second-order derivative space for generating a comfort-optimal reference trajectory. Second, by combining abstraction and refinement, an algorithm is proposed to construct a convex feasible space for optimization. Finally, a piecewise Bézier polynomial optimization approach with trapezoidal corridors is presented, which theoretically guarantees safety and significantly enlarges the solution space compared with the existing rectangular corridors-based approach. We validate the efficiency and effectiveness of the proposed approach in simulations.

IROS Conference 2022 Conference Paper

Online Adaptive Compensation for Model Uncertainty Using Extreme Learning Machine-based Control Barrier Functions

Emanuel Munoz
Dvij Kalaria
Qin Lin 0001
John M. Dolan

A control barrier functions-based quadratic programming (CBF-QP) method has emerged as a controller synthesis tool to assure safety of autonomous systems owing to the appealing safe forward invariant set. However, the provable safety relies on a precisely described dynamic model, which is not always available in practice. Recent works leverage learning to compensate model uncertainty for a CBF controller. However, these approaches based on reinforcement learning or episodic learning are limited to dealing with time-invariant uncertainty. Also, the reinforcement learning approach learns the uncertainty offline, while episodic learning only updates the controller after a batch of data is available by the end of an episode. Instead, we propose a novel tuning extreme learning machine (tELM)-based CBF controller that can compensate time-variant and time-invariant model uncertainty adaptively in an online manner. We validate our approach's effectiveness in a simulation of an Adaptive Cruise Control (ACC) system.

ICRA Conference 2022 Conference Paper

Speed Planning in Dynamic Environments over a Fixed Path for Autonomous Vehicles

Wenda Xu
John M. Dolan

In this paper, we present a novel convex optimization approach to address the minimum-time speed planning problem over a fixed path with dynamic obstacle constraints and point-wise speed and acceleration constraints. The contributions of this paper are three-fold. First, we formulate the speed planning as an iterative convex optimization problem based on space discretization. Our formulation allows imposing dynamic obstacle constraints and point-wise speed and acceleration constraints simultaneously. Second, we propose a modified vertical cell decomposition method to handle dynamic obstacles. It divides the freespace into channels, where each channel represents a homotopy of free paths and defines convex constraints for dynamic obstacles. Third, we demonstrate significant improvement over previous work on speed planning for typical driving scenarios such as following, merging, and crossing.

IROS Conference 2022 Conference Paper

State Dropout-Based Curriculum Reinforcement Learning for Self-Driving at Unsignalized Intersections

Shivesh Khaitan
John M. Dolan

Traversing intersections is a challenging problem for autonomous vehicles, especially when the intersections do not have traffic control. Recently deep reinforcement learning has received massive attention due to its success in dealing with autonomous driving tasks. In this work, we address the problem of traversing unsignalized intersections using a novel curriculum for deep reinforcement learning. The proposed curriculum leads to: 1) A faster training process for the reinforcement learning agent, and 2) Better performance compared to an agent trained without curriculum. Our main contribution is two-fold: 1) Presenting a unique curriculum for training deep reinforcement learning agents, and 2) demonstrating the performance improvement using the proposed curriculum in the unsignalized intersection traversal task. The framework expects processed observations of the surroundings from the perception system of the autonomous vehicle. We test our method in the CommonRoad motion planning simulator on T-intersections and four-way intersections.

ICRA Conference 2021 Conference Paper

Autonomous Vehicle Motion Planning via Recurrent Spline Optimization

Wenda Xu
Qian Wang 0010
John M. Dolan

Trajectory planning in dynamic environments can be decomposed into two sub-problems: 1) planning a path to avoid static obstacles, 2) then planning a speed profile to avoid dynamic obstacles. This is also called path-speed decomposition. In this work, we present a novel approach to solve the first sub-problem, motion planning with static obstacles. From an optimization perspective, motion planning for autonomous vehicles can be viewed as non-convex constrained nonlinear optimization, which requires a good enough initial guess to start and is often sensitive to algorithm parameters. We formulate motion planning as convex spline optimization. The convexity of the formulated problem makes it able to be solved fast and reliably, while guaranteeing a global optimum. We then reorganize the constrained spline optimization into a recurrent formulation, which further reduces the computational time to be linear in the optimization horizon size. The proposed method can be applied to both trajectory generation and motion planning problems. Its effectiveness is demonstrated in challenging scenarios such as tight lane changes and sharp turns.

ICRA Conference 2021 Conference Paper

Behavior Planning at Urban Intersections through Hierarchical Reinforcement Learning *

Zhiqian Qiao
Jeff G. Schneider
John M. Dolan

For autonomous vehicles, effective behavior planning is crucial to ensure safety of the ego car. In many urban scenarios, it is hard to create sufficiently general heuristic rules, especially for challenging scenarios that some new human drivers find difficult. In this work, we propose a behavior planning structure based on reinforcement learning (RL) which is capable of performing autonomous vehicle behavior planning with a hierarchical structure in simulated urban environments. Application of the hierarchical structure [1] allows the various layers of the behavior planning system to be satisfied. Our algorithms can perform better than heuristic-rule-based methods for elective decisions such as when to turn left between vehicles approaching from the opposite direction or possible lane-change when approaching an intersection due to lane blockage or delay in front of the ego car. Such behavior is hard to evaluate as correct or incorrect, but some aggressive expert human drivers handle such scenarios effectively and quickly. On the other hand, compared to traditional RL methods, our algorithm is more sample-efficient, due to the use of a hybrid reward mechanism and heuristic exploration during the training process. The results also show that the proposed method converges to an optimal policy faster than traditional RL methods.

ICRA Conference 2021 Conference Paper

ENCODE: a dEep poiNt Cloud ODometry nEtwork

Yihuan Zhang
Liang Wang
Chen Fu
Yifan Dai
John M. Dolan

Ego-motion estimation is a key requirement for the simultaneous localization and mapping (SLAM) problem. The traditional pipeline goes through feature extraction, feature matching and pose estimation, whose performance depends on the manually designed features. In this paper, we are motivated by the strong performance of deep learning methods in other computer vision and robotics tasks. We replace hand-crafted features with a neural network and directly estimate the relative pose between two adjacent scans from a LiDAR sensor using ENCODE: a dEep poiNt Cloud ODometry nEtwork. Firstly, a spherical projection of the input point cloud is performed to acquire a multi-channel vertex map. Then a multi-layer network backbone is applied to learn the abstracted features and a fully connected layer is adopted to estimate the 6-DoF ego-motion. Additionally, a map-to-map optimization module is applied to update the local poses and output a smooth map. Experiments on multiple datasets demonstrate that the proposed method achieves the best performance in comparison to state-of-the-art methods and is capable of providing accurate poses with low drift in various kinds of scenarios.

ICRA Conference 2021 Conference Paper

Learning to Robustly Negotiate Bi-Directional Lane Usage in High-Conflict Driving Scenarios

Christoph Killing
Adam Villaflor
John M. Dolan

Recently, autonomous driving has made substantial progress in addressing the most common traffic scenarios like intersection navigation and lane changing. However, most of these successes have been limited to scenarios with well-defined traffic rules and require minimal negotiation with other vehicles. In this paper, we introduce a previously unconsidered, yet everyday, high-conflict driving scenario requiring negotiations between agents of equal rights and priorities. There exists no centralized control structure and we do not allow communications. Therefore, it is unknown if other drivers are willing to cooperate, and if so to what extent. We train policies to robustly negotiate with opposing vehicles of an unobservable degree of cooperativeness using multi-agent reinforcement learning (MARL). We propose Discrete Asymmetric Soft Actor-Critic (DASAC), a maximum- entropy off-policy MARL algorithm allowing for centralized training with decentralized execution. We show that using DASAC we are able to successfully negotiate and traverse the scenario considered over 99% of the time. Our agents are robust to an unknown timing of opponent decisions, an unobservable degree of cooperativeness of the opposing vehicle, and previously unencountered policies. Furthermore, they learn to exhibit human-like behaviors such as defensive driving, anticipating solution options and interpreting the behavior of other agents.

ICRA Conference 2021 Conference Paper

Linear Inverse Problem for Depth Completion with RGB Image and Sparse LIDAR Fusion

Chen Fu
Christoph Mertz
John M. Dolan

Comprehensive depth information from surrounding scenes is important for perception in autonomous driving and robots. Sparse LIDAR sensors give a low-density point cloud of the environment, but are more affordable than their high-density counterparts. In this paper, we propose a novel sensor fusion architecture for sparse LIDAR depth completion. Instead of the traditional end-to-end neural network-based algorithm, we formulate depth completion as a Linear Inverse Problem (LIP) with a multi-modal proximal operator. This sensor fusion architecture allows a better signal prior and finds the unique optimal solution to the LIP. Instead of learning a unified network for the sparse input which treats pixels evenly, the proposed architecture guarantees both the data consistency and smoothness of the predicted depth map. To demonstrate the performance of our algorithm, we benchmark on the simulation dataset TartanAir, and the real indoor NYUdepthv2 and real outdoor KITTI datasets. Our proposed method outperforms previous methods and uses fewer parameters in both indoor and outdoor datasets.

ICRA Conference 2021 Conference Paper

Probabilistic Safety-Assured Adaptive Merging Control for Autonomous Vehicles

Yiwei Lyu 0002
Wenhao Luo
John M. Dolan

Autonomous vehicles face tremendous challenges while interacting with human drivers in different kinds of scenarios. Developing control methods with safety guarantees while performing interactions with uncertainty is an ongoing research goal. In this paper, we present a real-time safe control framework using bi-level optimization with Control Barrier Function (CBF) that enables an autonomous ego vehicle to interact with human-driven cars in ramp merging scenarios with a consistent safety guarantee. In order to explicitly address motion uncertainty, we propose a novel extension of control barrier functions to a probabilistic setting with provable chance-constrained safety and analyze the feasibility of our control design. The formulated bi-level optimization framework entails first choosing the ego vehicle's optimal driving style in terms of safety and primary objective, and then minimally modifying a nominal controller in the context of quadratic programming subject to the probabilistic safety constraints. This allows for adaptation to different driving strategies with a formally provable feasibility guarantee for the ego vehicle's safe controller. Experimental results are provided to demonstrate the effectiveness of our proposed approach.

IROS Conference 2020 Conference Paper

Depth Completion via Inductive Fusion of Planar LIDAR and Monocular Camera

Chen Fu
Chiyu Dong
Christoph Mertz
John M. Dolan

Modern high-definition LIDAR is expensive for commercial autonomous driving vehicles and small indoor robots. An affordable solution to this problem is fusion of planar LIDAR with RGB images to provide a similar level of perception capability. Even though state-of-the-art methods provide approaches to predict depth information from limited sensor input, they are usually a simple concatenation of sparse LIDAR features and dense RGB features through an end-to-end fusion architecture. In this paper, we introduce an inductive late-fusion block which better fuses different sensor modalities inspired by a probability model. The proposed demonstration and aggregation network propagates the mixed context and depth features to the prediction network and serves as a prior knowledge of the depth completion. This late-fusion block uses the dense context features to guide the depth prediction based on demonstrations by sparse depth features. In addition to evaluating the proposed method on benchmark depth completion datasets including NYUDepthV2 and KITTI, we also test the proposed method on a simulated planar LIDAR dataset. Our method shows promising results compared to previous approaches on both the benchmark datasets and simulated dataset with various 3D densities.

ICRA Conference 2020 Conference Paper

FG-GMM-based Interactive Behavior Estimation for Autonomous Driving Vehicles in Ramp Merging Control *

Yiwei Lyu 0002
Chiyu Dong
John M. Dolan

Interactive behavior is important for autonomous driving vehicles, especially for scenarios like ramp merging which require significant social interaction between autonomous driving vehicles and human-driven cars. This paper enhances our previous Probabilistic Graphical Model (PGM) merging control model for the interactive behavior of autonomous driving vehicles. To better estimate the interactive behavior for autonomous driving cars, a Factor Graph (FG) is used to describe the dependency among observations and estimate other cars’ intentions. Real trajectories are used to approximate the model instead of human-designed models or cost functions. Forgetting factors and a Gaussian Mixture Model (GMM) are also applied in the intention estimation process for stabilization, interpolation and smoothness. The advantage of the factor graph is that the relationship between its nodes can be described by self-defined functions, instead of probabilistic relationships as in PGM, giving more flexibility. Continuity of GMM also provides higher accuracy than the previous discrete speed transition model. The proposed method enhances the overall performance of intention estimation, in terms of collision rate and average distance between cars after merging, which means it is safer and more efficient.

IROS Conference 2020 Conference Paper

Hierarchical Reinforcement Learning Method for Autonomous Vehicle Behavior Planning

Zhiqian Qiao
Zachariah Tyree
Priyantha Mudalige
Jeff G. Schneider
John M. Dolan

Behavioral decision making is an important aspect of autonomous vehicles (AV). In this work, we propose a behavior planning structure based on hierarchical reinforcement learning (HRL) which is capable of performing autonomous vehicle planning tasks in simulated environments with multiple sub-goals. In this hierarchical structure, the network is capable of 1) learning one task with multiple sub-goals simultaneously; 2) extracting attentions of states according to changing sub-goals during the learning process; 3) reusing the well-trained network of sub-goals for other tasks with the same sub-goals. A hybrid reward mechanism is designed for different hierarchical layers in the proposed HRL structure. Compared to traditional RL methods, our algorithm is more sample-efficient, since its modular design allows reusing the policies of sub-goals across similar tasks for various transportation scenarios. The results show that the proposed method converges to an optimal policy faster than traditional RL methods.

ICRA Conference 2020 Conference Paper

Human Driver Behavior Prediction based on UrbanFlow *

Zhiqian Qiao
Jing Zhao
Jin Zhu
Zachariah Tyree
Priyantha Mudalige
Jeff G. Schneider
John M. Dolan

How autonomous vehicles and human drivers share public transportation systems is an important problem, as fully automatic transportation environments are still a long way off. Understanding human drivers’ behavior can be beneficial for autonomous vehicle decision making and planning, especially when the autonomous vehicle is surrounded by human drivers who have various driving behaviors and patterns of interaction with other vehicles. In this paper, we propose an LSTM-based trajectory prediction method for human drivers which can help the autonomous vehicle make better decisions, especially in urban intersection scenarios. Meanwhile, in order to collect human drivers’ driving behavior data in the urban scenario, we describe a system called UrbanFlow which includes the whole procedure from raw bird’s-eye view data collection via drone to the final processed trajectories. The system is mainly intended for urban scenarios but can be extended to be used for any traffic scenarios.

IROS Conference 2020 Conference Paper

ReachFlow: An Online Safety Assurance Framework for Waypoint-Following of Self-driving Cars

Qin Lin 0001
Xin Chen 0002
Aman Khurana
John M. Dolan

Learning-enabled components have been widely deployed in autonomous systems. However, due to the weak interpretability and the prohibitively high complexity of large-scale machine learning models such as neural networks, reliability has been a crucial concern for safety-critical autonomous systems. This work proposes an online monitor called Reach-Flow for fault prevention of waypoint-following tasks for self-driving cars. It mainly consists of two components: (a) an online verification tool which conservatively checks the safety of the system behavior in the near future, and (b) a fallback controller which steers the system back to a desired state when the system is potentially unsafe. We implement ReachFlow in a self-driving racing car governed by a reinforcement learning-based controller. We demonstrate the effectiveness by rigorously verifying a safe waypoint-following control and providing a fallback control for an unsafe situation in which a large deviation from the planned path is predicted.

IROS Conference 2020 Conference Paper

Safe Planning for Self-Driving Via Adaptive Constrained ILQR

Yanjun Pan 0002
Qin Lin 0001
Het Shah
John M. Dolan

Constrained Iterative Linear Quadratic Regulator (CILQR), a variant of ILQR, has been recently proposed for motion planning problems of autonomous vehicles to deal with constraints such as obstacle avoidance and reference tracking. However, the previous work considers either deterministic trajectories or persistent prediction for target dynamical obstacles. The other drawback is lack of generality - it requires manual weight tuning for different scenarios. In this paper, two significant improvements are achieved. Firstly, a two-stage uncertainty-aware prediction is proposed. The short-term prediction with safety guarantee based on reachability analysis is responsible for dealing with extreme maneuvers conducted by target vehicles. The long-term prediction leveraging an adaptive least square filter preserves the long-term optimality of the planned trajectory since using reachability only for long-term prediction is too pessimistic and makes the planner over-conservative. Secondly, to allow a wider coverage over different scenarios and to avoid tedious parameter tuning case by case, this paper designs a scenario-based analytical function taking the states from the ego vehicle and the target vehicle as input, and carrying weights of a cost function as output. It allows the ego vehicle to execute multiple behaviors (such as lane-keeping and overtaking) under a single planner. We demonstrate safety, effectiveness, and real-time performance of the proposed planner in simulations.

IROS Conference 2019 Conference Paper

Attention-based Hierarchical Deep Reinforcement Learning for Lane Change Behaviors in Autonomous Driving

Yilun Chen
Chiyu Dong
Praveen Palanisamy
Priyantha Mudalige
Katharina Muelling
John M. Dolan

Performing safe and efficient lane changes is a crucial feature for creating fully autonomous vehicles. Recent advances have demonstrated successful lane following behavior using deep reinforcement learning, yet the interactions with other vehicles on-road for lane changes are rarely considered. In this paper, we design a hierarchical Deep Reinforcement Learning (DRL) algorithm to learn lane change behaviors in dense traffic. By breaking down overall behavior to sub-policies, faster and safer lane change actions can be learned. We also apply temporal and spatial attention to the DRL architecture, which helps the vehicle focus more on surrounding vehicles and leads to smoother lane change behavior. We conduct our experiments in the TORCS simulator and the results outperform the state-of-the-art deep reinforcement learning algorithm in various lane change scenarios.

ICRA Conference 2019 Conference Paper

Interactive Trajectory Prediction for Autonomous Driving via Recurrent Meta Induction Neural Network

Chiyu Dong
Yilun Chen
John M. Dolan

Interactive driving is challenging but essential for autonomous cars in dense traffic or urban areas. Proper interaction requires understanding and prediction of future trajectories of all neighboring cars around a target vehicle. Current solutions typically assume a certain distribution or stochastic process to approximate human-driven cars' behaviors. To relax this assumption, a Recurrent Meta Induction Network (RMIN) framework is developed. The original Conditional Neural Process (CNP) on which this is based does not consider the sequence of the conditions, due to the permutation invariance requirements for stochastic processes. However, the sequential information is important for the driving behavior estimation. Therefore, in the proposed method, a recurrent neural cell replaces the original demonstration sub-net. The behavior estimation is conditioned on the historical observations for all related cars, including the target car and its surrounding cars. The method is applied to predict the lane change trajectory of a target car in dense traffic areas. The proposed method achieves better results than previous methods and thanks to the meta-learning framework, it can use a smaller dataset, putting fewer demands on autonomous driving data collection.

IROS Conference 2017 Conference Paper

Lane-change social behavior generator for autonomous driving car by non-parametric regression in Reproducing Kernel Hilbert Space

Chiyu Dong
Yihuan Zhang
John M. Dolan

Nowadays, self-driving cars are being applied to more complex urban scenarios including intersections, merging ramps or lane changes. It is, therefore, important for self-driving cars to behave socially with human-driven cars. In this paper, we focus on generating the lane change behavior for self-driving cars: perform a safe and effective lane change behavior once a lane-change command is received. Our method bridges the gap between higher-level behavior commands and the trajectory planner. There are two challenges in the task: 1) Analyzing the surrounding vehicles' mutual effects from their trajectories. 2) Estimating the proper lane change start point and end point according to the analysis of surrounding vehicles. We propose a learning-based approach to understand surrounding traffic and make decisions for a safe lane change. Our contributions and advantages of the approach are: 1 Considers the behavior generator as a continuous function in Reproducing Kernel Hilbert Space (RKHS) which contains a family of behavior generators; 2 Constructs the behavior generator function in RKHS by non-parametric regressions on training data; 3 Takes past trajectories of all related surrounding cars as input to capture mutual interactions and output continuous values to represent behaviors. Experimental results show that the proposed approach is able to generate feasible and human-like lane-change behavior (represented by start and end points) in multi-car environments. The experiments also verified that our suggested kernel outperforms the ones which were used in a previous method.

IROS Conference 2016 Conference Paper

Automated tactical maneuver discovery, reasoning and trajectory planning for autonomous driving

Tianyu Gu
John M. Dolan
Jin-Woo Lee 0003

In a hierarchical motion planning system for urban autonomous driving, it is a common practice to separate tactical reasoning from the lower-level trajectory planning. This separation makes it difficult to achieve robust maneuver-based tactical reasoning, which is intrinsically linked to trajectory planning. We therefore propose a planning method that automatically discovers tactical maneuver patterns, and fuses pattern reasoning and sampling-based trajectory planning. The results demonstrate enhanced planning feasibility, coherency and scalability.

IROS Conference 2015 Conference Paper

COLREGS-compliant target following for an Unmanned Surface Vehicle in dynamic environments

Pranay Agrawal
John M. Dolan

This paper presents the autonomous tracking and following of a marine vessel by an Unmanned Surface Vehicle in the presence of dynamic obstacles while following the International Regulations for Preventing Collisions at Sea (COLREGS) rules. The motion prediction for the target vessel is based on Monte-Carlo sampling of dynamically feasible and collision-free paths with fuzzy weights, leading to a predicted path resembling anthropomorphic driving behavior. This prediction is continuously optimized for a particular target by learning the necessary parameters for a 3-degree-of-freedom model of the vessel and its maneuvering behavior from its path history without any prior knowledge. The path planning for the USV with COLREGS is achieved on a grid-based map in a single stage by incorporating A* path planning with Artificial Terrain Costs for dynamically changing obstacles. Various scenarios for interaction, including multiple civilian and adversarial vessels, are handled by the planner with ease. The effectiveness of the algorithms has been demonstrated both in representative simulations and on-water experiments.

IROS Conference 2015 Conference Paper

Tunable and stable real-time trajectory planning for urban autonomous driving

Tianyu Gu
Jason Atwood
Chiyu Dong
John M. Dolan
Jin-Woo Lee 0003

This paper investigates real-time on-road motion planning algorithms for autonomous passenger vehicles (APV) in urban environments, and propose a computationally efficient planning formulation. Two key properties, tunability and stability, are emphasized when designing the proposed planner. The main contributions of this paper are: • A computationally efficient decoupled space-time trajectory planning structure. • The formulation of optimization-free elastic-band-based path planning and speed-constraint-based temporal planning routines with pre-determined runtime. • Identification of continuity problems with previous cost-based planners that cause tunability and stability issues.

ICRA Conference 2014 Conference Paper

Motion planning under uncertainty for on-road autonomous driving

Wenda Xu
Jia Pan 0001
Junqing Wei
John M. Dolan

We present a motion planning framework for autonomous on-road driving considering both the uncertainty caused by an autonomous vehicle and other traffic participants. The future motion of traffic participants is predicted using a local planner, and the uncertainty along the predicted trajectory is computed based on Gaussian propagation. For the autonomous vehicle, the uncertainty from localization and control is estimated based on a Linear-Quadratic Gaussian (LQG) framework. Compared with other safety assessment methods, our framework allows the planner to avoid unsafe situations more efficiently, thanks to the direct uncertainty information feedback to the planner. We also demonstrate our planner's ability to generate safer trajectories compared to planning only with a LQG framework.

IROS Conference 2013 Conference Paper

Learning-based event response for marine robotics

Matthew Bernstein
Rishi Graham
Danelle Cline
John M. Dolan
Kanna Rajan

Robotic vehicles have become a critical tool for studying the under-sampled coastal ocean. This has led to new paradigms in scientific discovery. The combination of agility, reactivity, and persistent presence makes autonomous robots ideal for targeted sampling of elusive, episodic events such as algal blooms. In order to achieve this goal, they need to be deployed at the right place and time. To that end, we have designed and will soon deploy a shore-based event recognition technology to continuously monitor remote sensing imagery for algal blooms as targets for robotic field experiments. A Support Vector Machine underlies a field-tested decision support system which scientists will consult prior to deploying robots in the coastal ocean. Our aim is to target oceanographic field experiments for evaluation and verification.

ICRA Conference 2012 Conference Paper

A real-time motion planner with trajectory optimization for autonomous vehicles

Wenda Xu
Junqing Wei
John M. Dolan
Huijing Zhao
Hongbin Zha

In this paper, an efficient real-time autonomous driving motion planner with trajectory optimization is proposed. The planner first discretizes the plan space and searches for the best trajectory based on a set of cost functions. Then an iterative optimization is applied to both the path and speed of the resultant trajectory. The post-optimization is of low computational complexity and is able to converge to a higher-quality solution within a few iterations. Compared with the planner without optimization, this framework can reduce the planning time by 52% and improve the trajectory quality. The proposed motion planner is implemented and tested both in simulation and on a real autonomous vehicle in three different scenarios. Experiments show that the planner outputs high-quality trajectories and performs intelligent driving behaviors.

UAI Conference 2012 Conference Paper

Decentralized Data Fusion and Active Sensing with Mobile Sensors for Modeling and Predicting Spatiotemporal Traffic Phenomena

Jie Chen 0027
Bryan Kian Hsiang Low
Colin Keng-Yan Tan
Ali Oran
Patrick Jaillet
John M. Dolan
Gaurav S. Sukhatme

The problem of modeling and predicting spatiotemporal traffic phenomena over an urban road network is important to many traffic applications such as detecting and forecasting congestion hotspots. This paper presents a decentralized data fusion and active sensing (D2 FAS) algorithm for mobile sensors to actively explore the road network to gather and assimilate the most informative data for predicting the traffic phenomenon. We analyze the time and communication complexity of D2 FAS and demonstrate that it can scale well with a large number of observations and sensors. We provide a theoretical guarantee on its predictive performance to be equivalent to that of a sophisticated centralized sparse approximation for the Gaussian process (GP) model: The computation of such a sparse approximate GP model can thus be parallelized and distributed among the mobile sensors (in a Google-like MapReduce paradigm), thereby achieving efficient and scalable prediction. We also theoretically guarantee its active sensing performance that improves under various practical environmental conditions. Empirical evaluation on real-world urban road network data shows that our D2 FAS algorithm is significantly more time-efficient and scalable than state-ofthe-art centralized algorithms while achieving comparable predictive performance.

ICRA Conference 2011 Conference Paper

A point-based MDP for robust single-lane autonomous driving behavior under uncertainties

Junqing Wei
John M. Dolan
Jarrod M. Snider
Bakhtiar Litkouhi

In this paper, a point-based Markov Decision Process (QMDP) algorithm is used for robust single-lane autonomous driving behavior control under uncertainties. Autonomous vehicle decision making is modeled as a Markov Decision Process (MDP), then extended to a QMDP framework. Based on MDP/QMDP, three kinds of uncertainties are taken into account: sensor noise, perception constraints and surrounding vehicles' behavior. In simulation, the QMDP-based reasoning framework makes the autonomous vehicle perform with differing levels of conservativeness corresponding to different perception confidence levels. Road tests also indicate that the proposed algorithm helps the vehicle in avoiding potentially unsafe situations under these uncertainties. In general, the results indicate that the proposed QMDP-based algorithm makes autonomous driving more robust to limited sensing ability and occasional sensor failures.

AAMAS Conference 2011 Conference Paper

Active Markov Information-Theoretic Path Planning for Robotic Environmental Sensing

Kian Hsiang Low
John M. Dolan
Pradeep Khosla

Recent research in multi-robot exploration and mapping has focused on sampling environmental fields, which are typically modeled using the Gaussian process (GP). Existing information-theoretic exploration strategies for learning GP-based environmental field maps adopt the non-Markovian problem structure and consequently scale poorly with the length of history of observations. Hence, it becomes computationally impractical to use these strategies for in situ, real-time active sampling. To ease this computational burden, this paper presents a Markov-based approach to efficient information-theoretic path planning for active sampling of GP-based fields. We analyze the time complexity of solving the Markov-based path planning problem, and demonstrate analytically that it scales better than that of deriving the non-Markovian strategies with increasing length of planning horizon. For a class of exploration tasks called the transect sampling task, we provide theoretical guarantees on the active sampling performance of our Markov-based policy, from which ideal environmental field conditions and sampling task settings can be established to limit its performance degradation due to violation of the Markov assumption. Empirical evaluation on real-world temperature and plankton density field data shows that our Markov-based policy can generally achieve active sampling performance comparable to that of the widely-used non-Markovian greedy policies under less favorable realistic field conditions and task settings while enjoying significant computational gain over them.

ICRA Conference 2011 Conference Paper

Motion planning for autonomous driving with a conformal spatiotemporal lattice

Matthew McNaughton
Chris Urmson
John M. Dolan
Jin-Woo Lee 0003

We present a motion planner for autonomous highway driving that adapts the state lattice framework pioneered for planetary rover navigation to the structured environment of public roadways. The main contribution of this paper is a search space representation that allows the search algorithm to systematically and efficiently explore both spatial and temporal dimensions in real time. This allows the low-level trajectory planner to assume greater responsibility in planning to follow a leading vehicle, perform lane changes, and merge between other vehicles. We show that our algorithm can readily be accelerated on a GPU, and demonstrate it on an autonomous passenger vehicle.

ICRA Conference 2011 Conference Paper

Toward adaptation and reuse of advanced robotic software

Christopher R. Baker
John M. Dolan
Shige Wang
Bakhtiar Litkouhi

As robotic software systems become larger and more complex, it is increasingly important to reuse existing software components to control development costs. For a broad class of relatively simple components, ranging from sensor and actuation interfaces to many simple perception and navigation algorithms, this can be a reasonably straightforward process, and many excellent frameworks have been developed in recent years that support reuse of such components in novel systems. However, there is also a class of more advanced software components, such as for modeling and interacting with complex environments, for which reuse can be a much more challenging problem. In particular, many advanced robotic algorithms can be highly sensitive to the perception and actuation capabilities of the specific robots they are deployed on, in turn requiring significant and invasive modifications to accommodate the specific capabilities of a different robotic system. This work examines the nature of this sensitivity and proposes a novel design methodology for isolating a stable, reusable "core" algorithm from any platform-specific enhancements, or "supplemental" effects. Modern software engineering techniques are used to encapsulate these supplemental effects separately from the core algorithm, allowing platform-specific details to be accommodated by modular substitution instead of direct modification of the "core" component. This methodology is experimentally evaluated on existing software for autonomous driving behaviors, yielding useful insights into the creation of highly adaptable robotic software components.

IROS Conference 2010 Conference Paper

Reliability impact on planetary robotic missions

David Asikin
John M. Dolan

In the mobile robotics literature, there is little formal discussion of reliability and failure. Moreover, current work focuses more on the assessment of existing robots. In contrast, our work predicts the impact on reliability on robotic missions. In our previous work, we presented a quantitative analysis to predict the probability of robot failure during a mission and use this to compare the performance of different robot team configurations. In order to comprehensively characterize robot failure, we proposed a taxonomy system which divides planetary robotic missions into three classes and showed how the taxonomy can be used as a framework to explore the reliability characteristics of each mission class. In this paper, we define and simulate common mission scenarios for each class in the taxonomy system and attempt to extract general reliability trends and mission characteristics for given robot and environment parameters. Our results show that, for comparable mission scopes with a fixed budget, exploration-type missions have maximum mission success probability for smaller team sizes than is the case for construction-type missions.

ICAPS Conference 2009 Conference Paper

Information-Theoretic Approach to Efficient Adaptive Path Planning for Mobile Robotic Environmental Sensing

Bryan Kian Hsiang Low
John M. Dolan
Pradeep K. Khosla

Recent research in robot exploration and mapping has focused on sampling environmental hotspot fields. This exploration task is formalized by Low, Dolan, and Khosla (2008) in a sequential decision-theoretic planning under uncertainty framework called MASP. The time complexity of solving MASP approximately depends on the map resolution, which limits its use in large-scale, high-resolution exploration and mapping. To alleviate this computational difficulty, this paper presents an information-theoretic approach to MASP (iMASP) for efficient adaptive path planning; by reformulating the cost-minimizing iMASP as a reward-maximizing problem, its time complexity becomes independent of map resolution and is less sensitive to increasing robot team size as demonstrated both theoretically and empirically. Using the reward-maximizing dual, we derive a novel adaptive variant of maximum entropy sampling, thus improving the induced exploration policy performance. It also allows us to establish theoretical bounds quantifying the performance advantage of optimal adaptive over non-adaptive policies and the performance quality of approximately optimal vs. optimal adaptive policies. We show analytically and empirically the superior performance of iMASP-based policies for sampling the log-Gaussian process to that of policies for the widely-used Gaussian process in mapping the hotspot field. Lastly, we provide sufficient conditions that, when met, guarantee adaptivity has no benefit under an assumed environment model.

ICRA Conference 2008 Conference Paper

Operation of robotic science boats using the telesupervised adaptive ocean sensor fleet system

Gregg Podnar
John M. Dolan
Alberto Elfes
Stephen B. Stancliff
Ellie Lin
Jeffrey C. Hosier
Troy J. Ames
John Moisan

This paper describes a multi-robot science exploration software architecture and system called the telesupervised adaptive ocean sensor fleet (TAOSF). TAOSF supervises and coordinates a group of robotic boats, the OASIS platforms, to enable in situ study of phenomena in the ocean/atmosphere interface, as well as on the ocean surface and sub-surface. The OASIS platforms are extended-deployment autonomous ocean surface vessels, whose development is funded separately by the National Oceanic and Atmospheric Administration (NOAA). TAOSF allows a human operator to effectively supervise and coordinate multiple robotic assets using a multi-level autonomy control architecture, where the operating mode of the vehicles ranges from autonomous control to teleoperated human control. TAOSF increases data-gathering effectiveness and science return while reducing demands on scientists for robotic asset tasking, control, and monitoring. The first field application chosen for TAOSF is the characterization of Harmful Algal Blooms (HABs). We discuss the overall TAOSF architecture, describe field tests conducted under controlled conditions using rhodamine dye as a HAB simulant, present initial results from these tests, and outline the next steps in the development of TAOSF.

IROS Conference 2008 Conference Paper

Traffic interaction in the urban challenge: Putting boss on its best behavior

Christopher R. Baker
John M. Dolan

We describe an autonomous robotic software subsystem for managing mission execution and discrete traffic interaction in the 2007 DARPA Urban Challenge. Its role is reviewed in the context of the software system that controls ldquoBossrdquo, Tartan Racingpsilas winning entry in the competition. Design criteria are presented, followed by the application of software design principles to derive an architecture well suited to the rigors of developing complex robotic systems. Combined with a discussion of robust behavioral algorithms, the designpsilas effectiveness is highlighted in its ability to manage complex autonomous driving behaviors while remaining adaptable to the systempsilas evolving capabilities.

ICRA Conference 2007 Conference Paper

Adaptive Sampling for Multi-Robot Wide-Area Exploration

Bryan Kian Hsiang Low
Geoffrey J. Gordon
John M. Dolan
Pradeep K. Khosla

The exploration problem is a central issue in mobile robotics. A complete coverage is not practical if the environment is large with a few small hotspots, and the sampling cost is high. So, it is desirable to build robot teams that can coordinate to maximize sampling at these hotspots while minimizing resource costs, and consequently learn more accurately about properties of such environmental phenomena. An important issue in designing such teams is the exploration strategy. The contribution of this paper is in the evaluation of an adaptive exploration strategy called adaptive cluster sampling (ACS), which is demonstrated to reduce the resource costs (i. e. , mission time and energy consumption) of a robot team, and yield more information about the environment by directing robot exploration towards hotspots. Due to the adaptive nature of the strategy, it is not obvious how the sampled data can be used to provide unbiased, low-variance estimates of the properties. This paper therefore discusses how estimators that are Rao-Blackwellized can be used to achieve low error. This paper also presents the first analysis of the characteristics of the environmental phenomena that favor the ACS strategy and estimators. Quantitative experimental results in a mineral prospecting task simulation show that our approach is more efficient in exploration by yielding more minerals and information with fewer resources and providing more precise mineral density estimates than previous methods.

IROS Conference 2007 Conference Paper

Scheduling for humans in multirobot supervisory control

Sandra Mau
John M. Dolan

This paper describes efficient utilization of human time by two means: prioritization of human tasks and maximizing multirobot team size. We propose an efficient scheduling algorithm for multirobot supervisory control that helps complete a mission faster. The proposed algorithm is superior to existing algorithms by prioritizing human tasks such that robots can regain autonomous control sooner. In simulations of a multirobot area surveying problem, we show that the rate of area coverage is much higher using our algorithm compared to first-in-first-out. We also show that the use of different scheduling algorithms can affect the maximum number of robots a human can manage on a team. Another significant finding related to maximum team size is that the size is always the same or higher than an often-cited estimate known as fan-out [5]. Since fan-out is derived from an ideal, average case, simulations show that the upper bound on team size is higher than that predicted by the fan-out equation. Fan-out is actually a lower bound on the maximum team size for any practical situation (i. e. , where task lengths and periodicity may vary or when robots are heterogeneous).

IROS Conference 2006 Conference Paper

Mission Reliability Estimation for Multirobot Team Design

Stephen B. Stancliff
John M. Dolan
Ashitey Trebi-Ollennu

One reason given for the use of multirobot systems is that many cheap robots are more reliable than one expensive robot. To date, however, there has been no quantitative analysis to support this assertion. This paper presents the first quantitative support for the argument that larger teams of less-reliable robots can perform certain missions more reliably than smaller teams of more-reliable robots. Our results show that for short missions, in fact, a team of four robots can provide greater mission reliability than a team of two robots, even when the individual robots in the team of four have reliability that is an order of magnitude lower. These results suggest that considerable cost reductions can be achieved for some missions by choosing larger teams of less-reliable robots over smaller teams of more-reliable robots

IROS Conference 2005 Conference Paper

Efficient mapping through exploitation of spatial dependencies

Yaron Rachlin
John M. Dolan
Pradeep K. Khosla

Occupancy grid mapping algorithms assume that grid block values are independently distributed. However, most environments of interest contain spatial patterns that are better characterized by models that capture dependencies among grid blocks. To account for such dependencies, we model the environment as a pairwise Markov random field. We specify a belief propagation-based mapping algorithm that takes these dependencies into account when estimating a map. To demonstrate the potential benefits of this approach, we simulate a simple multi-robot minefield mapping scenario. Minefields contain spatial dependencies since some landmine configurations are more likely than others, and since clutter, which causes false alarms, can be concentrated in certain regions and completely absent in others. Our belief propagation-based approach outperforms conventional occupancy grid mapping algorithms in the sense that better maps can be obtained with significantly fewer robot measurements. The belief propagation algorithm requires a modest amount of increased computation, but we contend that in applications where significant energy and time expenditure is associated with robot movement and active sensing, the reduction in the required number of samples justified the increased computation.

ICRA Conference 2004 Conference Paper

Optimal Sensor Placement for Cooperative Distributed Vision

Luis Ernesto Navarro-Serment
John M. Dolan
Pradeep K. Khosla

This work describes a method for observing maneuvering targets using a group of mobile robots equipped with video cameras. These robots are part of a team of small-size (7/spl times/7/spl times/7 cm) robots configured from modular components that collaborate to accomplish a given task. The cameras seek to observe the target while facing it as much as possible from their respective viewpoints. This work considers the problem of scheduling and maneuvering the cameras based on the evaluation of their current positions in terms of how well can they maintain a frontal view of the target. We describe our approach, which distributes the task among several robots and avoids extensive energy consumption on a single robot. We explore the concept in simulation and present results.

IROS Conference 2003 Conference Paper

Crucial factors affecting cooperative multirobot learning

Poj Tangamchit
John M. Dolan
Pradeep K. Khosla

The effectiveness of multirobot learning in achieving optimal, cooperative solutions is potentially affected by various factors having to do with the nature and configuration of the robots and the nature and configuration of the robots and the nature of the learning entities. Varying one factor wrongly may lead to undesirable results. There is no reported work on how systematically to set up these factors. In this paper, we methodically test the effect of varying four common factors (reward scope, global information delay, diversity of robots, and number of robots) in a decentralized multirobot system, first in simulation and then on real robots. The results show that two of these factors, reward scope and global information delay, if set up incorrectly, can prevent optimal, cooperative solutions.

IROS Conference 2003 Conference Paper

Learning to detect partially labeled people

Yaron Rachlin
John M. Dolan
Pradeep K. Khosla

Deployed vision systems often encounter image variations poorly represented in their training data. While observing their environment, such vision systems obtain unlabeled data that could be used to compensate for incomplete training. In order to exploit these relatively cheap and abundant unlabeled data we present a family of algorithms called /spl lambda/MEEM. Using these algorithms, we train an appearance-based people detection model. In contrast to approaches that rely on a large number of manually labeled training points, we use a partially labeled data set to capture appearance variation. One can both avoid the tedium of additional manual labeling and obtain improved detection performance by augmenting a labeled training set with unlabeled data. Further, enlarging the original training set with new unlabeled points enables the update of detection models after deployment without human intervention. To support these claim we show people detection results, and compare our performance to a purely generative expectation maximization-based approach to learning over partially labeled data.

ICRA Conference 2002 Conference Paper

The Necessity of Average Rewards in Cooperative Multirobot Learning

Poj Tangamchit
John M. Dolan
Pradeep K. Khosla

Learning can be an effective way for robot systems to deal with dynamic environments and changing task conditions. However, popular single-robot learning algorithms based on discounted rewards, such as Q learning, do not achieve cooperation (i. e. , purposeful division of labor) when applied to task-level multirobot systems. A task-level system is defined as one performing a mission that is decomposed into subtasks shared among robots. We demonstrate the superiority of average-reward-based learning such as the Monte Carlo algorithm for task-level multirobot systems, and suggest an explanation for this superiority.

IROS Conference 1999 Conference Paper

RAVE: a real and virtual environment for multiple mobile robot systems

Kevin R. Dixon
John M. Dolan
Wesley Huang
Christiaan J. J. Paredis
Pradeep K. Khosla

To focus on the research issues surrounding collaborative behavior in multiple mobile-robotic systems, a great amount of low-level infrastructure is required. To facilitate our on-going research into multi-robot systems, we have developed RAVE, a software framework that provides a real and virtual environment for running and managing multiple heterogeneous mobile-robot systems. This framework simplifies the implementation and development of collaborative robotic systems by providing the following capabilities: the ability to run systems off-line in simulation, user-interfaces for observing and commanding simulated and real robots, transparent transference of simulated robot programs to real robots, the ability to have simulated robots interact with real robots, and the ability to place virtual sensors on real robots to augment or experiment with their performance.