Author name cluster

Kai Gao

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

17 papers

2 author rows

EAAI Journal 2025 Journal Article

Evolutionary dynamic multiobjective optimization using a Bayesian vector autoregression prediction model

Kai Gao
Wenxiang Jiang
Lihong Xu

Dynamic multiobjective optimization problems (DMOPs), inherently involving the simultaneous optimization of conflicting objectives under time-varying environments, exhibit ubiquitous presence in real-world applications spanning diverse engineering disciplines. A prevalent limitation in existing dynamic multiobjective optimization evolutionary algorithms (DMOEAs) lies in inadequate utilization of historical information and neglect of interdependencies among decision variables, which frequently induces suboptimal initial population predictions deviating from true Pareto optimal sets. To mitigate these limitations, we propose MOEA/D-BVAR, a novel DMOEA framework incorporating a Bayesian vector autoregressive (BVAR) model that conceptualizes solution dynamics through holistic vector forecasting rather than isolated variable-specific prediction. The algorithm initially clusters variables via mutual information correlation analysis, subsequently constructing BVAR models for each cluster to project their evolutionary trajectories. Independently varying variables are rapidly predicted through differential forecasting models. A multivariate interaction optimization mechanism enhances search efficiency. Comprehensive empirical evaluations on 14 benchmark suites compare MOEA/D-BVAR against six state-of-the-art DMOEAs developed over the past five years. Statistical analysis of experimental outcomes demonstrates the proposed algorithm’s superior competitiveness in handling complex DMOPs.

Details DOI

ICRA Conference 2025 Conference Paper

MuST: Multi-Head Skill Transformer for Long-Horizon Dexterous Manipulation with Skill Progress

Kai Gao
Fan Wang
Erica Aduh
Dylan Randle
Jane Shi

Robot picking and packing tasks require dexterous manipulation skills, such as rearranging objects to establish a good grasping pose, or placing and pushing items to achieve tight packing. These tasks are challenging for robots due to the complexity and variability of the required actions. To tackle the difficulty of learning and executing long-horizon tasks, we propose a novel framework called the Multi-Head Skill Transformer (MuST). This model is designed to learn and sequentially chain together multiple motion primitives (skills), enabling robots to perform complex sequences of actions effectively. MuST introduces a “progress value” for each skill, guiding the robot on which skill to execute next and ensuring smooth transitions between skills. Additionally, our model is capable of expanding its skill set and managing various sequences of sub-tasks efficiently. Extensive experiments in both simulated and real-world environments demonstrate that MuST significantly enhances the robot's ability to perform long-horizon dexterous manipulation tasks.

Details

ICRA Conference 2025 Conference Paper

ORLA*: Mobile Manipulator-Based Object Rearrangement with Lazy A

Kai Gao
Zhaxizhuoma
Yan Ding 0002
Shiqi Zhang 0001
Jingjin Yu

Effectively performing object rearrangement is an essential skill for mobile manipulators, e. g. , setting up a dinner table. A key challenge in such problems is deciding an appropriate ordering to effectively untangle object-object dependencies while considering the necessary motions for realizing manipulation tasks (e. g. , pick and place). Computing time-optimal multi-object rearrangement solutions for mobile manipulators remains a largely untapped research direction. In this work, we propose ORLA*, which leverages delayed/lazy evaluation in searching for a high-quality object pick-n-place sequence that considers both end-effector and mobile robot base travel. ORLA* readily handles multi-layered rearrangement tasks powered by learning-based stability predictions. Employing an optimal solver for finding temporary locations for displacing objects, ORLA* can achieve global optimality. Through extensive simulation and ablation study, we confirm the effectiveness of ORLA* delivering quality solutions for challenging rearrangement instances. Supplementary materials are available at: gaokai15.github.io/ORLA-Star/

Details

IROS Conference 2024 Conference Paper

LGMCTS: Language-Guided Monte-Carlo Tree Search for Executable Semantic Object Rearrangement

Haonan Chang
Kai Gao
Kowndinya Boyalakuntla
Alex Lee
Baichuan Huang
Jingjin Yu
Abdeslam Boularias

We present LGMCTS, a framework that uniquely combines language guidance with geometrically informed sampling distributions to effectively rearrange objects according to geometric patterns dictated by natural language descriptions. LGMCTS uses Monte Carlo Tree Search (MCTS) to create feasible action plans that ensure executable semantic object rearrangement. We present a comprehensive comparison with leading approaches that use language to generate goal rearrangements independently of actionable planning, including Structformer, StructDiffusion, and Code as policies. We also present a new benchmark, the Executable Language Guided Rearrangement (ELGR) Bench, containing tasks involving intricate geometry. With the ELGR bench, we show limitations of task and motion planning (TAMP) solutions that are purely based on Large Language Models (LLM) such as Code as Policies and Progprompt on such tasks. Our findings advocate for using LLMs to generate intermediary representations rather than direct action planning in geometrically complex rearrangement scenarios, aligning with perspectives from recent literature. Our code and supplementary materials are accessible at https://lgmcts.github.io/.

Details

AAAI Conference 2024 Conference Paper

Token-Level Contrastive Learning with Modality-Aware Prompting for Multimodal Intent Recognition

Qianrui Zhou
Hua Xu
Hao Li
Hanlei Zhang
Xiaohan Zhang
Yifan Wang
Kai Gao

Multimodal intent recognition aims to leverage diverse modalities such as expressions, body movements and tone of speech to comprehend user's intent, constituting a critical task for understanding human language and behavior in real-world multimodal scenarios. Nevertheless, the majority of existing methods ignore potential correlations among different modalities and own limitations in effectively learning semantic features from nonverbal modalities. In this paper, we introduce a token-level contrastive learning method with modality-aware prompting (TCL-MAP) to address the above challenges. To establish an optimal multimodal semantic environment for text modality, we develop a modality-aware prompting module (MAP), which effectively aligns and fuses features from text, video and audio modalities with similarity-based modality alignment and cross-modality attention mechanism. Based on the modality-aware prompt and ground truth labels, the proposed token-level contrastive learning framework (TCL) constructs augmented samples and employs NT-Xent loss on the label token. Specifically, TCL capitalizes on the optimal textual semantic insights derived from intent labels to guide the learning processes of other modalities in return. Extensive experiments show that our method achieves remarkable improvements compared to state-of-the-art methods. Additionally, ablation analyses demonstrate the superiority of the modality-aware prompt over the handcrafted prompt, which holds substantial significance for multimodal prompt learning. The codes are released at https://github.com/thuiar/TCL-MAP.

PDF Details DOI

IROS Conference 2023 Conference Paper

Effectively Rearranging Heterogeneous Objects on Cluttered Tabletops

Kai Gao
Justin Yu
Tanay Sandeep Punjabi
Jingjin Yu

Effectively rearranging heterogeneous objects constitutes a high-utility skill that an intelligent robot should master. Whereas significant work has been devoted to the grasp synthesis of heterogeneous objects, little attention has been given to the planning for sequentially manipulating such objects. In this work, we examine the long-horizon sequential rearrangement of heterogeneous objects in a tabletop setting, addressing not just generating feasible plans but near-optimal ones. Toward that end, and building on previous methods, including combinatorial algorithms and Monte Carlo tree search-based solutions, we develop state-of-the-art solvers for optimizing two practical objective functions considering key object properties such as size and weight. Thorough simulation studies show that our methods provide significant advantages in handling challenging heterogeneous object rearrangement problems, especially in cluttered settings. Real robot experiments further demonstrate and confirm these advantages. Source code and evaluation data associated with this research will be available at https//github.com/arc-l/TRLB upon the publication of this manuscript

Details

ICRA Conference 2023 Conference Paper

On the Utility of Buffers in Pick-n-Swap Based Lattice Rearrangement

Kai Gao
Jingjin Yu

We investigate the utility of employing multiple buffers in solving a class of rearrangement problems with pick- n-swap manipulation primitives. In this problem, objects stored randomly in a lattice are to be sorted using a robot arm with k 1 swap spaces or buffers, capable of holding up to $k$ objects on its end-effector simultaneously. On the structural side, we show that the addition of each new buffer brings diminishing returns in saving the end-effector travel distance while holding the total number of pick-n-swap operations at a minimum. This is due to an interesting recursive cycle structure in random m-permutation, where the largest cycle covers over 60% of objects. On the algorithmic side, we propose fast algorithms for 1D and 2D lattice rearrangement problems that can effectively use multiple buffers to boost solution optimality. Numerical experiments demonstrate the efficiency and scalability of our methods, as well as confirm the diminishing return structure as more buffers are employed. Introduction video: https://youtu.be/KtBxoARGaVQ

Details

IROS Conference 2023 Conference Paper

Optimal and Stable Multi-Layer Object Rearrangement on a Tabletop

Andy Xu
Kai Gao
Si Wei Feng
Jingjin Yu

Object rearrangement is a fundamental sub-task in accomplishing a great many physical tasks. As such, effectively executing rearrangement is an important skill for intelligent robots to master. In this study, we conduct the first algorithmic study on optimally solving the problem of Multi-layer Object Rearrangement on a Tabletop (MORT), in which one object may be relocated at a time, and an object can only be moved if other objects do not block its top surface. In addition, any intermediate structure during the reconfiguration process must be physically stable, i. e. , it should stand without external support. To tackle the dual challenges of untangling the dependencies between objects and ensuring structural stability, we develop an algorithm that interleaves the computation of the optimal rearrangement plan and structural stability checking. Using a carefully constructed integer linear programming (ILP) model, our algorithm, Stability-Aware Rearrangement Programming (SARP), readily scales to optimally solve complex rearrangement problems of 3D structures with over 60 building blocks, with solution quality significantly outperforming natural greedy best-first approaches. Upon the publication of the manuscript source code and data will be available at https//github.com/arc-1/mort/.

Details

AAAI Conference 2022 System Paper

An End-to-End Traditional Chinese Medicine Constitution Assessment System Based on Multimodal Clinical Feature Representation and Fusion

Huisheng Mao
Baozheng Zhang
Hua Xu
Kai Gao

Traditional Chinese Medicine (TCM) constitution is a fundamental concept in TCM theory. It is determined by multimodal TCM clinical features which, in turn, are obtained from TCM clinical information of image (face, tongue, etc.), audio (pulse and voice), and text (inquiry) modality. The auto assessment of TCM constitution is faced with two major challenges: (1) learning discriminative TCM clinical feature representations; (2) jointly processing the features using multimodal fusion techniques. The TCM Constitution Assessment System (TCM-CAS) is proposed to provide an end-to-end solution to this task, along with auxiliary functions to aid TCM researchers. To improve the results of TCM constitution prediction, the system combines multiple machine learning algorithms such as facial landmark detection, image segmentation, graph neural networks and multimodal fusion. Extensive experiments are conducted on a four-category multimodal TCM constitution dataset, and the proposed method achieves state-of-the-art accuracy. Provided with datasets containing annotations of diseases, the system can also perform automatic disease diagnosis from a TCM perspective.

PDF Details

ICRA Conference 2022 Conference Paper

Fast High-Quality Tabletop Rearrangement in Bounded Workspace

Kai Gao
Darren Lau
Baichuan Huang
Kostas E. Bekris
Jingjin Yu

In this paper, we examine the problem of rearranging many objects on a tabletop in a cluttered setting using overhand grasps. Efficient solutions for the problem, which capture a common task that we solve on a daily basis, are essential in enabling truly intelligent robotic manipulation. In a given instance, objects may need to be placed at temporary positions (“buffers”) to complete the rearrangement, but allocating these buffer locations can be highly challenging in a cluttered environment. To tackle the challenge, a two-step baseline planner is first developed, which generates a primitive plan based on inherent combinatorial constraints induced by start and goal poses of the objects and then selects buffer locations assisted by the primitive plan. We then employ the “lazy” planner in a tree search framework which is further sped up by adapting a novel preprocessing routine. Simulation experiments show our methods can quickly generate high-quality solutions and are more robust in solving large-scale instances than existing state-of-the-art approaches. source: github.com/arc-l/TRLB

Details

ICAPS Conference 2022 Conference Paper

Lazy Rearrangement Planning in Confined Spaces

Rui Wang 0087
Kai Gao
Jingjin Yu
Kostas E. Bekris

Object rearrangement is important for many applications but remains challenging, especially in confined spaces, such as shelves, where objects cannot be accessed from above and they block reachability to each other. Such constraints require many motion planning and collision checking calls, which are computationally expensive. In addition, the arrangement space grows exponentially with the number of objects. To address these issues, this work introduces a lazy evaluation framework with a local monotone solver and a global planner. Monotone instances are those that can be solved by moving each object at most once. A key insight is that reachability constraints at the grasps for objects' starts and goals can quickly reveal dependencies between objects without having to execute expensive motion planning queries. Given that, the local solver builds lazily a search tree that respects these reachability constraints without verifying that the arm paths are collision free. It only collision checks when a promising solution is found. If a monotone solution is not found, the non-monotone planner loads the lazy search tree and explores ways to move objects to intermediate locations from where monotone solutions to the goal can be found. Results show that the proposed framework can solve difficult instances in confined spaces with up to 16 objects, which state-of-the-art methods fail to solve. It also solves problems faster than alternatives, when the alternatives find a solution. It also achieves high-quality solutions, i. e. , only 1. 8 additional actions on average are needed for non-monotone instances.

Details

ICRA Conference 2022 Conference Paper

Persistent Homology for Effective Non-Prehensile Manipulation

Ewerton R. Vieira
Daniel Nakhimovich
Kai Gao
Rui Wang 0087
Jingjin Yu
Kostas E. Bekris

This work explores the use of topological tools for achieving effective non-prehensile manipulation in cluttered, constrained workspaces. In particular, it proposes the use of persistent homology as a guiding principle in identifying the appropriate non-prehensile actions, such as pushing, to clean a cluttered space with a robotic arm so as to allow the retrieval of a target object. Persistent homology enables the automatic identification of connected components of blocking objects in the space without the need for manual input or tuning of parameters. The proposed algorithm uses this information to push groups of cylindrical objects together and aims to minimize the number of pushing actions needed to reach to the target. Simulated experiments in a physics engine using a model of the Baxter robot show that the proposed topology-driven solution is achieving significantly higher success rate in solving such constrained problems relatively to state-of-the-art alternatives from the literature. It manages to keep the number of pushing actions low, is computationally efficient and the resulting decisions and motion appear natural for effectively solving such tasks.

Details

IROS Conference 2022 Conference Paper

Toward Efficient Task Planning for Dual-Arm Tabletop Object Rearrangement

Kai Gao
Jingjin Yu

We investigate the problem of coordinating two robot arms to solve non-monotone tabletop multi-object re- arrangement tasks. In a non-monotone rearrangement task, complex object-object dependencies exist that require moving some objects multiple times to solve an instance. In working with two arms in a large workspace, some objects must be handed off between the robots, which further complicates the planning process. For the challenging dual-arm tabletop rearrangement problem, we develop effective task planning algorithms for scheduling the pick-n-place sequence that can be properly distributed between the two arms. We show that, even without using a sophisticated motion planner, our method achieves significant time savings in comparison to greedy approaches and naive parallelization of single-robot plans.

Details

IROS Conference 2021 Conference Paper

Capacitated Vehicle Routing with Target Geometric Constraints

Kai Gao
Jingjin Yu

We investigate the capacitated vehicle routing problem (CVRP) under a robotics context, where a vehicle with limited payload must complete delivery (or pickup) tasks to serve a set of geographically distributed customers with varying demands. In classical CVRP, a customer location is modeled as a point. In many robotics applications, however, it is more appropriate to model such "customer locations" as 2D regions. For example, in aerial delivery, a drone may drop a package anywhere in a customer’s lot. This yields the problem of CVRG (Capacitated Vehicle Routing with Target Geometric Constraints). Computationally, CVRP is already strongly NPhard; CVRG is therefore more challenging. Nevertheless, we develop fast algorithms for CVRG, capable of computing high quality solutions for hundreds of regions. Our algorithmic solution is guaranteed to be optimal when customer regions are convex. Numerical evaluations show that our proposed methods significantly outperform greedy best-first approaches. Comprehensive simulation studies confirm the effectiveness of our methods.

Details

ICRA Conference 2021 Conference Paper

Sensor Placement for Globally Optimal Coverage of 3D-Embedded Surfaces

Si Wei Feng
Kai Gao
Jie Gong
Jingjin Yu

We carry out a structural and algorithmic study of a mobile sensor coverage optimization problem targeting 2D surfaces embedded in a 3D workspace. The investigated settings model multiple important applications including camera net- work deployment for surveillance, geological monitoring/survey of 3D terrains, and UVC-based surface disinfection for the prevention of the spread of disease agents (e. g. , SARS-CoV-2). Under a unified general "sensor coverage" problem, three concrete formulations are examined, focusing on optimizing visibility, single-best coverage quality, and cumulative quality, respectively. After demonstrating the computational intractability of all these formulations, we describe approximation schemes and mathematical programming models for near-optimally solving them. The effectiveness of our methods is thoroughly evaluated under realistic and practical scenarios.

Details

ICRA Conference 2021 Conference Paper

Uniform Object Rearrangement: From Complete Monotone Primitives to Efficient Non-Monotone Informed Search

Rui Wang 0087
Kai Gao
Daniel Nakhimovich
Jingjin Yu
Kostas E. Bekris

Object rearrangement is a widely-applicable and challenging task for robots. Geometric constraints must be carefully examined to avoid collisions and combinatorial issues arise as the number of objects increases. This work studies the algorithmic structure of rearranging uniform objects, where robot-object collisions do not occur but object-object collisions have to be avoided. The objective is minimizing the number of object transfers under the assumption that the robot can manipulate one object at a time. An efficiently computable decomposition of the configuration space is used to create a "region graph", which classifies all continuous paths of equivalent collision possibilities. Based on this compact but rich representation, a complete dynamic programming primitive DFS DP performs a recursive depth first search to solve monotone problems quickly, i. e. , those instances that do not require objects to be moved first to an intermediate buffer. DFS DP is extended to solve single-buffer, non-monotone instances, given a choice of an object and a buffer. This work utilizes these primitives as local planners in an informed search framework for more general, non-monotone instances. The search utilizes partial solutions from the primitives to identify the most promising choice of objects and buffers. Experiments demonstrate that the proposed solution returns near-optimal paths with higher success rate, even for challenging non-monotone instances, than other leading alternatives.

Details

AAAI Conference 2020 Short Paper

Multi-Channel Convolutional Neural Networks with Adversarial Training for Few-Shot Relation Classification (Student Abstract)

Yuxiang Xie
Hua Xu
Congcong Yang
Kai Gao

The distant supervised (DS) method has improved the performance of relation classiﬁcation (RC) by means of extending the dataset. However, DS also brings the problem of wrong labeling. Contrary to DS, the few-shot method relies on few supervised data to predict the unseen classes. In this paper, we use word embedding and position embedding to construct multi-channel vector representation and use the multi-channel convolutional method to extract features of sentences. Moreover, in order to alleviate few-shot learning to be sensitive to overﬁtting, we introduce adversarial learning for training a robust model. Experiments on the FewRel dataset show that our model achieves signiﬁcant and consistent improvements on few-shot RC as compared with baselines.

PDF Details