Author name cluster

Yu Shen

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

22 papers

2 author rows

JBHI Journal 2026 Journal Article

Vision Sensing-Driven Intelligent Ocular Disease Detection Using Conformer-Based Dual Fusion

Zhiwei Guo
Qin Zhang
Peng Xu
Yu Shen
Chinmay Chakraborty
Osama Alfarraj
Keping Yu

The deep vision sensing has been a practical tool in early disease detection, and this work aims at an important branch of ocular disease recognition. Although a number of researchers had paid attention to it during past years, fine-grained ocular feature extraction always remains a challenge. To handle with this issue, this work benefits from comprehensive ability of the convolution-Transformer structure (Conformer), and proposes vision sensing-driven intelligent ocular disease detection using conformer-based dual fusion. On the one hand, the proposal combines technical advantages of convolution and visual Transformer to more accurately fuse local subtle features and global representation information in images. On the other hand, the proposal significantly improves accuracy and robustness of the model by optimizing depth and width. Simulation experiments on real-world ocular disease image datasets show that the proposed model exhibits higher performance in ocular disease detection compared to other methods. Numerical results show that it improves the detection accuracy by 1% to 3. 7% compared to several mainstream baseline methods. This research result not only promotes the development of ocular disease detection, but also provides more reliable technical support for accurate diagnosis of ophthalmic diseases.

Details DOI

ICML Conference 2025 Conference Paper

Adaptive Sensitivity Analysis for Robust Augmentation against Natural Corruptions in Image Segmentation

Laura Y. Zheng
Wenjie Wei
Tony Wu
Jacob Clements
Shreelekha Revankar
Andre Harrison
Yu Shen
Ming Lin 0003

Achieving robustness in image segmentation models is challenging due to the fine-grained nature of pixel-level classification. These models, which are crucial for many real-time perception applications, particularly struggle when faced with natural corruptions in the wild for autonomous systems. While sensitivity analysis can help us understand how input variables influence model outputs, its application to natural and uncontrollable corruptions in training data is computationally expensive. In this work, we present an adaptive, sensitivity-guided augmentation method to enhance robustness against natural corruptions. Our sensitivity analysis on average runs 10 times faster and requires about 200 times less storage than previous sensitivity analysis, enabling practical, on-the-fly estimation during training for a model-free augmentation policy. With minimal fine-tuning, our sensitivity-guided augmentation method achieves improved robustness on both real-world and synthetic datasets compared to state-of-the-art data augmentation techniques in image segmentation.

Details

NeurIPS Conference 2025 Conference Paper

CAML: Collaborative Auxiliary Modality Learning for Multi-Agent Systems

Rui Liu
Yu Shen
Peng Gao
Pratap Tokekar
Ming C. Lin

Multi-modal learning has emerged as a key technique for improving performance across domains such as autonomous driving, robotics, and reasoning. However, in certain scenarios, particularly in resource-constrained environments, some modalities available during training may be absent during inference. While existing frameworks effectively utilize multiple data sources during training and enable inference with reduced modalities, they are primarily designed for single-agent settings. This poses a critical limitation in dynamic environments such as connected autonomous vehicles (CAV), where incomplete data coverage can lead to decision-making blind spots. Conversely, some works explore multi-agent collaboration but without addressing missing modality at test time. To overcome these limitations, we propose Collaborative Auxiliary Modality Learning (CAML), a novel multi-modal multi-agent framework that enables agents to collaborate and share multi-modal data during training, while allowing inference with reduced modalities during testing. Experimental results in collaborative decision-making for CAV in accident-prone scenarios demonstrate that CAML achieves up to a 58. 1% improvement in accident detection. Additionally, we validate CAML on real-world aerial-ground robot data for collaborative semantic segmentation, achieving up to a 10. 6% improvement in mIoU.

PDF Details

IROS Conference 2025 Conference Paper

MMCD: Multi-Modal Collaborative Decision-Making for Connected Autonomy with Knowledge Distillation

Rui Liu 0040
Zikang Wang
Peng Gao 0007
Yu Shen
Pratap Tokekar
Ming Lin 0003

Autonomous systems have advanced significantly, but challenges persist in accident-prone environments where robust decision-making is crucial. A single vehicle’s limited sensor range and obstructed views increase the likelihood of accidents. Multi-vehicle connected systems and multi-modal approaches, leveraging RGB images and LiDAR point clouds, have emerged as promising solutions. However, existing methods often assume the availability of all data modalities and connected vehicles during both training and testing, which is impractical due to potential sensor failures or missing connected vehicles. To address these challenges, we introduce a novel framework MMCD (Multi-Modal Collaborative Decision-making) for connected autonomy. Our framework fuses multi-modal observations from ego and collaborative vehicles to enhance decision-making under challenging conditions. To ensure robust performance when certain data modalities are unavailable during testing, we propose an approach based on cross-modal knowledge distillation with a teacher-student model structure. The teacher model is trained with multiple data modalities, while the student model is designed to operate effectively with reduced modalities. In experiments on connected autonomous driving with ground vehicles and aerial-ground vehicles collaboration, our method improves driving safety by up to 20. 7%, surpassing the best-existing baseline in detecting potential accidents and making safe driving decisions. More information can be found on our website https://ruiiu.github.io/mmcd.

Details

JBHI Journal 2025 Journal Article

Multiview Deep Learning-Based Efficient Medical Data Management for Survival Time Forecasting

Keping Yu
Lijuan Quan
Chinmay Chakraborty
Xin Qi
Yu Shen
Zhiwei Guo
Osama Alfarraj
Amr Tolba

In recent years, data-driven remote medical management has received much attention, especially in application of survival time forecasting. By monitoring the physical characteristics indexes of patients, intelligent algorithms can be deployed to implement efficient healthcare management. However, such pure medical data-driven scenes generally lack multimedia information, which brings challenge to analysis tasks. To deal with this issue, this paper introduces the idea of ensemble deep learning to enhance feature representation ability, thus enhancing knowledge discovery in remote healthcare management. Therefore, a multiview deep learning-based efficient medical data management framework for survival time forecasting is proposed in this paper, which is named as “ MDL-MDM ” for short. Firstly, basic monitoring data for body indexes of patients is encoded, which serves as the data foundation for forecasting tasks. Then, three different neural network models, convolution neural network, graph attention network, and graph convolution network, are selected to build a hybrid computing framework. Their combination can bring a multiview feature learning framework to realize an efficient medical data management framework. In addition, experiments are conducted on a realistic medical dataset about cancer patients in the US. Results show that the proposal can predict survival time with 1% to 2% reduction in prediction error.

Details DOI

IROS Conference 2024 Conference Paper

AutoJoin: Efficient Adversarial Training against Gradient-Free Perturbations for Robust Maneuvering via Denoising Autoencoder and Joint Learning

Michael Villarreal
Bibek Poudel
Ryan Wickman
Yu Shen
Weizi Li

With the growing use of machine learning algorithms and ubiquitous sensors, many ‘perception-to-control’ systems are being developed and deployed. To ensure their trustworthiness, improving their robustness through adversarial training is one potential approach. We propose a gradient-free adversarial training technique, named AutoJoin, to effectively and efficiently produce robust models for image-based maneuvering. Compared to other state-of-the-art methods with testing on over 5M images, AutoJoin achieves significant performance increases up to the 40% range against perturbations while improving on clean performance up to 300%. AutoJoin is also highly efficient, saving up to 86% time per training epoch and 90% training data over other state-of-the-art techniques. The core idea of AutoJoin is to use a decoder attachment to the original regression model creating a denoising autoencoder within the architecture. This architecture allows the tasks ‘maneuvering’ and ‘denoising sensor input’ to be jointly learnt and reinforce each other’s performance.

Details

ICRA Conference 2024 Conference Paper

Collaborative Decision-Making Using Spatiotemporal Graphs in Connected Autonomy

Peng Gao 0007
Yu Shen
Ming Lin 0003

Collaborative decision-making is an essential capability for multi-robot systems, such as connected vehicles, to collaboratively control autonomous vehicles in accident-prone scenarios. Under limited communication bandwidth, capturing comprehensive situational awareness by integrating connected agents’ observation is very challenging. In this paper, we propose a novel collaborative decision-making method that efficiently and effectively integrates collaborators’ representations to control the ego vehicle in accident-prone scenarios. Our approach formulates collaborative decision-making as a classification problem. We first represent sequences of raw observations as spatiotemporal graphs, which significantly reduce the package size to share among connected vehicles. Then we design a novel spatiotemporal graph neural network based on heterogeneous graph learning, which analyzes spatial and temporal connections of objects in a unified way for collaborative decision-making. We evaluate our approach using a high-fidelity simulator that considers realistic traffic, communication bandwidth, and vehicle sensing among connected autonomous vehicles. The experimental results show that our representation achieves over 100x reduction in the shared data size that meets the requirements of communication bandwidth for connected autonomous driving. In addition, our approach achieves over 30% improvements in driving safety.

Details

JMLR Journal 2024 Journal Article

OpenBox: A Python Toolkit for Generalized Black-box Optimization

Huaijun Jiang
Yu Shen
Yang Li
Beicheng Xu
Sixian Du
Wentao Zhang
Ce Zhang
Bin Cui

Black-box optimization (BBO) has a broad range of applications, including automatic machine learning, experimental design, and database knob tuning. However, users still face challenges when applying BBO methods to their problems at hand with existing software packages in terms of applicability, performance, and efficiency. This paper presents OpenBox, an open-source BBO toolkit with improved usability. It implements user-friendly interfaces and visualization for users to define and manage their tasks. The modular design behind OpenBox facilitates its flexible deployment in existing systems. Experimental results demonstrate the effectiveness and efficiency of OpenBox over existing systems. The source code of OpenBox is available at https://github.com/PKU-DAIR/open-box. [abs] [ pdf ][ bib ] [ code ] &copy JMLR 2024. ( edit, beta )

PDF Details

ICRA Conference 2024 Conference Paper

Task-Driven Domain-Agnostic Learning with Information Bottleneck for Autonomous Steering

Yu Shen
Laura Zheng
Tianyi Zhou 0001
Ming Lin 0003

Environments for autonomous driving can vary from place to place, leading to challenges in designing a learning model for a new scene. Transfer learning can leverage knowledge from a learned domain to a new domain with limited data. In this work, we focus on end-to-end autonomous driving as the target task, consisting of both perception and control. We first utilize information bottleneck analysis to build a causal graph that defines our framework and the loss function; then we propose a novel domain-agnostic learning method for autonomous steering based on our analysis of training data, network architecture, and training paradigm. Experiments show that our method outperforms other SOTA methods.

Details

ICML Conference 2023 Conference Paper

Auxiliary Modality Learning with Generalized Curriculum Distillation

Yu Shen
Xijun Wang 0002
Peng Gao 0007
Ming Lin 0003

Driven by the need from real-world applications, Auxiliary Modality Learning (AML) offers the possibility to utilize more information from auxiliary data in training, while only requiring data from one or fewer modalities in test, to save the overall computational cost and reduce the amount of input data for inferencing. In this work, we formally define “Auxiliary Modality Learning” (AML), systematically classify types of auxiliary modality (in visual computing) and architectures for AML, and analyze their performance. We also analyze the conditions under which AML works well from the optimization and data distribution perspectives. To guide various choices to achieve optimal performance using AML, we propose a novel method to assist in choosing the best auxiliary modality and estimating an upper bound performance before executing AML. In addition, we propose a new AML method using generalized curriculum distillation to enable more effective curriculum learning. Our method achieves the best performance compared to other SOTA methods.

Details

EAAI Journal 2023 Journal Article

Prediction and control of water quality in Recirculating Aquaculture System based on hybrid neural network

Junchao Yang
Lulu Jia
Zhiwei Guo
Yu Shen
Xianwei Li
Zhenping Mou
Keping Yu
Jerry Chun-Wei Lin

In the Recirculating Aquaculture Systems (RAS), the control of water quality indices remains essential to survival and growth of aquaculture objects. This requires effect prediction of future water status in advance, which can be adopted to help the generation of following control strategies. However, conventional methods of water quality prediction were mostly dependent on redundant parameters of model, which leads to inefficiency and low accuracy. In addition, the complexity of the RAS multi-units requires intelligent control of the water quality unit. Thus, a prediction and control framework for predicting water quality in RAS is proposed in this paper. Specifically, a hybrid deep learning structure which combines the Convolutional Neural Network (CNN), Gated Recurrent Unit (GRU) and Attention mechanism is presented. To begin with, the CNN is utilized to extract local features for different timestamped water quality parameter. After the local features have been extracted, the proposed GRU model replicates the global sequential features of the parameters. The attention mechanism is then applied to focus on more critical features to promote the efficiency and accuracy of prediction. Finally, to demonstrate the efficiency and stability of the prediction and control framework with the mixture of CNN, GRU and Attention (PC-CGA), multiple groups of experiments and evaluations are carried out in a medium size RAS.

Details DOI

AAAI Conference 2023 Conference Paper

ProxyBO: Accelerating Neural Architecture Search via Bayesian Optimization with Zero-Cost Proxies

Yu Shen
Yang Li
Jian Zheng
Wentao Zhang
Peng Yao
Jixiang Li
Sen Yang
Ji Liu

Designing neural architectures requires immense manual efforts. This has promoted the development of neural architecture search (NAS) to automate the design. While previous NAS methods achieve promising results but run slowly, zero-cost proxies run extremely fast but are less promising. Therefore, it’s of great potential to accelerate NAS via those zero-cost proxies. The existing method has two limitations, which are unforeseeable reliability and one-shot usage. To address the limitations, we present ProxyBO, an efficient Bayesian optimization (BO) framework that utilizes the zero-cost proxies to accelerate neural architecture search. We apply the generalization ability measurement to estimate the fitness of proxies on the task during each iteration and design a novel acquisition function to combine BO with zero-cost proxies based on their dynamic influence. Extensive empirical studies show that ProxyBO consistently outperforms competitive baselines on five tasks from three public benchmarks. Concretely, ProxyBO achieves up to 5.41× and 3.86× speedups over the state-of-the-art approaches REA and BRP-NAS.

PDF Details DOI

ICRA Conference 2023 Conference Paper

Small-shot Multi-modal Distillation for Vision-based Autonomous Steering

Yu Shen
Luyu Yang
Xijun Wang 0002
Ming Lin 0003

In this paper, we propose a novel learning framework for autonomous systems that uses a small amount of “auxiliary information” that complements the learning of the main modality, called “small-shot auxiliary modality distillation network (AMD-S-Net)”. The AMD-S-Net contains a two-stream framework design that can fully extract information from different types of data (i. e. , paired/unpaired multi-modality data) to distill knowledge more effectively. We also propose a novel training paradigm based on the “reset operation” that enables the teacher to explore the local loss landscape near the student domain iteratively, providing local landscape information and potential directions to discover better solutions by the student, thus achieving higher learning performance. Our experiments show that AMD-S-Net and our training paradigm outperform other SOTA methods by up to 12. 7% and 18. 1% improvement in autonomous steering, respectively.

Details

IROS Conference 2023 Conference Paper

Visual, Spatial, Geometric-Preserved Place Recognition for Cross-View and Cross-Modal Collaborative Perception

Peng Gao 0007
Jing Liang 0006
Yu Shen
Sanghyun Son 0003
Ming Lin 0003

Place recognition plays an important role in multi-robot collaborative perception, such as aerial-ground search and rescue, in order to identify the same place they have visited. Recently, approaches based on semantics showed the promising performance to address cross-view and cross-modal challenges in place recognition, which can be further categorized as graph-based and geometric-based methods. However, both methods have shortcomings, including ignoring geometric cues and affecting by large non-overlapped regions between observations. In this paper, we introduce a novel approach that integrates semantic graph matching and distance fields (DF) matching for cross-view and cross-modal place recognition. Our method uses a graph representation to encode visual-spatial cues of semantics and uses a set of class-wise DFs to encode geometric cues of a scene. Then, we formulate place recognition as a two-step matching problem. We first perform semantic graph matching to identify the correspondence of semantic objects. Then, we estimate the overlapped regions based on the identified correspondences and further align these regions to compute their geometric-based DF similarity. Finally, we integrate graph-based similarity and geometry-based DF similarity to match places. We evaluate our approach over two public benchmark datasets, including KITTI and AirSim. Compared with the previous methods, our approach achieves around 10% improvement in ground-ground place recognition in KITTI and 35% improvement in aerial-ground place recognition in AirSim.

Details

NeurIPS Conference 2022 Conference Paper

DivBO: Diversity-aware CASH for Ensemble Learning

Yu Shen
Yupeng Lu
Yang Li
Yaofeng Tu
Wentao Zhang
Bin Cui

The Combined Algorithm Selection and Hyperparameters optimization (CASH) problem is one of the fundamental problems in Automated Machine Learning (AutoML). Motivated by the success of ensemble learning, recent AutoML systems build post-hoc ensembles to output the final predictions instead of using the best single learner. However, while most CASH methods focus on searching for a single learner with the best performance, they neglect the diversity among base learners (i. e. , they may suggest similar configurations to previously evaluated ones), which is also a crucial consideration when building an ensemble. To tackle this issue and further enhance the ensemble performance, we propose DivBO, a diversity-aware framework to inject explicit search of diversity into the CASH problems. In the framework, we propose to use a diversity surrogate to predict the pair-wise diversity of two unseen configurations. Furthermore, we introduce a temporary pool and a weighted acquisition function to guide the search of both performance and diversity based on Bayesian optimization. Empirical results on 15 public datasets show that DivBO achieves the best average ranks (1. 82 and 1. 73) on both validation and test errors among 10 compared methods, including post-hoc designs in recent AutoML systems and state-of-the-art baselines for ensemble learning on CASH problems.

PDF Details

JBHI Journal 2022 Journal Article

Hybrid Intelligence-Driven Medical Image Recognition for Remote Patient Diagnosis in Internet of Medical Things

Zhiwei Guo
Yu Shen
Shaohua Wan
Wen-Long Shang
Keping Yu

In ear of smart cities, intelligent medical image recognition technique has become a promising way to solve remote patient diagnosis in IoMT. Although deep learning-based recognition approaches have received great development during the past decade, explainability always acts as a main obstacle to promote recognition approaches to higher levels. Because it is always hard to clearly grasp internal principles of deep learning models. In contrast, the conventional machine learning (CML)-based methods are well explainable, as they give relatively certain meanings to parameters. Motivated by the above view, this paper combines deep learning with the CML, and proposes a hybrid intelligence-driven medical image recognition framework in IoMT. On the one hand, the convolution neural network is utilized to extract deep and abstract features for initial images. On the other hand, the CML-based techniques are employed to reduce dimensions for extracted features and construct a strong classifier that output recognition results. A real dataset about pathologic myopia is selected to establish simulative scenario, in order to assess the proposed recognition framework. Results reveal that the proposal that improves recognition accuracy about two to three percent.

Details DOI

IROS Conference 2022 Conference Paper

Inverse Reinforcement Learning with Hybrid-weight Trust-region Optimization and Curriculum Learning for Autonomous Maneuvering

Yu Shen
Weizi Li
Ming Lin 0003

Despite significant advancements, collision-free navigation in autonomous driving is still challenging, considering the navigation module needs to balance learning and planning to achieve efficient and effective control of the vehicle. We propose a novel framework of inverse reinforcement learning with hybrid-weight trust-region optimization and curriculum learning (IRL-HC) for autonomous maneuvering. Our method can incorporate both expert demonstration (from real driving) and domain knowledge (hard constraints such as collision avoidance, goal reaching, etc. encoded in reward functions) to learn an effective control policy. The hybrid-weight trustregion optimization is used to determine the difficulty of the task curriculum for fast incremental curriculum learning and improve the efficiency of inverse reinforcement learning by hybrid weight tuning of different sets of hyperparameters. IRL-HC is also compatible with domain-dependent techniques such as learn-from-accident, which can further boost performance. Overall, IRL-HC can reduce the number of collisions up to 48%, increase the training efficiency by 2. 8x, and enable the vehicle to drive 10x further compared to other methods.

Details

IS Journal 2022 Journal Article

MetaSensing in Metaverses: See There, Be There, and Know There

Yuhang Liu
Yu Shen
Chao Guo
Yonglin Tian
Xiao Wang
Yin Zhu
Fei-Yue Wang

The concept of metaverses has received extensive attention recently and cyber-physical-social systems (CPSS) is its academic foundation. In almost all the applications of metaverses, the sensing system is an essential part and intelligent sensing capacity must be provided. However, due to the insufficient consideration of human factors in most of the studies, digital twins’ sensing in cyber-physical systems cannot achieve smart sensing in metaverses. For this reason, a novel framework for intelligent sensing in metaverses, MetaSensing, is proposed based on parallel intelligence in CPSS. Within the framework of MetaSensing, there are four states of sensing: physical sensing, descriptive sensing, predictive sensing, and prescriptive sensing. To protect sensors’ data privacy in metaverses, DAO-based decentralized sensing is introduced as a mechanism of the operation and maintenance for smart sensing industries.

Details DOI

ICRA Conference 2021 Conference Paper

Adversarial Differentiable Data Augmentation for Autonomous Systems

Manli Shu
Yu Shen
Ming Lin 0003
Tom Goldstein

Autonomous systems often rely on neural networks to achieve high performance on planning and control problems. Unfortunately, neural networks suffer severely when input images become degraded in ways that are not reflected in the training data. This is particularly problematic for robotic systems like autonomous vehicles (AV) for which reliability is paramount. In this work, we consider robust optimization methods for hardening control systems against image corruptions and other unexpected domain shifts. Recent work on robust optimization for neural nets has been focused largely on combating adversarial attacks. In this work, we borrow ideas from the adversarial training and data augmentation literature to enhance robustness to image corruptions and domain shifts. To this end, we train networks while augmenting image data with a battery of image degradations. Unlike traditional augmentation methods, we choose the parameters for each degradation adversarially so as to maximize system performance. By formulating image degradations in a way that is differentiable with respect to degradation parameters, we enable the use of efficient optimization methods (PGD) for choosing worst-case augmentation parameters. We demonstrate the efficacy of this method on the learning to steer task for AVs. By adversarially training against image corruptions, we produce networks that are highly robust to image corruptions. We show that the proposed differentiable augmentation schemes result in higher levels of robustness and accuracy for a range of settings as compared to baseline and state-of-the-art augmentation methods.

Details

NeurIPS Conference 2021 Conference Paper

Gradient-Free Adversarial Training Against Image Corruption for Learning-based Steering

Yu Shen
Laura Zheng
Manli Shu
Weizi Li
Tom Goldstein
Ming Lin

We introduce a simple yet effective framework for improving the robustness of learning algorithms against image corruptions for autonomous driving. These corruptions can occur due to both internal (e. g. , sensor noises and hardware abnormalities) and external factors (e. g. , lighting, weather, visibility, and other environmental effects). Using sensitivity analysis with FID-based parameterization, we propose a novel algorithm exploiting basis perturbations to improve the overall performance of autonomous steering and other image processing tasks, such as classification and detection, for self-driving cars. Our model not only improves the performance on the original dataset, but also achieves significant performance improvement on datasets with multiple and unseen perturbations, up to 87% and 77%, respectively. A comparison between our approach and other SOTA techniques confirms the effectiveness of our technique in improving the robustness of neural network training for learning-based steering and other image processing tasks.

PDF Details

AAAI Conference 2021 Conference Paper

MFES-HB: Efficient Hyperband with Multi-Fidelity Quality Measurements

Yang Li
Yu Shen
Jiawei Jiang
Jinyang Gao
Ce Zhang
Bin Cui

Hyperparameter optimization (HPO) is a fundamental problem in automatic machine learning (AutoML). However, due to the expensive evaluation cost of models (e. g. , training deep learning models or training models on large datasets), vanilla Bayesian optimization (BO) is typically computationally infeasible. To alleviate this issue, Hyperband (HB) utilizes the early stopping mechanism to speed up configuration evaluations by terminating those badly-performing configurations in advance. This leads to two kinds of quality measurements: (1) many low-fidelity measurements for configurations that get early-stopped, and (2) few high-fidelity measurements for configurations that are evaluated without being early stopped. The state-of-the-art HB-style method, BOHB, aims to combine the benefits of both BO and HB. Instead of sampling configurations randomly in HB, BOHB samples configurations based on a BO surrogate model, which is constructed with the high-fidelity measurements only. However, the scarcity of high-fidelity measurements greatly hampers the efficiency of BO to guide the configuration search. In this paper, we present MFES-HB, an efficient Hyperband method that is capable of utilizing both the high-fidelity and low-fidelity measurements to accelerate the convergence of HPO tasks. Designing MFES-HB is not trivial as the lowfidelity measurements can be biased yet informative to guide the configuration search. Thus we propose to build a Multi- Fidelity Ensemble Surrogate (MFES) based on the generalized Product of Experts framework, which can integrate useful information from multi-fidelity measurements effectively. The empirical studies on the real-world AutoML tasks demonstrate that MFES-HB can achieve 3. 3−8. 9× speedups over the state-of-the-art approach — BOHB.

PDF Details

ICRA Conference 2017 Conference Paper

Illumination insensitive efficient second-order minimization for planar object tracking

Lin Chen 0030
Fan Zhou 0007
Yu Shen
Xiang Tian 0002
Haibin Ling
Yaowu Chen

Tracking for planar objects is an important issue to vision-based robotic applications. In direct visual tracking (DVT) methods, the similarity between two images is often measured through the sum of squared differences (SSD) especially with the efficient second-order minimization (ESM) due to its simplicity and efficiency. However, SSD-based ESM is not robust to illumination changes since it is usually built upon the brightness constancy assumption. Contrast to image brightness, gradient orientations (GO) are invariant to both linear and non-linear illumination changes as verified in practice. Based on GO, we propose an illumination insensitive ESM method for planar object tracking in this paper. In order to introduce GO into the ESM, we generalized the original ESM formulas for multi-dimensional features. In addition, a denoising method based on the Perona-Malik function and a mask image were suggested to improve GO's robustness against image noise and low texture. Our experimental results on dataset for planar objects with illumination changes and a benchmark dataset confirm the proposed method is robust to illumination variations and capable to deal with the general tracking challenges.

Details