Author name cluster

Su Lu

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

5 papers

2 author rows

ICML Conference 2024 Conference Paper

Revisit the Essence of Distilling Knowledge through Calibration

Wen-Shu Fan
Su Lu
Xin-Chun Li
De-Chuan Zhan
Le Gan

Knowledge Distillation (KD) has evolved into a practical technology for transferring knowledge from a well-performing model (teacher) to a weak model (student). A counter-intuitive phenomenon known as capacity mismatch has been identified, wherein KD performance may not be good when a better teacher instructs the student. Various preliminary methods have been proposed to alleviate capacity mismatch, but a unifying explanation for its cause remains lacking. In this paper, we propose a unifying analytical framework to pinpoint the core of capacity mismatch based on calibration. Through extensive analytical experiments, we observe a positive correlation between the calibration of the teacher model and the KD performance with original KD methods. As this correlation arises due to the sensitivity of metrics (e. g. , KL divergence) to calibration, we recommend employing measurements insensitive to calibration such as ranking-based loss. Our experiments demonstrate that ranking-based loss can effectively replace KL divergence, aiding large models with poor calibration to teach better.

Details

ICRA Conference 2021 Conference Paper

Dual-Arm Needle Manipulation with the da Vinci ® Surgical Robot Under Uncertainty

Su Lu
Thomas Shkurti
Murat Cenk Çavusoglu

This paper proposes a path correction method for surgical robotic systems performing needle handoff manipulations as part of autonomous execution of surgical suturing. During handoff motions, the position and orientation of the needle is subject to perturbations from the idealized planned pose due to uncertainties in camera-robot calibration and needle localization. If, after a perturbation, the system needs to perform subsequent needle regrasp(s), but the robot still follows the originally planned trajectory out of the path planner [1], it has a lower chance of gripping the needle properly. In order to accommodate these unpredictable needle pose perturbations, the proposed path correction method works locally to direct the needle from the wrong pose to the original pose by partial replanning of the robot motion. The reliability of the proposed method is evaluated with three sets of experiments in a simulation environment.

Details

AAAI Conference 2021 Conference Paper

Tailoring Embedding Function to Heterogeneous Few-Shot Tasks by Global and Local Feature Adaptors

Su Lu
Han-Jia Ye
De-Chuan Zhan

Few-Shot Learning (FSL) is essential for visual recognition. Many methods tackle this challenging problem via learning an embedding function from seen classes and transfer it to unseen classes with a few labeled instances. Researchers recently found it beneficial to incorporate task-specific feature adaptation into FSL models, which produces the most representative features for each task. However, these methods ignore the diversity of classes and apply a global transformation to the task. In this paper, we propose Global and Local Feature Adaptor (GLoFA), a unifying framework that tailors the instance representation to specific tasks by global and local feature adaptors. We claim that class-specific local transformation helps to improve the representation ability of feature adaptor. Global masks tend to capture sketchy patterns, while local masks focus on detailed characteristics. A strategy to measure the relationship between instances adaptively based on the characteristics of both tasks and classes endow GLoFA with the ability to handle mix-grained tasks. GLoFA outperforms other methods on a heterogeneous task distribution and achieves competitive results on benchmark datasets.

PDF Details

NeurIPS Conference 2021 Conference Paper

Towards Enabling Meta-Learning from Target Models

Su Lu
Han-Jia Ye
Le Gan
De-Chuan Zhan

Meta-learning can extract an inductive bias from previous learning experience and assist the training of new tasks. It is often realized through optimizing a meta-model with the evaluation loss of task-specific solvers. Most existing algorithms sample non-overlapping $\mathit{support}$ sets and $\mathit{query}$ sets to train and evaluate the solvers respectively due to simplicity ($\mathcal{S}$/$\mathcal{Q}$ protocol). Different from $\mathcal{S}$/$\mathcal{Q}$ protocol, we can also evaluate a task-specific solver by comparing it to a target model $\mathcal{T}$, which is the optimal model for this task or a model that behaves well enough on this task ($\mathcal{S}$/$\mathcal{T}$ protocol). Although being short of research, $\mathcal{S}$/$\mathcal{T}$ protocol has unique advantages such as offering more informative supervision, but it is computationally expensive. This paper looks into this special evaluation method and takes a step towards putting it into practice. We find that with a small ratio of tasks armed with target models, classic meta-learning algorithms can be improved a lot without consuming many resources. We empirically verify the effectiveness of $\mathcal{S}$/$\mathcal{T}$ protocol in a typical application of meta-learning, $\mathit{i. e. }$, few-shot learning. In detail, after constructing target models by fine-tuning the pre-trained network on those hard tasks, we match the task-specific solvers and target models via knowledge distillation.

PDF Details

AAAI Conference 2019 Conference Paper

Multi-View Anomaly Detection: Neighborhood in Locality Matters

Xiang-Rong Sheng
De-Chuan Zhan
Su Lu
Yuan Jiang

Identifying anomalies in multi-view data is a difficult task due to the complicated data characteristics of anomalies. Specifically, there are two types of anomalies in multi-view data–anomalies that have inconsistent features across multiple views and anomalies that are consistently anomalous in each view. Existing multi-view anomaly detection approaches have some issues, e. g. , they assume multiple views of a normal instance share consistent and normal clustering structures while anomaly exhibits anomalous clustering characteristics across multiple views. When there are no clusters in data, it is difficult for existing approaches to detect anomalies. Besides, existing approaches construct a profile of normal instances, then identify instances that do not conform to the normal profile as anomalies. The objective is formulated to profile normal instances, but not to estimate the set of normal instances, which results in sub-optimal detectors. In addition, the model trained to profile normal instances uses the entire dataset including anomalies. However, anomalies could undermine the model, i. e. , the model is not robust to anomalies. To address these issues, we propose the nearest neighborbased MUlti-View Anomaly Detection (MUVAD) approach. Specifically, we first propose an anomaly measurement criterion and utilize this criterion to formulate the objective of MUVAD to estimate the set of normal instances explicitly. We further develop two concrete relaxations for implementing the MUVAD as MUVAD-QPR and MUVAD-FSR. Experimental results validate the superiority of the proposed MU- VAD approaches.

PDF Details