Author name cluster

Xue Li

Possible papers associated with this exact author name in Arrow. This page groups case-insensitive exact name matches and is not a full identity disambiguation profile.

27 papers

2 author rows

EAAI Journal 2026 Journal Article

A topology-aware segment anything model for domain-invariant crack segmentation

Xue Li
Siyi Yu
Shiyun Xiao
Xiangbo Lin
Zihan Zhao

Structural cracks play a vital role in prolonging the life of engineering structures and assessing safety risks. The precise segmentation and analysis of cracks has become challenging because of the complexity of background and diversity of morphology. In this study, we offer TopoSAM, a topology-aware model based on the Segment Anything Model (SAM) for domain-invariant crack segmentation. Particularly, we add a branch called Topology Serpentine Convolution Branch (TSCB) for recognizing the topological geometry and detailed features of the cracks in accordance with the characteristics of structural cracks. Cross Branch Fusor (CBF) is designed to fuse the features from TSCB and Image Encoder of SAM. Additionally, we present Background Adversarial Twin Learning(BATL) to remove the impact of background noise on the segmentation performance. The constructed twin samples with the same crack content but different backgrounds are sent into TopoSAM to participate in the training together, making the model actively ignore the background changes and concentrate on the structural cracks. TopoSAM is evaluated with seven publicly datasets. The experimental results demonstrate that TopoSAM outperforms the current state-of-the-art algorithms in generalization and has intense competition in crack segmentation accuracy.

AAAI Conference 2026 Conference Paper

Learning Structurally Stabilized Representations for Lossless DNA Storage

Ben Cao
Xue Li
Tiantian He
Bin Wang
Shihua Zhou
Xiaohu Wu
Qiang Zhang

This paper presents Reed-Solomon coded single-stranded representation learning (RSRL), a novel end-to-end model for learning representations for lossless DNA data storage. In contrast to existing learning-based methods, RSRL is inspired by both error-correction codec and structural biology. Specifically, RSRL first learns the representations for the subsequent storage from the binary data transformed by the Reed-Solomon codec (RS code). Then, the representations are masked by an RS-code-informed mask to focus on correcting the burst errors occurring in the learning process. The synergy of RS masks and graph attention enables active error localization, breaking through the limitations of traditional passive error correction. With the decoded representations with error corrections, a novel biologically stabilized loss is formulated to regularize the data representations to possess stable single-stranded structures. By incorporating these novel strategies, RSRL can learn highly durable, dense, and lossless representations for subsequent storage tasks in DNA sequences. The proposed RSRL has been compared with a number of baselines in real-world tasks of multi-type data storage. The experimental results obtained demonstrate that RSRL can store diverse types of data with much higher information density and durability, but much lower error rates.

PDF Details DOI

ICLR Conference 2025 Conference Paper

Chain-of-region: Visual Language Models Need Details for Diagram Analysis

Xue Li
Yiyou Sun
Wei Cheng 0002
Yinglun Zhu
Haifeng Chen

Visual Language Models (VLMs) like GPT-4V have broadened the scope of LLM applications, yet they face significant challenges in accurately processing visual details, particularly in scientific diagrams. This paper explores the necessity of meticulous visual detail collection and region decomposition for enhancing the performance of VLMs in scientific diagram analysis. We propose a novel approach that combines traditional computer vision techniques with VLMs to systematically decompose diagrams into discernible visual elements and aggregate essential metadata. Our method employs techniques in OpenCV library to identify and label regions, followed by a refinement process using shape detection and region merging algorithms, which are particularly suited to the structured nature of scientific diagrams. This strategy not only improves the granularity and accuracy of visual information processing but also extends the capabilities of VLMs beyond their current limitations. We validate our approach through a series of experiments that demonstrate enhanced performance in diagram analysis tasks, setting a new standard for integrating visual and language processing in a multimodal context.

EAAI Journal 2025 Journal Article

Machine learning applications for risk assessment in maritime transport: Current status and future directions

Yuqing Lin
Xue Li
Kum Fai Yuen

As the maritime transportation system is inherently complex and vulnerable to potential hazards, it is critical to conduct risk assessments via applicable methodologies. Recently, machine learning (ML) algorithms have attracted tremendous attention due to their ability to analyze risks effectively. Nevertheless, there is a lack of a systematic summarization of ML applications in maritime transport risk assessment (MTRA). Hence, this review aims to encapsulate the current status, issues, considerations, and future directions of ML applications using the systematic reviews and meta-analyses method. In particular, the status is summarized from the following three dimensions: advantages, disadvantages, and corresponding applications. Moreover, several issues are recognized, including dataset processing and methods utilization, and considerations from the perspective of sensitivity analysis and evaluation methods. Regarding future directions, promising opportunities in terms of data and method improvements are identified. Overall, this review contributes to MTRA by presenting the existing research status with a framework and providing suggestions on model selection and method improvement for future research.

EAAI Journal 2025 Journal Article

Multi-scale low-frequency enhanced spectral neural operator for reducing low-frequency error in partial differential equations solving

Fengrui Jing
Chuchu Zhai
Peizhi Zhao
Xue Li
Peifu Han
Hongzhen Ding
Yunlong Dong
Long Hao

Designing universal artificial intelligence (AI) solver for partial differential equations (PDEs) is an open problem and a significant challenge in science and engineering. AI-inspired data-driven solvers, such as neural operators, have achieved great success in PDE solving. However, insufficient low-frequency learning ability and inability to utilize physical prior knowledge remain an obstacle for the PDEs solver which designed by neural operator. To tackle this challenge, we drew inspiration from the multigrid method and developed the multi-scale low-frequency enhanced spectral neural operator. Our approach broadens the neural operator’s learnable range in the frequency domain by folding the frequency spectrum, thereby reducing low-frequency error through a carefully designed residual structure. Additionally, to adapt to the varying spectral distributions of different PDEs, we propose a neural operator correction strategy based on the correspondence between the PDE spectrum distribution pattern and the neural operator learning pattern in the low-frequency region, which we summarized, to correct the results by utilizing the prior knowledge of the PDE. Extensive experiments on benchmark fluid datasets, including the Darcy equation, the Navier–Stokes equation, and their variants, demonstrate that our model achieves a 26. 7% reduction in low-frequency error and a 25. 6% improvement in accuracy, outperforming traditional neural operators and verifying the effectiveness of the proposed method.

EAAI Journal 2025 Journal Article

Personalized and safe medication recommendation based on convolutional neural network and transformer architecture

Fengru Ge
Xiaomei Yu
Xue Li
Xingxu Fan
Yanjie Zhao

With the accumulation of electronic health records (EHRs), artificial intelligence (AI) based medical services such as medication recommendation (MR) has aroused widespread concern. However, existing drug recommendation models suffer from inadequate patient representation and adverse drug–drug interactions (DDIs). To address these challenges, we propose an AI-based personalized and safe medication recommendation method based on convolutional neural network and transformer architecture (CT-PASMR). Specifically, convolutional neural network (CNN) and transformer are combined in parallel (CAT) to model local relationships in a patient’s single visit and long-term dependencies in sequential EHR data, respectively. Subsequently, graph attention networks (GATs) are deployed to capture drug co-occurrences and adverse DDIs with various weights, generating safe drug representations. Moreover, a joint loss function is introduced to balance accuracy and safety in CT-PASMR. Finally, the experimental results on MIMIC (Medical Information Mart for Intensive Care)-III and MIMIC-IV datasets demonstrate that CT-PASMR achieves competitive performance on seven evaluation metrics such as DDI rate, Jaccard index and F1 score, compared with nine state-of-the-art (SOTA) baseline models. Ultimately, ablation studies and further analysis confirm the efficacy of CATs and GATs in providing personalized and safe medication recommendations. Code, qualitative results, and trained weights will be available at the link: https: //github. com/gefengru/CT-PASMR.

EAAI Journal 2025 Journal Article

Polyhedral representations with high-frequency for three-dimensional point cloud classification

Xiaoxin Mao
Xue Li
Puming Wang
Xin Jin
Shengfa Miao
Peng Liu
Shaowen Yao
Siwang Yang

Point clouds have become increasingly important due to their wide applications in fields such as autonomous driving and robotic navigation. However, the disordered and irregular nature of point clouds poses a challenge to traditional coordinate-based processing methods. These methods often struggle to accurately capture the geometric features of point clouds obtained in real environments, and the high-frequency structural information inherent in point clouds is often overlooked. In this study, a novel feature extraction method called Polyhedral Representations with High-Frequency (HF-Poly) is developed for point cloud classification. The HF-Poly method constructs tangent plane features for each point and creates polyhedral representations by neighboring tangent plane features. This approach enhances the capacity of the model to capture local features. Secondly, we design a novel high-frequency coding function that enhances the edge structure features using a high-frequency mapping strategy. This approach effectively delays the decay of these features in the deep network. In addition, we propose a general Local Channel Attention module (LCA), which assigns varying attention weights to different channels of the fused high-frequency polyhedral features to extract global information from the point cloud more efficiently. Extensive experiments on the ModelNet40 and ScanObjectNN datasets demonstrate that HF-Poly maintains superior classification accuracy while reducing computational costs compared to traditional methods, particularly achieving an impressive 88. 7 % accuracy on the ScanObjectNN dataset, with a 10. 8 % increase compared to PointNet++.

JBHI Journal 2025 Journal Article

Predicting Mutation-Disease Associations Through Protein Interactions Via Deep Learning

Xue Li
Ben Cao
Jianmin Wang
Xiangyu Meng
Shuang Wang
Yu Huang
Enrico Petretto
Tao Song

Disease is one of the primary factors affecting life activities, with complex etiologies often influenced by gene expression and mutation. Currently, wet lab experiments have analyzed the mechanisms of mutations, but these are usually limited by the costs of wet experiments and constraints in sample types and scales. Therefore, this paper constructs a real-world mutation-induced disease dataset and proposes Capsule and Graph topology networks with Multi-head attention (CGM) to predict the mutation-disease associations. CGM can accurately predict protein mutation-disease associations, and to further elucidate the pathogenicity of protein mutations, we also verified that protein mutations lead to protein structural alterations by the model, which suggests that mutation-induced conformational changes may be an important pathogenic factor. Limited by the size of the mutated protein dataset, we also performed experiments on benchmark and imbalanced datasets, where CGM mined 22 unknown protein interaction pairs from the benchmark dataset, better illustrating the potential of CGM in predicting mutation-disease associations. In summary, this paper curates a real dataset. It proposes that CGM predicts protein mutations and disease associations, providing a novel tool for further understanding of biomolecular pathways and disease mechanisms.

JBHI Journal 2025 Journal Article

Self-Supervised Monocular Depth Estimation for Endoscopic Imaging

Changsheng Li
Xue Li
Kaifeng Wang
Wenxin Chen
Qingyao Liu
Xingguang Duan

Endoscopy holds a pivotal role in the early detection and treatment of diverse diseases, with artificial intelligence (AI)-assisted methods increasingly gaining prominence in disease screening. Among them, the depth estimation from endoscopic sequences is crucial for a spectrum of AI-assisted surgical techniques. However, the development of endoscopic depth estimation algorithms presents a formidable challenge due to the unique environmental intricacies and constraints within the dataset. This paper proposes a self-supervised depth estimation network to comprehensively explore the brightness changes in endoscopic images, and fuse different features at multiple levels to achieve an accurate prediction of endoscopic depth. First, a FlowNet is designed to evaluate the brightness changes of adjacent frames by calculating the multi-scale structural similarity. Second, a feature fusion module is presented to capture multi-scale contextual information. Experiments show that the average accuracy of the algorithm is 97. 03% in the Stereo Correspondence and Reconstruction of Endoscopic Data (SCARED dataset). Based on the training parameters of the SCARED dataset, the algorithm achieves superior performance on the other two datasets (EndoSLAM and KVASIR dataset), indicating that the algorithm has good generalization performance.

NeurIPS Conference 2025 Conference Paper

When One Moment Isn't Enough: Multi-Moment Retrieval with Cross-Moment Interactions

Zhuo Cao
Heming Du
Bingqing Zhang
Xin Yu
Xue Li
Sen Wang

Existing Moment retrieval (MR) methods focus on Single-Moment Retrieval (SMR). However, one query can correspond to multiple relevant moments in real-world applications. This makes the existing datasets and methods insufficient for video temporal grounding. By revisiting the gap between current MR tasks and real-world applications, we introduce a high-quality datasets called QVHighlights Multi-Moment Dataset (QV-M$^2$), along with new evaluation metrics tailored for multi-moment retrieval (MMR). QV-M$^2$ consists of 2, 212 annotations covering 6, 384 video segments. Building on existing efforts in MMR, we propose a framework called FlashMMR. Specifically, we propose a Multi-moment Post-verification module to refine the moment boundaries. We introduce constrained temporal adjustment and subsequently leverage a verification module to re-evaluate the candidate segments. Through this sophisticated filtering pipeline, low-confidence proposals are pruned, and robust multi-moment alignment is achieved. We retrain and evaluate 6 existing MR methods on QV-M$^2$ and QVHighlights under both SMR and MMR settings. Results show that QV-M$^2$ serves as an effective benchmark for training and evaluating MMR models, while FlashMMR provides a strong baseline. Specifically, on QV-M$^2$, it achieves improvements over prior SOTA method by 3. 00% on G-mAP, 2. 70% on mAP@3+tgt, and 2. 56% on mR@3. The proposed benchmark and method establish a foundation for advancing research in more realistic and challenging video temporal grounding scenarios. Code is released at https: //github. com/Zhuo-Cao/QV-M2.

AAAI Conference 2024 Conference Paper

Dialogues Are Not Just Text: Modeling Cognition for Dialogue Coherence Evaluation

Xue Li
Jia Su
Yang Yang
Zipeng Gao
Xinyu Duan
Yi Guan

The generation of logically coherent dialogues by humans relies on underlying cognitive abilities. Based on this, we redefine the dialogue coherence evaluation process, combining cognitive judgment with the basic text to achieve a more human-like evaluation. We propose a novel dialogue evaluation framework based on Dialogue Cognition Graph (DCGEval) to implement the fusion by in-depth interaction between cognition modeling and text modeling. The proposed Abstract Meaning Representation (AMR) based graph structure called DCG aims to uniformly model four dialogue cognitive abilities. Specifically, core-semantic cognition is modeled by converting the utterance into an AMR graph, which can extract essential semantic information without redundancy. The temporal and role cognition are modeled by establishing logical relationships among the different AMR graphs. Finally, the commonsense knowledge from ConceptNet is fused to express commonsense cognition. Experiments demonstrate the necessity of modeling human cognition for dialogue evaluation, and our DCGEval presents stronger correlations with human judgments compared to other state-of-the-art evaluation metrics.

PDF Details DOI

JBHI Journal 2024 Journal Article

Multi-Loss Disentangled Generative-Discriminative Learning for Multimodal Representation in Schizophrenia

Peilun Song
Xiuxia Yuan
Xue Li
Xueqin Song
Yaping Wang

Schizophrenia (SCZ) is a multifactorial mental illness, thus it will be beneficial for exploring this disease using multimodal data, including functional magnetic resonance imaging (fMRI), genes, and the gut microbiome. Previous studies reported combining multimodal data can offer complementary information for better depicting the abnormalities of SCZ. However, the existing multimodal-based methods have multiple limitations. First, most approaches cannot fully use the relationships among different modalities for the downstream tasks. Second, representing multimodal data by the modality-common and modality-specific components can improve the performance of multimodal analysis but often be ignored. Third, most methods conduct the model for classification or regression, thus a unified model is needed for finishing these tasks simultaneously. To this end, a multi-loss disentangled generative-discriminative learning (MDGDL) model was developed to tackle these issues. Specifically, using disentangled learning method, the genes and gut microbial biomarkers were represented and separated into two modality-specific vectors and one modality-common vector. Then, a generative-discriminative framework was introduced to uncover the relationships between fMRI features and these three latent vectors, further producing the attentive vectors, which can help fMRI features for the downstream tasks. To validate the performance of MDGDL, an SCZ classification task and a cognitive score regression task were conducted. Results showed the MDGDL achieved superior performance and identified the most important multimodal biomarkers for the SCZ. Our proposed model could be a supplementary approach for multimodal data analysis. Based on this method, we could analyze the SCZ by combining multimodal data, and further obtain some interesting findings.

YNICL Journal 2024 Journal Article

Relationship of irisin with disease severity and dopamine uptake in Parkinson's disease patients

Xiaoxue Shi
Qi Gu
Chang Fu
Jianjun Ma
Dongsheng Li
Jinhua Zheng
Siyuan Chen
Zonghan She

BACKGROUND: This study was designed to investigate the relationship of irisin with the severity of Parkinson's disease (PD) and dopamine (DOPA) uptake in patients with PD and to understand the role of irisin in PD. METHODS: The plasma levels of irisin and α-syn were measured by enzyme-linked immunosorbent assay (ELISA). Motor and nonmotor symptoms were assessed with the relevant scales. DOPA uptake was measured with DOPA positron emission tomography (PET)/magnetic resonance imaging (MRI). RESULTS: The plasma levels of α-syn and irisin in patients with PD gradually increased and decreased, respectively, with the progression of the disease. There was a negative correlation between plasma α-syn and irisin levels in patients with PD. The level of irisin in plasma was negatively correlated with Unified Parkinson's Disease Rating Scale (UPDRS)-III scores and positively correlated with Montreal Cognitive Assessment (MoCA) scores. The striatal/occipital lobe uptake ratios (SORs) of the ipsilateral and contralateral caudate nucleus and anterior and posterior putamen in the high-irisin group were significantly higher than those in the low-irisin group, and irisin levels in the caudate nucleus and anterior and posterior putamen contralateral to the affected limb were lower than those on the ipsilateral side. The level of irisin was positively correlated with the SORs of the ipsilateral and contralateral caudate nucleus and putamen in PD patients. CONCLUSIONS: Irisin plays a neuroprotective role by decreasing the level of α-syn. Irisin is negatively correlated with the severity of motor symptoms and cognitive impairment. More importantly, irisin can improve DOPA uptake in the striatum of patients with PD, especially on the side contralateral to the affected limb.

YNICL Journal 2024 Journal Article

Right superior frontal gyrus: A potential neuroimaging biomarker for predicting short-term efficacy in schizophrenia

Yongfeng Yang
Xueyan Jin
Yongjiang Xue
Xue Li
Yi Chen
Ning Kang
Wei Yan
Peng Li

Antipsychotic drug treatment for schizophrenia (SZ) can alter brain structure and function, but it is unclear if specific regional changes are associated with treatment outcome. Therefore, we examined the effects of antipsychotic drug treatment on regional grey matter (GM) density, white matter (WM) density, and functional connectivity (FC) as well as associations between regional changes and treatment efficacy. SZ patients (n = 163) and health controls (HCs) (n = 131) were examined by structural magnetic resonance imaging (sMRI) at baseline, and a subset of SZ patients (n = 77) were re-examined after 8 weeks of second-generation antipsychotic treatment to assess changes in regional GM and WM density. In addition, 88 SZ patients and 81 HCs were examined by resting-state functional MRI (rs-fMRI) at baseline and the patients were re-examined post-treatment to examine FC changes. The Positive and Negative Syndrome Scale (PANSS) and MATRICS Consensus Cognitive Battery (MCCB) were applied to measure psychiatric symptoms and cognitive impairments in SZ. SZ patients were then stratified into response and non-response groups according to PANSS score change (≥50 % decrease or <50 % decrease, respectively). The GM density of the right cingulate gyrus, WM density of the right superior frontal gyrus (SFG) plus 5 other WM tracts were reduced in the response group compared to the non-response group. The FC values between the right anterior cingulate and paracingulate gyrus and left thalamus were reduced in the entire SZ group (n = 88) after treatment, while FC between the right inferior temporal gyrus (ITG) and right medial superior frontal gyrus (SFGmed) was increased in the response group. There were no significant changes in regional FC among the non-response group after treatment and no correlations with symptom or cognition test scores. These findings suggest that the right SFG is a critical target of antipsychotic drugs and that WM density and FC alterations within this region could be used as potential indicators in predicting the treatment outcome of antipsychotics of SZ.

AAMAS Conference 2023 Conference Paper

A Web-based Tool for Detecting Argument Validity and Novelty

Sandrine Chausson
Ameer Saadat-Yazdi
Xue Li
Jeff Z. Pan
Vaishak Belle
Nadin Kökciyan
Björn Ross

Individuals engage in arguments on an everyday basis as they seek to obtain information about current affairs and engage with social media. While fact-checkers are available to help dispel misinformation, it is almost impossible for users to verify every single claim they encounter. This means that oftentimes, it is left to the user to decide whether a claim is well supported. To address this, we have developed a Web interface that allows users to input an argument, and our developed framework automatically detects its validity (soundness of logical deduction) and novelty (whether the argument is non-circular). Our Web-based tool could be used by social media users who wish to evaluate the information they consume. As part of one of the modules developed at the University of Edinburgh, our tool will be deployed as a teaching tool for the students who study argumentation.

JBHI Journal 2023 Journal Article

Metadata and Image Features Co-Aware Personalized Federated Learning for Smart Healthcare

Tong Jin
Shujia Pan
Xue Li
Siguang Chen

Recently, artificial intelligence has been widely used in intelligent disease diagnosis and has achieved great success. However, most of the works mainly rely on the extraction of image features but ignore the use of clinical text information of patients, which may limit the diagnosis accuracy fundamentally. In this paper, we propose a metadata and image features co-aware personalized federated learning scheme for smart healthcare. Specifically, we construct an intelligent diagnosis model, by which users can obtain fast and accurate diagnosis services. Meanwhile, a personalized federated learning scheme is designed to utilize the knowledge learned from other edge nodes with larger contributions and customize high-quality personalized classification models for each edge node. Subsequently, a Naïve Bayes classifier is devised for classifying patient metadata. And then the image and metadata diagnosis results are jointly aggregated by different weights to improve the accuracy of intelligent diagnosis. Finally, the simulation results illustrate that, compared with the existing methods, our proposed algorithm achieves better classification accuracy, reaching about 97. 16% on PAD-UFES-20 dataset.

TIST Journal 2023 Journal Article

Skin Lesion Intelligent Diagnosis in Edge Computing Networks: An FCL Approach

Yanhang Shi
Xue Li
Siguang Chen

In recent years, automatic skin lesion diagnosis methods based on artificial intelligence have achieved great success. However, the lack of labeled data, visual similarity between skin diseases, and restriction on private data sharing remain the major challenges in skin lesion diagnosis. In this article, first, we propose a federated contrastive learning framework to break down data silos and enhance the generalizability of diagnostic model to unseen data. Subsequently, by combining data features from different participated nodes, the proposed framework can improve the performance of contrastive training. To extract discriminative features during on-device training, we propose a contrastive learning based intelligent skin lesion diagnosis scheme in edge computing networks. Specifically, a contrastive learning based dual encoder network is designed to overcome training sample scarcity by fully leveraging unlabeled samples for performance improvement. Meanwhile, we devise a maximum mean discrepancy based supervised contrastive loss function, which can efficiently explore complex intra-class and inter-class variances of samples. Finally, the diagnosis simulations demonstrate that compared with existing methods, our proposed scheme can achieve superior accuracy in both on-device training and distributed training scenarios.

IS Journal 2022 Journal Article

Dynamic Sampling and Selective Masking for Communication-Efficient Federated Learning

Shaoxiong Ji
Wenqi Jiang
Anwar Walid
Xue Li

Federated learning (FL) is a novel machine learning setting that enables on-device intelligence via decentralized training and federated optimization. Deep neural networks’ rapid development facilitates the learning techniques for modeling complex problems and emerges into federated deep learning under the federated setting. However, the tremendous amount of model parameters burdens the communication network with a high load of transportation. This article introduces two approaches for improving communication efficiency by dynamic sampling and top-$k$k selective masking. The former controls the fraction of selected client models dynamically, while the latter selects parameters with top-$k$k largest values of difference for federated updating. Experiments on convolutional image classification and recurrent language modeling are conducted on three public datasets to show our proposed methods’ effectiveness.

IJCAI Conference 2022 Conference Paper

KUNet: Imaging Knowledge-Inspired Single HDR Image Reconstruction

Hu Wang
Mao Ye
Xiatian Zhu
Shuai Li
Ce Zhu
Xue Li

Recently, with the rise of high dynamic range (HDR) display devices, there is a great demand to transfer traditional low dynamic range (LDR) images into HDR versions. The key to success is how to solve the many-to-many mapping problem. However, the existing approaches either do not consider constraining solution space or just simply imitate the inverse camera imaging pipeline in stages, without directly formulating the HDR image generation process. In this work, we address this problem by integrating LDR-to-HDR imaging knowledge into an UNet architecture, dubbed as Knowledge-inspired UNet (KUNet). The conversion from LDR-to-HDR image is mathematically formulated, and can be conceptually divided into recovering missing details, adjusting imaging parameters and reducing imaging noise. Accordingly, we develop a basic knowledge-inspired block (KIB) including three subnetworks corresponding to the three procedures in this HDR imaging process. The KIB blocks are cascaded in the similar way to the UNet to construct HDR image with rich global information. In addition, we also propose a knowledge inspired jump-connect structure to fit a dynamic range gap between HDR and LDR images. Experimental results demonstrate that the proposed KUNet achieves superior performance compared with the state-of-the-art methods. The code, dataset and appendix materials are available at https: //github. com/wanghu178/KUNet. git.

PDF Details DOI

IS Journal 2021 Journal Article

Differentially Private Collaborative Coupling Learning for Recommender Systems

Yanjun Zhang
Guangdong Bai
Mingyang Zhong
Xue Li
Ryan K. L. Ko

Coupling learning is designed to estimate, discover, and extract the interactions and relationships among learning components. It provides insights into complex interactive data, and has been extensively incorporated into recommender systems to enhance the interpretability of sophisticated relationships between users and items. Coupling learning can be further fostered once the trending collaborative learning can be engaged to take advantage of the cross-platform data. To facilitate this, privacy-preserving solutions are in high demand—it is desired that the collaboration should not expose either the private data of each individual owner or the model parameters trained on their datasets. In this article, we develop a distributed collaborative coupling learning system, which enables differential privacy. The proposed system defends against the adversary who has gained full knowledge of the training mechanism and the access to the model trained collaboratively. It also addresses the privacy-utility tradeoff by a provable tight sensitivity bound. Our experiments demonstrate that the proposed system guarantees favorable privacy gains at a modest cost in recommendation quality, even in scenarios with a large number of training epochs.

AAAI Conference 2020 Conference Paper

Adaptive Two-Dimensional Embedded Image Clustering

Zhihui Li
Lina Yao
Sen Wang
Salil Kanhere
Xue Li
Huaxiang Zhang

With the rapid development of mobile devices, people are generating huge volumes of images data every day for sharing on social media, which draws much research attention to understanding the contents of images. Image clustering plays an important role in image understanding systems. Often, most of the existing image clustering algorithms ﬂatten digital images that are originally represented by matrices into 1D vectors as the image representation for the subsequent learning. The drawbacks of vector-based algorithms include limited consideration of spatial relationship between pixels and computational complexity, both of which blame to the simple vectorized representation. To overcome the drawbacks, we propose a novel image clustering framework that can work directly on matrices of images instead of ﬂattened vectors. Speciﬁcally, the proposed algorithm simultaneously learn the clustering results and preserve the original correlation information within the image matrix. To solve the challenging objective function, we propose a fast iterative solution. Extensive experiments have been conducted on various benchmark datasets. The experimental results conﬁrm the superiority of the proposed algorithm.

AIJ Journal 2020 Journal Article

Robust learning with imperfect privileged information

Xue Li
Bo Du
Chang Xu
Yipeng Zhang
Lefei Zhang
Dacheng Tao

In the learning using privileged information (LUPI) paradigm, example data cannot always be clean, while the gathered privileged information can be imperfect in practice. Here, imperfect privileged information can refer to auxiliary information that is not always accurate or perturbed by noise, or alternatively to incomplete privileged information, where privileged information is only available for part of the training data. Because of the lack of clear strategies for handling noise in example data and imperfect privileged information, existing learning using privileged information (LUPI) methods may encounter serious issues. Accordingly, in this paper, we propose a Robust SVM+ method to tackle imperfect data in LUPI. In order to make the SVM+ model robust to noise in example data and privileged information, Robust SVM+ maximizes the lower bound of the perturbations that may influence the judgement based on a rigorous theoretical analysis. Moreover, in order to deal with the incomplete privileged information, we use the available privileged information to help us in approximating the missing privileged information of training data. The optimization problem of the proposed method can be efficiently solved by employing a two-step alternating optimization strategy, based on iteratively deploying off-the-shelf quadratic programming solvers and the alternating direction method of multipliers (ADMM) technique. Comprehensive experiments on real-world datasets demonstrate the effectiveness of the proposed Robust SVM+ method in handling imperfect privileged information.

AIIM Journal 2020 Journal Article

Temporal tree representation for similarity computation between medical patients

Suresh Pokharel
Guido Zuccon
Xue Li
Chandra Prasetyo Utomo
Yu Li

Objective The aim of this study is to compute similarities between patient records in an electronic health record (EHR). This is an important problem because the availability of effective methods for the computation of patient similarity would allow for assistance with and automation of tasks such as patients stratification, medical prognosis and cohort selection, and for unlocking the potential of medical analytics methods for healthcare intelligence. However, health data in EHRs presents many challenges that make the automatic computation of patient similarity difficult; these include: temporal aspects, multivariate, heterogeneous and irregular data, and data sparsity. Materials and methods We propose a new method for EHR data representation called Temporal Tree: a temporal hierarchical representation which, based on temporal co-occurrence, preserves the compound information found at different levels in health data. In addition, this representation is augmented using the doc2vec embedding technique which here is exploited for patient similarity computation. We empirically investigate our proposed method, along with several state-of-the-art benchmarks, on a dataset of real world Intensive Care Unit (ICU) EHRs, for the task of identifying patients with a specific target diagnosis. Results Our empirical results show that the Temporal Trees representation is significantly better than other traditional and state-of-the-art methods for representing patients and computing their similarities. Conclusion Temporal trees capture the temporal relationships between medical, hierarchical data: this enables to effectively model the rich information provided within EHRs and thus the identification of similar patients.

IJCAI Conference 2018 Conference Paper

R-SVM+: Robust Learning with Privileged Information

Xue Li
Bo Du
Chang Xu
Yipeng Zhang
Lefei Zhang
Dacheng Tao

In practice, the circumstance that training and test data are clean is not always satisfied. The performance of existing methods in the learning using privileged information (LUPI) paradigm may be seriously challenged, due to the lack of clear strategies to address potential noises in the data. This paper proposes a novel Robust SVM+ (RSVM+) algorithm based on a rigorous theoretical analysis. Under the SVM+ framework in the LUPI paradigm, we study the lower bound of perturbations of both example feature data and privileged feature data, which will mislead the model to make wrong decisions. By maximizing the lower bound, tolerance of the learned model over perturbations will be increased. Accordingly, a novel regularization function is introduced to upgrade a variant form of SVM+. The objective function of RSVM+ is transformed into a quadratic programming problem, which can be efficiently optimized using off-the-shelf solvers. Experiments on real-world datasets demonstrate the necessity of studying robust SVM+ and the effectiveness of the proposed algorithm.

IJCAI Conference 2017 Conference Paper

Dual Track Multimodal Automatic Learning through Human-Robot Interaction

Shuqiang Jiang
Weiqing Min
Xue Li
Huayang Wang
Jian Sun
Jiaqi Zhou

Human beings are constantly improving their cognitive ability via automatic learning from the interaction with the environment. Two important aspects of automatic learning are the visual perception and knowledge acquisition. The fusion of these two aspects is vital for improving the intelligence and interaction performance of robots. Many automatic knowledge extraction and recognition methods have been widely studied. However, little work focuses on integrating automatic knowledge extraction and recognition into a unified framework to enable jointly visual perception and knowledge acquisition. To solve this problem, we propose a Dual Track Multimodal Automatic Learning (DTMAL) system, which consists of two components: Hybrid Incremental Learning (HIL) from the vision track and Multimodal Knowledge Extraction (MKE) from the knowledge track. HIL can incrementally improve recognition ability of the system by learning new object samples and new object concepts. MKE is capable of constructing and updating the multimodal knowledge items based on the recognized new objects from HIL and other knowledge by exploring the multimodal signals. The fusion of the two tracks is a mutual promotion process and jointly devote to the dual track learning. We have conducted the experiments through human-machine interaction and the experimental results validated the effectiveness of our proposed system.

TIST Journal 2017 Journal Article

Unveiling Correlations via Mining Human-Thing Interactions in the Web of Things

Lina Yao
Quan Z. Sheng
Anne H. H. Ngu
Xue Li
Boualem Benattalah

With recent advances in radio-frequency identification (RFID), wireless sensor networks, and Web services, physical things are becoming an integral part of the emerging ubiquitous Web. Finding correlations among ubiquitous things is a crucial prerequisite for many important applications such as things search, discovery, classification, recommendation, and composition. This article presents DisCor-T, a novel graph-based approach for discovering underlying connections of things via mining the rich content embodied in the human-thing interactions in terms of user, temporal, and spatial information. We model this various information using two graphs, namely a spatio-temporal graph and a social graph. Then, random walk with restart (RWR) is applied to find proximities among things, and a relational graph of things (RGT) indicating implicit correlations of things is learned. The correlation analysis lays a solid foundation contributing to improved effectiveness in things management and analytics. To demonstrate the utility of the proposed approach, we develop a flexible feature-based classification framework on top of RGT and perform a systematic case study. Our evaluation exhibits the strength and feasibility of the proposed approach.

AAAI Conference 2017 Conference Paper

Video Captioning with Listwise Supervision

Yuan Liu
Xue Li
Zhongchao Shi

Automatically describing video content with natural language is a fundamental challenging that has received increasing attention. However, existing techniques restrict the model learning on the pairs of each video and its own sentences, and thus fail to capture more holistically semantic relationships among all sentences. In this paper, we propose to model relative relationships of different video-sentence pairs and present a novel framework, named Long Short-Term Memory with Listwise Supervision (LSTM-LS), for video captioning. Given each video in training data, we obtain a ranking list of sentences w. r. t. a given sentence associated with the video using nearest-neighbor search. The ranking information is represented by a set of rank triplets that can be used to assess the quality of ranking list. The video captioning problem is then solved by learning LSTM model for sentence generation, through maximizing the ranking quality over all the sentences in the list. The experiments on MSVD dataset show that our proposed LSTM-LS produces better performance than the state of the art in generating natural sentences: 51. 1% and 32. 6% in terms of BLEU@4 and METEOR, respectively. Superior performances are also reported on the movie description M-VAD dataset.