LinLin Shen

JBHI Journal 2026 Journal Article

ACGM: Attribute-Centric Graph Modeling Network for Concurrent Missing Tabular Data Imputation and COVID-19 Prognosis

• Zhuoru Wu
• Wenting Chen
• Xuechen Li
• Filippo Ruffini
• Shaonan Liu
• Lorenzo Tronchin
• Domenico Albano
• Eliodoro Faiella

COVID-19 prognosis using clinical tabular data faces significant challenges due to missing values and class imbalance issues. Existing methods often overlook the complex high-order interrelationship among clinicalattributes and struggle with training stability on imbalanced datasets. We propose ACGM, an attribute-centric graph modeling network that simultaneously addresses missing data imputation and COVID-19 prognosis. ACGM consists of three key modules: an attributes preprocessing module (APM) for coarse-grained imputation initialization, a graph-enhanced attributes imputation module (GEAIM) that models high-order inter-attribute relationships through graph structures, and a graph-enhanced disease prognosis module (GEDPM) that leverages these complex attribute interactions for final prediction. GEAIM and GEDPM employ a mean-teacher strategy with attributes graph matching to preserve high-order relationships, enhance training stability, and maintain structural integrity of attribute interactions. Extensive experiments are conducted on four public COVID-19 tabular datasets, demonstrating the superiority of our ACGM over existing methods. Through comprehensive interpretability analysis, we identify that attributes such as LDH, Difficulty In Breathing, and SaO 2 significantly impact COVID-19 prognosis, aligning well with clinical insights and radiologist assessments.

AAAI Conference 2026 Conference Paper

FineXtrol: Controllable Motion Generation via Fine-Grained Text

• Keming Shen
• Bizhu Wu
• Junliang Chen
• Xiaoqin Wang
• Linlin Shen

Recent works have sought to enhance the controllability and precision of text-driven motion generation. Some approaches leverage large language models (LLMs) to produce more detailed texts, while others incorporate global 3D coordinate sequences as additional control signals. However, the former often introduces misaligned details and lacks explicit temporal cues, and the latter incurs significant computational cost when converting coordinates to standard motion representations. To address these issues, we propose FineXtrol, a novel control framework for efficient motion generation guided by temporally-aware, precise, user-friendly and fine-grained textual control signals that describe specific body part movements over time. In support of this framework, we design a hierarchical contrastive learning module that encourages the text encoder to produce more discriminative embeddings for our novel control signals, thereby improving motion controllability. Quantitative results show that FineXtrol achieves strong performance in controllable motion generation, while qualitative analysis demonstrates its flexibility in directing specific body part movements.

AAAI Conference 2026 Conference Paper

Incomplete Multi-view Diabetic Retinopathy Grading via Self-Supervised Inter- and Intra-View Restoration

• Zhihao Wu
• Yuxin Lin
• Jie Wen
• Wuzhen Shi
• Linlin Shen

Multi-view diabetic retinopathy (DR) grading has achieved remarkable performance by capturing more comprehensive pathological features than single-view methods. However, complete multi-view fundus images are often difficult to obtain in clinical practice, and the performance degrades significantly when fewer views are available. To overcome this limitation, we propose the first incomplete multi-view DR grading framework, aiming to provide accurate diagnosis regardless of the number of available views. It introduces two novel modules. First, cross-view spatial correlation attention (CSCA) captures region correlations across views, automatically identifying and fusing diagnostically relevant spatial features to improve feature representation. Second, self-supervised mask consistency learning (SMCL) formulates a novel pretext task of missing-view information reconstruction by strategically masking inter- and intra-view regions, enabling the model to infer complete features from incomplete views. Benefiting from CSCA and SMCL, our method enhances structural feature consistency across views and effectively compensates for missing information during DR grading. Extensive experiments demonstrate that our method achieves state-of-the-art grading performance, particularly under realistic conditions where some views are unavailable.

JBHI Journal 2026 Journal Article

LUS-DET: Liver Ultrasound Open-Vocabulary Object Detection

• Jiansong Zhang
• Xiaoling Luo
• Guorong Lyu
• Yongjian Chen
• Linlin Shen

In liver ultrasound, the acquisition of standard scanning planes serves as a prerequisite for reliable lesion assessment. In clinical practice, physicians make diagnostic decisions by jointly interpreting the spatial configuration of key anatomical structures within standard planes and local lesion features. However, existing studies commonly treat standard plane recognition and lesion detection as two separate tasks, lacking a unified modeling approach that reflects their semantic continuity and clinical interdependence. Inspired by the diagnostic workflow of liver ultrasound, we propose LUS-DET, an open-vocabulary object detection framework designed to semantically bridge liver ultrasound standard plane analysis (LUSP) and liver ultrasound disease diagnosis (LUDD) through text-guided modeling. Specifically, we curate a retrospective LUSP dataset and develop a region–text alignment mechanism linking 44, 669 region–caption pairs across 12 anatomical categories to enable in-domain open-vocabulary pretraining. Building upon this alignment, we introduce object prompts to guide zero-shot lesion detection in an open-source LUDD task without using any lesion-specific annotations. Experimental results demonstrate that LUS-DET not only achieves competitive zero-shot performance, but also exhibits superior accuracy and robustness during end-to-end fine-tuning compared to conventional detection baselines. To the best of our knowledge, this is the first study to propose a clinically coherent modelling paradigm that unifies standard plane localisation and lesion analysis in liver ultrasound, providing a new direction for structure-aware and workflow-aligned AI systems in medical imaging.

EAAI Journal 2026 Journal Article

Shape-aware and feature fused power line detection network

• Shengdong Zhang
• Xiaoqin Zhang
• Wenqi Ren
• Linlin Shen
• Jun Zhang

Unmanned Aerial Vehicles (UAVs) have demonstrated flexibility and efficiency in power system inspection. Automatic power line detection is crucial to prevent collisions of UAVs, typically using visible or infrared (IR) sensors. In practice, infrared sensors are typically employed under adverse weather conditions due to their robustness to atmospheric interference, while visible-light sensors are preferred in clear weather to leverage their high resolution and detailed spectral information. However, this task remains highly challenging due to complex backgrounds and the inherently thin, elongated structure of power lines, which occupy minimal space in aerial imagery. These factors make power line detection a significant and ongoing problem in computer vision. Recently, convolutional neural networks (CNNs) have shown strong performance in semantic segmentation tasks. Yet, they often struggle with severe class imbalance, such as in power line detection, where background pixels dominate. To address these challenges, we propose a novel Shape-aware and Feature Fused Power Line Detection Network (SFFPLDN). Our approach includes: Shape-Aware Block (SAB) that enhances feature propagation from power line regions, and Spatial and Channel Attention Fusion Block (SCAFB) that adaptively merges multi-scale features by jointly evaluating spatial and channel-wise importance. We evaluate SFFPLDN on a public dataset containing both Infrared-IR and Visible-Light image sets. Extensive experiments demonstrate the superiority of our method. For the Infrared-IR subset, SFFPLDN achieves a sensitivity ( S e ) of 0. 859, a dice coefficient of 0. 882, and an Area Under Curve-Precision Recall ( A U C p r ) of 0. 921. For the Visible-Light subset, it attains a sensitivity of 0. 863, a dice coefficient of 0. 860, and an A U C p r of 0. 931. These results not only validate the effectiveness of each proposed module but also establish a new state-of-the-art benchmark for power line detection.

EAAI Journal 2025 Journal Article

A codebook-driven approach for low-light image enhancement

• Xu Wu
• Xianxu Hou
• Zhihui Lai
• Jie Zhou
• Ya-nan Zhang
• Witold Pedrycz
• Linlin Shen

Low-light image enhancement (LLIE) aims to improve low-illumination images. However, existing methods face two challenges: (1) uncertainty in restoration from diverse brightness degradations; (2) loss of texture and color information caused by noise suppression and light enhancement. In this paper, we propose a novel enhancement approach, CodeEnhance, by leveraging discrete codebook priors and image refinement to address these challenges. In particular, we reframe LLIE as learning an image-to-code mapping from low-light images to discrete codebook, which has been learned from high-quality images. To enhance this process, a Semantic Embedding Module (SEM) is introduced to integrate semantic information with low-level features, and a Codebook Shift (CS) mechanism, designed to adapt the pre-learned codebook to better suit the distinct characteristics of our low-light dataset. Additionally, we present an Interactive Feature Transformation (IFT) module to refine texture and color information during image reconstruction, allowing for interactive enhancement based on user preferences. Extensive experiments on both real-world and synthetic benchmarks demonstrate that the incorporation of prior knowledge and controllable information transfer significantly enhances LLIE performance in terms of quality and fidelity. The proposed CodeEnhance exhibits superior robustness to various degradations, including uneven illumination, noise, and color distortion. The code can be obtained from https: //github. com/csxuwu/CodeEnhance or https: //www. scholat. com/laizhihui. cn.

ICML Conference 2025 Conference Paper

BCE vs. CE in Deep Feature Learning

• Qiufu Li
• Huibin Xiao
• LinLin Shen

When training classification models, it expects that the learned features are compact within classes, and can well separate different classes. As the dominant loss function for training classification models, minimizing cross-entropy (CE) loss maximizes the compactness and distinctiveness, i. e. , reaching neural collapse (NC). The recent works show that binary CE (BCE) performs also well in multi-class tasks. In this paper, we compare BCE and CE in deep feature learning. For the first time, we prove that BCE can also maximize the intra-class compactness and inter-class distinctiveness when reaching its minimum, i. e. , leading to NC. We point out that CE measures the relative values of decision scores in the model training, implicitly enhancing the feature properties by classifying samples one-by-one. In contrast, BCE measures the absolute values of decision scores and adjust the positive/negative decision scores across all samples to uniformly high/low levels. Meanwhile, the classifier biases in BCE present a substantial constraint on the decision scores to explicitly enhance the feature properties in the training. The experimental results are aligned with above analysis, and show that BCE could improve the classification and leads to better compactness and distinctiveness among sample features. The codes have be released.

AAAI Conference 2025 Conference Paper

CA-Edit: Causality-Aware Condition Adapter for High-Fidelity Local Facial Attribute Editing

• Xiaole Xian
• Xilin He
• Zenghao Niu
• Junliang Zhang
• Weicheng Xie
• Siyang Song
• Zitong Yu
• Linlin Shen

For efficient and high-fidelity local facial attribute editing, most existing editing methods either require additional fine-tuning for different editing effects or tend to affect beyond the editing regions. Alternatively, inpainting methods can edit the target image region while preserving external areas. However, current inpainting methods still suffer from the generation misalignment with facial attributes description and the loss of facial skin details. To address these challenges, (i) a novel data utilization strategy is introduced to construct datasets consisting of attribute-text-image triples from a data-driven perspective, (ii) a Causality-Aware Condition Adapter is proposed to enhance the contextual causality modeling of specific details, which encodes the skin details from the original image while preventing conflicts between these cues and textual conditions. In addition, a Skin Transition Frequency Guidance technique is introduced for the local modeling of contextual causality via sampling guidance driven by low-frequency alignment. Extensive quantitative and qualitative experiments demonstrate the effectiveness of our method in boosting both fidelity and editability for localized attribute editing. Our codes will be made publicly available.

ECAI Conference 2025 Conference Paper

D 3 -Talker: Dual-Branch Decoupled Deformation Fields for Few-Shot 3D Talking Head Synthesis

• Yuhang Guo
• Kaijun Deng
• Siyang Song
• Jindong Xie
• Wenhui Ma
• LinLin Shen

A key challenge in 3D talking head synthesis lies in the reliance on a long-duration talking head video to train a new model for each target identity from scratch. Recent methods have attempted to address this issue by extracting general features from audio through pre-training models. However, since audio contains information irrelevant to lip motion, existing approaches typically struggle to map the given audio to realistic lip behaviors in the target face when trained on only a few frames, causing poor lip synchronization and talking head image quality. This paper proposes D3-Talker, a novel approach that constructs a static 3D Gaussian attribute field and employs audio and Facial Motion signals to independently control two distinct Gaussian attribute deformation fields, effectively decoupling the predictions of general and personalized deformations. We design a novel similarity contrastive loss function during pre-training to achieve more thorough decoupling. Furthermore, we integrate a Coarse-to-Fine module to refine the rendered images, alleviating blurriness caused by head movements and enhancing overall image quality. Extensive experiments demonstrate that D3-Talker outperforms state-of-the-art methods in both high-fidelity rendering and accurate audio-lip synchronization with limited training data.

AAAI Conference 2025 Conference Paper

DAMPER: A Dual-Stage Medical Report Generation Framework with Coarse-Grained MeSH Alignment and Fine-Grained Hypergraph Matching

• Xiaofei Huang
• Wenting Chen
• Jie Liu
• Qisheng Lu
• Xiaoling Luo
• Linlin Shen

Medical report generation is crucial for clinical diagnosis and patient management, summarizing diagnoses and recommendations based on medical imaging. However, existing work often overlook the clinical pipeline involved in report writing, where physicians typically conduct an initial quick review followed by a detailed examination. Moreover, current alignment methods may lead to misaligned relationships. To address these issues, we propose DAMPER, a dual-stage framework for medical report generation that mimics the clinical pipeline of report writing in two stages. In the first stage, a MeSH-Guided Coarse-Grained Alignment (MCG) stage that aligns chest X-ray (CXR) image features with medical subject headings (MeSH) features to generate a rough keyphrase representation of the overall impression. In the second stage, a Hypergraph-Enhanced Fine-Grained Alignment (HFG) stage that constructs hypergraphs for image patches and report annotations, modeling high-order relationships within each modality and performing hypergraph matching to capture semantic correlations between image regions and textual phrases. Finally,the coarse-grained visual features, generated MeSH representations, and visual hypergraph features are fed into a report decoder to produce the final medical report. Extensive experiments on public datasets demonstrate the effectiveness of DAMPER in generating comprehensive and accurate medical reports, outperforming state-of-the-art methods across various evaluation metrics.

ICLR Conference 2025 Conference Paper

DeeperForward: Enhanced Forward-Forward Training for Deeper and Better Performance

• Liang Sun
• Yang Zhang 0012
• Weizhao He
• Jiajun Wen 0001
• LinLin Shen
• Weicheng Xie 0001

While backpropagation effectively trains models, it presents challenges related to bio-plausibility, resulting in high memory demands and limited parallelism. Recently, Hinton (2022) proposed the Forward-Forward (FF) algorithm for high-parallel local updates. FF leverages squared sums as the local update target, termed goodness, and decouples goodness by normalizing the vector length to extract new features. However, this design encounters issues with feature scaling and deactivated neurons, limiting its application mainly to shallow networks. This paper proposes a novel goodness design utilizing **layer normalization** and **mean goodness** to overcome these challenges, demonstrating performance improvements even in 17-layer CNNs. Experiments on CIFAR-10, MNIST, and Fashion-MNIST show significant advantages over existing FF-based algorithms, highlighting the potential of FF in deep models. Furthermore, the model parallel strategy is proposed to achieve highly efficient training based on the property of local updates.

EAAI Journal 2025 Journal Article

Enhancing anomaly detection with few-shot fine-tuned long text-to-image models

• Jiachen Liu
• Jiajia An
• Junbin Lu
• Zhuoqin Yang
• Jinbao Wang
• Ping Lu
• Yuying Wang
• Linlin Shen

Industrial anomaly detection plays a crucial role in the industrial manufacturing field. Currently, utilizing generated data to improve the performance of the anomaly detection model is an effective approach. However, most existing methods often rely on mask-guided synthesis, where the distribution of the generated defects is limited by masks that are typically random or learned by a model. In addition, the scarcity of real anomalous samples makes it difficult for generative models to capture genuine defect patterns and align with the real anomaly distribution. To tackle these issues, we propose DefectGen, the first long-text-guided few-shot text-to-image data generation pipeline for industrial anomaly detection. To improve distribution alignment under limited anomaly samples, DefectGen incorporates a Prompt Generation and Variation Module, which uses MLLMs (Multimodal Large Language Models) to expand few-shot image–text pairs into diverse and semantically rich prompts, and DoKr (Weight-Decomposed Low-Rank Adaptation with Kronecker product), a lightweight fine-tuning strategy with structured low-rank adaptation. To ensure the quality of synthetic data, DefectGen further introduces the Real-Guided Clustering Filter, which selects high-quality generated samples by comparing their features with those of real anomalies. Experiments on the MVTec AD(MVTec AnomalyDetection) dataset show that DefectGen generates more diverse and realistic synthetic anomalies and achieves a 5. 58% average improvement in anomaly classification accuracy compared to state-of-the-art methods. Code and data are available at: https: //anonymous. 4open. science/r/DefectGen-CD04/.

AAAI Conference 2025 Conference Paper

Like an Ophthalmologist: Dynamic Selection Driven Multi-View Learning for Diabetic Retinopathy Grading

• Xiaoling Luo
• Qihao Xu
• Huisi Wu
• Chengliang Liu
• Zhihui Lai
• Linlin Shen

Diabetic retinopathy (DR), with its large patient population, has become a formidable threat to human visual health. In the clinical diagnosis of DR, multi-view fundus images are considered to be more suitable for DR diagnosis because of the wide coverage of the field of view. Therefore, different from most of the previous single-view DR grading methods, we design a dynamic selection-driven multi-view DR grading method to fit clinical scenarios better. Since lesion information plays a key role in DR diagnosis, previous methods usually boost the model performance by enhancing the lesion feature. However, during the actual diagnosis, ophthalmologists not only focus on the crucial parts, but also exclude irrelevant features to ensure the accuracy of judgment. To this end, we introduce the idea of dynamic selection and design a series of selection mechanisms from fine granularity to coarse granularity. In this work, we first introduce an Ophthalmic Image Reader (OIR) agent to provide the model with pixel-level prompts of suspected lesion areas. Moreover, a Multi-View Token Selection Module (MVTSM) is designed to prune redundant feature tokens and realize dynamic selection of key information. In the final decision stage, we dynamically fuse multi-view features through the novel Multi-View Mixture of Experts Module (MVMoEM), to enhance key views and reduce the impact of conflicting views. Extensive experiments on a large multi-view fundus image dataset with 34,452 images demonstrate that our method performs favorably against state-of-the-art models.

NeurIPS Conference 2025 Conference Paper

MedChain: Bridging the Gap Between LLM Agents and Clinical Practice with Interactive Sequence

• Jie Liu
• Wenxuan Wang
• Zizhan Ma
• Guolin Huang
• Yihang SU
• Kao-Jung Chang
• Haoliang Li
• Linlin Shen

Clinical decision making (CDM) is a complex, dynamic process crucial to healthcare delivery, yet it remains a significant challenge for artificial intelligence systems. While Large Language Model (LLM)-based agents have been tested on general medical knowledge using licensing exams and knowledge question-answering tasks, their performance in the CDM in real-world scenarios is limited due to the lack of comprehensive benchmark that mirror actual medical practice. To address this gap, we present MedChain, a dataset of 12, 163 clinical cases that covers five key stages of clinical workflow. MedChain distinguishes itself from existing benchmarks with three key features of real-world clinical practice: personalization, interactivity, and sequentiality. Further, to tackle real-world CDM challenges, we also propose MedChain-Agent, an AI system that integrates a feedback mechanism and a MedCase-RAG module to learn from previous cases and adapt its responses. MedChain-Agent demonstrates remarkable adaptability in gathering information dynamically and handling sequential clinical tasks, significantly outperforming existing approaches. The relevant dataset and code will be released upon acceptance of this paper.

AAAI Conference 2025 Conference Paper

PerReactor: Offline Personalised Multiple Appropriate Facial Reaction Generation

• Hengde Zhu
• Xiangyu Kong
• Weicheng Xie
• Xin Huang
• Xilin He
• Lu Liu
• Linlin Shen
• Wei Zhang

In dyadic human-human interactions, individuals may express multiple different facial reactions in response to the same/similar behaviours expressed by their conversational partners depending on their personalised behaviour patterns. As a result, frequently-employed reconstruction loss-based strategies lead the training of previous automatic facial reaction generation (FRG) models to not only suffer from the 'one-to-many mapping' problem, but also fail to comprehensively consider the quality of the generated facial reactions. Besides, none of them considered such personalised behaviour patterns in generating facial reactions. In this paper, we propose the first adversarial FRG model training strategy which jointly learns appropriateness and realism discriminators to provide comprehensive task-specific supervision for training the target facial reaction generators, and reformulates the 'one-to-many (facial reactions) mapping' training problem as a 'one-to-one (distribution) mapping' training task, i.e., the FRG model is trained to output a distribution representing multiple appropriate/plausible facial reaction from each input human behaviour. In addition, our approach also serves as the first offline FRG approach that considers personalised behaviour patterns in generating of target individuals' facial reactions. Experiments show that our PerReactor not only largely outperformed all existing offline solutions for generating more appropriate, diverse and realistic facial reactions, but also is the first approach that can effectively generate personalised appropriate facial reactions.

AAAI Conference 2025 Conference Paper

S³-Mamba: Small-Size-Sensitive Mamba for Lesion Segmentation

• Gui Wang
• Yuexiang Li
• Wenting Chen
• Meidan Ding
• Wooi Ping Cheah
• Rong Qu
• Jianfeng Ren
• Linlin Shen

Small lesions play a critical role in early disease diagnosis and intervention of severe infections. Popular models often face challenges in segmenting small lesions, as it occupies only a minor portion of an image, while down-sampling operations may inevitably lose focus on local features of small lesions. To tackle the challenges, we propose a Small-Size-Sensitive Mamba (S³-Mamba), which promotes the sensitivity to small lesions across three dimensions: channel, spatial, and training strategy. Specifically, an Enhanced Visual State Space block is designed to focus on small lesions through multiple residual connections to preserve local features, and selectively amplify important details while suppressing irrelevant ones through channel-wise attention. A Tensor-based Cross-feature Multi-scale Attention is designed to integrate input image features and intermediate-layer features with edge features and exploit the attentive support of features across multiple scales, thereby retaining spatial details of small lesions at various granularities. Finally, we introduce a novel regularized curriculum learning to automatically assess lesion size and sample difficulty, and gradually focus from easy samples to hard ones like small lesions. Extensive experiments on three medical image segmentation datasets show the superiority of our S³-Mamba, especially in segmenting small lesions.

AAAI Conference 2024 Conference Paper

Boosting Adversarial Transferability across Model Genus by Deformation-Constrained Warping

• Qinliang Lin
• Cheng Luo
• Zenghao Niu
• Xilin He
• Weicheng Xie
• Yuanbo Hou
• Linlin Shen
• Siyang Song

Adversarial examples generated by a surrogate model typically exhibit limited transferability to unknown target systems. To address this problem, many transferability enhancement approaches (e.g., input transformation and model augmentation) have been proposed. However, they show poor performances in attacking systems having different model genera from the surrogate model. In this paper, we propose a novel and generic attacking strategy, called Deformation-Constrained Warping Attack (DeCoWA), that can be effectively applied to cross model genus attack. Specifically, DeCoWA firstly augments input examples via an elastic deformation, namely Deformation-Constrained Warping (DeCoW), to obtain rich local details of the augmented input. To avoid severe distortion of global semantics led by random deformation, DeCoW further constrains the strength and direction of the warping transformation by a novel adaptive control strategy. Extensive experiments demonstrate that the transferable examples crafted by our DeCoWA on CNN surrogates can significantly hinder the performance of Transformers (and vice versa) on various tasks, including image classification, video action recognition, and audio recognition. Code is made available at https://github.com/LinQinLiang/DeCoWA.

NeurIPS Conference 2024 Conference Paper

HairDiffusion: Vivid Multi-Colored Hair Editing via Latent Diffusion

• Yu Zeng
• Yang Zhang
• Jiachen Liu
• Linlin Shen
• Kaijun Deng
• Weizhao He
• Jinbao Wang

Hair editing is a critical image synthesis task that aims to edit hair color and hairstyle using text descriptions or reference images, while preserving irrelevant attributes (e. g. , identity, background, cloth). Many existing methods are based on StyleGAN to address this task. However, due to the limited spatial distribution of StyleGAN, it struggles with multiple hair color editing and facial preservation. Considering the advancements in diffusion models, we utilize Latent Diffusion Models (LDMs) for hairstyle editing. Our approach introduces Multi-stage Hairstyle Blend (MHB), effectively separating control of hair color and hairstyle in diffusion latent space. Additionally, we train a warping module to align the hair color with the target region. To further enhance multi-color hairstyle editing, we fine-tuned a CLIP model using a multi-color hairstyle dataset. Our method not only tackles the complexity of multi-color hairstyles but also addresses the challenge of preserving original colors during diffusion editing. Extensive experiments showcase the superiority of our method in editing multi-color hairstyles while preserving facial attributes given textual descriptions and reference images.

AIIM Journal 2024 Journal Article

Introduction to the special issue on IEEE CBMS 2022 mining healthcare: AI and machine learning for biomedicine

• Rosa Sicilia
• Linlin Shen
• Alejandro Rodríguez-González
• KC Santosh
• Peter J.F. Lucas

EAAI Journal 2024 Journal Article

Photo realistic synthetic dataset and multi-scale attention dehazing network

• Shengdong Zhang
• Xiaoqin Zhang
• Wenqi Ren
• Linlin Shen
• Li Zhao
• Jun Zhang

Deep learning is a powerful tool in the realm of low-level computer vision and has achieved significant success in image dehazing. However, previous works have predominantly focused on synthetic hazy images, thereby overlooking the inherent differences between real-world hazy images and their synthetic counterparts. These prior approaches face a performance decline when models trained with synthetic hazy images are applied to naturally hazy scenes. In this context, we propose a novel method aimed at minimizing the discrepancies between real and synthetic hazy images, thus enhancing the dehazing performance for real-world scenarios. Specifically, our approach includes refining the synthesized transmission map, which often misses details around object boundaries. Furthermore, we implement a technique that transfers the visual appearance of natural haze onto simulated images. Additionally, we introduce noise into the synthesized hazy images to enhance realism. To demonstrate the effectiveness of our dataset, we introduce a multi-scale attention dehazing network, which delivers state-of-the-art dehazing results. Extensive experiments robustly attest to the superior performance of our proposed method. We further conduct experiments to validate the efficacy of the proposed dataset in addressing the challenges posed by real-world haze.

NeurIPS Conference 2024 Conference Paper

Towards Combating Frequency Simplicity-biased Learning for Domain Generalization

• Xilin He
• Jingyu Hu
• Qinliang Lin
• Cheng Luo
• Weicheng Xie
• Siyang Song
• Muhammad Haris Khan
• Linlin Shen

Domain generalization methods aim to learn transferable knowledge from source domains that can generalize well to unseen target domains. Recent studies show that neural networks frequently suffer from a simplicity-biased learning behavior which leads to over-reliance on specific frequency sets, namely as frequency shortcuts, instead of semantic information, resulting in poor generalization performance. Despite previous data augmentation techniques successfully enhancing generalization performances, they intend to apply more frequency shortcuts, thereby causing hallucinations of generalization improvement. In this paper, we aim to prevent such learning behavior of applying frequency shortcuts from a data-driven perspective. Given the theoretical justification of models' biased learning behavior on different spatial frequency components, which is based on the dataset frequency properties, we argue that the learning behavior on various frequency components could be manipulated by changing the dataset statistical structure in the Fourier domain. Intuitively, as frequency shortcuts are hidden in the dominant and highly dependent frequencies of dataset structure, dynamically perturbating the over-reliance frequency components could prevent the application of frequency shortcuts. To this end, we propose two effective data augmentation modules designed to collaboratively and adaptively adjust the frequency characteristic of the dataset, aiming to dynamically influence the learning behavior of the model and ultimately serving as a strategy to mitigate shortcut learning. Our code will be made publicly available.

NeurIPS Conference 2023 Conference Paper

Learning Visual Prior via Generative Pre-Training

• Jinheng Xie
• Kai Ye
• Yudong Li
• Yuexiang Li
• Kevin Qinghong Lin
• Yefeng Zheng
• Linlin Shen
• Mike Zheng Shou

Various stuff and things in visual data possess specific traits, which can be learned by deep neural networks and are implicitly represented as the visual prior, e. g. , object location and shape, in the model. Such prior potentially impacts many vision tasks. For example, in conditional image synthesis, spatial conditions failing to adhere to the prior can result in visually inaccurate synthetic results. This work aims to explicitly learn the visual prior and enable the customization of sampling. Inspired by advances in language modeling, we propose to learn Visual prior via Generative Pre-Training, dubbed VisorGPT. By discretizing visual locations, e. g. , bounding boxes, human pose, and instance masks, into sequences, VisorGPT can model visual prior through likelihood maximization. Besides, prompt engineering is investigated to unify various visual locations and enable customized sampling of sequential outputs from the learned prior. Experimental results demonstrate the effectiveness of VisorGPT in modeling visual prior and extrapolating to novel scenes, potentially motivating that discrete visual locations can be integrated into the learning paradigm of current language models to further perceive visual world. Code is available at https: //sierkinhane. github. io/visor-gpt.

NeurIPS Conference 2023 Conference Paper

UniTSFace: Unified Threshold Integrated Sample-to-Sample Loss for Face Recognition

• Qiufu Li
• Xi Jia
• Jiancan Zhou
• Linlin Shen
• Jinming Duan

Sample-to-class-based face recognition models can not fully explore the cross-sample relationship among large amounts of facial images, while sample-to-sample-based models require sophisticated pairing processes for training. Furthermore, neither method satisfies the requirements of real-world face verification applications, which expect a unified threshold separating positive from negative facial pairs. In this paper, we propose a unified threshold integrated sample-to-sample based loss (USS loss), which features an explicit unified threshold for distinguishing positive from negative pairs. Inspired by our USS loss, we also derive the sample-to-sample based softmax and BCE losses, and discuss their relationship. Extensive evaluation on multiple benchmark datasets, including MFR, IJB-C, LFW, CFP-FP, AgeDB, and MegaFace, demonstrates that the proposed USS loss is highly efficient and can work seamlessly with sample-to-class-based losses. The embedded loss (USS and sample-to-class Softmax loss) overcomes the pitfalls of previous approaches and the trained facial model UniTSFace exhibits exceptional performance, outperforming state-of-the-art methods, such as CosFace, ArcFace, VPL, AnchorFace, and UNPG. Our code is available at https: //github. com/CVI-SZU/UniTSFace.

IJCAI Conference 2022 Conference Paper

Learning Multi-dimensional Edge Feature-based AU Relation Graph for Facial Action Unit Recognition

• Cheng Luo
• Siyang Song
• Weicheng Xie
• Linlin Shen
• Hatice Gunes

The activations of Facial Action Units (AUs) mutually influence one another. While the relationship between a pair of AUs can be complex and unique, existing approaches fail to specifically and explicitly represent such cues for each pair of AUs in each facial display. This paper proposes an AU relationship modelling approach that deep learns a unique graph to explicitly describe the relationship between each pair of AUs of the target facial display. Our approach first encodes each AU's activation status and its association with other AUs into a node feature. Then, it learns a pair of multi-dimensional edge features to describe multiple task-specific relationship cues between each pair of AUs. During both node and edge feature learning, our approach also considers the influence of the unique facial display on AUs' relationship by taking the full face representation as an input. Experimental results on BP4D and DISFA datasets show that both node and edge feature learning modules provide large performance improvements for CNN and transformer-based backbones, with our best systems achieving the state-of-the-art AU recognition results. Our approach not only has a strong capability in modelling relationship cues for AU recognition but also can be easily incorporated into various backbones. Our PyTorch code is made available at https: //github. com/CVI-SZU/ME-GraphAU.

AAAI Conference 2021 Conference Paper

Adversarial Defence by Diversified Simultaneous Training of Deep Ensembles

• Bo Huang
• Zhiwei Ke
• Yi Wang
• Wei Wang
• Linlin Shen
• Feng Liu

Learning-based classifiers are susceptible to adversarial examples. Existing defence methods are mostly devised on individual classifiers. Recent studies showed that it is viable to increase adversarial robustness by promoting diversity over an ensemble of models. In this paper, we propose adversarial defence by encouraging ensemble diversity on learning high-level feature representations and gradient dispersion in simultaneous training of deep ensemble networks. We perform extensive evaluations under white-box and blackbox attacks including transferred examples and adaptive attacks. Our approach achieves a significant gain of up to 52% in adversarial robustness, compared with the baseline and the state-of-the-art method on image benchmarks with complex data scenes. The proposed approach complements the defence paradigm of adversarial training, and can further boost the performance. The source code is available at https: //github. com/ALIS-Lab/AAAI2021-PDD.

AAAI Conference 2021 Conference Paper

Translate the Facial Regions You Like Using Self-Adaptive Region Translation

• Wenshuang Liu
• Wenting Chen
• Zhanjia Yang
• Linlin Shen

With the progression of Generative Adversarial Networks (GANs), image translation methods has achieved increasingly remarkable performance. However, most available methods can only achieve image level translation, which is unable to precisely control the regions to be translated. In this paper, we propose a novel self-adaptive region translation network (SART) for region-level translation, which uses regionadaptive instance normalization (RIN) and a region matching loss (RML) for this task. We first encode the style and content image for each region with style and content encoder. To translate both shape and texture of the target region, we inject region-adaptive style features into the decoder by RIN. To ensure independent translation among different regions, RML is proposed to measure the similarity between the nontranslated/translated regions of content and translated images. Extensive experiments on three publicly available datasets, i. e. Morph, RaFD and CelebAMask-HQ, suggest that our approach demonstrate obvious improvement over state-of-theart methods like StarGAN, SEAN and FUNIT. Our approach has further advantages in precise control of the regions to be translated. As a result, region level expression changes and step-by-step make-up can be achieved. The video demo is available at (https: //youtu. be/DvIdmcR2LEc).

AAAI Conference 2020 Conference Paper

Group-Wise Dynamic Dropout Based on Latent Semantic Variations

• Zhiwei Ke
• Zhiwei Wen
• Weicheng Xie
• Yi Wang
• Linlin Shen

Dropout regularization has been widely used in various deep neural networks to combat overfitting. It works by training a network to be more robust on information-degraded data points for better generalization. Conventional dropout and variants are often applied to individual hidden units in a layer to break up co-adaptations of feature detectors. In this paper, we propose an adaptive dropout to reduce the co-adaptations in a group-wise manner by coarse semantic information to improve feature discriminability. In particular, we showed that adjusting the dropout probability based on local feature densities can not only improve the classification performance significantly but also enhance the network robustness against adversarial examples in some cases. The proposed approach was evaluated in comparison with the baseline and several state-of-the-art adaptive dropouts over four public datasets of Fashion-MNIST, CIFAR-10, CIFAR-100 and SVHN.

AIIM Journal 2020 Journal Article

Multi-resolution convolutional networks for chest X-ray radiograph based lung nodule detection

• Xuechen Li
• Linlin Shen
• Xinpeng Xie
• Shiyun Huang
• Zhien Xie
• Xian Hong
• Juan Yu

Lung cancer is the leading cause of cancer death worldwide. Early detection of lung cancer is helpful to provide the best possible clinical treatment for patients. Due to the limited number of radiologist and the huge number of chest x-ray radiographs (CXR) available for observation, a computer-aided detection scheme should be developed to assist radiologists in decision-making. While deep learning showed state-of-the-art performance in several computer vision applications, it has not been used for lung nodule detection on CXR. In this paper, a deep learning-based lung nodule detection method was proposed. We employed patch-based multi-resolution convolutional networks to extract the features and employed four different fusion methods for classification. The proposed method shows much better performance and is much more robust than those previously reported researches. For publicly available Japanese Society of Radiological Technology (JSRT) database, more than 99% of lung nodules can be detected when the false positives per image (FPs/image) was 0. 2. The FAUC and R-CPM of the proposed method were 0. 982 and 0. 987, respectively. The proposed approach has the potential of applications in clinical practice.

JBHI Journal 2018 Journal Article

A Solitary Feature-Based Lung Nodule Detection Approach for Chest X-Ray Radiographs

• Xuechen Li
• Linlin Shen
• Suhuai Luo

Lung cancer is one of the most deadly diseases. It has a high death rate and its incidence rate has been increasing all over the world. Lung cancer appears as a solitary nodule in chest x-ray radiograph (CXR). Therefore, lung nodule detection in CXR could have a significant impact on early detection of lung cancer. Radiologists define a lung nodule in CXR as “solitary white nodule-like blob. ” However, the solitary feature has not been employed for lung nodule detection before. In this paper, a solitary feature-based lung nodule detection method was proposed. We employed stationary wavelet transform and convergence index filter to extract the texture features and used AdaBoost to generate white nodule-likeness map. A solitary feature was defined to evaluate the isolation degree of candidates. Both the isolation degree and the white nodule likeness were used as final evaluation of lung nodule candidates. The proposed method shows better performance and robustness than those reported in previous research. More than 80% and 93% of lung nodules in the lung field in the Japanese Society of Radiological Technology (JSRT) database were detected when the false positives per image were two and five, respectively. The proposed approach has the potential of being used in clinical practice.

ICML Conference 2017 Conference Paper

Kernelized Support Tensor Machines

• Lifang He 0001
• Chun-Ta Lu
• Guixiang Ma
• Shen Wang 0005
• LinLin Shen
• Philip S. Yu
• Ann B. Ragin

In the context of supervised tensor learning, preserving the structural information and exploiting the discriminative nonlinear relationships of tensor data are crucial for improving the performance of learning tasks. Based on tensor factorization theory and kernel methods, we propose a novel Kernelized Support Tensor Machine (KSTM) which integrates kernelized tensor factorization with maximum-margin criterion. Specifically, the kernelized factorization technique is introduced to approximate the tensor data in kernel space such that the complex nonlinear relationships within tensor data can be explored. Further, dual structural preserving kernels are devised to learn the nonlinear boundary between tensor data. As a result of joint optimization, the kernels obtained in KSTM exhibit better generalization power to discriminative analysis. The experimental results on real-world neuroimaging datasets show the superiority of KSTM over the state-of-the-art techniques.

AAAI Conference 2016 Conference Paper

Analysis-Synthesis Dictionary Learning for Universality-Particularity Representation Based Classification

• Meng Yang
• Weiyang Liu
• Weixin Luo
• Linlin Shen

Dictionary learning has played an important role in the success of sparse representation. Although synthesis dictionary learning for sparse representation has been well studied for universality representation (i. e. , the dictionary is universal to all classes) and particularity representation (i. e. , the dictionary is class-particular), jointly learning an analysis dictionary and a synthesis dictionary is still in its infant stage. Universality-particularity representation can well match the intrinsic characteristics of data (i. e. , different classes share commonality and distinctness), while analysis-synthesis dictionary can give a more complete view of data representation (i. e. , analysis dictionary is a dual-viewpoint of synthesis dictionary). In this paper, we proposed a novel model of analysis-synthesis dictionary learning for universalityparticularity (ASDL-UP) representation based classification. The discrimination of universality and particularity representation is jointly exploited by simultaneously learning a pair of analysis dictionary and synthesis dictionary. More specifically, we impose a label preserving term to analysis coding coefficients for universality representation. Fisher-like regularizations for analysis coding coefficients and the subsequent synthesis representation are introduced to particularity representation. Compared with other state-of-the-art dictionary learning methods, ASDL-UP has shown better or competitive performance in various classification tasks.